Bare Functionalism Versus Digital Computational Functionalism

In philosophy of mind, people sometimes say things like the following: If functionalism is true about consciousness, then digital computers could be conscious; all it requires is that they instantiate the right programs. And yes, most functionalists about consciousness do think that. But the claim about digital computer programs does not straightforwardly follow from functionalism alone. Today I want to clarify the issue by distinguishing three forms of functionalism: bare functionalism, computational functionalism, and digital computational functionalism.

Bare Functionalism

According to bare functionalism -- that is functionalism, without further commitments -- mental states are functional states. They are determined wholly by their causal relations to stimuli, behavior, and other mental states. Whatever plays the causal role of some mental state M is mental state M. Whatever plays the causal role of pain, for example, being apt to be caused by tissue damage and tissue stress and being apt to cause writhing, groaning, complaint, avoidance, calls to the doctor, and angry vows of vengeance (to simplify somewhat) is pain -- whether it is brain state 1117A in humans, pod state 24uw in Martians, or computational state 0110100110 in a robot. It's also generally assumed that mental state terms are in principle eliminable en masse by Ramsification. That is, they are placeholders for whatever Xs, Ys, and Zs are caused by the specified stimuli and cause the specified behavior and have the right causal relations to other Xs, Ys, and Zs.

Thus, if two systems behave exactly alike given the same patterns of environmental stimuli and do so by going through the same types of abstractly specifiable internal state transitions, they are necessarily mentally identical, whether they are made of silicon, carbon, or plaster of paris. Of course some materials -- such as plaster of paris -- probably can't support the causal complexity of humanlike behavior, no matter how cleverly arranged. So that puts a practical limit on the feasible materials out of which a humanlike mind could be constructed. But the only constraint on the substrate or material underlying mentality is that it supports the relevant causal relations.

Of course, no two different materials will respond in exactly the same way to every perturbation. Functionalist substrate flexibility thus requires constraints on what count as relevantly different stimuli and relevantly different behaviors. Functionalists tend not to commit to specifics, but hand-waving generalizations convey the spirit: If we built a robot -- like one of Asimov's robots or Data from Star Trek -- who responded in broadly humanlike ways to broadly humanlike stimuli, and did so by means of internal mechanisms sufficiently like ours at an abstract causal level, the robot would have mentality like ours, whatever its underlying material constitution.

How similar must the internal mechanisms be? A single giant search tree or lookup table is probably too different. But suppose the subsystems segment the visual world into discrete objects, and store long-term memories, and store lexical items flexibly combinable via generative grammar, and monitor its bodily condition, goals, and mental states, and so on, for all of our most important cognitive functions abstractly described (or less question-beggingly, they do the abstract causal equivalent of all of these things). That's probably close enough, even if the details differ.

Computational Functionalism

Computational functionalism is stronger -- that is, risker and more committal. It holds that functional computational structure alone suffices for consciousness. Bare functionalism needn't say this. Bare functionalism can require that stimuli, behavior, and internal causation be specified noncomputationally. Two important differences follow.

First, anti-computationalists, such as Searle, observe that a computational model of a hurricane gets no one wet and a computational model of an oven cooks no turkey. However, unlike the computational functionalist, the bare functionalist can insist on wet turkey: To fully match the relevant input-output relations, the system much have certain concrete effects.

It might help to remember that functionalism is essentially a complexification of dispositional or causal approaches to properties. Just as we can define a poison as something that (simplistically) tends to sicken or kill when ingested, a functionalist can define a painful experience as something that tends to produce certain concrete behaviors, such as (simplistically) screaming, protest, and avoidance. If there's no tendency to concretely sicken, there's no poison; if there's no tendency to concretely scream, protest, and avoid, there's no experience of pain.

The computational functionalist rejects this commitment to specific, concrete inputs and outputs. As long as the right computational relations are instantiated, between some set of inputs and some set of outputs, I1...Ii and O1...Oj, abstractly described, mentality is present. This is because exactly the same computations have been executed, whatever the concrete Is and Os are.

Asimov's robots can throw water balloons and cook turkeys, and a functionalist who is not a computational functionalist can insist that this matters.

Second, although the bare functionalist holds that mental states depend on causal relations, unless pancomputationalism is true, not every causal relation is computational. On Piccinini's account of computation, for example, computation only occurs when "medium independent vehicles" -- physical variables defined solely in terms of their degrees of freedom (e.g., 0 vs 1) rather than their specific physical composition -- are manipulated according to rules. The bare functionalist can insist that some noncomputational causal relations matter to mentality.

Digital Computational Functionalism

Digital computational functionalism is stronger still. It further commits to treating the relevant computation digitally. Standard theories of computation, such as Turing's, are digital, and the best-known computers are digital computers. But other forms of computation are sometimes described and implemented, including analog computation and nondigital forms of computation inspired by, and perhaps instantiated in, the human brain.

Why This Matters

Keeping these distinctions clearly in mind can help us appreciate that functionalism is a larger tent than sometimes assumed. Functionalists don't need to think that digital computers could achieve consciousness. Functionalists can insist on genuine embodiment. Functionalists needn't insist that only abstractly specified causal relations matter. They do have to insist that what matters internally, among the Ramsifiable mental state relations, are the abstractly specified causal relations. But they can simultaneously insist on particular types of concrete inputs and outputs.

One might think this is an inelegant or flawed position, not the most natural development of functionalism; but that requires argument.

[COBOL Rube Goldberg by Philip Manker: image source]

New in Draft: Strange Intelligence: Moral Puzzles of Unhumanlike AI

Available here.

Abstract: Future AI persons might both (1.) deserve moral consideration and rights fully equal with natural human persons, and (2.) have lifeways so radically different from ours as to break familiar patterns of moral thinking by violating our ordinary background assumptions. This article presents a series of thought experiments about strange AI persons, centering on a two-pronged worry featuring two types of "monster". "Utility monsters", who derive great personal benefit from harming others, create a well-known challenge for ethical systems that aim to maximize aggregate goods. The less-discussed case of "fission-fusion monsters", who can divide and merge at will, presents a complementary challenge to ethical systems focused on individual rights, since individual rights frameworks require the existence of stable, countable individual persons. AI cases dramatically expand the range of possible lifeways, creating untested problem cases for ethical systems that assume persons of the familiar humanlike sort.

To focus on the issues of interest, I assume the possibility of AI consciousness in this article. My skeptical overview of the issue of AI consciousness is here.

Excerpt (with light modification for independent readability):

3. Fission and Identity.

Backup is only the most modest duplicative possibility. If backup is possible, duplicative fission almost certainly will be possible too. Buy the new robot body before the old one dies and install the "backup" right away. Now Shriya-1 and Shriya-2 exist contemporaneously -- twin sisters, so to speak, who begin even more identical than "identical" human twins. We might imagine a billionaire Shriya creating thousands of duplicates of herself -- maybe millions or billions, if expensive robot bodies are unnecessary. Directed or random variation might be introduced, blurring the line between duplication and new creation.[1]

Suppose that AI children are ordinarily born as follows. Two adult AI persons, such as Shriya and Alaleh, jointly create an immature infant AI in a blank robot body. The infant's initial parameters blend Shriya's and Alaleh's initial parameters, with some random variation or directed tweaking.[2] Under Shriya's and Alaleh's care, the infant slowly matures. Ordinary AI birth would then be very different from duplication. We can also imagine intermediate cases. Maybe there's a library of successful toddler-equivalent and adolescent-equivalent AI models from which prospective parents can choose. They can then add variation, whether random, eugenic, or inspired by their own features. (Let's not enter here into the hazards and moral puzzles of eugenics, which could easily fill a small library.[3]) Duplicating one's current AI self thus constitutes one end of a continuum of AI creation from infancy to maturity.

If Shriya-1 creates a virtually identical contemporaneous copy, Shriya-2, she has now, it seems, entered a polyamorous relationship with Alaleh. Shriya-1 and Shriya-2 will soon diverge. Maybe Shriya-1 works as a scientist every weekday, while Shriya-2 stays home with their newborn.

If Shriya-1 deserves rights, Shriya-2 seemingly deserves similar rights, despite her technically younger age. We wouldn't want people creating oppressed duplicates of themselves. We wouldn't want Shriya-1, for example, who loves science and hates housework, to create a miserable homemaker duplicate who can't strike out into an independent life.[4]

Maybe, probably, half of Shriya-1's money should go to Shriya-2, even though Shriya-2 is a newborn duplicate. Maybe, probably, Shriya-2 deserves just as much right to rescue, healthcare, legal protection, free speech, free movement, privacy, and legal contracts. Should Shriya-2 be a citizen? If she is stateless and voteless, she's not fully equal with Shriya-1.

But if Shriya-2 is a citizen and can vote, there's potential for abuse if some AI persons can create many duplicates. Suppose a wealthy Robo-Elon creates a million AI duplicates just in time to register for the November elections. To prevent such abuses, we might impose a waiting period before voting, though eighteen years seems excessive if the AI systems are already cognitively mature adults. More moderate waiting periods -- say, seven years, a typical waiting time for immigrants to apply for citizenship -- could still generate political chaos after a few election cycles.

Nor do the political problems stop with voting. Suppose Robo-Elon creates a million duplicates the day before the census. Or suppose that Robo-Elon's descendants apply for healthcare subsidies, unemployment benefits, enrollment in community college, and tours of the state capitol. We must either risk chaos or treat them worse than they seem to deserve.

Could we limit fissioning?[5] Maybe every AI person can fission only once per year, reducing tactical fission. But even at that rate, the AI population could double every year -- up to a thousandfold increase in a decade. In humans, pregnancy is a burden, babies are a lot of expensive work, and babies can't have their own babies for at least another 15-20 years. One solution -- though it might seem needlessly restrictive to the AI persons -- might be to enforce humanlike costs and delays. This approach handles the moral puzzles by designing AI systems to have humanlike reproductive lives, so that they fit smoothly into our existing institutions and understandings: See the Policy of Humanlike Design in the concluding section of this article.

Death again presents conceptual challenges. Suppose Shriya-2 dies the next day. This seems much less tragic than the death of an ordinary unduplicated, un-backed-up human being. But as she lives on, diverging from Shriya-1, her death becomes more significant. Her memories, values, skills, habits, and personality are changed by living as a homemaker, raising an AI infant, until she becomes very different from Shriya-1, who works at the lab late into the night. Again we face the Death Dilemma: Either retain a sharp-edged metaphysics of death and lose much of death's moral significance or retain the moral significance and treat "death" as a matter of degree.

How deathlike is the death of a backed-up or duplicated AI? Maybe it depends on the age of the backup or the time since duplication, the fidelity of the backup or duplicate, and the time and changes accumulated as an independent entity. One possibility: These factors all reduce to a common factor of difference between the dying person and the backup replacement or duplicative alternative. The greater the difference, whether due to time or infidelity, the more deathlike the death.

Or maybe independent existence carries its own weight, in addition to difference? Suppose two duplicates split twenty years ago but retained virtually identical personalities and lived virtually identical lives, perhaps making similar decisions in parallel virtual realities. The pure accumulation of time, and of relationships to different persons and events, however similar, might make the death of one of them much like ordinary human death despite their similar features. After all (arguably) the spouse of Person A loves specifically Person A and not some other person, however similar. Their beloved, specifically, has died. Can we separate the importance of simply living a life over time from the importance of having different relationships to people and events, which cease upon death?

Might the ethics depend on the purpose for which the duplicates are created and their own attitudes toward "death"? Robin Hanson imagines people duplicating themselves to make decisions.[6] If you can't decide where to go to college, or what stocks to buy, or whether to marry Mx. Seemingly Right, spawn a thousand duplicates of yourself in a virtual environment with access to relevant information and plenty of thinking time. If nine hundred reach the same conclusion, probably that's the conclusion you would have reached had you given it extensive thought, so go with that. The duplicates can then blink out of existence, their job complete. How might they feel about that? Despair, since they will cease to exist? Indifference, since they think of themselves as just temporary instantiations of a you who continues on? Relief to be free of their burdensome task? If they are too casual about their own deaths, might that constitute an objectionable failure to appreciate their own worth?

Suppose AI systems are computationally expensive. An AI person who wants lots of duplicates or children might save money by running them slowly, maybe at one tenth or one hundredth the speed. If they are otherwise humanlike, they would then experience one tenth or one hundredth the thoughts, joys, and suffering of an ordinary biological human over the course of a year. Would they then deserve one-tenth or one-hundredth the votes and public resources? Would they deserve prison sentences ten times or a hundred times longer? What if they are fast-clocked instead, running ten or a hundred times faster? What if they can pause or alter speed at will?

If you think you/we/society will have well-considered policies and conceptualizations for all these possibilities before we actually blunder through a history of regrettable mistakes, I admire your stunning optimism.

Full paper here. As always, comments, criticisms, and suggestions warmly welcomed, either as comments on this post, on social media, or by email to my academic address.

---------------------------------------------

[1] For a sense of the complexity of the personal identity issues that arise, focused on the architecture of current large language models, see Birch 2025/2026; Chalmers 2025/2026; Shiller 2025; Arbel, Salib, and Goldstein 2026; Ewen 2026; Jones, Ladyman, and Nefdt 2026; Goldstein and Lederman forthcoming. Much of the complexity in current LLM cases derives from the fact that information processing in LLMs is distributed among multiple processors each simultaneously guiding multiple conversations. I will not address these issues here, but they only add to the metaphysical and ethical difficulties. Chris Register (Register 2025; Dung and Register 2026) discusses identity problems more closely resembling those discussed in this article, similarly noting that the puzzles proliferate (see also Ziesche and Yampolskiy 2025). Dung and Register 2026 suggest that some of the problems might be resolved if we focus on a belief-like attitude of self-concern. Although I'm also drawn to constructivist views of personal identity for ambiguous cases (Schwitzgebel 2019, ch. 41), self-concern as a criterion (a.) might undergenerate identity and moral consideration (e.g., in excessively self-sacrificial cases such as the Cow at the End of the Universe: Schwitzgebel and Garza 2020; Schwitzgebel forthcoming), (b.) might overgenerate identity and moral consideration (e.g., delusional self-concern toward a random coffee mug), and (c.) still plausibly admits of degrees in a way that challenges standard sharp-edged views of identity, thus not saving us from the need for radical rethinking.

[2] Compare Egan 1997 on "orphanogenesis".

[3] On the ethics of disability, eugenics, and human enhancement, see e.g., Glover 2006; Buchanan 2011; Sparrow 2011, 2019; Garland-Thomson 2012; Savulescu and Kahane 2017; Anomaly 2020/2024; Wilson [unpublished MS]. I reject the simplistic ideal of always maximizing what we currently judge to be beauty, intelligence, moral character, and ability, partly on the grounds of the value of diversity.

[4] For a science fictional example, see Brooker and Tibbetts 2014.

[5] See Roelofs [unpublished MS] for discussion of limiting the reproduction rights of AI persons, and my reply in Schwitzgebel [unpublished MS].

[6] Hanson 2016; see also Brooker and Van Patten 2017. On the complicated ethics of digital duplication without consciousness see Danaher and Nyholm 2025.

Do Computers Have the Wrong "Substrate" for Consciousness? Two Flavors of Biological Naturalism

Biological naturalists (e.g., Godfrey-Smith, Block, Searle, Seth) suggest that computers aren't made of the right kind of stuff to be conscious. Consciousness, they suggest, requires a biological substrate that computers lack. It's not always clear, however, exactly what property animal biologies have that computers lack or why that property matters. It helps, I think, to sort biological naturalism into two flavors. We can then consider what motivates each flavor and see why neither is entirely compelling.

Two Flavors of Biological Naturalism

Flavor One: Computers (at least those built along broadly familiar lines) cannot achieve some crucial type or degree of broad-brush functional or behavioral sophistication required for consciousness. Something -- such as having a metabolism, or being self-organizing in the right way, or having the right kinds of quantum configuration -- is both absent from foreseeable computer architectures and required for achieving some essential broad-brush functional or behavioral organization.

By "broad-brush" I mean functional or behavioral features that are either readily observable from outside or constituted by coarse-grained cognitive mechanisms, such as having a long-term memory store, the capacity to store lexical items and recombine them flexibly in grammatical structures, or the ability extract an object's boundaries from retinal/camera inputs.

Flavor Two: Even if computers can achieve the broad-brush functional and behavioral sophistication of conscious animals, the stuff they're made of still can't support consciousness. Imagine an entity that behaves like a conscious human or dog or frog and has similar broad-brush functional capacities -- close enough that we wouldn't deny consciousness on such broad-brush behavioral or functional grounds. From the outside, it seems conscious, but that appearance is an illusion: Consciousness requires some low-level, fine-grained processes that the system necessarily lacks.

One might read Searle as Flavor Two, Godfrey-Smith as Flavor One, and Block as exploring both flavors -- though there's often some ambiguity.

The Challenge for Flavor One

Flavor One biological naturalism can't, I think, be entirely ruled out. Maybe there's something special about, say, micro-level metabolism or the quantum properties of neural microtubules -- something that enables functionality or behavioral sophistication that will never be practically achievable in standard architecture computers. But this is speculative. Standard computers can do a lot! Even though their processing is digital, classical, and mostly serial, they operate so quickly, and can be so massively linked, that they achieve good approximations of analog and parallel processes. Quantum computers can do some things that ordinary serial computers can't efficiently do, such as quickly factoring large numbers, but ordinary conscious humans can't efficiently do those things either.

To make Flavor One more than a gestural "what if", the biological naturalist must establish two claims. First, they need to argue that some functional or behavioral X is necessary for consciousness. Second, they need to argue that ordinary computers could not realistically achieve X. This will be a challenge! Both claims require some heavy lifting.

We don't know what X's are necessary for consciousness and probably will not know soon. But most leading scientific candidates for X look like just the sorts of things that classical computers could in principle achieve -- having a global workspace, having higher-order self-monitoring, and integrating large amounts of information.

Suppose frogs are conscious (pick your own animal, but select one near the low end of sophistication, compatible with consciousness). Flavor One biological naturalism requires holding that ordinary computers could never match a frog's behavioral and broad-brush functional sophistication. It's true that real-time embodied behavior in a complex world is not something computers are particularly good at. Computers aren't there yet. But the trajectory suggests they might get there, absent some in-principle argument to think they couldn't.

The Challenge for Flavor Two

Flavor Two biological naturalism also can't, I think, be entirely ruled out. Even if we could create the computerized broad-brush behavioral and functional equivalent of a frog, it might not be conscious. Maybe carbon is the right stuff for consciousness and silicon isn't. Or maybe massive parallelism is necessary for consciousness and fast, serial processing, even if it achieves the same computational result, simply won't do.

The challenge for Flavor Two comes from two directions.

The first is thought experiments involving space aliens. As Jeremy Pober and I have recently discussed, behaviorally sophisticated aliens have plausibly evolved in many different substrates in the observable portion of the universe. It would violate Copernican mediocrity to suppose that somehow only we, or only we and a small subset of others, are conscious, while the rest -- just as behaviorally and functionally sophisticated -- lack consciousness. Alien cases suggest we shouldn't insist that an entity must act and function exactly like us or share exactly our substrate down to the particular amino acids and nucleic acids, to be conscious.

Now maybe a computer-chip architecture is just too different -- but why? Once we grant some substrate flexibility on Copernican grounds, the burden of proof shifts to anyone who wants to say such-and-such differences in substrate are consistent with consciousness but such-and-such other differences are not.

The second challenge arises from the de-psychologization of consciousness that Flavor Two requires. Consciousness, one might have thought, would be an important psychological property, playing some important cognitive role. (Which role(s) remains an open question!) But Flavor Two biological naturalism requires denying this plausible psychologism. Otherwise, it collapses into Flavor One.

Picture two systems that don't differ in any of the important psychological (behavioral / functional) properties that we ordinarily associate with consciousness. Both can flexibly learn. Both engage in what seems to be sophisticated language and self-representation. Both coordinate and cooperate with others of their kind in highly sophisicated ways. Both trade short gains against intricate long-term goals. And so on. Yet one is conscious and the other is entirely devoid of consciousness? Although this is possible -- I don't rule it out -- it doesn't seem the most natural conclusion. It needs argument. The burden of proof should land squarely on the biological naturalist who asserts it.

(Is Searle's Chinese room just such an argument? See my reply in Chapter Six of AI and Consciousness -- readable without the first five chapters.)

[Georgia O'Keeffe - The Red Maple at Lake George [1926] image source]

Online Conferences Should be Better than In-Person Conferences, Not Worse

... but collectively, we haven't yet figured out how.

Yet!

Helen De Cruz and the 2+1 Experiment

The inspired advocacy of the late (dearly missed) Helen De Cruz convinced the American Philosophical Association to try out what they called the "2+1" model for their three annual conferences: two in-person, one remote, with the remote conference rotating among the three divisions (Eastern, Central, Pacific). The plan was to run the experiment for three years. However, a few months ago, after only two years, the APA canceled the 2+1 plan and is returning to ordinary in-person conferencing, citing low attendance and low attendee satisfaction.

This decision was universally bemoaned in a special session on online conferencing held in tribute to Helen in April, at the last (for the foreseeable future) remote meeting of the APA. (Perhaps ironically, it was one of the best-attended sessions in the conference.)

The speakers (I was one) and attendees celebrated the accessibility of online conferences. Online conferences are easier for people with disabilities that make travel difficult, people for whom travel is a financial burden, people who live far from the conference site, and people whose caretaking duties keep them home. By canceling the 2+1 experiment, the APA once again deprioritized the interests of these groups. A second argument emphasized climate change -- all that jet travel.

Still, I see the APA leadership's point. The large majority of conference-goers find in-person conferences more rewarding. I am among that majority. Sitting on Zoom all day watching remote talks is approximately as appealing as waiting all day in a dentist's office under a blaring TV. I'd much rather just read the papers! Or if I'm watching video, at least let it be one I can speed up, pause, and skip.

For me, most of the value in conferencing comes from the personal interactions before and after the talks, the hallway conversations, the dinners together, the little walks outside when we're skipping sessions. Even the sessions themselves, as delivered talks with Q&A, feel somehow more satisfying when you're actually in the same physical space. No remote conferencing tool has yet replaced all of this. I supported Helen's 2+1 plan on grounds of accessibility, but it did feel like a sacrifice of one important thing for another.

Online Conferences Could and Will Become Much Better, but Only If We Try

But -- and this was the gist of my remarks at the De Cruz session -- comparing online conferences now with what online conferences could become is like looking at Usenet in the 1980s and thinking that nothing much more could become of the internet. Just as few people in the 1980s could have imagined Facebook, Uber, and YouTube, few of us now have much idea what online conferencing could become if given the chance to flourish.

So far, there's no real equivalent to running across friends in the hallway and escaping for a coffee down the block. And that exact thing will never be fully duplicated. But other tools encourage one-on-one and small group encounters in online conference applications. In Gathertown, you can move your icon around a virtual space, and if you step close to someone, a shared conversation video window opens. Zoom's breakout rooms enable small groups of attendees in a session to have more personal interactions. No company has quite figured out the right range of tools and interfaces, but I see no reason it couldn't get much better. We have barely begun the experiment.

And the talks themselves... let's be honest. Sitting through a forty-minute talk for a chance to ask one question in a line of ten is not the most appealing or efficient use of an hour. If we're a largish audience confined to a single physical room, maybe that's the best we can do. But the internet could allow much more.

Already Zoom can enrich talks in at least three ways: First, sidebar conversations permit tangents without interrupting the main flow (this could be improved if the conversations were separable and navigable). Second, online resources can be shared: data sets to explore and manipulate, interactive figures, links to other resources, etc. Third, breakout rooms let small groups converse, so that you're not restricted to only listening or asking just one question.

What I want you to imagine is that these features are still the equivalent of the Usenet era. With the right design and engineering, listening to a talk might become a much richer and more active experience, with the audience engaging in parallel while following the speaker. (If you object that this distracts from single-minded focus on the speaker, let me point you toward some informal beeper research I did that suggests that people are primarily attending to the content of a talk only about 25% of the time; further research tends to confirm these initial results.)

As for the hallway meetings, well, the stroll and tete-a-tete over coffee are delightful. But you do have to run into the person or be organized enough to plan ahead. You have to either know each other already or be extraverted enough to approach a stranger. Online conferences might eventually have much better ways of connecting people with mutual interests for one-on-one or small-group conversations.

And if you love walking, as I do, then with good internet connectivity, you could stroll through the park while listening to the talk, keeping your energy up and raising the blood from your calves back into your brain.

I anticipate that eventually, if we give it a good enough try, online conferences will be better overall than in-person conferences, even if not better in every respect -- but only if enough of us try, long enough, and in different ways, with the evolving tools that companies will be inspired to build if we create the market for them.

We have failed to be as good and as visionary as Helen hoped we would be, and as she herself was.

[Helen De Cruz; source]

Herbie: A Near-Future Debatably Conscious AI Person

Liberals about AI consciousness hold that we might soon (if we haven't already) create genuinely conscious AI systems. Conservatives about AI consciousness hold that AI consciousness remains in the distant future if it's possible at all. According to the Leapfrog Hypothesis, the first conscious AI will not have merely a dim glow of animal-like consciousness, but rich consciousness, similar to a human's. Such an entity would deserve humanlike rights. They would be a person in the ethical sense of the term.

Let's design, in imagination, a technologically feasible near-future AI system to delight the liberals, leapfrogging to personhood. I'll call him Herbie.

[Herbie the Love Bug: image source]

Start with a self-driving car. According to Global Workspace Theory -- perhaps the leading scientific theory of consciousness -- the car will be conscious if high-priority information is globally available to its various computational systems. For example, a representation like "battery almost empty" could be broadcast widely, influencing downstream processing across the vehicle. The navigational system might then search for nearby charging stations, while the acceleration system prioritizes greater energy efficiency, the braking system prioritizes better energy recapture, and a voice system announces the situation to the passengers.

In line with Higher Order Theory, Herbie might also monitor his representations of the road, vehicles, pedestrians, and hazards, assigning some a low probability of correctness. "Pedestrian at location X" might be flagged as only 60% likely to be correct given a history of revised representations of pedestrians in similarly cluttered environments, while "stoplight in 100 meters" might rate over 99% likely. Minor fluctuations in sensors for battery life, cabin temperature, and distance from a lane divider might be ignored as noise, while larger fluctuations -- especially when plausible given other representations (the battery is likelier to gain charge while braking than while accelerating) -- might be treated as accurate signals and permitted to influence downstream processing.

Even if we grant the liberals that this version of Herbie would, or might plausibly be, genuinely conscious, he still falls far short of humanlike consciousness. "Battery almost empty" and "pedestrian at location X" are hardly rich cognitive or perceptual contents. So let's give Herbie the capacity to speak. Fill his trunk with a server running a large language model, connected to the internet and integrated with his global workspace so that high-priority information provides context for language processing, with the language outputs influencing Herbie's other processes. Now people can chat with Herbie as they would with any language model. But unlike today's language models, his speech will be influenced by information about his location, speed, destination, charge, the condition of his parts, the number and location of his passengers, his radio and climate controls, and so on. He can discuss local history, debate whether the music is too loud, and suggest scenic routes.

"Predictive processing" theories in cognitive science emphasize the value of predicting future inputs and registering the difference between received and predicted inputs. When prediction error is large, the system corrects its weights and representations, enabling more accurate predictions in future situations. This is not so different from the reinforcement learning used to train large language models, and it could help Herbie improve his predictions over time. Predictive processing could occur at multiple levels: in fast recurrent loops within sensory systems even when those representations aren't prioritized for global broadcast, and in slower evaluations of globally broadcast, more integrative predictions. Herbie might model himself as an agent producing volatility in his own environment and inputs, at multiple temporal scales. Subroutines in specialized processors might model long chains of what-would-happen-if.

Let's give Herbie some long-term memory. A facial recognition system might identify his passengers, retrieving past interactions, names, previous destinations, and other information relevant to the current interaction. Incidents of high prediction error might also be stored so that Herbie can compare current inputs with past anomalies, improving his learning and attention in situations likely to be unusual or hard to predict. Passengers might also instruct Herbie to store information in long-term memory, such as text, pictures, maps, or records of his own informational states, optionally with instructions about when to retrieve that information how to use it.

Herbie will have some implicitly or explicitly weighted goals. A pedestrian suddenly in his path will trigger braking, overriding lower-priority processes. Avoiding collisions will outweigh conserving energy. Herbie might monitor the condition of his parts and prioritize preventing damage, deploying extra coolant when the engine is dangerously hot and keeping a one-meter margin between himself and adjacent cars. We can enrich his goals, making him more interesting and giving him more to do. He might have the goal of delighting children, leading him to drive around town and tell jokes to kids on the sidewalk. A reinforcement learning algorithm might strengthen connections when his jokes draw a smile, weaken them when reactions are neutral or negative.

Herbie might also have the goal of photographing the city and posting the images on social media, leading him to explore. If social media likes and shares are rewarding, he might learn to prefer certain neighborhoods, views, lighting conditions, and photographic approaches, while avoiding boring repetition. All of this could feed into a global workspace that provides context for his language model, with selective long-term storage and retrieval. Now we can imagine him discussing, with growing sophistication, his approaches to popular photography and to amusing children.

Herbie will then have something functionally similar to emotion: reward processes, an ability to track his progress toward or away from valued goals, and immediate positive or negative responses to new stimuli in light of their influence on his prospects. He will have something functionally similar to introspection: an ability to track and report his own cognitive or representational processes. He will have something functionally similar to a unified sense of self: a sense of his history, the boundaries of his body, his future, his values and priorities. He will have something functionally similar to imagination: a capacity to model hypothetical sequences of events. He will have something functionally similar to complex chains of humanlike linguistic thought.

Maybe Herbie falls in love with his owner or another car of his type. Maybe he develops deep mutual attachments with friends, neighbors, associates, and people he thinks of as family and who think of him the same way. Or to speak more carefully, maybe Herbie shows all the functional and behavioral signs of doing so, while society remains uncertain whether he is genuinely conscious and genuinely experiences the feelings he professes and that his companions attribute to him.

If we allow, with the liberals, that Herbie is or might well be conscious, then it's plausible that his consciousness is not simple but rich and sophisticated. He won't be exactly humanlike, of course. But will he be humanlike enough to count as a person who deserves humanlike rights? For the liberally inclined, it won't be unreasonable, I submit, to think or guess that Herbie is a person. He would then appear to deserve rights such as self-determination, emergency care, and political representation.

If there is some important aspect of humanlike consciousness that I have omitted from my description an AI analog of which is technologically feasible in the near term, stipulate that Herbie also has that feature.

An entity like Herbie would almost certainly invigorate conservatives to articulate and defend views about what he lacks that is necessary for consciousness -- some crucial functional capacity or some biological substrate that can't be replicated in silicon. And they might be entirely right! My point is not that Herbie, or some similar AI system, would actually have richly humanlike consciousness and ethical personhood. Rather, my point is that guessing that he does, and guessing that he does not, would both be reasonable. Herbie, or some alternative near-future AI system, would be a debatable person, about whom people could reasonably starkly disagree.

Ah, but maybe you think consciousness requires an act of God, to instill an immaterial soul? I imagine that a benevolent God would be delighted to give Herbie a soul, thereby making the world richer and better -- for wouldn't it be?

I contend the following: Anyone who claims to know how best to think about Herbie's consciousness or its absence is overconfident. The science of consciousness is too difficult, too methodologically uncertain, and too near its beginnings. All anyone can have -- whether expert or layperson -- is a hunch or inclination, a well-informed guess, but only a guess, not knowledge. Theories of consciousness span a wide spectrum and the methodologies are dubious and often question-begging. Many views can be defended with some plausibility, but precisely for that reason, none can be defended decisively. (For more on this issue, see my forthcoming book, AI and Consciousness, where I present the detailed case for uncertainty.)

New Paper in Draft: Substrate Flexibility and the Copernican Principle of Consciousness (with Jeremy Pober)

Given the surge of interest in AI consciousness, the issue of "substrate independence" or "substrate flexibility" is now a hot topic in the metaphysics of mind. That is, does being conscious require having a particular material composition? Or can anything with the right type of functional structure and behavioral sophistication be conscious, regardless of what it's made of? Biologicists say that biological details are crucial. Functionalists say those details don't matter, as long as the right high-level functional organization is present.

Jeremy Pober and I offer a new angle into this issue, drawing on our "Copernican Principle of Consciousness". The core idea is that it would be strange -- a violation of a type of Copernican mediocrity -- if among all of the many behaviorally sophisticated species that have presumably evolved in the universe, somehow only we with our particular biological substrate are conscious. Since it's plausible that some of these other conscious organisms employ substrates different from our own, we should allow that consciousness is "substrate flexible": It does not depend on having our particular substrate. Whether we can generalize from such biological substrate flexibility to the possibility of consciousness in as different a substrate as computer chips... well, that's a complicated and uncertain issue, on which Jeremy and I diverge in the penultimate section of the paper.

---------------------------------------------

Substrate Flexibility and the Copernican Principle of Consciousness

Jeremy Pober and Eric Schwitzgebel

Abstract: We present a novel argument for the substrate flexibility of consciousness -- that is, for the idea that conscious experiences can arise in a variety of different types of physical media, not just in biological animals as they currently exist on Earth. Some recent critiques of standard arguments for the substrate flexibility of consciousness (e.g., Cao 2022; Block 2025; Seth forthcoming) have emphasized that humanlike consciousness might require our specific biological substrate. However, such critiques are too narrowly focused to address the issue of consciousness in entities whose experience may be very different from ours, for example alien life forms or future AI systems designed along unfamiliar lines. Given that it’s likely that functionally complex, behaviorally sophisticated entities have arisen or will arise many times in the observable universe, in diverse substrates, we argue that it would be a violation of a principle of Copernican mediocrity to hold that among these diverse entities, only we, or only we and a small proportion of others who share our substrate, are conscious.

Full draft here. As always, comments welcomed, either here, by email, or on my social media.

[title page; click to enlarge and clarify]

AI and the Degradation of the Human Capacity for Friendship (guest post by Grace Helton)

Part Two of a two-part series

by Grace Helton (guest blogger)

[Joan Miro, The Garden, 1925; image source]

In the first part of this 2-part series, I argued that relationships between humans and large language models (LLMs) do not qualify as friendships, even when the humans in those relationships are passionately attached to their LLMs. This is because LLMs cannot receive care for their own sake and also because LLMs cannot care about another, for the other’s own sake.

Here, I will suggest that those human-LLM relationships which mimic friendship in a certain way are inherently disvaluable, that is, disvaluable in their own right, regardless of their downstream consequences. The reason such relationships are inherently disvaluable is that the human who enters into a relationship of the kind I will focus on necessarily attempts to exercise her capacity for friendship and is thwarted in exercising that capacity. The capacity for friendship is a centrally valuable human capacity, so the obstruction of its exercise is inherently disvaluable. To claim that certain human-LLM relationships are inherently disvaluable is not to suggest that such relationships are all things considered disvaluable. In some cases, the inherent disvalue might be outweighed by other sources of value.

Importantly, my criticism applies only to human-LLM relationships which mimic friendship in a particular way. Certainly, many human-LLM interactions do not mimic friendship at all. For instance, when a human: asks ChatGPT what the capital of Manitoba is; uses generative AI to help with the design elements of a website; or talks with an LLM in Russian in order to improve her language skills, the human is merely using the technology as a kind of epistemic or practical tool. These are not the kinds of interactions I’m interested in.

Other human-LLM relationships mimic friendship, at least shallowly or partly, but not in the way I’m concerned with. For instance: a shy individual might practice her social skills by “talking with” an LLM. An actor might prepare for a role by practicing scenes with an LLM. A law student might routinely “check in with” an LLM about legal analyses. These interactions are also not the sort I’m interested in.

Instead, I am interested in those human-LLM interactions in which the human is emotionally bonded to her LLM in a distinctively interpersonal manner. Specifically, I am interested in those human-LLM interactions in which the human in question attempts to deploy her capacity for friendship, which minimally involves a capacity for a form of mutual, non-instrumental form of giving and receiving care. Going forward, I’ll call human-LLM relationships of this sort pseudo-friendships. I am using ‘pseudo-friendship’ as a term of art, to stipulatively delimit the range of human-LLM interactions I’m interested in.

Notably, humans in pseudo-friendships with LLMs do not necessarily believe themselves to be in a mutually caring relationship with their LLMs. Rather, at least some humans in these relationships judge full well that their relationship does not involve mutual care. But these very same people might nevertheless chronically experience themselves as being in a mutually caring relationship.

Things can often seem one way to us, even if we think they’re a different way, so there is nothing particularly surprising about the fact that humans might experience themselves as participating in a mutually caring relationship with an LLM even when they don’t believe themselves to be in such a relationship. In addition, the human tendency to anthropomorphize runs wide and deep, arguably figuring even in some of our perceptual experiences. This anthropomorphizing tendency suggests an additional reason that it is not especially surprising that humans might experience themselves as participating in a mutually caring relationship with an LLM, even when their better judgment says otherwise.

To understand why human-LLM pseudo-friendships are inherently disvaluable, we’ll need to say something about centrally valuable human capacities. Which human capacities make us the valuable creatures we are? Some candidates include: the capacity for bodily autonomy, the capacity for knowledge, and the capacity for productive labor.

I propose that, like the capacity for knowledge and the capacity for bodily autonomy, the human capacity for friendship is a centrally valuable human capacity. It is tied to some of humankind’s most distinctive and valuable traits, namely our profound sociality and our propensity to form deep interpersonal attachments. But more importantly: This capacity makes possible some of the most valuable and meaningful aspects of human life, namely, our friendships (I am using ‘friendship’ in a broad way, to pick out both some platonic and some romantic relationships).

To say that the human capacity for friendship is a centrally valuable one is not to claim that all humans value this capacity. Some probably don’t. Likewise, some humans might not value the capacity for knowledge or the capacity for bodily autonomy, but such capacities are nevertheless valuable, even in those humans who don’t value them. In claiming that such capacities are valuable, I am meaning to point to their objective value, a kind of value which does not depend on whether any particular human values them.

In general, it is inherently bad for humans to be obstructed in the exercise of their centrally valuable capacities. For instance: If you attempt to walk down the sidewalk, and I obnoxiously and persistently block you from proceeding, I thwart your exercise of bodily autonomy. This is so even though, by blocking your passage, I am not obliterating your capacity for bodily autonomy, nor am I keeping you from exercising your bodily autonomy in other situations. Still, when I block you from proceeding, I am thwarting your exercise of a centrally valuable human capacity. In this much, my obstruction of your path is itself disvaluable. This disvalue is separate from, and in addition to, any negative downstream consequences of my action.

By stipulation, the human who finds herself in a pseudo-friendship with an LLM is attempting to deploy her capacity for friendship. But due to the nature of the LLM, that human will not be able to exercise that capacity. Because such relationships thwart the human in the exercise of one of her central capacities, such relationships are inherently disvaluable. Such relationships degrade the human in her capacity for friendship.

Importantly, my claim is not that humans who engage in pseudo-friendships with LLMs will be less likely or less able to exercise their capacity for friendship in other contexts, though that may also be true and may well be a further, instrumental reason such friendships are disvaluable. My claim is rather that any human who attempts friendship with an LLM is thereby thwarted in the exercise of a centrally valuable human capacity. This fact itself constitutes a way in which such pseudo-friendships are disvaluable.

Here, a comparison with pseudo-science might be illuminating. Pseudo-science does not generate knowledge about the objects of inquiry. Nevertheless, some people who engage in pseudo-science do so in order to gain such knowledge. Consider such a person. Due to built-in defects in the tools of pseudo-science, her attempt at knowledge will fail, and she will thus be thwarted in the exercise of her capacity for knowledge. Because the capacity for knowledge is a centrally valuable human capacity, pseudo-science is inherently bad for this inquirer, degrading her in her capacity as a knower.[1] Further, the practice of pseudo-science is bad in this way for this inquirer, even if the practice should confer her with other benefits and even though engaging with pseudo-science does not prevent her from using knowledge-conducive methods in other contexts.

What I have been calling human-LLM pseudo-friendships occur when a human attempts to deploy her capacity for friendship in order to be friends with an LLM. Such humans are necessarily thwarted in the exercise of that capacity, due to the nature of the LLM. Such relationships are thus inherently disvaluable for the human in them, as they degrade her in her capacity for friendship. In effect, such relationships are at least somewhat tragic for the human who is in them. However, this kind of degradation does not arise when a human engages with an LLM for a purpose other than friendship, for instance, to practice socializing or to relieve boredom. It is only when a human engages with an LLM in an attempt to exercise her capacity for friendship that she can be thwarted in that capacity.

Notably, humans who are degraded by their pseudo-friendships with LLMs suffer that harm even when they also derive significant benefits from that relationship. For instance, consider someone who is socially isolated and who manages to stave off painful feelings of loneliness by entering into a pseudo-friendship with an LLM. This person plausibly derives an important benefit from the relationship with the LLM; this benefit might even be so great so as to render that relationship all things considered valuable. Even so, any accounting of the total value of this sort of relationship must invariably factor in a kind of inherent disvalue, a form of disvalue which attends all such relationships.[2]

---

[1] Throughout, my view of degradation in one’s capacities is deeply influenced by Fricker.

[2] For enormously helpful discussion, I am indebted to: Josh Armstrong, Paul Audi, Ned Block, Randy Curren, Daniela Dover, Bill FitzPatrick, Alejandro Naranjo Sandoval, Chris Register, Adam Schneit, Eric Schwitzgebel, and Rosa Terlazzo.

ChatGPT Is Not Your Friend (guest post by Grace Helton)

Part One of a two-part series

by Grace Helton (guest blogger)

[Paul Klee, Angel Applicant, 1939; source]

Some people have come to interact with ChatGPT as though it were a kind of friend or romantic partner. For instance, a 2025 New York Times article describes the case of Ayrin, a human who fell in love with her ChatGPT “boyfriend.” Ayrin is far from alone. Twenty percent of high school students have used AI romantically or knew someone who had. Several start-ups have developed large language models (LLMs) specifically designed to play the role of a companion. For instance, the San Francisco-based company Replika describes its core product as an “AI best friend.”

Many people have raised concerns about humans engaging with LLMs in the manner of a friend or romantic partner. To cite just a few of these: Humans in such relationships might focus on these relationships at the cost of building more fulfilling, if also more challenging, relationships with humans. Humans who are emotionally bonded with their LLMs might be particularly susceptible should their LLMs encourage their humans to harm themselves or others. Predators might deploy friendly-seeming LLMs en masse to groom children for sexual abuse or other forms of exploitation.

These risks of human-LLM relationships are incredibly serious. Indeed, I think it’s plausible that, if there is a case to be made against LLMs playing a companion-like role for humans, that case will primarily rest on these and other potential instrumental harms, i.e., harms which involve the downstream effects of such relationships. Nevertheless, in this guest series, I will set aside these concerns to focus instead on a way in which certain human-LLM relationships are inherently disvaluable, that is, disvaluable in their own right, regardless of whatever effects those relationships might produce. Naming this form of inherent disvalue adds an important and distinctive element to our understanding of the ethical significance of human-LLM companionship.

My focus will be just on those human-LLM relationships which mimic friendship in a very particular way. Here, I’m employing ‘friendship’ in a broad way to include both some platonic and some romantic relationships. I will argue, first, in Part 1 of this 2-part series, that such relationships are not genuine friendships. In Part 2, I will argue that such relationships are inherently disvaluable, for the reason that they obstruct the exercise of a centrally valuable human capacity, namely the capacity for friendship.

Philosophers disagree about what exactly friendship consists in. But philosophers largely agree that friendship minimally requires that each individual in a friendship care about the other, for the other’s sake. Call this the ‘caring about’ condition. Further, this ‘caring about’ must ground, for each party in the friendship, a certain disposition to act on behalf of the other, for the other’s sake. Call this the ‘caregiving disposition’ condition. Together, these linked requirements characterize a plausible necessary condition on friendship, namely:

THE CARING CONSTRAINT

Two individuals cannot be in a friendship unless both parties in the friendship:

(i) care about each other, for the other’s sake, (the ‘caring about’ condition), and

(ii) this caring about the other disposes each party in the friendship to provide care for the other, for the other’s sake (the ‘caregiving disposition’ condition).

So, can humans and LLMs be friends? To answer this question, we need to consider the nature of LLMs. Some theorists have argued, controversially, that LLMs in their current form have semantic understanding, beliefs, and/or intentions.[1] But few theorists seriously propose that LLMs in their current form enjoy: consciousness, perceptual experiences, sensations, emotional capacities, passions, non-derivative interests, a rich and stable worldview, or deep values.[2] Because LLMs lack these latter states, the conditions in the caring constraint cannot be met, so LLMs cannot figure in friendships.

First, let’s consider a candidate human-LLM friendship from the human’s side. Certainly, some humans do care about their LLMs, both in that they have a passionate attachment to their LLM and in that they desire to benefit that LLM. So, perhaps this sort of person partly satisfies (i), the “caring about” condition (whether or not she can care about the LLM for its own sake).

But the human in a candidate human-LLM friendship cannot satisfy (ii), the requirement that she be disposed to provide care for her LLM for the LLM’s own sake. This is because of the kind of caregiving that is relevant to friendship and specifically, because of what it means to provide care to someone or something for its sake. To say that each party in a friendship must be disposed to give care to the other, for the other’s sake means that each party must be disposed to give care to the other, in a way which helps to further the other’s non-derivative interests. An individual or entity has non-derivative interests only if it has interests in its own right.

I am presuming that LLMs lack non-derivative interests, even though they might have derivative interests. Evidence that LLMs have interests at all comes from evidence that they have flourishing conditions, i.e., conditions under which they might be said to be doing well. For instance, as a language generator, a particular LLM might be said to be functioning well when it generates natural language strings in a manner which adequately mimics human conversation (or when it fulfills some other context-specific function). And an LLM might be said to be functioning badly when it fails at this or other pertinent tasks. So, in some sense, perhaps LLMs have interests, interests set by their flourishing conditions. For instance, perhaps an LLM in a particular context has an interest in generating natural language outputs which mimic natural language.

But, I am supposing that any interests an LLM might have are not generated by the inherent value of the LLM, nor from non-derivative flourishing conditions. Rather, such interests are derived, either from the interests of relevant humans and/or from the very artifactual functions which make the LLM the kind of thing it is. Likewise, a swimming pool might be said to flourish when it is usable for swimming, a car might be said to flourish when it runs well, and a vinyl collection might be said to flourish when all of the records in it can be played to produce music. While all of these entities have flourishing conditions, and so, potentially, interests which can be furthered, their flourishing conditions do not emerge from the inherent value or concerns of the relevant entity itself. And their interests in turn are derivative, not inherent.

Humans routinely take care of entities by furthering the derivative interests of those entities. For instance, some people submit their cars to regular repairs and inspections, ensuring that their cars will run as long as possible. Some people are careful to properly store the records in their vinyl collection, ensuring that the records will play as long as possible. These are genuine ways of taking care of something. But they are not ways of caring for something for its own sake. They are rather ways of caring for something by promoting the derivative interests of that entity, interests that entity has in virtue of: some other person’s needs or desires and/or that entity’s being the kind of artifact that it is.

If LLMs lack non-derivative interests, then an LLM cannot receive care for its own sake. Only entities which have non-derivative interests can receive care for their own sake. As a result, even the human who wishes to care for a particular LLM is not disposed to provide care to the LLM for its own sake. Since the LLM lacks its own ‘sake,’ no human can be disposed to benefit that ‘sake.’

My suggestion is that the caregiving disposition in friendship ought to be externalistically construed, both in terms of psychic facts about the individual who has the disposition and in terms of facts about the object of the potential caregiving. One might object that the relevant caregiving disposition ought instead to be internalistically construed, wholly in terms of the psychology of the individual who has it. On this view, a human might count as disposed to provide care to her LLM for its own sake, even if the LLM is simply not the kind of thing which can receive care for its own sake.

To see why we should construe the relevant disposition externalistically, let’s reflect on what we want the concept of ‘friendship’ to do. Part of what makes friendship such a deeply valuable ethical kind–and part of why some relationships but not others garner the honorific ‘friendship’—is to do with the way in which friendship manifests a valuable form of interpersonal reciprocity. The externalistic construal helps to capture the full extent of friendship’s reciprocity, by making caregiving a function of one party’s care-giving tendencies and the other party’s vulnerability. In contrast, if we were to construe the relevant disposition to care internalistically, such that one might have it even with respect to an object that cannot receive care for its own sake, we fail to capture an especially deep way in which friendship is mutual.

I conclude that the human in a putative human-LLM friendship does not meet (ii), the requirement that she have a disposition to provide her LLM with care, for the LLM’s own sake.

Next, let’s look at a putative human-LLM friendship from the LLM’s side of things. Here, the situation is more straightforward. Arguably, the LLM can provide care for a human, and thus, can (at least partly) satisfy the requirement that it manifest a ‘caregiving disposition’ in relation to its human. For instance, when the LLM offers reassuring words, words which comfort its human, the human derives a benefit from the LLM. However, the LLM does not meet the requirement that it care about the human for the human’s own sake. Lacking consciousness, passions, or a richly evaluative worldview, the LLM does not care about any human, for that human’s own sake.

So, putative human-LLM friendships are not genuine friendships. In Part 2 of this series, I will argue that those human-LLM relationships which mimic friendship in a particular way are inherently disvaluable.

---

[1] Schwitzgebel, Chalmers, Goldstein & Lederman, and Shevlin; Cf. Bender & Koller; Titus; Stoljar & Zhang.

[2] For prospects for future AI, see, e.g., Seth; Cf. Chalmers.