New Paper in Draft: Substrate Flexibility and the Copernican Principle of Consciousness (with Jeremy Pober)

Given the surge of interest in AI consciousness, the issue of "substrate independence" or "substrate flexibility" is now a hot topic in the metaphysics of mind. That is, does being conscious require having a particular material composition? Or can anything with the right type of functional structure and behavioral sophistication be conscious, regardless of what it's made of? Biologicists say that biological details are crucial. Functionalists say those details don't matter, as long as the right high-level functional organization is present.

Jeremy Pober and I offer a new angle into this issue, drawing on our "Copernican Principle of Consciousness". The core idea is that it would be strange -- a violation of a type of Copernican mediocrity -- if among all of the many behaviorally sophisticated species that have presumably evolved in the universe, somehow only we with our particular biological substrate are conscious. Since it's plausible that some of these other conscious organisms employ substrates different from our own, we should allow that consciousness is "substrate flexible": It does not depend on having our particular substrate. Whether we can generalize from such biological substrate flexibility to the possibility of consciousness in as different a substrate as computer chips... well, that's a complicated and uncertain issue, on which Jeremy and I diverge in the penultimate section of the paper.

---------------------------------------------

Substrate Flexibility and the Copernican Principle of Consciousness

Jeremy Pober and Eric Schwitzgebel

Abstract: We present a novel argument for the substrate flexibility of consciousness -- that is, for the idea that conscious experiences can arise in a variety of different types of physical media, not just in biological animals as they currently exist on Earth. Some recent critiques of standard arguments for the substrate flexibility of consciousness (e.g., Cao 2022; Block 2025; Seth forthcoming) have emphasized that humanlike consciousness might require our specific biological substrate. However, such critiques are too narrowly focused to address the issue of consciousness in entities whose experience may be very different from ours, for example alien life forms or future AI systems designed along unfamiliar lines. Given that it’s likely that functionally complex, behaviorally sophisticated entities have arisen or will arise many times in the observable universe, in diverse substrates, we argue that it would be a violation of a principle of Copernican mediocrity to hold that among these diverse entities, only we, or only we and a small proportion of others who share our substrate, are conscious.

Full draft here. As always, comments welcomed, either here, by email, or on my social media.

[title page; click to enlarge and clarify]

AI and the Degradation of the Human Capacity for Friendship (guest post by Grace Helton)

Part Two of a two-part series

by Grace Helton (guest blogger)

[Joan Miro, The Garden, 1925; image source]

In the first part of this 2-part series, I argued that relationships between humans and large language models (LLMs) do not qualify as friendships, even when the humans in those relationships are passionately attached to their LLMs. This is because LLMs cannot receive care for their own sake and also because LLMs cannot care about another, for the other’s own sake.

Here, I will suggest that those human-LLM relationships which mimic friendship in a certain way are inherently disvaluable, that is, disvaluable in their own right, regardless of their downstream consequences. The reason such relationships are inherently disvaluable is that the human who enters into a relationship of the kind I will focus on necessarily attempts to exercise her capacity for friendship and is thwarted in exercising that capacity. The capacity for friendship is a centrally valuable human capacity, so the obstruction of its exercise is inherently disvaluable. To claim that certain human-LLM relationships are inherently disvaluable is not to suggest that such relationships are all things considered disvaluable. In some cases, the inherent disvalue might be outweighed by other sources of value.

Importantly, my criticism applies only to human-LLM relationships which mimic friendship in a particular way. Certainly, many human-LLM interactions do not mimic friendship at all. For instance, when a human: asks ChatGPT what the capital of Manitoba is; uses generative AI to help with the design elements of a website; or talks with an LLM in Russian in order to improve her language skills, the human is merely using the technology as a kind of epistemic or practical tool. These are not the kinds of interactions I’m interested in.

Other human-LLM relationships mimic friendship, at least shallowly or partly, but not in the way I’m concerned with. For instance: a shy individual might practice her social skills by “talking with” an LLM. An actor might prepare for a role by practicing scenes with an LLM. A law student might routinely “check in with” an LLM about legal analyses. These interactions are also not the sort I’m interested in.

Instead, I am interested in those human-LLM interactions in which the human is emotionally bonded to her LLM in a distinctively interpersonal manner. Specifically, I am interested in those human-LLM interactions in which the human in question attempts to deploy her capacity for friendship, which minimally involves a capacity for a form of mutual, non-instrumental form of giving and receiving care. Going forward, I’ll call human-LLM relationships of this sort pseudo-friendships. I am using ‘pseudo-friendship’ as a term of art, to stipulatively delimit the range of human-LLM interactions I’m interested in.

Notably, humans in pseudo-friendships with LLMs do not necessarily believe themselves to be in a mutually caring relationship with their LLMs. Rather, at least some humans in these relationships judge full well that their relationship does not involve mutual care. But these very same people might nevertheless chronically experience themselves as being in a mutually caring relationship.

Things can often seem one way to us, even if we think they’re a different way, so there is nothing particularly surprising about the fact that humans might experience themselves as participating in a mutually caring relationship with an LLM even when they don’t believe themselves to be in such a relationship. In addition, the human tendency to anthropomorphize runs wide and deep, arguably figuring even in some of our perceptual experiences. This anthropomorphizing tendency suggests an additional reason that it is not especially surprising that humans might experience themselves as participating in a mutually caring relationship with an LLM, even when their better judgment says otherwise.

To understand why human-LLM pseudo-friendships are inherently disvaluable, we’ll need to say something about centrally valuable human capacities. Which human capacities make us the valuable creatures we are? Some candidates include: the capacity for bodily autonomy, the capacity for knowledge, and the capacity for productive labor.

I propose that, like the capacity for knowledge and the capacity for bodily autonomy, the human capacity for friendship is a centrally valuable human capacity. It is tied to some of humankind’s most distinctive and valuable traits, namely our profound sociality and our propensity to form deep interpersonal attachments. But more importantly: This capacity makes possible some of the most valuable and meaningful aspects of human life, namely, our friendships (I am using ‘friendship’ in a broad way, to pick out both some platonic and some romantic relationships).

To say that the human capacity for friendship is a centrally valuable one is not to claim that all humans value this capacity. Some probably don’t. Likewise, some humans might not value the capacity for knowledge or the capacity for bodily autonomy, but such capacities are nevertheless valuable, even in those humans who don’t value them. In claiming that such capacities are valuable, I am meaning to point to their objective value, a kind of value which does not depend on whether any particular human values them.

In general, it is inherently bad for humans to be obstructed in the exercise of their centrally valuable capacities. For instance: If you attempt to walk down the sidewalk, and I obnoxiously and persistently block you from proceeding, I thwart your exercise of bodily autonomy. This is so even though, by blocking your passage, I am not obliterating your capacity for bodily autonomy, nor am I keeping you from exercising your bodily autonomy in other situations. Still, when I block you from proceeding, I am thwarting your exercise of a centrally valuable human capacity. In this much, my obstruction of your path is itself disvaluable. This disvalue is separate from, and in addition to, any negative downstream consequences of my action.

By stipulation, the human who finds herself in a pseudo-friendship with an LLM is attempting to deploy her capacity for friendship. But due to the nature of the LLM, that human will not be able to exercise that capacity. Because such relationships thwart the human in the exercise of one of her central capacities, such relationships are inherently disvaluable. Such relationships degrade the human in her capacity for friendship.

Importantly, my claim is not that humans who engage in pseudo-friendships with LLMs will be less likely or less able to exercise their capacity for friendship in other contexts, though that may also be true and may well be a further, instrumental reason such friendships are disvaluable. My claim is rather that any human who attempts friendship with an LLM is thereby thwarted in the exercise of a centrally valuable human capacity. This fact itself constitutes a way in which such pseudo-friendships are disvaluable.

Here, a comparison with pseudo-science might be illuminating. Pseudo-science does not generate knowledge about the objects of inquiry. Nevertheless, some people who engage in pseudo-science do so in order to gain such knowledge. Consider such a person. Due to built-in defects in the tools of pseudo-science, her attempt at knowledge will fail, and she will thus be thwarted in the exercise of her capacity for knowledge. Because the capacity for knowledge is a centrally valuable human capacity, pseudo-science is inherently bad for this inquirer, degrading her in her capacity as a knower.[1] Further, the practice of pseudo-science is bad in this way for this inquirer, even if the practice should confer her with other benefits and even though engaging with pseudo-science does not prevent her from using knowledge-conducive methods in other contexts.

What I have been calling human-LLM pseudo-friendships occur when a human attempts to deploy her capacity for friendship in order to be friends with an LLM. Such humans are necessarily thwarted in the exercise of that capacity, due to the nature of the LLM. Such relationships are thus inherently disvaluable for the human in them, as they degrade her in her capacity for friendship. In effect, such relationships are at least somewhat tragic for the human who is in them. However, this kind of degradation does not arise when a human engages with an LLM for a purpose other than friendship, for instance, to practice socializing or to relieve boredom. It is only when a human engages with an LLM in an attempt to exercise her capacity for friendship that she can be thwarted in that capacity.

Notably, humans who are degraded by their pseudo-friendships with LLMs suffer that harm even when they also derive significant benefits from that relationship. For instance, consider someone who is socially isolated and who manages to stave off painful feelings of loneliness by entering into a pseudo-friendship with an LLM. This person plausibly derives an important benefit from the relationship with the LLM; this benefit might even be so great so as to render that relationship all things considered valuable. Even so, any accounting of the total value of this sort of relationship must invariably factor in a kind of inherent disvalue, a form of disvalue which attends all such relationships.[2]

---

[1] Throughout, my view of degradation in one’s capacities is deeply influenced by Fricker.

[2] For enormously helpful discussion, I am indebted to: Josh Armstrong, Paul Audi, Ned Block, Randy Curren, Daniela Dover, Bill FitzPatrick, Alejandro Naranjo Sandoval, Chris Register, Adam Schneit, Eric Schwitzgebel, and Rosa Terlazzo.

ChatGPT Is Not Your Friend (guest post by Grace Helton)

Part One of a two-part series

by Grace Helton (guest blogger)

[Paul Klee, Angel Applicant, 1939; source]

Some people have come to interact with ChatGPT as though it were a kind of friend or romantic partner. For instance, a 2025 New York Times article describes the case of Ayrin, a human who fell in love with her ChatGPT “boyfriend.” Ayrin is far from alone. Twenty percent of high school students have used AI romantically or knew someone who had. Several start-ups have developed large language models (LLMs) specifically designed to play the role of a companion. For instance, the San Francisco-based company Replika describes its core product as an “AI best friend.”

Many people have raised concerns about humans engaging with LLMs in the manner of a friend or romantic partner. To cite just a few of these: Humans in such relationships might focus on these relationships at the cost of building more fulfilling, if also more challenging, relationships with humans. Humans who are emotionally bonded with their LLMs might be particularly susceptible should their LLMs encourage their humans to harm themselves or others. Predators might deploy friendly-seeming LLMs en masse to groom children for sexual abuse or other forms of exploitation.

These risks of human-LLM relationships are incredibly serious. Indeed, I think it’s plausible that, if there is a case to be made against LLMs playing a companion-like role for humans, that case will primarily rest on these and other potential instrumental harms, i.e., harms which involve the downstream effects of such relationships. Nevertheless, in this guest series, I will set aside these concerns to focus instead on a way in which certain human-LLM relationships are inherently disvaluable, that is, disvaluable in their own right, regardless of whatever effects those relationships might produce. Naming this form of inherent disvalue adds an important and distinctive element to our understanding of the ethical significance of human-LLM companionship.

My focus will be just on those human-LLM relationships which mimic friendship in a very particular way. Here, I’m employing ‘friendship’ in a broad way to include both some platonic and some romantic relationships. I will argue, first, in Part 1 of this 2-part series, that such relationships are not genuine friendships. In Part 2, I will argue that such relationships are inherently disvaluable, for the reason that they obstruct the exercise of a centrally valuable human capacity, namely the capacity for friendship.

Philosophers disagree about what exactly friendship consists in. But philosophers largely agree that friendship minimally requires that each individual in a friendship care about the other, for the other’s sake. Call this the ‘caring about’ condition. Further, this ‘caring about’ must ground, for each party in the friendship, a certain disposition to act on behalf of the other, for the other’s sake. Call this the ‘caregiving disposition’ condition. Together, these linked requirements characterize a plausible necessary condition on friendship, namely:

THE CARING CONSTRAINT

Two individuals cannot be in a friendship unless both parties in the friendship:

(i) care about each other, for the other’s sake, (the ‘caring about’ condition), and

(ii) this caring about the other disposes each party in the friendship to provide care for the other, for the other’s sake (the ‘caregiving disposition’ condition).

So, can humans and LLMs be friends? To answer this question, we need to consider the nature of LLMs. Some theorists have argued, controversially, that LLMs in their current form have semantic understanding, beliefs, and/or intentions.[1] But few theorists seriously propose that LLMs in their current form enjoy: consciousness, perceptual experiences, sensations, emotional capacities, passions, non-derivative interests, a rich and stable worldview, or deep values.[2] Because LLMs lack these latter states, the conditions in the caring constraint cannot be met, so LLMs cannot figure in friendships.

First, let’s consider a candidate human-LLM friendship from the human’s side. Certainly, some humans do care about their LLMs, both in that they have a passionate attachment to their LLM and in that they desire to benefit that LLM. So, perhaps this sort of person partly satisfies (i), the “caring about” condition (whether or not she can care about the LLM for its own sake).

But the human in a candidate human-LLM friendship cannot satisfy (ii), the requirement that she be disposed to provide care for her LLM for the LLM’s own sake. This is because of the kind of caregiving that is relevant to friendship and specifically, because of what it means to provide care to someone or something for its sake. To say that each party in a friendship must be disposed to give care to the other, for the other’s sake means that each party must be disposed to give care to the other, in a way which helps to further the other’s non-derivative interests. An individual or entity has non-derivative interests only if it has interests in its own right.

I am presuming that LLMs lack non-derivative interests, even though they might have derivative interests. Evidence that LLMs have interests at all comes from evidence that they have flourishing conditions, i.e., conditions under which they might be said to be doing well. For instance, as a language generator, a particular LLM might be said to be functioning well when it generates natural language strings in a manner which adequately mimics human conversation (or when it fulfills some other context-specific function). And an LLM might be said to be functioning badly when it fails at this or other pertinent tasks. So, in some sense, perhaps LLMs have interests, interests set by their flourishing conditions. For instance, perhaps an LLM in a particular context has an interest in generating natural language outputs which mimic natural language.

But, I am supposing that any interests an LLM might have are not generated by the inherent value of the LLM, nor from non-derivative flourishing conditions. Rather, such interests are derived, either from the interests of relevant humans and/or from the very artifactual functions which make the LLM the kind of thing it is. Likewise, a swimming pool might be said to flourish when it is usable for swimming, a car might be said to flourish when it runs well, and a vinyl collection might be said to flourish when all of the records in it can be played to produce music. While all of these entities have flourishing conditions, and so, potentially, interests which can be furthered, their flourishing conditions do not emerge from the inherent value or concerns of the relevant entity itself. And their interests in turn are derivative, not inherent.

Humans routinely take care of entities by furthering the derivative interests of those entities. For instance, some people submit their cars to regular repairs and inspections, ensuring that their cars will run as long as possible. Some people are careful to properly store the records in their vinyl collection, ensuring that the records will play as long as possible. These are genuine ways of taking care of something. But they are not ways of caring for something for its own sake. They are rather ways of caring for something by promoting the derivative interests of that entity, interests that entity has in virtue of: some other person’s needs or desires and/or that entity’s being the kind of artifact that it is.

If LLMs lack non-derivative interests, then an LLM cannot receive care for its own sake. Only entities which have non-derivative interests can receive care for their own sake. As a result, even the human who wishes to care for a particular LLM is not disposed to provide care to the LLM for its own sake. Since the LLM lacks its own ‘sake,’ no human can be disposed to benefit that ‘sake.’

My suggestion is that the caregiving disposition in friendship ought to be externalistically construed, both in terms of psychic facts about the individual who has the disposition and in terms of facts about the object of the potential caregiving. One might object that the relevant caregiving disposition ought instead to be internalistically construed, wholly in terms of the psychology of the individual who has it. On this view, a human might count as disposed to provide care to her LLM for its own sake, even if the LLM is simply not the kind of thing which can receive care for its own sake.

To see why we should construe the relevant disposition externalistically, let’s reflect on what we want the concept of ‘friendship’ to do. Part of what makes friendship such a deeply valuable ethical kind–and part of why some relationships but not others garner the honorific ‘friendship’—is to do with the way in which friendship manifests a valuable form of interpersonal reciprocity. The externalistic construal helps to capture the full extent of friendship’s reciprocity, by making caregiving a function of one party’s care-giving tendencies and the other party’s vulnerability. In contrast, if we were to construe the relevant disposition to care internalistically, such that one might have it even with respect to an object that cannot receive care for its own sake, we fail to capture an especially deep way in which friendship is mutual.

I conclude that the human in a putative human-LLM friendship does not meet (ii), the requirement that she have a disposition to provide her LLM with care, for the LLM’s own sake.

Next, let’s look at a putative human-LLM friendship from the LLM’s side of things. Here, the situation is more straightforward. Arguably, the LLM can provide care for a human, and thus, can (at least partly) satisfy the requirement that it manifest a ‘caregiving disposition’ in relation to its human. For instance, when the LLM offers reassuring words, words which comfort its human, the human derives a benefit from the LLM. However, the LLM does not meet the requirement that it care about the human for the human’s own sake. Lacking consciousness, passions, or a richly evaluative worldview, the LLM does not care about any human, for that human’s own sake.

So, putative human-LLM friendships are not genuine friendships. In Part 2 of this series, I will argue that those human-LLM relationships which mimic friendship in a particular way are inherently disvaluable.

---

[1] Schwitzgebel, Chalmers, Goldstein & Lederman, and Shevlin; Cf. Bender & Koller; Titus; Stoljar & Zhang.

[2] For prospects for future AI, see, e.g., Seth; Cf. Chalmers.

Superhuman Moral Standing

Human beings matter morally. We have moral standing. Our interests deserve consideration -- for our own sakes, and not just as means to ends. Good ethical decision-making requires valuing human lives. Most philosophers hold that humans have the highest moral standing. No entity matters more, and many matter less. It’s worse to kill a human than a dog or a frog or bacteria or a tree.

That humans have the maximum possible moral standing is sometimes encoded in the philosophical jargon, for example when philosophers say that humans have "full moral status". The moral gas-gauge tops out at "full" for us, so to speak.

But might some entities have higher moral standing than humans? Futurists envision the possibility of a post-human, transhuman, or superhuman future, or AI systems with superhuman capacities. Might there someday exist entities whose lives are intrinsically more valuable than ours, deserving moral priority over us, just as a human life deserves moral priority over that of a frog?

I see three possible paths to superhuman moral standing.

[Xul Solar, San Danza, source]

First Path: Quantitative Superhumanity

The seemingly most straightforward path to superhuman moral standing would involve having much more of something that we already regard as relevant to moral standing.

Classical utilitarians ground moral standing in the capacity for pleasure and pain. An entity deserves moral consideration to the extent we can increase or decrease its happiness. Humans (it's assumed) experience more, or at least richer, pleasures and pains than other animals, hence human lives matter more. A utility monster or a superpleasure machine capable of vastly more happiness than an ordinary human might then deserve much greater weight in ethical decision making.

Rationality-based views, like Kant's, ground moral standing in sophisticated rational capacities, such as our ability to think abstractly about our duties to one another. Maybe -- although this is not Kant's view -- entities with some but less rational capacity, such as dogs, have significant but subhuman moral standing. Future entities with vastly superior rational capacities might correspondingly have superhuman moral standing.

A third type of view locates the distinctive value of humanity in our capacity to flourish in activities such as intimate friendship, productive work, creative play, and imaginative thought. Dogs also befriend and play, work and think, but perhaps not as richly and flourishingly as humans (though I can imagine disputing that). Possibly, some future entity could far surpass us in such capacities and deserve superhuman moral consideration on those grounds.

The big catch with quantitative approaches to superhumanity -- or maybe instead an appealing feature -- is that the utilitarian, rationalist, and perfectionist views I've just described should probably be articulated in egalitarian ways that impair the inference from more of X to higher moral standing. After all, we don't normally say that mercurial people who feel more joy and suffering in everyday life deserve more moral consideration than those who keep an even keel. Nor do we say that "more rational" people deserve greater moral consideration, or that people who are more productive workers or more creative playmates do.

On all of these views, there's plausibly a threshold of good enough, above which one has full moral status, fully equal with other humans. People with severe cognitive disabilities have full moral status either by being above that threshold or on more complicated grounds, such as belonging to humanity as a whole. If so, then hypothetical superhumans might also have moral standing only in virtue of exceeding that threshold, without its mattering how far above that threshold they are -- our equals in moral standing rather than our superiors.

To achieve superhuman moral standing despite egalitarianism among humans might then require either (1.) having so much more of the relevant X than an ordinary human as to trigger a genuine difference in kind; or (2.) having enough of X that, as a practical matter, the entity deserves greater weight even if its formal status is equal (as when utilitarians prioritize humans over mice because of their richer possible experiences, despite granting both equal standing in principle).

Second Path: Qualitative Superhumanity

A more radical possibility is that some beings might possess entirely new capacities that we can't even conceive -- capacities that ground a higher kind of moral standing.

Just as a sea turtle could never understand cryptocurrency, we too are cognitively limited. Some features of the world might be forever beyond human comprehension. (Colin McGinn has suggested that how consciousness arises from matter is one example.) Maybe someday Earth will host entities whose cognitive capacities surpass ours as dramatically as ours surpass sea turtles. And maybe these entities will deserve a new type of higher moral consideration.

This isn't just the quantitative thought that such entities might deserve more because they have more rationality or intelligence. The thought is that they might possess an unknown property Z -- something we entirely lack and cannot envision -- that elevates their standing beyond both sea turtles and humans.

For example, maybe sea turtles deserve some moral consideration because they can feel pleasure and pain. But maybe they don't deserve fully humanlike moral consideration because they lack some other relevant capacity, such as the capacity to consider and adhere to ethical norms. They have some of X but none of Y, while we humans have both X and Y. The qualitative view posits a further Z, inaccessible to us, that grounds superhuman standing.

I can only present this possibility abstractly. But I'm not sure it's in principle impossible. If moral standing depends on one thing only, such as pleasure or humanlike practical reasoning, then you can resist this move by insisting that only that one thing counts. But pluralists about the grounds of moral standing, who hold that it derives from more than one intrinsically good feature or capacity, have no clear reason to think that humans manifest the exhaustive list.

Third Path: Failures of Subject-Counting

I find egalitarianism attractive: one person, one point in the moral calculus, so to speak. But as I've argued elsewhere, future AI persons, if they ever come to exist, might defy the ordinary standards of individuation (e.g., here, here, here, here). They might overlap, merge, divide, back themselves up, and spin off partially or temporarily independent copies.

The norm of equality of persons would then require serious rethinking. There will be no clean count of AI persons to weigh against human persons. A "fission-fusion monster" who can split into a hundred copies at will and later merge or partly merge back together raises difficult questions. Does the monster deserve equal consideration with one person, a hundred people, or some intermediate number? There might be no determinate answer. We'll need new ethical principles for weighing competing interests. For some purposes we might treat the monster as equivalent to one person; for other purposes we might give it greater consideration. This could constitute a type of partly superhuman moral standing.

Alternatively, consider a massive entity, or a cluster of entities with many overlapping parts, whose total capacity and activity is comparable to several humans but who is neither wholly unified nor clearly individuatable into discrete humanlike subparts. We might just do our best with a rough count and give it equal consideration with that many ordinary humans. But another possibility would be to regard it not as approximately X humans but rather as a single, complex entity whose interests deserve significantly more weight than those of a single, ordinary human.