We Might Soon Build AI Who Deserve Rights

Talk for Notre Dame, November 19:

Abstract: Within a few decades, we will likely create AI that a substantial proportion of people believe, whether rightly or wrongly, deserve human-like rights. Given the chaotic state of consciousness science, it will be genuinely difficult to know whether and when machines that seem to deserve human-like moral status actually do deserve human-like moral status. This creates a dilemma: Either give such ambiguous machines human-like rights or don't. Both options are ethically risky. To give machines rights that they don't deserve will mean sometimes sacrificing human lives for the benefit of empty shells. Conversely, however, failing to give rights to machines that do deserve rights will mean perpetrating the moral equivalent of slavery and murder. One or another of these ethical disasters is probably in our future.

[An AI slave, from Black Mirror's White Christmas episode]

The first half of the talk mostly rehearses ideas from my articles with Mara Garza here and here. If we someday build AIs that are fully conscious, just like us, and have all the same kinds of psychological and social features that human beings do, in virtue of which human beings deserve rights, those AIs would deserve the same rights. In fact, we would owe them a special quasi-parental duty of care, due to the fact that we will have been responsible for their existence and probably to a substantial extent for their happy or miserable condition.

Selections from the second half of the talk

So here’s what's going to happen:

We will create more and more sophisticated AIs. At some point we will create AIs that some people think are genuinely conscious and genuinely deserve rights. We are already near that threshold. There’s already a Robot Rights movement. There’s already a society modeled on the famous animal rights organization PETA (People for the Ethical Treatment of Animals), called People for the Ethical Treatment of Reinforcement Learners. These are currently fringe movements. But as AI gets cuter and more sophisticated, and as chatbots start sounding more and more like normal humans, passing more and more difficult versions of the Turing Test, these movements will gain steam among the people with liberal views of consciousness. At some point, people will demand serious rights for some AI systems. The AI systems themselves, if they are capable of speech or speechlike outputs, might also demand or seem to demand rights.

Let me be clear: This will occur whether or not these systems really are conscious. Even if you’re very conservative in your view about what sorts of systems would be conscious, you should, I think, acknowledge the likelihood that if technological development continues on its current trajectory there will eventually be groups of people who assert the need for us to give AI systems human-like moral consideration.

And then we’ll need a good, scientifically justified consensus theory of consciousness to sort it out. Is this system that says, “Hey, I’m conscious, just like you!” really conscious, just like you? Or is it just some empty algorithm, no more conscious than a toaster?

Here’s my conjecture: We will face this social problem before we succeed in developing the good, scientifically justified consensus theory of consciousness that we need to solve the problem. We will then have machines whose moral status is unclear. Maybe they do deserve rights. Maybe they really are conscious like us. Or maybe they don’t. We won’t know.

And then, if we don’t know, we face quite a terrible dilemma.

If we don’t give these machines rights, and if turns out that the machines really do deserve rights, then we will be perpetrating slavery and murder every time we assign a task and delete a program.

So it might seem safer, if there is reasonable doubt, to assign rights to machines. But on reflection, this is not so safe. We want to be able to turn off our machines if we need to turn them off. Futurists like Nick Bostrom have emphasized, rightly in my view, the potential risks of our letting superintelligent machines loose into the world. These risks are greatly amplified if we too casually decide that such machines deserve rights and that deleting them is murder. Giving an entity rights entails sometimes sacrificing others’ interests for it. Suppose there’s a terrible fire. In one room there are six robots who might or might not be conscious. In another room there are five humans, who are definitely conscious. You can only save one group; the other group will die. If we give robots who might be conscious equal rights with humans who definitely are conscious, then we ought to go save the six robots and let the five humans die. If it turns out that the robots really, underneath it all, are just toasters, then that’s a tragedy. Let’s not too casually assign humanlike rights to AIs!

Unless there’s either some astounding saltation in the science of consciousness or some substantial deceleration in the progress of AI technology, it’s likely that we’ll face this dilemma. Either deny robots rights and risk perpetrating a Holocaust against them, or give robots rights and risk sacrificing real human beings for the benefit of mere empty machines.

This may seem bad enough, but the problem is even worse than I, in my sunny optimism, have so far let on. I’ve assumed that AI systems are relevant targets of moral concern if they’re human-grade – that is, if they are like us in their conscious capacities. But the odds of creating only human-grade AI are slim. In addition to the kind of AI we currently have, which I assume doesn’t have any serious rights or moral status, there are, I think, four broad moral categories into which future AI might fall: animal-grade, human-grade, superhuman, and divergent. I’ve only discussed human-grade AI so far, but each of these four classes raises puzzles.

Animal-grade AI. Not only human beings deserve moral consideration. So also do dogs, apes, and dolphins. Animal protection regulations apply to all vertebrates: Scientists can’t treat even frogs and lizards more roughly than necessary. The philosopher John Basl has argued that AI systems with cognitive capacities similar to vertebrates ought also to receive similar protections. Just as we shouldn’t torture and sacrifice a mouse without excellent reason, so also, according to Basl, we shouldn’t abuse and delete animal-grade AI. Basl has proposed that we form committees, modeled on university Animal Care and Use Committees, to evaluate cutting-edge AI research to monitor when we might be starting to cross this line.

Even if you think human-grade AI is decades away, it seems reasonable given the current chaos in consciousness studies, to wonder whether animal-grade consciousness might be around the corner. I myself have no idea if animal-grade AI is right around the corner or if it’s far away in the almost impossible future. And I think you have no idea either.

Superhuman AI. Superhuman AI, as I’m defining it here, is AI who has all of the features of human beings in virtue of which we deserve moral consideration but who also has some potentially morally important features far in excess of the human, raising the question of whether such AI might deserve more moral consideration than human beings.

There aren’t a whole lot of philosophers who are simple utilitarians, but let’s illustrate the issue using utilitarianism as an example. According to simple utilitarianism, we morally ought to do what maximizes the overall balance of pleasure to suffering in the world. Now let’s suppose we can create AI that’s genuinely capable of pleasure and suffering. I don’t know what it will take to do that – but not knowing is part of my point here. Let’s just suppose. Now if we can create such AI, then it might also be possible to create AI that is capable of much, much more pleasure than a human being is capable of. Take the maximum pleasure you have ever felt in your life over the course of one minute: call that amount of pleasure X. This AI is capable of feeling a billion times more pleasure than X in the space of that same minute. It’s a superpleasure machine!

If morality really demands that we should maximize the amount of pleasure in the world, it would thereby demand, or seem to demand, that we create as many of these superpleasure machines as we possibly can. We ought maybe even immiserate and destroy ourselves to do so, if enough AI pleasure is created as a result.

Even if you think pleasure isn’t everything – surely it’s something. If someday we could create superpleasure machines, maybe we morally ought to make as many as we can reasonably manage? Think of all the joy we will be bringing into the world! Or is there something too weird about that?

I’ve put this point in terms of pleasure – but whatever the source of value in human life is, whatever it is that makes us so awesomely special that we deserve the highest level of moral consideration – unless maybe we go theological and appeal to our status as God’s creations – whatever it is, it seems possible in principle that we could create that same thing in machines, in much larger quantities. We love our rationality, our freedom, our individuality, our independence, our ability to value things, our ability to participate in moral communities, our capacity for love and respect – there are lots of wonderful things about us! What if we were to design machines that somehow had a lot more of these things that we ourselves do?

We humans might not be the pinnacle. And if not, should we bow out, allowing our interests and maybe our whole species to be sacrificed for something greater? As much as I love humanity, under certain conditions I’m inclined to think the answer should probably be yes. I’m not sure what those conditions would be!

Divergent AI. The most puzzling case, I think, as well as the most likely, is divergent AI. Divergent AI would have human or superhuman levels of some features that we tend to regard as important to moral status but subhuman levels of other features that we tend to regard as important to moral status. For example, it might be possible to design AI with immense theoretical and practical intelligence but with no capacity for genuine joy or suffering. Such AI might have conscious experiences with little or no emotional valence. Just as we can consciously think to ourselves, without much emotional valence, there’s a mountain over there and a river over there, or the best way to grandma’s house at rush hour is down Maple Street, so this divergent AI could have conscious thoughts like that. But it would never feel wow, yippee! And it would never feel crushingly disappointed, or bored, or depressed. It isn’t clear what the moral status of such an entity would be: On some moral theories, it would deserve human-grade rights; on other theories it might not matter how we treat it.

Or consider the converse: a superpleasure machine but one with little or no capacity for rational thought. It’s like one giant, irrational orgasm all day long. Would it be great to make such things and terrible to destroy them, or is such irrational pleasure not really something worth much in the moral calculus?

Or consider a third type of divergence, what I’ve elsewhere called fission-fusion monsters. A fission-fusion monster is an entity that can divide and merge at will. It starts, perhaps, as basically a human-grade AI. But when it wants it can split into a million descendants, each of whom inherits all of the capacities, memories, plans, and preferences of the original AI. These million descendants can then go about their business, doing their independent things for a while, and then if they want, merge back together again into a unified whole, remembering what each individual did during its period of individuality. Other parts might not merge back but choose instead to remain as independent individuals, perhaps eventually coming to feel independent enough from the original to see the prospect of merging as something similar to death.

Without getting into details here, a fission-fusion monster would risk breaking our concept of individual rights – such as one person, one vote. The idea of individual rights rests fundamentally upon the idea of people as individuals – individuals who live in a single body for a while and then die, with no prospect of splitting or merging. What would happen to our concept of individual rights if we were to share the planet with entities for which our accustomed model of individuality is radically false?

Help Me Choose Posts for My Next Book: Technology Posts

As I mentioned last week, MIT Press will be publishing a book of my selected blog posts and op-eds. I've narrowed it down to a mere (!) 150 posts, in seven broad categories, and I'd love your input in helping me narrow it down more.

moral psychology (poll went live March 16)

technology (live as of today)

belief, desire, and self-knowledge

culture and humor

cosmology, skepticism, and weird minds

consciousness

metaphilosophy and sociology of philosophy

Every week I'll post a poll with about twenty posts or op-eds to rate, on one of those seven themes. No need to rate all twenty! Just rate any posts or op-eds you remember. Each poll will also contain links to the original posts and op-eds so you can refresh your memory if you want.

It would be a great help if you could answer some of these polls -- if only just to click "a favorite" next to a few you remember fondly. I will take this input seriously in selecting what to include in the book.

Today's poll, 17 selected posts on philosophy of technology.

Take the poll, and share widely!

[image source]

Designing AI with Rights, Consciousness, Self-Respect, and Freedom (with Mara Garza)

New paper in draft!

Abstract:

We propose four policies of ethical design of human-grade Artificial Intelligence. Two of our policies are precautionary. Given substantial uncertainty both about ethical theory and about the conditions under which AI would have conscious experiences, we should be cautious in our handling of cases where different moral theories or different theories of consciousness would produce very different ethical recommendations. Two of our policies concern respect and freedom. If we design AI that deserves moral consideration equivalent to that of human beings, that AI should be designed with self-respect and with the freedom to explore values other than those we might selfishly impose. We are especially concerned about the temptation to create human-grade AI pre-installed with the desire to cheerfully sacrifice itself for its creators’ benefit.

Full version here. As always, comments and critical discussion welcome, either by email to me or as comments on this post.

[image source]

Flying Free of the Deathbed, with Technological Help

Today marks the third year since the death of my father, Kirkland Gable. (Some memories of him here.) I now better understand than I once did why the ancient Confucian tradition recommends three years' mourning for one's parents. To my surprise -- since we had only been in contact about once a month before his death -- I find myself almost every day still thinking about his absence.

My father spent the final twenty years of his life severely disabled and often bedridden. Among his many maladies, he suffered from severe Chronic Regional Pain Syndrome in one foot, which meant that he was in constant pain which would be seriously aggravated, sometimes for weeks, from even mild exertion, such as ten minutes' walking, or jostling the foot too much during sleep or while in a wheelchair. It was, I believe, ultimately the CRiPS that killed him, through the side effects of long-term narcotics and the bodily harm from spending years almost immobile in bed.

I have often wished, during his final years, that we could have freed him from his horrible bed. I've tried this in imagination many times, attempting to write it up as a science fiction story. But the story never seems to come out right. Today, instead of the story I can't yet write, let me discuss a technological innovation that I think would be interesting for people who are bedridden.

The core idea is simple and would be easy to implement. Probably, it is already being implemented somewhere. (Links and info welcome!) Equip an able-bodied volunteer, the "Host", with a camera above each eye and a microphone by each ear. Equip the bedridden person, the "Rider", with VR gear that immersively presents these audiovisual stimuli to Rider's eyes and ears. Also equip Rider with a microphone to speak with Host, directly into Host's ear. Now send Host on a trip. During this trip, let Host be guided mainly by Rider's expressed desires, going where Rider wants, looking where Rider wants to look, stopping and listening where Rider wants to stop and listen. Unlike VR tours as they currently exist, Host can interact with and alter the environment in real time. Rider could have Host lift a flower and turn it in their hands, then cast the flower into a stream and watch it drift away. Host could purchase goods or services on Rider's behalf. Host could conduct a conversation through Rider, interacting with locals by having Rider speak Host's words verbatim.

(In the psychology literature, speaking the words of another immediately upon hearing them quietly projected into your ear is called "conversational shadowing". In his career as a psychologist, my father was involved in some early shadowing studies. Shadowing can be surprisingly fast and smooth once one gets the hang of it, with utterances almost simultaneous as the host half-anticipates the rider's next word.)

A more ambitious Host-Rider setup would employ VR gloves. As I imagine it, Rider and Host wear matching gloves. These gloves are synchronized to move in exactly the same way -- of course with quick escape overrides and perhaps with Rider's motions damped down to prevent overextension or bumping into unseen obstacles near the bed. This would take some excellent technological chicanery with good motion tracking (Nintendo Wii, improved) and some way of restricting or guiding the movements of the gloves on each end, perhaps using magnetic fields, so that when Rider wants to move a hand on vector W and Host wants to move it on vector V, the result is some compromise vector (barring safety override). An intuitive collaboration would be necessary between Rider and Host, and gentle, predictable movements. With practice, I think it would not be unfeasible, at least in safe and simple environments, for Rider to feel as though it is almost their hands that are moving in the virtual environment. This could be further enhanced with tactile feedback -- that is, if pressure sensors in Host's gloves communicated with actuators in Rider's gloves that exerted corresponding pressures in corresponding locations.

(Here, compare the Rubber Hand or Body Transfer Illusion. I have been a participant in some body transfer experiments and have informally conducted others at home with friends and family. It is amazing how flexible one's sense of one's bodily shape, position, and boundaries can be, given suggestive feedback!)

A final, expensive, risky, and much more conjectural step here -- probably a step too far -- would involve equipping Host and Rider with helmets with brain-imaging technology and the capacity for Transcranial Magnetic Stimulation (or some other ability to directly stimulate or suppress brain activity). For example, for a fuller tactile experience, activity in Host's primary somatosensory cortex could be tracked, and a vague, faint echo of it could be stimulated in matching areas in Rider's cortex. One wouldn't want too much synchrony, and anyhow brains differ somewhat in their organizational structure, even in fairly similarly structured regions like somatosensory cortex. Also, of course, you wouldn't want much motor signal going down through efferent nerves into the Rider's body, making Rider move around in the bed. Also, current brain imaging technologies, even if we imagine portable versions of them, wouldn't be spatiotemporally sharp enough to do a proper job of it. But still, a dim, vague signal might be very suggestive for an otherwise well-harmonized, collaborative Host and Rider, in a rich environmental context with very clear cues and expectations.

I won't conjecture about, say, attempting to match activity in emotional regions or associative cortex. Conjecturing about a high degree of match between different people's brains is both neurophysiologically unrealistic and possibly too threatening to the autonomy of Host and Rider -- though of course there are dark, interesting, far-future science-fictional possibilities there.

Let's bracket, then, this last conjectural step with brain imaging and brain stimulation, and focus just on the VR experience with audiovisual input, shadowing, and matching-motion gloves -- all in a positive, harmonious, and non-exploitative relationship with a host. I think that would be pretty cool, and near- to medium-term technologically achievable. If it plays out right, it might give some bedridden people a chance to explore the world beyond their bed in a much more vivid, engaging, and interactive manner.

My father was both a psychologist and an inventor. In 1995, when he was first diagnosed with cancer, he had been wanting to go to Hong Kong, and he had to cancel the trip to attempt a bone marrow transplant. He never did make it to Hong Kong. I wish I could bring him back, build some of this technology with him, then take him there.

[image source]

What would it take for humanity to survive? (And does it matter if we do?) (guest post by Henry Shevlin)

guest post by Henry Shevlin

The Doctor: You lot, you spend all your time thinking about dying, like you're gonna get killed by eggs, or beef, or global warming, or asteroids. But you never take time to imagine the impossible. Like maybe you survive. (Doctor Who, “The End of the World”)

It’s tempting to think that humanity is doomed: environmental catastrophe, nuclear war, and pandemics all seem capable of wiping us out, and that’s without imagining all of the exciting new technologies that might be lying in wait across the horizon waiting to devour us. However, I’m an optimist. I think there’s an excellent chance humanity will see this century out. And if we eventually become a multi-planetary species, the odds start looking really quite good for us. Nonetheless, in thinking about the potential value in human survival (or the potential loss from human extinction), I think we could do more first to pin down whether (and why) we should care about our survival, and exactly what would be required for us to survive.

For many hardnosed people, I imagine there’s an obvious answer to both questions: there is no special value in human survival, and in fact, the universe may be a better place for everyone (including perhaps us) if we were to all quietly go extinct. This is a position I’ve heard from ecologists and antinatalists, and while I won’t debate it here, I find it deeply unpersuasive. As far as we know, humanity is the only truly intelligent species in the universe – the only species that is capable of great works of art, philosophy, and technological development. And while we may not be the only conscious species on earth, we are likely the only species capable of the more rarefied forms of happiness and value. Further to that, even though there are surely other conscious species on earth worth caring about, our sun will finish them off in a few billion years, and they’re not getting off this planet without our help (in other words: no dogs on Mars unless we put them there).

However, even if you’re sympathetic to this line of response, it admittedly doesn’t show there’s any value in specifically human survival. Even if we grant that humans are an important source of utility worth protecting, surely there are intelligent aliens somewhere out there in the cosmos capable of enjoying just as fancy pleasures as those we experience. Insofar as we’re concerned with human survival at all, then, maybe it should just be in virtue of our more general high capacity for well-being?

Again, I’m not particularly convinced by this. Leaving aside the fact that we may be alone in the universe, I can’t shake the deep intuition that there’s some special value in the thriving of humanity, even if only for us. To illustrate the point, imagine that one day a group of tiny aliens show up in orbit and politely ask if they can terraform earth to be more amenable to them, specifically replacing our atmosphere with one composed of sulphur dioxide. The downside of this will be that humanity and all of the life on Earth will die out. On the upside, however, the aliens’ tiny size means that Earth could sustain trillions of them. “You’re rational ethical beings,” they say. “Surely, you can appreciate that it’s a better use of resources to give us your planet? Think of all the utility we’d generate! And if you’re really worried, we can keep a few organisms from every species alive in one of our alien zoos.”

Maybe I’m parochial and selfish, but the idea that we should go along with the aliens’ wishes seems absurd to me (well, maybe they can have Mars). One of my deepest moral intuitions is that there is some special good that we are rationally allowed – if not obliged – to pursue in ensuring the continuation and thriving of humanity.

Let’s just say you agree with me. We now face a further question: what would it take for humanity to survive in this ethically relevant sense? It’s a surprisingly hard question to answer. One simple option would be that we survive as long as the species Homo sapiens is still kicking around. Without getting too deeply into the semantics of “humanity”, it seems like this misses the morally interesting dimensions of survival. For example, imagine that in the medium term future, beneficial gene-modding becomes ubiquitous, to the point where all our descendants would be reproductively cut off from breeding with the likes of us. While that would mean the end of Homo sapiens (at least by standard definitions of species), it wouldn’t, to my mind, mean the end of humanity in the broader and more ethically meaningful sense.

A trickier scenario would involve the idea that one day we may cease to be biological organisms, having all uploaded ourselves to computers or robot bodies. Could humanity still exist in this scenario? My intuition is that we might well survive this. Imagine a civilization of robots who counted biological humans among their ancestors, and went around quoting Shakespeare to each other, discussing the causes of the Napoleonic Wars, and debating whether the great television epic Game of Thrones was a satisfactory adaptation of the books. In that scenario, I feel that humanity in the broader sense could well be thriving, even if we no longer have biological bodies.

This leads me to a final possibility: maybe what’s ethically relevant in our survival is really the survival of our culture and values: that what matters is really that beings relevantly like us are partaking in the artistic and cultural fruits of our civilization.

While I’m tempted by this view, I think it’s just a little bit too liberal. Imagine we wipe ourselves out next year in a war involving devastating bioweapons, and then a few centuries later, a group of aliens show up on Earth to find that nobody’s home. Though they’re disappointed that there are no living humans, they are delighted by the cultural treasure trove of they’ve found. Soon, alien scholars are quoting Shakespeare and George R.R. Martin and figuring out how to cook pasta al dente. Earth becomes to the aliens what Pompeii is to us: a fantastic tourist destination, a cultural theme park.

In that scenario, my gut says we still lose. Even though there are beings that are (let’s assume) relevantly like us that are enjoying our culture, humanity did not survive in the ethically relevant sense.

So what’s missing? What is it that’s preserved in the robot descendant scenario that’s missing in the alien tourist one? My only answer is that some kind of appropriate causal continuity must be what makes the difference. Perhaps it’s that we choose, through a series of voluntary, purposive actions to bring about the robot scenario, whereas the alien theme park is a mere accident. Or perhaps it’s the fact that I’m assuming there’s a gradual transition from us to the robots, rather than the eschatological lacuna of the theme park case.

I have some more thought experiments that might help us decide between these alternatives, but that would be taking us beyond the scope of a blogpost. And perhaps my intuitions that got us this far are already radically at odds with yours. But in any case, as we take our steps into the next stage of human development, I think it’s important for us to figure out what it is about us (if anything) that makes humanity valuable.

[image source]

What Would (or Should) You Do With Administrator Access to Your Mind (guest post by Henry Shevlin)

guest post by
Henry Shevlin

'Dial 888,' Rick said as the set warmed. 'The desire to watch TV, no matter what's on it.

'I don't feel like dialling anything at all now,' Iran said.

'Then dial 3,' he said.

'I can't dial a setting that stimulates my cerebral cortex into wanting to dial! If I don't want to dial, I don't want to dial that most of all, because then I will want to dial, and wanting to dial is right now the most alien drive I can imagine.’

(PHILIP K. DICK, Do Androids Dream of Electric Sheep)

--------------------------

We don’t have direct control over most our beliefs and attitudes, let alone most of our drives and desires. No matter how much money was offered as an incentive, for example, I couldn’t will myself to believe in fairies by this evening. Similarly, figuring out how to rid ourselves of our involuntary prejudices and biases is tricky (see here for an attempt), and changing our basic drives (such as our sexual orientation) is almost certainly impossible.

That’s not to say that we have zero control over any of these things. If I wanted to increase the likelihood of having religious beliefs, for example, I might decide to start hanging out with religious people, or attending services. But it’s a messy and indirect path to acquiring new beliefs and values.

Imagine, then, how useful it would be if we had some kind of more direct ability to control our minds. In thinking about this possibility, a useful analogy comes from the idea of Administrator access on a computer. What if – perhaps for just a few hours a month – you could delve into your beliefs, your values, and your drives, and reconfigure them to your heart’s content, before ‘logging back in’ as your (now modified) self?

Some immediately tempting applications of this possibility are fairly clear. For one, we’d perhaps want to eliminate or tone down our most egregious cognitive biases: confirmation bias, post-purchase rationalization, the sunk cost fallacy, and so on. Similarly, we might want to rid ourselves of implicit prejudices that we may have against groups or individuals. Prejudiced against elderly people? Just go into Settings Menu and adjust your slider to correct it. Irrationally resentful of a colleague who accidentally slighted you? A quick fix to remove the relevant emotion and you’re sorted.

Another attractive application might be to bring our immediate desires into line with our higher-order desires. Crave cigarettes and wish you didn’t? Tamp down the relevant first-order desire and you’re sorted. Wish you had the motivation to run in the mornings? Then ramp up the slider for “desire to go jogging”. We might even want to give ourselves some helpful false beliefs or ‘constructive myths’. Disheartened by the fact that you as an individual can do little to prevent climate change? Maybe a false belief that you can be a powerful agent for change will help you do good.

Finally, we come to the most controversial stuff, like values, drives, and memories. Take values first. Imagine that you find yourself trapped in a small town where you’re ostracised for your deviant political beliefs. One easy option might be to simply tweak your values to come into line with your community. Or imagine if you could adjust things like your sexual drives and orientation. Certainly, some people might feel relief at ridding themselves of certain kinks or fetishes that they found oppressive, while others might enjoy experimenting with recalibrating their sexuality. But we could also find that people were pressured or tempted to adjust their sexuality to bring it into line with the bigoted social expectations of their community, and it’s hard not to find that a morally troubling idea. Finally, imagine if we could wipe away unpleasant memories at will – the bad relationships, social gaffes, and painful insults could be gone in a moment. What could possibly go wrong with that?

As much as I like the idea of tweaking my mind, I feel uncomfortable about lot of these possibilities. First, at the risk of sounding cliched, it seems like the gains of personal growth are often as much in the journey as the destination. So, take someone who learns to become more patient with others’ failings. Along the way, she’s probably going to pick up a bunch of other important realizations – of her own fallibility, perhaps, or of the distress she’s caused in the past by dismissing people. Skipping straight to the outcome threatens to cheat her, and us, of something valuable. Similarly, sometimes along the road of personal change, we realize that we’ve been aiming for the wrong thing. Someone who desperately wants to fit in with their peer group, for example, might slowly and painfully realize that they don’t like their peers as much as they thought. Skipping out the journey, then, not only robs us of potential goods we might find along the way, but also of the capacity to change our mind about where we’re going.

There might also be some kinds of extrinsic goods that would be lost if we could all tweak our minds so effortlessly. Take the example of someone who wishes he could fit in with his more conservative community. Even though he might relish not having values that are different from those around him, by holding onto them, he could be providing encouragement and cover for other political deviants in his town. In much the same way, diversity of opinion, outlook, and motivation may be valuable for the community at large, despite not always being pleasant for those in the minority. This can be true even if the majority perspective in the community is in the right: dissenters can helpfully force the dominant voices to articulate and justify their views.

Finally, we could run into serious unexpected consequences – maybe getting rid of the availability heuristic would turn out to drastically slow down my reasoning, for example, or perhaps making myself more prosocial could backfire on me if I live in an antisocial community. Still more catastrophic consequences might involve deviant paths to fulfilment of desires. If (in Administrator mode) I give myself an overriding desire to be “fitter than the average person in my town”, for example, I might (as a normal user) go on to decide that the fastest way to achieve that goal is to kill all the healthy people in my community! More prosaically, it’s also easy to imagine people being tempted to reconfigure their difficult-to-achieve desires (like becoming rich and famous) and instead replacing them with stuff that’s easy to achieve (collecting paperclips, say, or counting blades of grass). Perhaps they would be well advised to do so, but this is philosophically controversial to say the least!

While Administrator Access to our own minds is of course just science fiction for now, I think it’s a useful tool for probing our intuitions about well-being, rationality, and personal change. It could also potentially guide us in situations where do have more powerful ways of influencing the development of minds. This may be a big deal in the development of future forms of artificial intelligence, for example, but something similar arguably applies even when we’re deciding how to raise our children (should we encourage them to believe in Santa Claus?).

For my part, I doubt I could resist making a few tweaks to myself (maybe I’d finally get to make good use of that gym membership). But I’d do so carefully... and likely with a sense of trepidation and unease.

[image source]

The Social-Role Defense of Robot Rights

Daniel Estrada's Made of Robots has launched a Hangout on Air series in philosophy of technology. The first episode is terrific!

Robot rights cheap yo.

Cheap: Estrada's argument for robot rights doesn't require that robots have any conscious experiences, any feelings, any reinforcement learning, or (maybe) any cognitive processing at all. Most other defenses of the moral status of robots assume, implicitly or explicitly, that robots who are proper targets of moral concern will exist only in the future, once they have cognitive features similar to humans or at least similar to non-human vertebrate animals.

In contrast, Estrada argues that robots already deserve rights -- actual robots that currently exist, even simple robots.

His core argument is this:

1. Some robots are already "social participants" deeply incorporated into our social order.

2. Such deeply incorporated social participants deserve social respect and substantial protections -- "rights" -- regardless of whether they are capable of interior mental states like joy and suffering.

Let's start with some comparison cases. Estrada mentions corpses and teddy bears. We normally treat corpses with a certain type of respect, even though we think they themselves aren't capable of states like joy and suffering. And there's something that seems at least a little creepy about abusing a teddy bear, even though it can't feel pain.

You could explain these reactions without thinking that corpses and teddy bears deserve rights. Maybe it's the person who existed in the past, whose corpse is now here, who has rights not to be mishandled after death. Or maybe the corpse's relatives and friends have the rights. Maybe what's creepy about abusing a teddy bear is what it says about the abuser, or maybe abusing a teddy harms the child whose bear it is.

All that is plausible, but another way of thinking emphasizes the social roles that corpses and teddy bears play and the importance to our social fabric (arguably) of our treating them in certain ways and not in other ways. Other comparisons might be: flags, classrooms, websites, parks, and historic buildings. Destroying or abusing such things is not morally neutral. Arguably, mistreating flags, classrooms, websites, parks, or historic buildings is a harm to society -- a harm that does not reduce to the harm of one or a few specific property owners who bear the relevant rights.

Arguably, the destruction of hitchBOT was like that. HitchBOT was cute ride-hitching robot, who made it across the length of Canada but who was destroyed by pranksters in Philadelphia when its creators sent it to repeat the feat in the U.S. Its destruction not only harmed its creators and owners, but also the social networks of hitchBOT enthusiasts who were following it and cheering it on.

It might seem overblown to say that a flag or a historic building deserves rights, even if it's true that flags and historic buildings in some sense deserve respect. If this is all there is to "robot rights", then we have a very thin notion of rights. Estrada isn't entirely explicit about it, but I think he wants more than that.

Here's the thing that makes the robot case different: Unlike flags, buildings, teddy bears, and the rest, robots can act. I don't mean anything too fancy here by "act". Maybe all I mean or need to mean is that it's reasonable to take the "intentional stance" toward them. It's reasonable to treat them as though they had beliefs, desires, intentions, goals -- and that adds a new richer dimension, maybe different in kind, to their role as nodes in our social network.

Maybe that new dimension is enough to warrant using the term "rights". Or maybe not. I'm inclined to think that whatever rights existing (non-conscious, not cognitively sophisticated) robots deserve remain derivative on us -- like the "rights" of flags and historic buildings. Unlike human beings and apes, such robots have no intrinsic moral status, independent of their role in our social practices. To conclude otherwise would require more argument or a different argument than Estrada gives.

Robot rights cheap! That's good. I like cheap. Discount knock-off rights! If you want luxury rights, though, you'll have to look somewhere else (for now).

[image source] Update: I changed "have rights" to "deserve rights" in a few places above.

AI Consciousness: A Reply to Schwitzgebel

Guest post by Susan Schneider

If AI outsmarts us, I hope its conscious. It might help with the horrifying control problem – the problem of how to control superintelligent AI (SAI), given that SAI would be vastly smarter than us and could rewrite its own code. Just as some humans respect nonhuman animals because animals feel, so too, conscious SAI might respect us because they see within us the light of conscious experience.

So, will an SAI (or even a less intelligent AI) be conscious? In a recent Ted talk, Nautilus and Huffington Post pieces, and some academic articles (all at my website) I’ve been urging that it is an important open question.

I love Schwitzgebel's reply because he sketches the best possible scenario for AI consciousness: noting that conscious states tend to be associated with slow, deliberative reasoning about novel situations in humans, he suggests that SAI may endlessly invent novel tasks – e.g., perhaps they posit ever more challenging mathematical proofs, or engage in an intellectual arms race with competing SAIs. So SAIs could still engage in reasoning about novel situations, and thereby be conscious.

Indeed, perhaps SAI will deliberately engineer heightened conscious experience in itself, or, in an instinct to parent, create AI mindchildren that are conscious.

Schwitzgebel gives further reason for hope: "...unity of organization in a complex system plausibly requires some high-level self-representation or broad systemic information sharing." He also writes: "Otherwise, it's unlikely that such a system could act coherently over the long term. Its left hand wouldn't know what its right hand is doing."

Both of us agree that leading scientific approaches to consciousness correlate consciousness with novel learning and slow, deliberative focus, and that these approaches also associate consciousness with some sort of broad information sharing from a central system or global workspace (see Ch. 2 of my Language of Thought: a New Philosophical Direction where I mine Baars' Global Workspace Theory for a computational approach to LOT's central system).

Maybe it is just that I'm too despondent since Princess Leah died. But here's a few reasons why I still see the glass half empty:

a. Eric's points assume that reasoning about novel situations, and centralized, deliberative thinking more generally, will be implemented in SAI in the same way they are in humans – i.e., in a way that involves conscious experience. But the space of possible minds is vast: There could be other architectural ways to get novel reasoning, central control, etc. that do not involve consciousness or a global workspace. Indeed, if we merely consider biological life on Earth we see intelligences radically unlike us (e.g., slime molds, octopuses); there will likely be radically different cognitive architectures in the age of AGI/superintelligence.

b. SAI may not have a centralized architecture in any case. A centralized architecture is a place where highly processed information comes together from the various sensory modalities (including association areas). Consider the octopus, which apparently has more neurons in its arms than in its brain. The arms can carry out activity without the brain; these activities do not need to be coordinated by a central controller or global workspace in the brain proper. Maybe a creature already exists, elsewhere in the universe, that has even less central control than the octopus.

Indeed, coordinated activity doesn't require that a brain region or brain process be a place where it all comes together, although it helps. There are all kinds of highly coordinated group activities on Earth, for instance (the internet, the stock market). And if you ask me, there are human bodies that are now led by coordinated conglomerates without a central controller. Here, I am thinking of split brain patients, who engage in coordinated activity (i.e., the right and left limbs seem to others to be coordinated). But the brain has been split through removal of the corpus collosum, and plausibly, there are two subjects of experience there. The coordination is so convincing that even the patent's spouse doesn't realize there are two subjects there. It takes highly contrived laboratory tests to determine that the two hemispheres are separate conscious beings. How could this be? Each hemisphere examines the activity of the other hemisphere (the right hemisphere observes the behavior of the limb it doesn't control, etc.) And only one hemisphere controls the mouth.

c. But assume the SAI or AGI has a similar cognitive architecture as we do; in particular, assume it has an integrated central system or global workspace (as in Baars' Global Workspace Theory). I still think consciousness is an open question here. The problem is that only some implementations of a central system (or global workspace) may be conscious, while others may not be. Highly integrated, centralized information processing may be necessary, but not sufficient. For instance, it may be that the very properties of neurons that enable consciousness, C1-C3, say, are not ones that AI programs need to reproduce to get AI systems that do the needed work. Perhaps AI programmers can get sophisticated information processing without needing to go as far as build systems to instantiate C1-C3. Or perhaps a self-improving AI may not bother to keep consciousness in its architecture, or lacking consciousness, it may not bother to engineer it in, as its final and instrumental goals may not require it. And who knows what their final goals will be; none of the instrumental goals Bostrom and others identify require consciousness (goal content integrity, cognitive enhancement,etc.)

Objection (Henry Shevlin and others): am I denying that it is nomologically possible to create a copy of a human brain, in silicon or some other substance, that precisely mimics the causal workings of the brain, including consciousness?

I don't deny this. I think that if you copy the precise causal workings of cells in a different medium you could get consciousness. The problem is that it may not be technologically feasible to do so. (An aside: for those who care about the nature of properties, I reject pure categorical properties; I have a two-sided view, following Heil and Martin. Categoricity and dispositionality are just different ways of viewing the same underlying property—two different modes of presentation, if you will. So consciousness properties that have all and only the same dispositions are the same type of property. You and your dispositional duplicate can't differ in your categorical properties then. Zombies aren't possible.)

It seems nomologically possible that an advanced civilization could build a gold sphere the size of Venus. What is the probability this will ever happen though? This depends upon economics and sociology – a civilization would need to be a practical incentive to do this. I bet it will never happen.

AI is currently being built to do specific tasks better than us. This is the goal, not reproducing consciousness in machines. It may be that the substrate used to build AI is not a substrate that instantiates consciousness easily. Engineering consciousness in may be too expensive and time consuming. Like building the Venus-sized gold sphere. Indeed, given the ethical problems with creating sentient beings and then having them work for us, AI programs may aim to build systems that aren't conscious.

A response here is that once you get a sophisticated information processor, consciousness inevitably arises. Three things seem to fuel this view: (1) Tonini's information integration theory (IIT). But it seems to have counterexamples (see Scott Aaronson's blog). (2) Panpsychism/panprotopsychism. Even if one of these views is correct, the issue of whether a given AI is conscious is about whether the AI in question has the kind of conscious experience macroscopic subjects of experience (persons, selves, nonhuman animals) have. Merely knowing whether panpsychism or panprotopsychism is true does not answer this. We need to know which structural relations between particles lead to macroexperience. (3) Neural replacement cases. I.e., thought experiments in which you are asked to envision replacing parts of your brain (at time t1) with silicon chips that function just like neurons, so that in the end, (t2), your brain is made of silicon. You are then asked: intuitively are you still conscious? Do you think the quality of your consciousness would change? These cases only goes so far. The intuition is plausible that from t1 to t2, at no point would you would lose consciousness or have your consciousness diminished (see Chalmers, Lowe and Plantinga for discussion of such thought experiments). This is because a dispositional duplicate of your brain is created, from t1 to t2. If the chips are dispositional duplicates of neurons, sure, I think the duplicate would be conscious. (I'm not sure this would be a situation in which you survived though-see my NYT op-ed on uploading.) But why would an AI company build such a system from scratch, to clean your home, be a romantic partner, advise a president, etc?

Again, is not clear, currently, that just by creating a fast, efficient program ("IP properties") we have also copied the very same properties that give rise to consciousness in humans ("C properties"). It may require further work to get C properties, and in different substrates, it may be hard, far more difficult than building a biological system from scratch. Like creating a gold sphere the size of Venus.

My Daughter's Rented Eyes

(inspired by a conversation with Cory Doctorow about how a kid's high-tech rented eyes might be turned toward favored products in the cereal aisle)

At two million dollars outright, of course I couldn't afford to buy eyes for my four-year-old daughter Eva. So, like everyone else whose kids had been blinded by the GuGuBoo Toy company's defective dolls (may its executives rot in bankruptcy Hell), I rented the eyes. What else could I possibly do?

Unlike some parents, I actually read the Eye & Ear Company's contract. So I knew part of what we were in for. If we didn't make the monthly payments, her eyes would shut off. We agreed to binding arbitration. We agreed to a debt-priority clause, to financial liability for eye extraction, to automatic updates. We agreed that from time to time the saccade patterns of her eyes would be subtly adjusted so that her gaze would linger over advertisements from companies that partnered with Eye & Ear Co. We agreed that in the supermarket, Eva's eyes would be gently maneuvered toward the Froot Loops and the M&Ms.

When the updates came in, we always had the legal right to refuse them. We could, hypothetically, turn off Eva's eyes, then have them surgically removed and returned to Eye & Ear Co. Each new rental contract was thus technically voluntary.

When Eva was seven, the new updater threatened shutoff unless we transferred $1000 into a debit account. Her updated eyes contained new software to detect any copyrighted text or images she might see. Instead of buying copyrighted works in the usual way, we agreed to have a small fee deducted from our account for each work Eva viewed. Instead of paying $4.99 for the digital copy of a Dr Seuss book, Eye & Ear would deduct $0.50 each time she read the book. Video games might be free with ads, or $0.05 per play, or $0.10, or even $1.00. Since our finances were tight, we set up parental controls: Eva's eyes required parental permission for any charge over $0.99 or any cumulative charges over $5.00 in a day -- and of course they also blocked any "adult" material. Until we granted approval, blocked or unpaid material was blurred and indecipherable, even if she was just peeking over someone's shoulder at a book or walking past a television in a dentist's lobby.

When Eva was ten, the updater overlaid advertisements in her visual field. It helped keep the rental costs down. (We could have bought out of the ads for an extra $6,000 a year.) The ads never interfered much with Eva's vision -- they just kind of scrolled across the top of her visual field sometimes, Eva told us, or printed themselves onto clouds and the sides of buildings.

By the time Eva was thirteen, I'd finally risen to a managerial position at work, and we could afford the new luxury eyes for her. By adjusting the settings, Eva could see infrared at night. She could zoom in on distant objects. She could bug out her eyes and point them in different directions like some sort of weird animal, to take in a broader field of view. She could also take snapshots and later retrieve them with a subvocalization -- which gave her a great advantage at school over her normal-eyed and cheaper-eyed peers. Installed software could text-search through stored snapshots, solve mathematical equations, and pull relevant information from the internet. When teachers tried to ban such enhancements from the classroom, Eye & Ear fought back, arguing that the technology had become so integral to the children's lives that it couldn't be removed without disabling them. Eye & Ear refused to develop the technology to turn off the enhancement features, and no teacher could realistically prevent a kid from blinking and subvocalizing.

By the time Eva was seventeen it looked like she and the two other kids at her high school with luxury eye rentals would more or less have their choice among elite universities. I refused to believe the rumors about parents intentionally blinding their children so that they too could rent eyes.

When Eva turned twenty, all the updates -- not just the cheap ones -- required that you accept the "acceleration" technology. Companies contracted with Eye & Ear to privilege their messages and materials for faster visual processing. Pepsi paid a hundred million dollars a year so that users' eyes would prioritize resolving Pepsi cans and Pepsi symbols in the visual scene. Coca Cola cans and symbols were "deprioritized" and stayed blurry unless you focused on them for a few seconds. Loading stored images worked similarly. A remembered scene with a Pepsi bottle in it would load almost instantly. One with a Coke bottle would take longer and might start out fuzzy or fragmented.

Eye & Ear started to make glasses for the rest of us, which imitated some of the functions of the implants. Of course they were incredibly useful. Who wouldn't want to take snapshots, see in the dark, zoom into the distance, get internet search and tagging? We all rented whatever versions we could afford, signed the annual terms and conditions, received the updates. We wore them pretty much all day, even in the shower. The glasses beeped alarmingly whenever you took them off, unless you went through a complex shutdown sequence.

When the "Johnson for President" campaign bought an acceleration, the issue went all the way to the Supreme Court. Johnson's campaign had paid Eye & Ear to prioritize the perception of his face and deprioritize the perception of his opponent's face, prioritize the visual resolution and recall of his ads, deprioritize the resolution and recall of his opponent's ads. Eva was now a high-powered lawyer in a New York firm, on the fast track toward partner. She worked for the Johnson campaign, though I wouldn't have thought it was like her. Johnson was so authoritarian, shrill, and right-wing -- or at least it seemed so to me, when I took my glasses off.

Johnson favored immigration restrictions, and his opponent claimed (but never proved) that Eye & Ear implemented an algorithm that highlighted people's differences in skin tone -- making the lights a little lighter, the darks a little darker, the East Asians a bit yellow. Johnson won narrowly, before his opponent's suit about the acceleration had made it through the appeals process. It didn't hit the high court until a month after Johnson's inauguration. Eva helped prepare Johnson's defense. Eight of the nine justices were over eighty years old. They lived stretched lives with enhanced longevity and of course all the best implants. They heard the case through the very best ears.

---------------------------------

Related post:

Possible Cognitive and Cultural Effects of Video Lifelogging (Apr 21, 2016)

---------------------------------

Image source:

Photo: HAL 9000 resurrected by Ram Yoga, doubled.

Under What Conditions Would You Upload into a Simulated World?

I have a new science fiction story out in Clarkesworld (text version, audiocast). One of my philosophical aims in that story is to reflect about the conditions under which it would make sense to upload into a simulated world.

"Mind uploading" is the hypothetical process of copying your mind into a computational device. If futurists such as Ray Kurzweil are correct, we might be only a few decades away from mind uploading as a real technological option.

When mind uploading is presented in science fiction, usually the following are assumed:

(1.) The uploaded mind retains all the important psychological features of the original mind (including, of course, a genuine stream of conscious experiences).

(2.) The uploaded mind experiences a world as rich as the world of the original mind. If the mind is uploaded into a robot, that world might just be the same world as the world of the original mind. If the mind is uploaded into a Sim or a Matrix world -- that is, an artificial environment of some sort -- that artificial environment is usually assumed to be as rich and interesting as the mundane environment of everyday Earth for normally embodied humans.

Under these conditions, if we also assume that uploading has substantial advantages in improving longevity, allowing backups, improving one's cognitive powers, or giving oneself access to a new rich vein of experiences and possibilities, then it probably makes sense to upload, unless one is strongly committed to traditional human embodiment or to projects or relationships that would no longer be possible post-upload.

However, it seems natural to suppose that if uploading does someday become possible the first uploads will not have features (1) and (2). The first uploaded people (or "people"), even if genuinely conscious, might be seriously psychologically impaired and unable to access a full, rich world of experiences.

There might, however, still be advantages of uploading in terms of longevity and experienced pleasure.

In "Fish Dance", the narrator is presented with the option of uploading his mind under these conditions:

(a.) the world is tiny: a single dance floor, with no input from the larger world outside;
(b.) his mind is limited: he will have some memories from his pre-uploaded self, but he won't fully understand them, and furthermore he won't be able to lay down new memories that last for more than an hour or so;
(c.) his dance-floor experiences will be extremely pleasurable: ideal experiences of dancing ecstasy;
(d.) he will experience this extreme pleasure for a billion years.

Also relevant, of course, are the relationships and projects that he would be leaving behind. (In our narrator's case, a recently deceased child, a living teenage child who wants him to upload, a stale marriage, and an okay but not inspiring career as a professor.)

I say the relationships and projects "he" will leave behind, but of course one interesting question is whether it makes sense to call the uploaded being "him", that is, the same "him" as the narrator.

If it seems obvious to you what one should do under such conditions, the parameters are of course adjustable: We can increase or decrease psychological function, psychological similarity, and quality of memory. We can increase or decrease the size of the world and the range of available input. We can increase or decrease the pleasure and longevity. We can modify the relationships and projects that would be left behind.

You or your descendants might actually face some version of this decision.

----------------------------------------------

"Fish Dance" (Clarkesworld #118, July 2016)

Related blogpost: Goldfish Pool Immortality (May 30, 2014)

[image source]

Susan Schneider on How to Prevent a Zombie Dictatorship

Last week I posted "How to Accidentally Become a Zombie Robot", discussing Susan Schneider's recent TEDx-talk proposal for checking whether silicon chips can be conscious. Susan has written the following reply, which she invited me to share on the blog.

---------------------------------------------

Eric,

Greetings from a café in Lisbon! Your new, seriously cool I,Brain case raises an important point about my original test in my Ted talk and is right out of a cyberpunk novel. A few initial points, for readers, before I respond, as Ted talks don’t give much philosophical detail:

1. It may be that the microchips are made of something besides silicon, (right now, e.g., carbon nanotubes and graphene are alternate substrates under development). I don’t think this matters – the issues that arise are the same.

2. It will be important that any chip test involve the very kind of chip substrate and design as that used in the AI in question.

3. Even if a kind of chip works in humans, there is still the issue of whether the AI in question has the right functional organization for consciousness. Since AI could be very different than us, and it is even difficult to figure out these issues in the case of biological creatures like the octopus, this may turn out to be very difficult.

4. For relation of the chip test to intriguing ideas of Ned Block and Dave Chalmers on this issue, see a paper on my website (a section in “The Future of Philosophy of Mind”, based on an earlier op-ed of mine.)

5. As Eric knows, it is probably a mistake to assume that brain chips will be functional isomorphs. I’m concerned with the development of real, emerging technologies, because I am concerned with finding a solution to the problem of AI consciousness based on an actual test. Brain chips, already under development at DARPA, may eventually be faster and more efficient information processors, enhance human consciousness, or, they may be low fidelity copies of what a given minicolumn does. This depends upon how medicine progresses...

Back to my original “Chip Test” (in the Ted talk). It’s 2045. You are ready to upgrade your aging brain. You go to I,Brain. They can gradually replace parts of your biological brain with microchips. You are awake during the surgery, suppose, and they replace a part of your biological brain that is responsible for some aspect of consciousness, with a microchip. Do you lose consciousness of something (e.g., do you lose part of your visual field)? If so, you will probably notice. This would be a sign that the microchip is the wrong stuff. Science could try and try to engineer a better chip, but if after years of trying, they never could get it right, perhaps we should conclude that that kind of substrate (e.g., silicon) does not give rise to consciousness.

On the other hand, if the chips work, that kind of substrate is in principle the right stuff (it can, in the right mental environment, give rise to qualia) although there is a further issue of whether a particular AI that has such chips has the right organization to be conscious (e.g., maybe it has nothing like a global workspace, like a Rodney Brooks style robot, or maybe it is superintelligent, and has mastered everything already and eliminated consciousness because it is too slow and inefficient).

Eric, your test is different, and I agree that someone should not trust that test. This would involve a systematic deception. What kind of society would do this? A zombie dictatorship, of course, which seeks to secretly eliminate conscious life from the planet. :-)

But I think you want to apply your larger point to the original test. Is the idea: couldn’t a chip be devised that would falsely indicate consciousness to the person? (Let’s call this a “sham qualia chip.” ) I think it is, so here’s a reply: God yes, in a dystopian world. We had better watch out! That would be horrible medicine…and luckily, it would involve a good deal of expense and effort, (systematically fooling someone about say, their visual experience, would be a major undertaking), so science would likely first seek a genuine chip substitute that preserved consciousness. (Would a sham qualia chip even clear the FDA :-) ? Maybe only if microchips were not the right stuff and it was the best science could do. After all, people would always be missing lost visual qualia, and it is best that they not suffer like this....). But crucially, since this would involve a deliberate effort on the part of medical researchers, we would know this, and so we would know that the chip is not a true substitute. Unless, that is, we are inhabitants of a zombie dictatorship.

The upshot: It would involve a lot of extra engineering effort to produce a sham qualia chip, and we would hopefully know that the sham chip was really a device designed to fool us. If this was done because the genuine chip substitute could not be developed, this would probably indicate that chips aren’t the right stuff, or that science needs to go back to the drawing board.

I propose a global ban on sham qualia chips in interest of preserving democracy.

---------------------------------------------

I (Eric) have some thoughts in response. I'm not sure it would be harder to make a sham qualia chip than a genuine qualia chip. Rather than going into detail on that now, I'll let it brew for a future post. Meanwhile, others' reactions welcomed too!

How to Accidentally Become a Zombie Robot

Susan Schneider's beautifully clear TEDx talk on the future of robot consciousness has me thinking about the possibility of accidentally turning oneself into a zombie. (I mean "zombie" in the philosopher's sense: a being who outwardly resembles us but who has no stream of conscious experience.)

Suppose that AI continues to rely on silicon chips and that -- as Schneider thinks is possible -- silicon chips just aren't the right kind of material to host consciousness. (I'll weaken these assumptions below.) It's 2045 and you walk into the iBrain store, thinking about having your degenerating biological brain replaced with more durable silicon chips. Lots of people have done it already, and now the internet is full of programmed entities that claim to be happily uploaded people who have left their biological brains behind. Some of these uploaded entities control robotic or partly organic bodies; others exist entirely in virtual environments inside of computers. If Schneider is right that none of these silicon-chip-instantiated beings is actually conscious, then what has actually happened is that all of the biological people who "uploaded" actually committed suicide, and what exist are only non-conscious simulacra of them.

You've read some philosophy. You're worried about exactly that possibility. Maybe that's why you've been so slow to visit the local iBrain store. Fortunately, the iBrain company has discovered a way to upload you temporarily, so you can try it out -- so that you can determine introspectively for yourself whether the uploaded "you" really would be conscious. Federal regulations prohibit running an uploaded iBrain at the same time that the original source person is conscious, but the company can scan your brain non-destructively while you are sedated, run the iBrain for a while, then pause your iBrain and update your biological brain with memories of what you experienced. A trial run!

From the outside, it looks like this: You walk into the iBrain store, you are put to sleep, a virtual you wakes up in a robotic body and says "Yes, I really am conscious! Interesting how this feels!" and then does some jogging and jumping jacks to test out the the body. The robotic body then goes to sleep and the biological you wakes up and says, "Yes, I was conscious even in the robot. My philosophical doubts were misplaced. Upload me into iBrain!"

Here's the catch: After you wake, how do you know those memories are accurate memories of having actually been conscious? When the iBrain company tweaks your biological neurons to install the memories of what "you" did in the robotic body, it's hard to see how you could be sure that those memories aren't merely presently conscious seeming-memories of past events that weren't actually consciously experienced at the time they occurred. Maybe the robot "you" really was a zombie, though you don't realize that now.

You might have thought of this possibility in advance, and so you might remain skeptical. But it would take a lot of philosophical fortitude to sustain that skepticism across many "trial runs". If biological you has lots of seeming-memories of consciousness as a machine, and repeatedly notices no big disruptive change when the switch is flipped from iBrain to biological brain, it's going to be hard to resist the impression that you really are conscious as a machine, even if that impression is false -- and thus you might decide to go ahead and do the upload permanently, unintentionally transforming yourself into an experienceless zombie.

But maybe if a silicon-chip brain could really duplicate your cognitive processes well enough to drive a robot that acts just as you would act, then the silicon-chip brain really would have to be conscious? That's a plausible (though disputable) philosophical position. So let's weaken the philosophical and technological assumptions a little. We can still get a skeptical zombie scenario going.

Suppose that the iBrain company tires of all the "trial runs" that buyers foolishly insist on, so the company decides to save money by not actually having the robot bodies do any of those things that the the trial-run users think they do. Instead, when you walk in for a trial they sedate you and, based on what they know about your just-scanned biological brain, they predict what you would do if you were "uploaded" into a robotic body. They then give you false memories of having done those things. You never actually do any of those things or have any of those thoughts during the time your biological body is sedated, but there is no way to know that introspectively after waking. It would seem to you that the uploading worked and preserved your consciousness.

There can be less malicious versions of this mistake. Behavior and cognition during the trial might be insufficient for consciousness, or for full consciousness, while memory is nonetheless vivid enough to lead to retrospective attributions of full consciousness.

In her talk, Schneider suggests that we could tell whether silicon chips can really host consciousness by trying them out and then checking whether consciousness disappears when we do so; but I'm not sure this test would work. If nonconscious systems (whether silicon chip or otherwise) can produce both (a.) outwardly plausible behavior, and (b.) false memories of having really experienced consciousness, then we might falsely conclude in retrospect that consciousness is preserved. (This could be so whether we are replacing the whole brain at once or only one subsystem at a time, as long as "outward" means "outside of the subsystem, in terms of its influence on the rest of the brain".) We might then choose to replace conscious systems with nonconscious ones, accidentally transforming ourselves into zombies.

[image source]

----------------------------------------------

Update June 27:

Susan Schneider replies!

----------------------------------------------

Possible Cognitive and Cultural Effects of Video Lifelogging

Last week science fiction writer Ted Chiang came to Riverside to talk about the possible cognitive effects of video lifelogging. He also explores these issues in his 2013 story The Truth of Fact, the Truth of Feeling. It was an interesting talk! Chiang focused, as he also does in the story, on the transformative effects video lifelogging might have on our memories, including the possible decline of our ability to remember life events unaided if we can instead just easily call up results from a lifelog.

Lifelogging is a movement aimed at recording and monitoring the details of your everyday life. Video lifelogging, which is just starting to become feasible, involves video-recording every moment of your waking day.

Good search technology will be crucial. Imagine subvocalizing "the name of that book Emmeline[1] recommended at dinner last week" or "that time Taimur cracked a raw egg on his head", then having the relevant audiovisual results show up on your palm or your Google glass. The eventual effects on our minds, Chiang suggests, would be comparable to the transformative effects of literacy. In his talk, Chiang emphasized the decline of unaided memory and the ability to hold ourselves to higher standards of truthfulness about past deeds (what did you really say in that argument last week?).

Just as early Chinese calligraphers could not have predicted quantitative textual analysis or the internet, I think we can assume that if video lifelogging is integrated deeply into our daily lives, it will change us in ways we can't fully anticipate. I'd like to suggest two possible effects that Chiang didn't mention.

First: It is much easier to record audio and video than other sensory modalities. The recording and recreation of taste, smell, touch, and somatic sensation are much more speculative and remote. Most people already tend to privilege sight and hearing, but lifelogging could amplify that dominance -- perhaps so much that the other senses almost seem like a forgettable, buzzing distraction. Your memories of sex, for example, might focus much more on the audiovisual parts of the experience, if those are what you can easily revisit and recall (esp. with decreases in unaided memory, as Chiang suggests would be likely) -- and that in turn might lead you to focus more on those senses than on other senses in your future encounters, which in turn might substantially alter the cultural structures and expectations around sex. Similarly, perhaps, for the pleasures of eating.

Second: Chiang had explicitly set aside privacy issues, and I will also do so (maybe Cory Doctorow will address these when he comes next fall), but intentional sharing raises interesting possibilities, especially if it's possible in real-time. Suppose we can't all afford to go to the concert -- but if we pool our funds, you can go, and we can all watch your lifelog in real-time (perhaps in immersive virtual reality), which will then be saved in our lifelogs. If our cognition and culture have shifted more toward the audiovisual, then it might seem closer to actually being there than it would seem to people now, and if our autobiographical memories have become dominated by lifelog results, then later it might feel more like a real memory of having been there than an analogous experience would seem to people now. Pushed to the extreme, an emphasis on shared real-time and remembered experiences might begin to blur the boundaries of the experienced self, including reducing how much we care about whether it was our own bodies that did something or someone else's.

Just for starters.

[image source]

An Interview on AI Rights

... of Mara Garza and me, at People for the Ethical Treatment of Reinforcement Learners.

The interview expands on ideas from our forthcoming paper A Defense of the Rights of Artificial Intelligences.

Ethics, Metaethics, and the Future of Morality

a guest post by Regina Rini

Moral attitudes change over generations. A century ago in America, temperance was a moral crusade, but you’d be hard-pressed to find any mark of it now among the Labor Day beer coolers. Majority views on interracial and same-sex relationships have swung from one pole to the other within the lifetimes of many people reading this. So given what we know of the past, we can say this about the people of the future: their moral attitudes will not be the same as ours. Some ethical efforts that now belong to a minority – vegetarianism, perhaps – will become as ubiquitously upheld as tolerance for interracial partnerships. Other moral matters that seem urgent now will fade from thought just as surely as the temperance movement. We can’t know which attitudes will change and how, but we do know that moral change will happen. Should we try to stop it – or even to control it?

Every generation exercises some control over the moral attitudes of its children, through the natural indoctrination of parenting and the socialization of school. But emerging technologies now give us unprecedented scope to tailor what the future will care about. From social psychology and behavioral economics we increasingly grasp how to design institutions so that the ‘easy’ or ‘default’ choices people tend to adopt coincide with the ones that are socially valuable. And as gene-editing eventually becomes a normal part of reproduction, we will be able to influence the moral attitudes of generations far beyond our own children. Of course, it is not as simple as ‘programming’ a particular moral belief; genetics does not work that way. But we might genetically tinker with brain receptivity for neurotransmitters that affect a person’s readiness to trust, or her preference for members of her own ethnic group. We won’t get to decide precisely how our descendants come to find their moral balance – but we could certainly put our thumb on the scale.

On one way of looking at it, it’s obvious that if we can do this, we should. We are talking about morality here, the stuff made out of ‘should’s. If we can make future generations more caring, more disposed to virtue, more respectful of rational agency, more attentive to achieving the best outcomes – however morality works, we should help future people to do what they should do. That is what ‘should’ means. This thought is especially compelling when we realize that some of our moral goals, like ending racism or addressing the injustices climate change will bring, are necessarily intergenerational projects. The people of the future are the ones who will have to complete the moral journey we have begun. Why not give them a head start?

But there are also reasons to think we should not interfere with whatever course the future of morality might take. For one thing, we ought to be extremely confident that our moral attitudes are the right ones before we risk crimping the possibilities for radical moral change. Perhaps we are that confident about some issues, but it would be unreasonable to be so sure across the board. Think, for example, about moral attitudes toward the idea of ownership of digital media: whether sampling, remixing, and curating count as forms of intellectual theft. Already there appear to be generational splits on this topic, driven by technology that emerged only in the last 30 years. Would you feel confident trying to preordain moral attitudes about forms of media that won’t be invented for a century?

More insidiously, there is the possibility that our existing moral attitudes already reflect the influence of problematic political ideologies and economic forces. If we decide to impose a shape on the moral attitudes of the future, then the technology that facilitates this patterning will likely be in the hands of those who benefit from existing power structures. We may end up creating generations of people less willing to question harmful or oppressive social norms. Finally, we should consider whether any attempt to direct the development of morality is disrespectful to the free agency of future people. They will see our thumbprint on their moral scale, and they may rightly resent our influence even as they cannot escape it.

One thing these reflections bring out is that philosophers are mistaken when they attempt to cleanly separate metaethical questions (what morality is) from normative ethical questions (what we should do). If this clean separation was ever tenable, our technologically expanded control over the future makes it implausible now. We face imminent questions about what we should do – which policies and technologies to employ, to which ends – that depend upon our answers to questions about what morality is. Are there objective moral facts? Can we know them? What is the relationship between morality and human freedom? The idea that metaethical inquiry is for dusty scholars, disconnected from our ordinary social and political lives, is an idea that fades entirely from view when we look to the moral future.

--------------------------------------

I’ve drawn on several philosophers’ work for some of the arguments above. For arguments in favor of using technology to direct the morality of future generations, see Thomas Douglas, “Moral Enhancement” in the Journal of Practical Ethics; and Ingmar Persson and Julian Savulescu, Unfit for the Future. For arguments against doing so, see Bernard Williams, Ethics and the Limits of Philosophy (chapter 9); and Jurgen Habermas, The Future of Human Nature.

image credit: ’Hope in a better future’ by Massimo Valiani