If someday we can create consciousness inside computers, what moral obligations will we have to the conscious beings we create?
R. Scott Bakker and I have written a short story about this, which came out today in Nature.
You might think that it would be a huge moral triumph to create a society of millions of actually conscious, happy beings inside one's computer, who think they are living, peacefully and comfortably, in the base level of reality -- Eden, but better! Divinity done right!
On the other hand, there might be something creepy and problematic about playing God in that way. Arguably, such creatures should be given self-knowledge, autonomy, and control over their own world -- but then we might end up, again, with evil, or even with an entity both intellectually superior to us and hostile.
[For Scott's and my first go-round on these issues, see here.]
11 comments:
Congrats on the publication. Nature... I think I've heard of that one. ;)
The best fiction I read on this was this: http://subterraneanpress.com/magazine/fall_2010/fiction_the_lifecycle_of_software_objects_by_ted_chiang
some great illustrations in the hard copy too.
Congrats, Eric, I'm pretty sure that will leave a long lasting mark!
But hey, in a way, aren't alot of issues raised by having kids, willy nilly?
Iain M Banks covered this in his last Culture Novel - The Hydrogen Sonata. But still a good story.
Fantastic story.
I've been exposed to your blog before but this might actually keep me a regular.
Cheers.
Thanks for the kind comments, folks, and for the suggestions of fiction on similar issues!
Arguably, such creatures should be given self-knowledge, autonomy, and control over their own world -- but then we might end up, again, with evil
One could simulate beings who traditionally don't have such rights, like chimps, and then make them happy according in their favorite ways.
Alternatively, one could simulate beings with self-knowledge and autonomy, but design their world so that they cannot commit evil against each other because their rights are always protected.
Alternatively, one could simulate beings with self-knowledge and autonomy, but design their nature so benevolent that they do not want to commit evil.
But it might be quite tricky. I would just simplify the problem and simulate unintelligent pleasure brains in constant activation, but not everybody like that idea.
Yes! Interesting options. These are the kinds of issues I imagine the first narrator to be considering when he is doing "theodicy, top-down". It might be interesting to try another story with a narrator who works explicitly through one or more of these other ways of going....
Have you seen Fassbinder's "World on a Wire"? http://www.imdb.com/title/tt0070904/
It's a wonderful and very creepy presentation of this idea in film. Way ahead of its time for early 70s German TV...
That world on a wire sounds cool!
Sounds interesting. Thanks for the tip!
Post a Comment