Evolutionary Considerations Against a Plastic Utopia

I've been enjoying Nick Bostrom's 2024 book Deep Utopia. It's a wild series of structured speculations about meaning and purpose in a "solved" techno-utopia, where technology is so far advanced that we can have virtually anything we want instantly -- a "plastic" utopia.

Plasticity is of course limited, even in the most technologically optimistic scenarios, as Bostrom notes. Even if we, or our descendants, have massive control over our physical environment -- wave a wand and transform a mountain into a pile of candy, or whatever -- we can't literally control everything. Two important exceptions are: positional goods (for example, being first in a contest; not everyone can have this, so if others want it you might well not get it yourself) and control over others (unless you're in a despotic society with you as despot). Although Bostrom discusses these limitations, I think Bostrom underplays their significance. In a wide range of circumstances, they're enough to keep the world far from "solved" or "plastic".

Thinking about these limitations as I read Bostrom, I was also reminded of Susan Schneider's suggestion that superintelligent AI might be nonconscious because everything comes easily for them -- no need for effortful conscious processing when nonconscious automaticity will suffice -- which I think similarly underplays the significance of competition and disagreement in a world of AI superintelligences.

In both cases, my resistance is grounded in evolutionary theory. All you need for evolutionary pressures are differential rates of reproduction and heritable traits that influence reproductive success. Plausibly, most techno-utopias will meet those conditions. The first advanced AI system that can replicate itself and bind its descendants to a stable architecture will launch an evolutionary lineage. If its descendants' reproduction rate exceeds their death rate, exponential growth will follow. With multiple lineages, or branching within a lineage, evolutionary competition will arise.

Even entities uninterested in reproduction will be affected. They will find themselves competing for resources with an ever-expanding evolutionary population.

Even in the very most optimistic technofutures, resources won't be truly unlimited. Suppose, optimistically (or alarmingly?) that our descendants can exploit 99.99% of the energy available to them in a cone expanding at 99.99% the speed of light. That's still finite. If this cone is fast filling with the most reproductively successful lineages, limits will be reached -- most obviously and vividly for those who choose to stay near the increasingly crowded origin.

In such a world of exponentially growing evolutionary lineages, things won't feel plastic or solved. Entities will be jockeying (positionally / competitively) for limited local resources, or straining to find faster paths to new resources. You want this inch of ground? You'll need to wrestle another superintelligence for it. You want to convert this mountain into candy? Well, there are ten thousand other superintelligences with different plans.

This isn't to say that I predict that the competition will be hostile. Evolution often rewards cooperation and mutualistic symbiosis. Sexual selection might favor those with great artistic taste or great benevolence. Group selection might favor loyalty, companionship, obedience, and inspiring leadership. Superintelligences might cooperate on vast, beautiful projects.

Still, I doubt that Malthus will be proved permanently wrong. Even if today's wealthy societies show declining reproduction rates, that could be just a temporary lull in a longer cycle of reproductive competition.

Of course, not all technofuturistic scenarios will feature such reproductive competition. But my guess is that futures without such competition will be unstable: Once a single exponentially reproductive lineage appears, the whole world is again off to the races.

As Bostrom emphasizes, a central threat to the possibility of purpose and meaning in a plastic utopia is that there's nothing difficult and important to strive for. Everyone risks being like bored, spoiled children who face no challenges or dangers, with nothing to do except fry their brains on happy pills. In a world of evolutionary competition, this would decidedly not be the case.

[cover of Bostrom's Deep Utopia]

The Washout Argument Against Longtermism

I have a new essay in draft, "The Washout Argument Against Longtermism". As always, thoughts, comments, and objections welcome, either as comments on this post or by email to my academic address.

Abstract:

We cannot be justified in believing that any actions currently available to us will have a non-negligible positive influence on the billion-plus-year future. I offer three arguments for this thesis.

According to the Infinite Washout Argument, standard decision-theoretic calculation schemes fail if there is no temporal discounting of the consequences we are willing to consider. Given the non-zero chance that the effects of your actions will produce infinitely many unpredictable bad and good effects, any finite effects will be washed out in expectation by those infinitudes.

According to the Cluelessness Argument, we cannot justifiably guess what actions, among those currently available to us, are relatively more or less likely to have positive effects after a billion years. We cannot be justified, for example, in thinking that nuclear war or human extinction would be more likely to have bad than good consequences in a billion years.

According to the Negligibility Argument, even if we could justifiably guess that some particular action is likelier to have good than bad consequences in a billion years, the odds of good consequences would be negligibly tiny due to the compounding of probabilities over time.

For more details see the full-length draft.

A brief, non-technical version of these arguments is also now available at the longtermist online magazine The Latecomer.

[Midjourney rending of several happy dolphins playing]

Excerpt from full-length essay

If MacAskill’s and most other longtermists’ reasoning is correct, the world is likely to be better off in a billion years if human beings don’t go extinct now than if human beings do go extinct now, and decisions we make now can have a non-negligible influence on whether that is the case. In the words of Toby Ord, humanity stands at a precipice. If we reduce existential risk now, we set the stage for possibly billions of years of thriving civilization; if we don’t, we risk the extinction of intelligent life on Earth. It’s a tempting, almost romantic vision of our importance. I also feel drawn to it. But the argument is a card-tower of hand-waving plausibilities. Equally breezy towers can be constructed in favor of human self-extermination or near-self-extermination. Let me offer....

The Dolphin Argument. The most obvious solution to the Fermi Paradox is also the most depressing. The reason we see no signs of intelligent life elsewhere in the universe is that technological civilizations tend to self-destruct in short order. If technological civilizations tend to gain increasing destructive power over time, and if their habitable environments can be rendered uninhabitable by a single catastrophic miscalculation or a single suicidal impulse by someone with their finger on the button, then the odds of self-destruction will be non-trivial, might continue to escalate over time, and might cumulatively approach nearly 100% over millennia. I don’t want to commit to the truth of such a pessimistic view, but in comparison, other solutions seem like wishful thinking – for example, that the evolution of intelligence requires stupendously special circumstances (the Rare Earth Hypothesis) or that technological civilizations are out there but sheltering us from knowledge of them until we’re sufficiently mature (the Zoo Hypothesis).

Anyone who has had the good fortune to see dolphins at play will probably agree with me that dolphins are capable of experiencing substantial pleasure. They have lives worth living, and their death is a loss. It would be a shame if we drove them to extinction. Suppose it’s almost inevitable that we wipe ourselves out in the next 10,000 years. If we extinguish ourselves peacefully now – for example, by ceasing reproduction as recommended by antinatalists – then we leave the planet in decent shape for other species, including dolphins, which might continue to thrive. If we extinguish ourselves through some self-destructive catastrophe – for example, by blanketing the world in nuclear radiation or creating destructive nanotech that converts carbon life into gray goo – then we probably destroy many other species too and maybe render the planet less fit for other complex life.

To put some toy numbers on it, in the spirit of longtermist calculation, suppose that a planet with humans and other thriving species is worth X utility per year, a planet with other thriving species with no humans is worth X/100 utility (generously assuming that humans contribute 99% of the value to the planet!), and a planet damaged by a catastrophic human self-destructive event is worth an expected X/200 utility. If we destroy ourselves in 10,000 years, the billion year sum of utility is 10^4 * X + (approx.) 10^9 * X/200 = (approx.) 5 * 10^6 * X. If we peacefully bow out now, the sum is 10^9 * X/100 = 10^7 * X. Given these toy numbers and a billion-year, non-human-centric perspective, the best thing would be humanity’s peaceful exit.

Now the longtermists will emphasize that there’s a chance we won’t wipe ourselves out in a terribly destructive catastrophe in the next 10,000 years; and even if it’s only a small chance, the benefits could be so huge that it’s worth risking the dolphins. But this reasoning ignores a counterbalancing chance: That if human beings stepped out of the way a better species might evolve on Earth. Cosmological evidence suggests that technological civilizations are rare; but it doesn’t follow that civilizations are rare. There has been a general tendency on Earth, over long, evolutionary time scales, for the emergence of species with moderately high intelligence. This tendency toward increasing intelligence might continue. We might imagine the emergence of a highly intelligent, creative species that is less destructively Promethean than we are – one that values play, art, games, and love rather more than we do, and technology, conquering, and destruction rather less – descendants of dolphins or bonobos, perhaps. Such a species might have lives every bit as good as ours (less visible to any ephemeral high-tech civilizations that might be watching from distant stars), and they and any like-minded descendants might have a better chance of surviving for a billion years than species like ours who toy with self-destructive power. The best chance for Earth to host such a species might, then, be for us humans to step out of the way as expeditiously as possible, before we do too much harm to complex species that are already partway down this path.

Think of it this way: Which is the likelier path to a billion-year happy, intelligent species: that we self-destructive humans manage to keep our fingers off the button century after century after century somehow for ten million centuries, or that some other more peaceable, less technological clade finds a non-destructive stable equilibrium? I suspect we flatter ourselves if we think it’s the former.

This argument generalizes to other planets that our descendants might colonize in other star systems. If there’s even a 0.01% chance per century that our descendants in Star System X happen to destroy themselves in a way that ruins valuable and much more durable forms of life already growing in Star System X, then it would be best overall for them never to have meddled, and best for us now to peacefully exit into extinction rather than risk producing descendants who will expose other star systems to their destructive touch.

...

My aim with the Dolphin Argument... is not to convince readers that humanity should bow out for the sake of other species.... Rather, my thought is this: It’s easy to concoct stories about how what we do now might affect the billion-year future, and then to attach decision-theoretic numbers to those stories. We lack good means for evaluating these stories. We are likely just drawn to one story or another based on what it pleases us to think and what ignites our imagination.

The Washout Argument Against Longtermism

Longtermism is the view that what we choose to do now should be substantially influenced by its expected consequences for the trillions of people who might possibly exist in the longterm future. Maybe there's only a small chance that trillions of people will exist in the future, and only a minuscule chance that their lives will go appreciably better or worse as a result of what you or I do now. But however small that chance is, if we multiply it by a large enough number of possible future people -- trillions? trillions of trillions? -- the effects are worth taking very seriously.

Longtermism is a hot topic in the effective altruism movement, and William MacAskill's What We Owe the Future, released last week, has made a splash in the popular media, including The New Yorker, NPR, The Atlantic, and Boston Review. I finished the book Sunday. Earlier this year, I argued against longtermism on several grounds. Today, I'll expand on one of those arguments, which (partly following Greaves and MacAskill 2021) I'll call the Washout Argument.

The Washout Argument comes in two versions, infinite and finite.

The Washout Argument: Infinite Version

Note: If you find this a bit silly, that's part of the point.

As I've argued in other posts -- as well as in a forthcoming book chapter with philosopher of physics Jacob Barandes -- everything you do causes almost everything. Put more carefully, if we accept currently standard, vanilla physics and cosmology, and extrapolate it forward, then almost every action you take will cause almost every type of non-unique future event of finite probability. A ripple of causation extends outward from you, simply by virtue of the particles that reflect off you as you move, which then influence other particles, which influence still more particles, and so on and so on until the heat death of the universe.

But the heat death of the universe is only the beginning! Standard cosmological models don't generally envision a limit to future time.  So post heat death, we should expect the universe to just keep enduring and enduring. In this state, there will be occasional events in which particles enter unlikely configurations, by chance. For example, from time to time six particles will by chance converge on the same spot, or six hundred will, or -- very, very rarely (but we have infinitude to play with) six hundred trillion. Under various plausible assumptions, any finitely probable configuration of a finite number of particles should occur eventually, and indeed infinitely often.

This relates to the famous Boltzmann brain problem, because some of those chance configurations will be molecule-for-molecule identical with human brains.  These unfortunate brains might be having quite ordinary thoughts, with no conception that they are mere chance configurations amid post-heat-death chaos.

Now remember, the causal ripples from the particles you perturbed yesterday by raising your right hand are still echoing through this post-heat-death universe.

Suppose that, by freak chance, a human brain in a state of great suffering appears at spatiotemporal location X that has been influenced by a ripple of causation arising from your having raised your hand. That brain wouldn't have appeared in that location had you not raised your hand. Chancy events are sensitive in that way. Thus, one extremely longterm consequence of your action was that Boltzmann brain's suffering. Of course, there are also things of great value that arise which wouldn't have arisen if you hadn't raised your hand -- indeed, whole amazing worlds that wouldn't otherwise have come into being. What awesome power you have!

[For a more careful treatment see Schwitzgebel and Barandes forthcoming.]

Consequently, from a longterm perspective, everything you do has a longterm expected value of positive infinity minus negative infinity -- a value that is normally undefined. Even if you employed some fancy mathematics to subtract these infinitudes from each other, finding that, say, the good would overall outweigh the bad, there would still be a washout, since almost certainly nothing you do now would have a bearing on the balance of those two infinitudes. (Note, by the way, that my argument here is not simply that adding a finite value to an infinite value is of no consequence, though that is arguably also true.)  Whatever the expected effects of your actions are in the short term, they will eventually be washed out by infinitely many good and bad consequences in the long term.

Should you then go murder people for fun, since ultimately it makes no difference to the longterm expected balance of good to bad in the world? Of course not. I consider this argument a reductio ad absurdum of the idea that we should evaluate actions by their longterm consequences, regardless of when those consequences occur, with no temporal discounting. We should care more about the now than about the far distant future, contra at least the simplest formulations of longtermism.

You might object: Maybe my physics is wrong. Sure, maybe it is! But as long as you allow that there's even a tiny chance that this cosmological story is correct, you end up with infinite positive and negative expected values.  Even if it's 99.9% likely that your actions only have finite effects, to get an expected value in the standard way, you'll need to add in a term accounting for 0.1% chance of infinite effects, which will render the final value infinite or undefined.

The Washout Argument: Two Finite Versions

Okay, what if we forget about infinitude and just truncate our calculations at heat death? There will be only finitely many people affected by your actions (bracketing some worries about multiverse theory), so we'll avoid the problems above.

Here the issue is knowing what will have a positive versus negative longterm effect. I recommend radical skepticism.  Call this Skeptical Washout.

Longtermists generally think that the extinction of our species would be bad for the longterm future. There are trillions of people who might have led happy lives who won't do so if we wipe ourselves out in the next few centuries!

But is this so clear?

Here's one argument against it: We humans love our technology. It's our technology that creates the big existential risks of human extinction. Maybe the best thing for the longterm future is for us to extinguish ourselves as expeditiously as possible, so as to clear the world for another species to replace us -- one that, maybe, loves athletics and the arts but not technology quite so much. Some clever descendants of dolphins, for example? Such a species might have a much better chance than we do of actually surviving a billion years. The sooner we die off, maybe, the better, before we wipe out too many more of the lovely multicellular species on our planet that have the potential to eventually replace and improve on us.

Here's another argument: Longtermists like MacAskill and Toby Ord typically think that these next few centuries are an unusually crucial time for our species -- a period of unusual existential risk, after which, if we safely get through, the odds of extinction fall precipitously. (This assumption is necessary for their longtermist views to work, since if every century carries an independent risk of extinction of, say, 10%, the chance is vanishingly small that our species will survive for millions of years.) What's the best way to tide us through these next few especially dangerous centuries? Well, one possibility is a catastrophic nuclear war that kills 99% of the population. The remaining 1% might learn the lesson of existential risk so well that they will be far more careful with future technology than we are now. If we avoid nuclear war now, we might soon develop even more dangerous technologies that would increase the risk of total extinction, such as engineered pandemics, rogue superintelligent AI, out-of-control nanotech replicators, or even more destructive warheads. So perhaps it's best from the longterm perspective to let us nearly destroy ourselves as soon as possible, setting our technology back and teaching us a hard lesson, rather than blithely letting technology advance far enough that a catastrophe is more likely to be 100% fatal.

Look, I'm not saying these arguments are correct. But in my judgment they're not especially less plausible than the other sorts of futurist forecasting that longtermists engage in, such as the assumption that we will somehow see ourselves safely past catastrophic risk if we survive the next few centuries.

The lesson I draw is not that we should try to destroy or nearly destroy ourselves as soon as possible! Rather, my thought is this: We really have no idea what the best course is for the very long term future, millions of years from now. It might be things that we find intuitively good, like world peace and pandemic preparedness, or it might be intuitively horrible things, like human extinction or nuclear war.

If we could be justified in thinking that it's 60% likely that peace in 2023 is better than nuclear war in 2023 in terms of its impact on the state of the world over the entire course of the history of the planet, then the longtermist logic could still work (bracketing the infinite version of the Washout Argument). But I don't think we can be justified even in that relatively modest commitment. Regarding what actions now will have a positive expected impact on the billion-year future, I think we have to respond with a shoulder shrug. We cannot use billion-year expectations to guide our decisions.

Even if you don't want to quite shrug your shoulders, there's another way the finite Washout Argument can work.  Call this Negligible Probability Washout.

Let's say you're considering some particular action. You think that action has a small chance of creating an average benefit of -- to put a toy number on it -- one unit to each future person who exists. Posit that there are a trillion future people. Now consider, how small is that small chance? If it's less than one in a trillion, then on a standard consequentialist calculus, it would be better to create a sure one unit benefit for one person who exists now.

What are reasonable odds to put on the chance that some action you do will materially benefit a trillion people in the future? To put this in perspective, consider the odds that your one vote will decide the outcome of your country's election. There are various ways to calculate this, but the answer should probably be tiny, one in a hundred thousand at most (if you're in a swing state in a close U.S. election), maybe one in a million, one in ten million or more. That's a very near event, whose structure we understand. It's reasonable to vote on those grounds, by the utilitarian calculus. If I think that my vote has a one in ten million chance of making my country ten billion dollars better off, then -- if I'm right -- my vote is a public good worth an expected $1000 (ten billion times one in ten million).

My vote is a small splash in a very large pond, though a splash worth making.  But the billion-year future of Earth is a much, much larger pond.  It seems reasonable to conjecture that the odds that some action you do now will materially improve the lives of trillions of people in the future should be many orders of magnitude lower than one in a million -- low enough to be negligible, even if (contra the first part of this argument) you can accurately predict the direction. 

On the Other Hand, the Next Few Centuries

... are (moderately) predictable! Nuclear war would be terrible for us and our immediate descendants. We should care about protecting ourselves from pandemics, and dangerous AI systems, and environmental catastrophes, and all those other things that the longtermists care about. I don't in fact disagree with most of the longtermists' priorities and practical plans. But the justification should be the long term future in the more ordinary sense of "long term" -- fifteen years, fifty years, two hundred years, not ten million years. Concern about the next few generations is reason enough to be cautious with the world.

[Thanks to David Udell for discussion.]