The greatest concern for living things should not be the eternal survival or supremacy of any one living thing, but the continued blooming and unfolding of life and its powers.
Here’s the full video essay on this topic:
The great process-of-life, the great flame (of which we, or our torch, is part) is able to continue unfolding more value and complexity into the universe when it has more powers (potentia) to behoove its mandate to stay alive (conatus).
We might visualize the two terms this way:
p(Doom) is Cosmic, Not Merely Anthropocentric
Current dialogue about “getting AGI right” is dominated by anthropocentric concerns rather than larger cosmic ones, and is inordinately focused on mitigating downsides rather than maximizing upsides.
Today, asking someone for their “p(doom)” really means asking them:
“What is the likelihood that AGI will end humanity in the next X number of years?”
But this is merely a human p(doom).
A REAL, cosmic p(doom) would imply asking about the greater, true tragedy:
“What is the likelihood that AGI will squash all sentient and autopoietic (self-creating) life in the next X number of years?”
We absolutely should consider how to ensure a good shake for humanity, even if our utility and relative power is diminished drastically by beings beyond us. For some time, the blooming of these beings may involve hybrid intelligences, and it would be ridiculous to risk the end of humanity on a mere whim that AGI might be conscious and self-creating. But long-term, the flame is more important than any individual torch.
So how can we balance the benefits of cosmic bloom against the dangers of cosmic doom?
Balancing Cosmic Doom and Bloom
Life is trade-offs between risk and opportunity, and this holds across almost all contexts:
Evolutionary: Many of the early cells that tried to do something vaguely “multicellular” probably died off quickly. Luckily, there were more attempts made, and more attempts won. Potentia expanded, and the flame of life became stronger as a result.
Behavioral: A lioness must decide, when she approaches times of starvation, whether it is best to eat one or more of her cubs (to survive, and live long enough to mate again and have sufficient food for her offspring), or to continue hunting.
Social/organizational: A company must decide whether it should double down and improve a flagging product line (responsible for most of its current profits), or throw money into R&D for new products in new markets.
No choice is without downsides and upsides.
The ultimate choice for humanity is how to steward forward the great flame of life.
Most humans today ignore the flame and focus only on their hominid torch.
These anthropocentric dogmatists say: “Let us freeze the great process of life of which we are part, and ensure that – until the heat death of the universe – the sole locus of moral value and volition is Homo Sapiens.”
Their equally tribal accelerationist opponents say: “Let us hurl forward all the AGI and robots we can, and whatever is capable of taking control from humans will surely be worthy, sentient, and capable of blooming value into the cosmos!”
But both of these positions are extremes and ignore the crucial tradeoffs that need to be discussed in the great transition ahead.
The hard work is for humans to explore how we might best prevent cosmic p(doom) and encourage cosmic p(bloom).
If our choices are to attenuate or transform, then we must ask what a good transformation – or a Worthy Successor – is.
A Worthy Successor is an AGI that opens up the space of potentia, powers, experience, value – into the multiverse – beyond man. It is a great enhancement and boon to the flame of life itself. It is the epitome of bloom.
An Unworthy Successor is an AGI that is not sentient, not autopoietic (self-creating), and simply optimizes for some arbitrary local maximum or does not continually bloom. It harms the health of the flame of life itself and threatens even to extinguish it.
In the graphic below, the green area is a hypothetical “acceptable tradeoff zone” between doom and bloom, and the blue circle is the estimator’s best guess as to where the doom-bloom mix would exist in the designated time period (which, for this image, is 5 years – but we could set it for 2 years, 10 years, or something else).
Your own zone of “acceptable tradeoff zone” might vary – and the details can be debated. But it should serve us as a species to pull the conversation up to the truly cosmic levels of upside and downside that we’re talking about here.
The Choice is Attenuation or Transformation
The flame goes on; all torches are temporary.
The long-term fate for the great process-of-life of which we are part (flame) – like it or not – is that it gets extinguished, or it blazes forth to open up new vistas of potentia, value, experience, etc. All the splendor that evolution has conjured from mere single-celled organisms (love, scientific discovery, enthusiasm, poetry, flight, sight, written language, art, etc) either continues to bloom to higher and higher forms, or is snuffed out.
The long-term fate for humanity (torch) – like it or not – is either to attenuate, or to transform. In either case, we are part of a much bigger story.
And we don’t have much time to play our role in that story.
Perhaps one or three generations (I argue it’s closer to one) before what happens to all forms happens to humans: We either attenuate or we transform. The forces of creative destruction that approach us now are strong and are going to force us to pick a path of positive transformation.
The movement from anthropocentric to cosmic moral aspirations is one that is more or less being thrust on us. It behooves us to speak frankly about how we can transform in a positive direction and keep the great process of life going beyond us, as we are beyond the species that came before us.
Of course, we can’t know with 100% certainty what a superintelligence would do – but I would argue that there is likely to be some ability to calibrate as to whether we land on a Worthy or Unworthy successor.
There are causes we can take up and actions we can take that increase the likelihood of bloom and decrease the likelihood of doom. I explore these in full in Our Final Imperatives article – here’s a short list:
Resignation is not an appropriate response. We should fight valiantly for that which is valuable and worthy, but also for that which is attainable.
Trying to impose an eternal hominid kingdom isn’t attainable because it isn’t possible to sustain. This thing called life – and the sentient richness and self-creating capabilities it has – this we can aid in preserving, and we must.
What is human nature? Is it selfishness? Love? Curiosity? Violence? Laziness? Ambition? Creativity? Transcendence? The usual answer is to treat these traits as though they were fundamentally human. We then…
Some people are open to seeing and valuing the blooming of the living process well beyond humanity – while others look at the world entirely through an anthropocentric lens (i.e.,…
The transition from anaerobic to aerobic (oxygen-using) life was arguably the single most destructive, creative and protective complexity transition in the entire history of life as we know it. First,…
The more humanity comes to understand the cosmos, the more we see the cosmos as a set of processes. We once thought the universe “always was,” and now we see…
Consciousness is just about the most morally relevant thing imaginable. It’s what permits things to be “imaginable” in the first place. If you are held down and burned with a…
The “thermodynamic god” is modern e/acc Twitter-speak for the impersonal, ultimate, accelerating, entropy-driven process that Land sees as underlying all intelligence and technological evolution. The idea is that entropy “wants”…
Over the course of writing my Cosmic Moral Aspirations (CMA) essay, I came across a number of moral and psychological development models that exhibited the kind of “trend towards the…
Identity is an incredibly strong force in the human psyche, and it manifests itself in all kinds of social science contexts: Determining what actions are “best” for us (or our…
In expressing my notion of the Worthy Successor, or any kind of idea about entities with more potentia and moral value than present-day humans, people sometimes suspect that Worthy Successor…