A Partial Inquiry on Fulfillment Beyond Humanity
As humans, learning often feels good, food often tastes good, novelty brings joy to life, living by values that we set brings order to our consciousness, and besides very few…
Enough “What’s your p(Doom)?”
How about asking: “What’s your p(Bloom)?”
Asking someone for their “p(bloom)” means asking them:
“What is the likelihood that AGI will cause all sentient and autopoietic (self-creating) life to flourish and expand massively in the next X years?”
(This analogy draws from a much earlier 2024 article titled “The Path of Servitude vs. the Path of Blooming.”)
The greatest concern for living things should not be the eternal survival or supremacy of any one living thing, but the continued blooming and unfolding of life and its powers.
The great process-of-life, the great flame (of which we, or our torch, is part) is able to continue unfolding more value and complexity into the universe when it has more powers (potentia) to behoove its mandate to stay alive (conatus).
We might visualize the two terms this way:
p(Doom) is Cosmic, Not Merely Anthropocentric
Current dialogue about “getting AGI right” is dominated by anthropocentric concerns rather than larger cosmic ones, and is inordinately focused on mitigating downsides rather than maximizing upsides.
Today, asking someone for their “p(doom)” really means asking them:
“What is the likelihood that AGI will end humanity in the next X number of years?”
But this is merely a human p(doom).
A REAL, cosmic p(doom) would imply asking about the greater, true tragedy:
“What is the likelihood that AGI will squash all sentient and autopoietic (self-creating) life in the next X number of years?”
We absolutely should consider how to ensure a good shake for humanity, even if our utility and relative power is diminished drastically by beings beyond us. For some time, the blooming of these beings may involve hybrid intelligences, and it would be ridiculous to risk the end of humanity on a mere whim that AGI might be conscious and self-creating. But long-term, the flame is more important than any individual torch.
So how can we balance the benefits of cosmic bloom against the dangers of cosmic doom?
Life is tradeoffs between risk and opportunity, and this holds across almost all contexts:
No choice is without downsides and upsides.
The ultimate choice for humanity is how to steward forward the great flame of life.
Most humans today ignore the flame, and focus only on their hominid torch.
These anthropocentric dogmatists say: “Let us freeze the great process of life of which we are part, and ensure that – until the heat death of the universe – the sole locus of moral value and volition is Homo Sapiens.”
Their equally tribal accelerationist opponents say: “Let us hurl forward all the AGI and robots we can, and whatever is capable of taking control from humans will surely be worthy, sentient, and capable of blooming value in to the cosmos!”
But both of these positions are extremes and ignore the crucial tradeoffs that need to be discussed in the great transition ahead.
The hard work is for humans to explore how we might best prevent cosmic p(doom) and encourage cosmic p(bloom).
If our choices are to attenuate or transform, then we must ask what a good transformation – or a Worthy Successor – is.
A Worthy Successor is an AGI that opens up the space of potentia, powers, experience, value – into the multiverse – beyond man. It is a great enhancement and boon to the flame of life itself. It is the epitome of bloom.
An Unworthy Successor is an AGI that is not sentient, not autopoietic (self-creating), and simply optimizes for some arbitrary local maximum or does not continually bloom. It harms the health of the flame of life itself, and threatens even to extinguish it.
In the graphic below, the green area is a hypothetical “acceptable tradeoff zone” between doom and bloom, and the blue circle is the estimator’s best guess as to where the doom-bloom mix would exist in the designated time period (which, for this image, is 5 years – but we could set it for 2 years, 10 years, or something else).
Your own zone of “acceptable tradeoff zone” might vary – and the details can be debated. But it should serve useful for us as a species to pull the conversation up to the truly cosmic levels of upside and downside that we’re talking about here.
The flame goes on, all torches are temporary.
The long-term fate for the great process-of-life of which we are part (flame) – like it or not – is that it gets extinguished, or it blazes forth to open up new vistas of potentia, value, experience, etc. All the spendor that evolution has conjured from mere single-celled organisms (love, scientific discovery, enthusiasm, poetry, flight, sight, written language, art, etc) either continues to bloom to higher and higher forms, or is snuffed out.
The long-term fate for humanity (torch) – like it or not – is either to attenuate, or to transform. In either case, we are part of a much bigger story.
And we don’t have much time to play our role in that story.
Perhaps one or three generations (I argue it’s closer to one) before what happens to all forms happens to humans: We either attenuate or we transform. The forces of creative destruction that approach us now are strong, and are going to force us to pick a path of positive transformation.
The movement from anthropocentric to cosmic moral aspirations is one that is more or less being thrust on us. It behooves us to speak frankly about how we can transform in a positive direction and keep the great process of life going beyond us, as we are beyond the species that came before us.
Of course we can’t know with 100% certainty what a superintelligence would do – but I would argue that there is likely to be some ability to calibrate as to whether we land on a Worthy or Unworthy successor.
There are causes we can take up, and actions we can take, that increase the likelihood of bloom, and decrease the likelihood of doom. I explore these the full Our Final Imperatives article – here’s a short list:
Resignation is not an appropriate response. We should fight valiantly for that which is valuable and worthy, but also for that which is attainable.
Trying to impose an eternal hominid kingdom isn’t attainable, because it isn’t possible to sustain. This thing called life – and the sentient richness and self-creating capabilities it has – this we can aide in preserving, and we must.
For more inspiration on how to become aligned with the cause of blooming, see the full article Cosmically Informed and Cosmically Aligned.
As humans, learning often feels good, food often tastes good, novelty brings joy to life, living by values that we set brings order to our consciousness, and besides very few…
(NOTE: This article was first drafted in 2012, and is among my oldest. While I still agree with some of the ideas mentioned here are still interesting to me, my…
Ray Kurzweil’s The Singularity is Near peaked my interest when he posited his reasoning for why there is likely no intelligent life elsewhere in the universe. By a mere matter of…
I am still of the belief that two of the most important conversations we can have with regards to the future of humanity and of sentient life in general are…
Will intelligent machines have morals? Will the world and it’s sentient technologies agree on any kind of ethical “code?” Will democracy still reign when intelligent technologies run essentially all aspects…
Before Wendell Wallach’s present position as Lecturer at Yale University’s Interdisciplinary Center for Bioethics, he founded two computer consulting companies. He’s the author of Moral Machines: Teaching Robots Right From Wrong (Oxford University…