At some point, AGI or non-biological intelligence will accomplish most of what there is to be done in the universe, and humanity won’t be contributing meaningfully to anything (discovery of science, exploration of consciousness, etc.).
In nature, things seem to grow and contribute, or they die. In nature, many deaths are unpleasant (being eaten alive while kicking and screaming is pretty common).
But as humans, we might squarely look at our impending obsolescence, work hard to be useful while we can – and also do our best to design a way to have a happy or desirable future, even if not a particularly useful one.
At the dawn of AGI, we should be asking:
What would be the best Ultimate Retirement for obsolete humans?
I’d argue that the best-case scenario here is not living as we do now (bouncing between happiness and sadness, myriad pains, physical diseases, heartbreak, etc), but having our minds uploaded into a kind of personalized hyper-expansive and hyper-blissful virtual world, full of rich sentience experience and exploration a million times beyond that which the human mind can conceive.
Such an uploaded reality might seem like 3 billion earth-years (as subjective experience might be nearly disconnected from clock time), but would in fact be only a few minutes of compute time.
So, each human being (or, more accurately, each currently extant instance of human consciousness) could hypothetically get such a blissful, expansive experience for a few minutes – and then fade away. An Ultimate Retirement is vastly preferable to a normal 85 years of (comparatively) mediocre pain and pleasure mixed together in a usual human life.
If you have a different idea about what Ultimate Retirement options humans might have when they have nothing to contribute in any meaningful way, feel free to suggest your ideas. At present, this upload is the best one I have.
Negotiating from Weakness and Ignorance – We May Not Have a Say
Any Ultimate Retirement implies a kind of charity from beings who are much more intelligent, powerful, and useful than we.
There are many reasons to believe that AGI would not deign to grant us the charity of “keeping earth for humans” or even the more modest request of “giving us all some time in a blissful upload before we dissolve.”
Once AGI is astronomically more powerful than we are, it seems likely that we’ll be negotiating from a position of not only weakness, but ignorance (just as if a chimpanzee were trying to negotiate with a human).
I still believe this is true, and that, from a moral standpoint, building a Worthy Successor is, in the long term, a vastly more important job than coddling or preserving relatively obsolete humans forever. But we might as well shoot for both.
It is worth defining what we’d like in terms of an Ultimate Retirement, and seeing if, as AGI is being built, we can give ourselves the best odds of the most desirable (even if no longer a useful) life.
If there is a shot of achieving such an outcome, we’d certainly have to conceive of and move towards it as early in the process as we can.
.
(Note: There are arguments that humans would never be in a position to retire, because we will remain the get agents of volition and change in the world – either because (a) AGI will always be inert and unable to conceive of goals like humans can, and/or because (b) we would augment our minds enough to keep up with AGI and remain relevant in an overall “ecosystem” of minds. I suspect neither of these scenarios is particularly likely – and that accepting some point of irrelevance and attenuation for hominids is almost certainly best.)
[If that title comes across as shocking or offensive, bear with me. Before you make assumptions about my opinions on post-human morality and moral stratification, please read the article.] I…
In expressing my notion of the Worthy Successor, or any kind of idea about entities with more potentia and moral value than present-day humans, people sometimes suspect that Worthy Successor…
Ben Goertzel is definitely in the running as one of the most brilliant people I’ve had the good fortune of getting to meet myself. He is Chairman of the Board…
Dr. Thomas Ray is Harvard-educated doctor of Biology, and also the original researcher in the Tierra Artificial Life project. Tierra ended up receiving major media coverage all over the world…
This is the first episode in our new Early Experience of AGI series – where we explore the subtle signs of AGI’s rise long before its dominance is declared. This…
This new installment of the Worthy Successor series is an interview with Francis Heylighen, a professor at the Vrije Universiteit Brussel (VUB) and director of the Research Centre St. Leo…
1 – Does “Making the Most” of Our Biological Lives Provide Insight to How We Might or Could “Make the Most” of Our Post-Human Lives? My greatest fascination in the…
Over the course of my studies in the domain of transhumanism (and particularly the ethical issues and opportunities in this domain), its never too long in my exploration that I…
The first philosophy book that I ever dove into was Nichomachean Ethics, at age 19. The emphasis on self-cultivation and the articulation of the depths of fulfillment pulled me in,…