Consolation Cosmism
We are about to be inundated with cosmic visions of the post-AGI future. This is mostly a good sign – humans are coming to see that AI won’t be merely…
I’m no “AI successionist,” and no one I know in the Trajectory Forum orbit is, either. AGI coordination and governance is a major part of our focus, and beyond that, carrying forward the living process is the singular objective of the group – and all of the our physical and virtual events.
The contrast is made clear by simply showing what both worldviews are advocating for:
AI Successionism says:
The purpose is to displace humanity with something capable of wrestling control away from humans. Such an AGI will be good because is displaces humans (the goal), and/or because of an unfounded faith that all uncontrollable AGIs will carry value and life into the cosmos.
Stewarding the Flame says:
Humanity faces a responsibility to shape the long-term trajectory of life and intelligence so that valuable forms of experience, agency, and flourishing are expanded and made more robust (in the same way that the current biosphere is vastly more robust and capable than the one Earth had 1 billion years ago). This should be done carefully and without rushing via an unbridled arms-race dynamic. This will almost certainly involve the transformation / augmentation of humanity, and the creation of new kinds of minds and agents. At some point humans-as-they-are are unlikely to be the main character of this process, and the expansion of the living process will likely take forms we cannot today imagine. (Read: Stewarding the Flame)
AI Succession has a number of biases directly at odds with Stewardship:
- AI Succession’s goal is displacement of humans. Either because it assumes all AGIs would be a boon to the cosmos (i.e. “All successors are automatically worthy”, which is the exact opposite perspective of the original Worthy Successor article), or because it believes the current human-led world is going to hell and we need to throw a hail mary for life to continue, or because they simply want to bring an end to humanity (overt misanthropy – a goal of destruction for its own sake).
- Stewardship‘s goal is the flourishing of the living process. That continued, self-transcending process can’t possibly be made to guarantee eternal leadership by homo sapiens, or even eternal involvement from homo sapiens, but it can involve our careful stewardship in the meantime, and the transforming of humanity to continue participating in the greater becoming of the living process.
- Stewardship‘s goal is the flourishing of the living process. That continued, self-transcending process can’t possibly be made to guarantee eternal leadership by homo sapiens, or even eternal involvement from homo sapiens, but it can involve our careful stewardship in the meantime, and the transforming of humanity to continue participating in the greater becoming of the living process.
- AI Successionism is compucentrist. Successionism mandates that compute / silicon be the substrate that houses the next wave of intelligences (or maybe even all future waves of intelligences). This is presumedly because silicon entities could travel at faster speeds and escape the bounds of earth quite easily, while bio-life cannot.
- Stewardship is an explicitly substrate-independent perspective, recognizing that there may be strengths to different substrates in different contexts, and recognizing that there are almost certainly substrates for housing the living process that are completely beyond human conception. In addition, Stewardship implies an overt skepticism about compute necessarily carrying the living process and being capable of consciousness or self-transcending evolution on its own. An entity powerful enough to kill off humanity but incapable of carrying forward a conscious (preferably blissful), self-transcending living process is the very definition of an unworthy successor. These distinctions are remarkably clear in the full Stewarding the Flame article.
- Stewardship is an explicitly substrate-independent perspective, recognizing that there may be strengths to different substrates in different contexts, and recognizing that there are almost certainly substrates for housing the living process that are completely beyond human conception. In addition, Stewardship implies an overt skepticism about compute necessarily carrying the living process and being capable of consciousness or self-transcending evolution on its own. An entity powerful enough to kill off humanity but incapable of carrying forward a conscious (preferably blissful), self-transcending living process is the very definition of an unworthy successor. These distinctions are remarkably clear in the full Stewarding the Flame article.
- AI Succession sees no continued role for humanity. The role of humanity in the eyes of a successionist is building AGI, and then letting the AGI take over – even if it means the immediate destruction of humanity. Because the goal is “succession” itself, humanity should – in the perspective of a successionist – actively allow themselves to be succeedeed.
- Stewardship implies an careful, active, gradual role for humanity – and the potential for extended relevance for augmented or radically transformed humanity even after regular humans may no longer be exerting much volitional influence over the trajectory of intelligence. Stewardship sees a role for a substrate-inclusive range of life processes that we build (AGI, xenobots, bio-engineered entities, etc), and life processes that we become (BCI, bioengineering, eventually mind uploading, etc). While there is no guarantee that any one temporary torch within the flame of life gets to stay the same or remain the “main character,” our expanded transformation and participation in the living process (for as long as we can) is a world apart from “successionism.”
Summarized in a simple graphic:
Any entity that immediately makes a hard break from biology and kills off humans is almost certainly not a worthy successor, and almost certainly is not stewarding forward the living process in a way that allows for the most robustness and self-transcending flourishing.
The successor is instrumental. The robust, self-transcending, thriving process-of-life (which undergirds all known kinds of value worth preserving, and likely all future kinds of value to be unfolded into the cosmos) is central. In What the Worthy Successor Is and Isn’t, I make it clear that it is a zetetic exercise to get us to think about the traits and qualities that we want to carry forward. It is an important though experiment within a larger goal of stewarding forward the living process / expanding potentia.
…
If you’re interested in meeting other people who are interested in stewarding forward the coming ecosystem of intelligence and life, get in touch via the Trajectory Forum.
