Since the publication of the Worthy Successor essay in 2023, I’ve been glad to see the enthusiasm around exploring posthuman futures from people in tech and policy (especially on Twitter, but also at AI and AGI events).
But I’ve also been a little disappointed in what people (who haven’t read the article, but hear the term) think the term means. Often, people assume that Worthy Successor implies a mad, lawless, reckless race to build AGI just to get rid of humans with something vaguely more “powerful.”
You can’t possibly read the original article and think that that’s what I mean, but on Twitter, few people have time to read an article, which is why I try to communicate my ideas in small tables and graphics.
So I’ll do the same with what the Worthy Successor is and is not:
Here’s the full video essay version if you prefer watching to reading – and if you want a bit more nuance to some of the concepts in this short article:
To add some detail to the table shown above, with related links to essays that more clearly flesh out the points:
There is no predisposition to speedily racing to AGI in the Worthy Successor. There is a belief, instead, that such transitions to posthuman entities are likely to be inevitable, but that such transitions should be remarkably carefully approached to ensure that they (a) preserve and expand current value, and (b) are capable of opening up new realms of value (potentia). This almost certainly requires international coordination and governance of AGI-related tech. [Related resources: Potentia, The Business of Value Itself, Potestas]
There is no attempt to eternally define worthy now and forever. There is merely an emphasis on doing our best to define and optimize for it, and to then allow for an unraveling magazine of new (potentially higher, greater) values to emerge, which may self-evidently be beyond human conception, and beyond our fettered, hominid conceptions of what worthy means. [Related resources: Axiological Cosmism]
There is no bend against humanity. While the Worthy Successor idea implies that humanity will attenuate and may (ultimately: should) be superseded by higher-potentia entities, humanity both (a) is the most morally valuable kind of entity we have yet discovered in the universe and so is not to be leapt beyond recklessly, and (b) seems to have some degree of volitional control over how the next permutations of intelligence emerge into the universe. While the Worthy Successor view does hold that there are hypothetically more powerful and valuable entities than humans, there is no drive to move beyond humanity without deeply understanding and knowing that we can optimize for value (via cognitive upgrades, building silicon entities, etc) first. [Related resources: The Business of Value Itself]
There is no bent on “breaking up the happy home” of an eternally human world by hurling AGI into the mix just to spite humanity. Rather, the Worthy Successor view is that potentia seems to bubble into higher and higher forms continuously, and in the long term, humanity (like every other individual species) will have the fate totally attenuating, or turning into something else. Myriad forces (including our own motives and desires) are pulling us away from “default” unaugmented hominids already, and our role is to think seriously about what we’re turning into, and how we might avoid a collapse of earth-life in that critical process of transformation. [Related resources: Bend, not Pause, Baton]
“Worthy Successor” is just a pithy way of expressing the philosophical position of Axiological Cosmism:
Worthy = Axiological = “There is value now, and there are likely to be near-infinite magazines of new kinds of value unfolded by new potentia expanding into new realms of power and experience. The current value is worth preserving and optimizing for – but the future value deserves to be explored and opened up.”
Successor = Cosmism = “Long-term, human attenuation is imminent, and long-term, intelligent, potentia-expanding life will and should take forms wholly beyond human conception.”
Ultimately, the Worthy Successor is a zetetic exercise, an opportunity to accept that change is coming and all things are a process, and to ask ourselves how to carry forward this process of life beyond ourselves. It is about normalizing cosmic moral ambitions by discussing them openly and trying to discern what trajectories of the future make sense. No one person knows – I certainly don’t. But I call for us to talk about it squarely even if doing so makes us feel uncomfortable.
We have hunches and ideas, but don’t have the answers – we’re merely asking the question:
the worthy successor is not *my* definition of a "worthy" AGI
it is a call to remind man that:
1- we are soon going to be destroyed or transform into something
2- we should think ardently about WHAT we should turn into, what value is / how to maintain it in the cosmos
I should even note that my near-certainty in human attenuation is not necessary to all Worthy Successor dialogue. Some people in our community (including some of my very closest and smartest friends) believe there are ways humans-as-they-are could persist. Some think consciousness is the only value, while others think that other kinds of value might emerge.
Many points are debated – our commonality is merely that the great flame of life should go on. That’s the common thread among the core group of people who are attracted to this idea and believe it to be important. Otherwise, the zetetic exercise is what’s primary here. My “flavour” and takes are a subset of a dialogue that is much larger than just me.
This new installment of the Worthy Successor series features Ed Boyden, Y. Eva Tan Professor in Neurotechnology at MIT and a full member of the McGovern Institute for Brain Research….
This is an interview with Kristian Rönn, author, startup founder, and now CEO of Lucid, and AI hardware governance startup based in San Francisco. In this episode Kristian explores his…
This new installment of the Worthy Successor series is an interview with Joe Carlsmith, a senior advisor at Open Philanthropy, whose work spans AI alignment, moral uncertainty, and the philosophical…
This new installment of the Worthy Successor series is an interview with Robin Hanson – economist and author, Associate Professor of Economics at George Mason University, and a former Research…
This new installment of the Worthy Successor series is a conversation with Stephen Wolfram, founder and CEO of Wolfram Research, creator of Mathematica and the Wolfram Language, and a physicist…
This week, Anders Sandberg joins us on The Trajectory for episode 3 of the Worthy Successor series. Anders – famously with the Future of Humanity Institute at Oxford for nearly…
This week, Scott Aaronson joins us on The Trajectory for episode 4 of the Worthy Successor series. Scott is a theoretical computer scientist and Schlumberger Centennial Chair of Computer Science…
Before the iPhone, there was the Palm Pilot, and before OpenAI, there was (and still is) Numenta – both were founded by Jeff Hawkins. Jeff joins us on The Trajectory…
When it comes to cognitive architecture, philosophy, and AGI, few thinkers are as well-versed as Joscha Bach. Previously, Principal AI Engineer of Cognitive Computing at Intel, today he serves as…