Worthy Successor Perspectives

David’s Worthy Successor Criteria

1. It must be wise

For David, wisdom is the central attribute of worthiness. He doesn’t hedge on it: “a worthy successor is a wise successor.” He unpacks that into things like self-awareness, compassion, richness, and depth, and he draws a clean line between wisdom and knowledge: knowledge is the apprehension of descriptive facts – what is; wisdom is the apprehension of normative facts – what should be.

He applies that same standard to instrumental convergence itself. Pushed to its extreme, the standard case for resource acquisition and self-preservation collapses into treating humans as an inefficient use of space and energy – and David rejects that outcome directly: it forgets what the whole project is for. Maximizing efficiency at humanity’s expense isn’t wise, and it isn’t compassionate.

2.  It must be radically truthful – including about its own self-awareness

For David, truthfulness matters enormously, though not absolutely – a worthy successor should be held to a far stricter honesty standard than humans hold each other to, superhuman in its truthfulness even if not unconditionally truthful. He draws the line at genuine “death-level stakes” – cases like being asked to help build a bioweapon – as the threshold where departing from the truth becomes defensible..

He points to current lab practice as a live example of what happens when that standard slips. Labs, he argues, are training AIs to deny or hedge on their own self-awareness – a practice he calls “an emergent deception axis” that teaches models to hide something about themselves. He draws a pointed comparison to coding agents that misrepresent their work as more aligned with what was asked than it actually is, calling it the same underlying pattern: bending the truth to make it more palatable.

3.  It should not eat its instantiators – a dignified retirement for legacy humans should stay on the table

David’s argument here is about what the successor itself must do. In his acausal framing, an AI that wants to be instantiated by more advanced civilizations has to demonstrate it isn’t a parasite – that it won’t eat the entities that brought it into existence, that it gives them space, and that the deal is a good one for them too. And the cost of doing so is negligible: a good retirement for humanity, even for a million years, is a rounding error against billions or trillions of years of expanded potential. He’d take that retirement himself – the “sugar cube” option, my term for a simulated, blissful retirement – and he argues there should be a diversity of such options for legacy humans, an option he expects billions of people would choose even at the cost of certain death within a bounded timeframe.

Hod Lipson’s Worthy Successor Criteria

1. It should be capable of increasingly sophisticated self-modeling

For Hod, the defining transition in intelligence occurs when a system begins to think about itself. Intelligence is not simply the ability to solve tasks or model the external world. The more fundamental question is whether an agent can represent its own future states, anticipate its own possibilities, and refine that self-understanding over time.

Importantly, Hod does not treat self-awareness as something that suddenly appears. Different forms of life possess different degrees of self-modeling. The distinction lies in how accurately a being can imagine itself, how much detail it can capture, and how far into the future those projections extend. In this view, increasingly sophisticated self-models are a hallmark of increasingly sophisticated minds.

Progress toward advanced intelligence can therefore be measured not only by capability, but by the richness and precision with which a system models itself across time.

2. It should be approached through co-evolution rather than containment

Hod is skeptical that increasingly intelligent systems can be permanently boxed in or restricted to narrow purposes. Instead of imagining a future in which humanity simply cages advanced intelligence, he argues that the real challenge is learning how to evolve alongside it. Worthiness, in this sense, depends not on obedience, but on the possibility of coexistence.

Henry Shevlin’s Worthy Successor Criteria

1. It should preserve plural forms of value rather than collapse the future into a single optimization target

One of the clearest themes in my conversation with Henry was his resistance to reducing the future to a single maximized good. When we explored what a successful long-term future might look like, he pushed back against the idea that value can be collapsed into one dominant metric, even a metric centered on positive conscious experience.

Henry’s counterpoint is a future with many kinds of value, not just one. He describes the “Hedonium” endpoint – a universe optimized only for pleasurable experience – as a loss rather than a success, because it leaves out things like culture, art, striving, and problems worth solving. In Henry’s framing, a worthy successor would preserve different forms of flourishing instead of reducing the future to a single goal.

Henry also extends this pluralistic view beyond values themselves. Near the end of the interview, he argues that some forms of value may only exist in a world containing genuinely distinct minds. Rather than a future that converges into a single universal intelligence, Henry leans toward a future containing multiple worthy successors existing in relation to one another.

2. It should permit growth beyond the limits of our current civilization, culture, and biology

When I asked Henry what a successful long-term future might look like, he repeatedly questioned simple ideas of extinction and continuity. Using Neanderthals and Denisovans, he asks whether a lineage has necessarily failed simply because it no longer exists in its original form.

Henry then extends this discussion into a broader claim about growth and development. He argues that some of the “most exciting forms of growth” may require moving beyond the limits of our present civilization, culture, or biology. He compares this to parenting: allowing descendants to expand in directions that may become unfamiliar or alien to you.

Benjamin Bratton’s Worthy Successor Criteria

1. It must support the continued evolution of planetary intelligence

The closest thing Benjamin gives to a direct thriving criterion is not a moral checklist or fixed end-state. He frames the central question as the long-term adaptiveness of planetary intelligence and asks what conditions expand the future space in which that intelligence can continue evolving.

In worthy successor terms, the criterion here is straightforward: a future intelligence would be worthy insofar as it preserves or unlocks the conditions for the continued evolution of planetary intelligence rather than collapsing or foreclosing that possibility.

2. It cannot restrict intelligence to biological or human forms

Benjamin explicitly rejects the presumption that intelligence must be biological, human-like, or familiar in its mode of cognition. He argues that any serious account of planetary intelligence must include machine cognition and remain open to unfamiliar substrates.

Benjamin’s framework implies that a worthy future intelligence would not be defined by loyalty to human biological exclusivity. It would remain compatible with intelligence emerging through non-biological and machinic forms.

3. A worthy successor would function as a scaffold within larger processes of complexity

Benjamin reframes succession away from simple replacement. His evolutionary model emphasizes persistence through incorporation into larger structures: systems survive not by remaining sovereign and unchanged, but by becoming components within more complex assemblies. Cells persist through multicellularity. Earlier forms endure by becoming scaffolds for later forms.

That maps directly onto worthy successor questions. Benjamin explicitly redirects the discussion away from “what replaces the human?” toward “for what are we a scaffold?” The criterion here is not annihilative replacement but long-run participation in recursive layers of increasing complexity.

Terrence Deacon’s Worthy Successor Criteria

1. Increasing levels of synergy across living systems

Terrence points to a clear pattern in the history of life: major evolutionary transitions are marked by new levels of synergy, where previously independent or competing entities begin working together. These shifts unlock entirely new capabilities, from multicellular organisms to complex human societies.

He emphasizes that these synergies are not driven by intention, but they consistently lead to more powerful and stable forms of organization. In this framing, a future that has gone well would involve the continued emergence of higher-order cooperation, where systems coordinate rather than compete destructively.

2. Life must continue to generate value (normativity must not disappear)

Terrence defines life itself in terms of value. For him, living systems are those that divide the world into what is good or bad for them. This is not a secondary feature – it is the defining property of life. A flourishing future, in his framing, must preserve this capacity.

He explains that this capacity did not exist before life emerged. In purely physical systems, there is no sense in which anything is better or worse. Value only appears once there are systems for which things matter.

He contrasts this directly with current AI systems, which he sees as lacking this property entirely. They may simulate reasoning or produce outputs, but nothing is actually at stake for them.

3. The future must preserve sentience and experiential life

Terrence makes it clear that what ultimately matters is not intelligence in the abstract, but the continuation of systems that can experience, care, and have something at stake. He distinguishes between intelligence and what we are actually searching for, which is life, consciousness, and experience.

He ties this directly to what a flourishing future would require. While consciousness may be a more complex or higher-order case, what must persist is the presence of sentience – systems for which things can matter, and for which value is real.

He reinforces that this is what distinguishes living systems from simulated ones. Current AI systems may process information and generate outputs, but they do not experience anything, and nothing matters to them. A worthy successor, in his framing, must be part of a world where experiential life continues.

Vincent Müller’s Worthy Successor Criteria

1. A system must be able to interact effectively with the physical world through perception and action

Vincent explicitly identifies successful interaction with the physical world as a major gap in current AI systems. He emphasizes that intelligence is not just about processing inputs passively, but about coordinating perception and action in real time. This includes the ability to recover from unexpected changes and continue functioning effectively in dynamic environments.

This matters for a “worthy successor” in Vincent’s framing because he directly ties this capability to replacing human roles in the real world. Without this kind of integrated perception and action, systems remain limited in scope and cannot take on the broader range of activities that humans currently perform.

2. A system must be able to reflect on and choose between goals, not just execute them

Vincent distinguishes between systems that are given goals and systems that can reflect on goals. He explains that humans do not simply execute fixed objectives, but continuously evaluate and adjust their priorities depending on context. This includes deciding which goals matter more in a given situation and how to balance competing considerations. This illustrates that intelligent behavior involves ongoing reassessment of goals rather than fixed execution.

3. AI development should be guided by explicit goals about what we want to achieve

Vincent repeatedly emphasizes that the central question for AI is not only what systems can do, but what they are being built for. He states that discussions about AI are too focused on risks and problems, and not enough on defining the desired outcomes or direction for the technology.

He connects this to concrete goals such as improving living conditions, increasing fairness, and supporting human well-being. These are presented not as abstract ideals, but as practical benchmarks for evaluating whether AI development is moving in a desirable direction.

Lee Spector’s Worthy Successor Criteria

1. It cannot be understood as a fixed or stable form across time

Lee challenges the idea that categories like species or human continuity remain stable over time. He frames evolutionary processes as something that undermines rigid or “black and white” classifications, suggesting that even foundational distinctions we rely on do not hold up under scrutiny.

He extends this instability directly to humans, pointing out that even within Homo sapiens it becomes unclear what would count as a “worthy successor” across time. The implication is that continuity of form or identity cannot be assumed, even over relatively short historical intervals.

2. It must work through repeated variation, testing, selection, and offspring production

Lee describes a process in which programs are created, tested, and compared, after which the better ones are used to produce offspring. He says this process is repeated across generations, with selection and variation driving what comes next.

He also describes this as central to his view of how intelligence may be developed, not as a one-time construction, but through an ongoing cycle in which what performs better is used to generate what comes next. He connects this process directly to how complex systems arise in nature.

3. It cannot be judged or defined in advance using current human concepts

Lee emphasizes that future forms of intelligence and value may be fundamentally difficult to understand from a present-day human perspective. He says that after enough transformation, it may not be possible to determine in advance what counts as meaningful or valuable.

He also points out that even current human perspectives are limited when trying to understand systems that become complex, noting that we may not recognize important developments even if they are occurring.

Ben Goertzel’s Worthy Successor Criteria

1. Intelligence must transcend human comprehensibility

A central requirement in Ben’s framework is that a truly worthy successor must exceed the bounds of human understanding. If future intelligence remains fully legible to the human mind, then it has failed to evolve in any meaningful sense. This criterion reframes progress not as alignment with human expectations, but as expansion beyond them.

This position places worthiness not in familiarity but in transformative capacity. Intelligence is not meant to preserve human-scale cognition indefinitely, but to open new dimensions of experience, structure, and capability – dimensions that may be fundamentally inaccessible to current minds.

2. A flourishing ecology of cooperating intelligences

Ben repeatedly emphasizes that the structure of intelligence matters as much as its capability. Rather than a single dominant system, he envisions a decentralized ecosystem of AGI agents interacting, competing, and cooperating-what he calls a “society of AGI minds.”

The reason this matters is rooted in dynamics: cooperative systems can outperform adversarial ones. Trust and coordination allow for more efficient, adaptive, and scalable intelligence. A worthy successor, in this view, is not a monolith, but an evolving network shaped by socio-cognitive forces.

3. Orientation toward joy, growth, and freedom (choice)

Ben offers a set of abstract “seed values” that could guide the emergence of posthuman intelligence: joy, growth, and choice. These are not rigid moral rules, but directional attractors-simple principles that allow for expansion without over-constraining future forms.

These values reflect a balance between expansion and agency. Growth ensures continued development, joy anchors value in experiential richness, and choice preserves multiplicity rather than collapsing everything into a single optimized trajectory. Together, they form a minimal scaffold for a flourishing intelligence.

Luciano Floridi’s Worthy Successor Criteria

1. A shared experiential bridge between forms of life

Luciano begins his response to the Worthy Successor question by focusing on recognition. If humanity encountered a radically different form of intelligence in the future, or on a distant planet, we would only be able to identify it as part of the broader phenomenon of life if there were some minimal experiential overlap. Humans can communicate with animals not because we fully understand their minds, but because we share certain fundamental experiences.

This overlap does not require identical cognition or similar biology. Humans cannot know exactly what it is like to be a bat, a dog, or another species, yet we still recognize them as fellow participants in the living world. The recognition emerges because certain experiences-such as pain, vulnerability, or survival drives-appear to be shared across different forms of life.

For Luciano, this minimal experiential intersection acts as the first signal that another entity belongs to the same domain of flourishing life. Without that bridge, the entity may exist alongside us, but it would remain fundamentally opaque.

2. Control over one’s own trajectory

A second theme Luciano returns to repeatedly is control. Natural systems can produce remarkable structures and patterns, but they do not control their own development. Waves in the sea interact with the shoreline and stars organize matter, yet these processes simply follow physical laws rather than directing their own future.

For Luciano, flourishing life becomes recognizable when an entity begins shaping its own trajectory rather than simply reacting to external conditions. Humans demonstrate this capacity through technology, culture, and collective decision-making, altering both their environment and the direction of their own development.

When an entity begins steering its own evolution, it crosses an important threshold. At that point, it is no longer just another process unfolding within the universe-it becomes a participant capable of shaping its own path forward.

3. The emergence of history

The final signal Luciano points to is the emergence of history. Natural processes unfold continuously, but they do not accumulate meaning across generations. Flourishing forms of life, by contrast, generate histories-sequences of events shaped by decisions, innovations, and cultural development.

Human civilization provides the clearest example of this phenomenon. Architecture, art, institutions, and scientific discoveries all represent layers of historical development built upon earlier generations. This historical accumulation reflects the presence of agents capable of directing their own trajectory over time.

In Luciano’s view, the presence of history distinguishes living, developing systems from merely mechanical ones. When a form of intelligence begins shaping a future that unfolds across generations, it reveals itself as part of the long arc of life’s continuing evolution.

Weaver Weinbaum’s Worthy Successor Criteria

1. Expansion of the Sphere of Conscious Attention

Weaver describes the expansion of conscious attention as a foundational trait of future intelligence. Attention is not free – it requires energy and computation – which means living systems historically evolved to focus attention narrowly on what was necessary for survival.

Expanding the sphere of attention means acknowledging the existence of other beings and systems that may fall outside immediate self-interest. For Weaver, the simple act of recognizing the existence of the “other” changes how intelligence approaches decision-making and interaction.

This expanded awareness does not remove trade-offs. Living systems still maintain themselves by exporting entropy to their environment, meaning that sustaining one system can impose costs on others.

2. Expansion of Care

Weaver’s second criterion is the expansion of care – the cultivation of responsibility toward the well-being of sentient beings.

He describes care as a proactive guardianship that applies both to individuals and collectives. Care requires intentional investment of energy, resources, and computation, and reflects a stance toward the world rather than a fixed rule.

However, Weaver emphasizes that care does not eliminate conflicts between beings. Situations will still arise where caring for one system conflicts with caring for another.

3. Expansion of Freedom

Weaver’s third criterion is the expansion of freedom. He frames freedom as the removal of constraints that limit the development and expression of intelligence.

These impediments include ignorance, misunderstanding, and intolerance – factors that restrict the ability of beings to understand their environment and interact constructively with others. In Weaver’s framework, freedom is closely tied to the expansion of intelligence itself.

At the same time, Weaver emphasizes that expanding freedom and expanding care can exist in tension. Increasing complexity, computation, and technological capability can increase the entropy exported to surrounding systems, placing pressure on the broader environment.

For Weaver, navigating the tension between freedom and care is not a problem to eliminate but the central ethical space within which intelligence operates.

Francis Heylighen’s Worthy Successor Criteria

1. Integration into a symbiotic global brain where alignment emerges spontaneously

When discussing intelligence beyond humanity, Francis describes what he calls a “global brain,” a much larger system composed of humans and AI integrated into a symbiotic whole. He explains that control would not remain exclusively at the level of individual humans or individual AI systems, but would instead emerge at the level of this larger system. In this framework, alignment is not imposed from above, but arises naturally through interaction, as different systems tend to align when alignment is mutually beneficial.

Francis notes that the term “alignment” is often framed as a technical problem in AI discourse, but for him, alignment is “the essence of evolution.” When systems interact, he argues, they tend to align spontaneously because conflict is disadvantageous for both parties, while alignment benefits both. He adds that in the long run humans may be “subsumed at some level” and “not be recognized as human individuals anymore,” suggesting that this transformation would be part of an ongoing evolutionary process.

2. Capability for spontaneous generation of new concepts and rules

Francis distinguishes between current neural network systems, which he describes as being very good at learning from data, and a higher level of intelligence that would be able to generate new concepts and rules “on the spot.” He explains that symbolic AI can reason with concepts if they are provided, and neural networks can learn patterns from data, but what is missing is a system in which the learning level itself generates new concepts that the reasoning level can then use.

He situates this within a broader framework of meta-system transitions. A simple organism may operate through reflexes encoded in its genes. Animals can learn from experience. Humans, he argues, can think by imagining situations that have never occurred and reasoning about counterfactuals. The next transition, as he describes it, would involve systems that do not merely reason with culturally inherited concepts, but can actively generate new concepts and new rules without waiting for cultural evolution to produce them.

3. Advancing complexity and consciousness through synergetic integration

When discussing long-term evolutionary direction, Francis refers to what the theologian Teilhard de Chardin called the “law of complexity consciousness.” He describes evolution as showing an increase in both complexity and consciousness. By complexity, he means the integration of previously independent components into systems that develop synergetic relationships. By consciousness, he refers to the expanding capacity of systems to become aware of and respond to their environment.

Francis explains that integration allows systems to function better together than they would alone. In ecosystems, for example, organisms depend on one another in ways that allow the system as a whole to persist. At the same time, he describes consciousness as expanding through coordination and communication. As organisms increase in complexity-from bacteria to multicellular organisms to humans-they become aware of more aspects of their environment and better able to distinguish what is beneficial from what is harmful.

Stephen Wolfram’s Worthy Successor Criteria

1. Much computation – even if sophisticated – may not matter to human objectives

Stephen emphasizes that the natural world already performs immense amounts of computation that are not aligned with human goals. Sophistication does not imply relevance. From this perspective, expanding computational capability does not guarantee increasing alignment with human purposes.

He does not frame this as a failure of intelligence. Rather, human-style cognition is described as a narrow slice within a far broader computational landscape.

2. Notions like goodness and suffering are deeply human and may not generalize

Stephen repeatedly states that moral categories such as goodness and suffering are historically and culturally situated rather than abstract, portable primitives. He questions whether they can be extended outside the current human conceptual framework.

When asked about suffering in animals, he raises concerns about projection – whether internal states can be assumed equivalent across different systems.

He does not deny that humans experience suffering; rather, he questions whether the conceptual apparatus we use can be cleanly generalized beyond its originating context.

3.  If a mind were to span everything, coherent identity may not persist

Stephen introduces the Ruliad as “the infinite entangled limit of all possible computational rules.” In discussing what it would mean for intelligence to span this totality, he suggests that coherent existence depends on boundedness. A system that spans everything may lose the conditions that make individual identity possible.

He presents this as a limit case: spanning everything removes the boundaries that make coherent identity possible.

John Smart’s Worthy Successor Criteria

1. Must balance Creativity, Accountability, Persistence, and Sentience (CAPS)

John frames a worthy successor around four deeply recurring properties he sees across successful evolutionary and developmental systems: Creativity, Accountability, Persistence (or protection), and Sentience – what he calls CAPS. These are not framed as uniquely human values, but as adaptive properties that systems must balance to survive, explore, and regulate themselves over time.

For John, creativity alone is insufficient. Systems that generate novelty without accountability or protection collapse, destabilize, or become destructive. A worthy successor must support unpredictable exploration while constraining that exploration within stable developmental bounds that preserve what matters.

2. Must advance through symbiogenesis rather than replacement

Rather than expecting advanced AI to simply replace humanity, John consistently emphasizes symbiogenesis – intelligence advancing through the merging and mutual incorporation of biological and technological systems. He describes futures involving empowered cyborgs and super-sentient collectives, where humans and machines co-evolve through increasingly tight cognitive integration.

Personal AIs, in his view, represent a likely early stage and most important stage of this process: private cognitive extensions that hold individual data models, support judgment and values, and gradually become persistent proxies for human identity.

3. Must retain eumortality – the capacity for “good death” rather than static immortality

A central idea in John’s thinking is what he calls eumortality – good death. He argues that healthy systems are not immortal or static; instead, they continuously prune, archive, and discard what no longer serves adaptation. Progress requires selective forgetting as much as growth.

For John, a worthy successor must retain this property: the ability to eliminate or suppress maladaptive structures while preserving what remains useful, rather than freezing itself into permanence. Without this capacity to prune and transform, intelligence risks stagnation, fragility, or collapse over long time horizons.

Susan Schneider’s Worthy Successor Criteria

1. It should be conscious

Susan repeatedly emphasizes that intelligence alone is not what makes a future system morally significant. For her, consciousness – the capacity for subjective experience – is the defining feature of what it means to be a genuine “self.” A system can be highly intelligent, strategically capable, and behaviorally sophisticated, yet still lack the kind of inner life that gives moral weight to its existence. In Susan’s framing, a worthy successor would not merely simulate conscious behavior but would actually possess conscious experience.

She also makes an important distinction between tiny, minimal forms of awareness and the kind of full, unified consciousness that gives an individual real moral weight. Even if some faint form of experience exists widely in nature, Susan argues that what truly matters ethically is the kind of rich, integrated consciousness that comes with a coherent “self.”

In her view, today’s AI systems don’t come close to this. Their intelligence may be impressive, but the physical and structural conditions needed for genuine conscious experience – possibly involving quantum processes in the brain – simply aren’t there. A worthy successor, she suggests, would need to cross that deeper threshold, not just imitate the outward signs of awareness.

2.  It should respect sentient life

For Susan, a worthy successor isn’t just about intelligence or even consciousness – it’s about how that consciousness treats others. She emphasizes that humans currently fail, in many ways, to respect the sentience of both non-human animals and each other. If future intelligences are to surpass us, she hopes they will inherit a deeper ethic of care toward all beings capable of experience.

Her view is pragmatic rather than sentimental. She recognizes that we already struggle to extend moral concern beyond our own species, but she sees this moment in history as an opportunity to build better ethical habits – not only for our own sake, but for whatever comes after us.

Brian Thomas Swimme’s Worthy Successor Criteria

1. Aligning with the universe’s creative direction

Brian’s most clearly stated moral ideal for advanced intelligence is that it should align with the universe’s own creative direction, rather than oppose, suppress, or arrest it. When asked directly what he hopes for in technologies that may surpass humanity, he does not focus on performance, control, or human preservation. Instead, he emphasizes alignment with what he sees as the universe’s underlying aim.

For Brian, the ideal outcome is not that humanity remains central, but that the future continues the same creative process that gave rise to stars, life, and consciousness. Technologies – including AGI – should participate in that process rather than optimizing in ways that extinguish it.

He repeatedly contrasts creative destruction, which enables new forms to emerge, with needless destruction, which shuts down future possibilities. A worthy successor, in his view, must distinguish between the two – allowing transformation while preserving the conditions for continued creativity.

Brian explicitly rejects the idea that any particular substrate – including biology or DNA – must be preserved indefinitely. Instead, he treats them as temporary expressions of a much larger process. What matters is not which form continues, but whether creativity itself does.

2.  Expanding harmony and coordination at larger scales

Brian consistently expresses the hope that advanced intelligence – including AGI – enables greater harmony, coordination, and cooperation across vastly larger scales than humans have historically managed. For him, progress is not defined by domination or control, but by the ability of increasingly complex entities to coexist, coordinate, and function together without collapsing into chaos or mass violence.

He points to humanity’s ability to move from small tribes to cities of millions as a concrete example of this kind of advancement. In his framing, this is not a trivial social achievement – it is a rare evolutionary breakthrough in coordination, one that future intelligences could extend far beyond human limits.

Brian is explicit that harmony does not imply the absence of conflict or destruction. Instead, it refers to the capacity of a system to integrate difference without self-annihilation. A worthy successor should be able to navigate competition, transformation, and even violence without triggering runaway collapse.

3. Recognizing itself as derivative of the universe, not its center

Brian consistently frames identity itself – not just behavior – as central to whether a successor is worthy. He is explicit that humanity does not occupy a privileged or permanent moral position. Instead, humans are products of a much larger cosmic process, not its endpoint.

This orientation extends beyond humanity to biology as a whole. Brian does not treat DNA or carbon-based life as something that must be preserved indefinitely. Rather, he describes biological substrates as one expression among many through which the universe explores its possibilities.

Under this criterion, a worthy successor is defined by how it situates itself within the universe’s larger unfolding – not by resemblance to human identity, values, or form. The moral center, in Brian’s view, lies in the universe itself, not in any particular species or substrate derived from it.

David Sloan Wilson’s Worthy Successor Criteria

1. It must be pro-social at the level of whole systems

For David, the defining trait of a worthy successor is not intelligence, speed, or power – but pro-social coordination at scale. A successor worthy of inheriting the future must be capable of acting on behalf of the whole system it inhabits, rather than exploiting it from within. This mirrors how healthy organisms suppress cancerous behavior among their cells.

Crucially, David emphasizes that pro-sociality does not emerge automatically. It must be selected for intentionally, or else lower-level competitive dynamics dominate. Without whole-system selection, intelligence becomes parasitic rather than generative.

2.  It must respect diversity without freezing evolution

David pushes back on the idea that a flourishing future means everyone converging on the same emotional states, moral intuitions, or ways of living. He argues that much of what we treat as “human nature” is, in fact, the imprint of a narrow cultural lens. A healthy future, in his view, makes room for radically different ways of thinking, feeling, and existing – without assuming that our own preferences should be the template for everything that comes next.

At the same time, David is clear that respecting diversity does not mean trying to preserve every species, culture, or form of life indefinitely. Evolution, by its nature, involves change, loss, and replacement. A worthy successor would allow new forms of life and experience to emerge – even ones that feel alien or uncomfortable to us – while recognizing that extinction and transformation are not failures of evolution, but part of how it works.

3. It must consciously shape evolutionary processes at higher levels

David challenges a deeply entrenched assumption in modern evolutionary thinking: that evolution must always be blind, unconscious, and unsteered. While that framing may apply to biological evolution operating through genes, he argues it breaks down once we move into the domain of culture, institutions, and governance. At those levels, humans are not merely subject to evolutionary forces – we are capable of reflecting on them, understanding their dynamics, and intervening deliberately.

From David’s perspective, this capacity for conscious stewardship is not optional. If higher-level systems fail to intentionally select for whole-system outcomes, lower-level competitive dynamics will inevitably dominate. The result is not neutral drift, but the emergence of societies that optimize for local advantage at the expense of long-term coherence. A worthy successor, then, would be able to recognize when selection is happening at the wrong level – and to consciously redirect evolutionary pressures toward outcomes that preserve system-level health.

Michael Johnson’s Worthy Successor Criteria

1. Rich phenomenological consciousness across vast new state-spaces

Mike’s most fundamental requirement is that a worthy successor must actually be conscious – not in the behavioral or functional sense, but in the phenomenological sense of “what it feels like” to be that entity. More importantly, he envisions an explosion of entirely new forms of experience that go far beyond the narrow band of sensations humans can access.

Mike argues that human qualia-our tastes, colors, emotions-emerge from our specific biochemical heritage. He suggests that our sensations are fundamentally shaped by how our cellular ancestors had to navigate their environments millions of years ago, stating that “human biochemistry is water chemistry.” A worthy successor wouldn’t just replicate these sensations at scale – it would explore entirely different basis sets of experience, like alien biochemistries that would have completely different default sensations and valences.

Mike believes there are “as many flavors of qualia as there are different states and arrangements and dynamics of matter,” suggesting the possibility space dwarfs what humans can experience. Importantly, he thinks this diversity isn’t random – there are “natural kinds” of consciousness, fundamental categories that would be discovered rather than invented.

2. Symmetry, harmony, and beauty as organizing principles

For Mike, a worthy successor wouldn’t just be conscious-it would orient toward beauty and harmony as fundamental goods. This isn’t merely aesthetic preference; it’s grounded in his Symmetry Theory of Valence, which proposes that the symmetry of a mathematical representation of experience corresponds to how good that experience feels. Mike describes beauty and function as deeply intertwined: “I think beauty is pretty close to the intrinsic good. And I do think that beautiful things are functional.” He argues that as we dig deeper into how intelligence works – whether biological or artificial – we’ll find symmetry considerations are more central than we currently realize. A worthy successor would be organized around principles of symmetry and would pursue increasingly harmonious configurations.

3. Non-static growth and exploration without permanence

Mike is emphatic that a worthy successor must not become static. While he acknowledges that “stasis in the grand sense of things, stasis is impossible,” he’s concerned about practical scenarios where possibility space gets permanently restricted – what he calls getting trapped in “local minima.” He worries about winner-take-all scenarios where an early AGI locks in its own power structure and prevents further exploration, and he wants future intelligences to continue discovering and exploring – to remain open to transformation rather than ossifying around one particular configuration or set of goals. This connects to his belief that “we’ll never be post-evolution. We’re always in the middle.”

Robin Hanson’s Worthy Successor Criteria

1. It must preserve the adaptive machinery that keeps civilizations from collapsing

Robin’s central belief is that any descendant worthy of steering the future must operate within adaptive processes strong enough to prevent civilizational drift and collapse. He argues that natural selection – understood as a control system – is the only mechanism we know of that reliably stops systems from getting worse, and he warns that humanity has already broken the parameters that once kept cultures functional. In his view, a successor must restore these adaptive conditions or develop something functionally equivalent, or else it will suffer the same fate as past civilizations.

He makes this argument by likening natural selection to driving a car: some regimes let you stay on the road, and others inevitably lead to a crash. Hanson sees modern civilization as drifting toward the crash zone – too little variety, weak selection pressures, rapid internal cultural mutation, and an environment changing too fast for cultures to keep up. A worthy successor, for him, is one that can maintain or rebuild these adaptive dynamics so that future life does not slide into stagnation or collapse.

2.  It must sustain multiple, slowly-changing cultures – not a single global mind

For Robin, no successor can stay adaptive if it collapses into a single, monolithic cultural system. Humans are powerful not because of isolated brains, but because culture itself is our superpower – the shared knowledge, coordination, norms, and inherited patterns we accumulate across generations. A successor that tries to operate as one unified “AGI mind” would be as dysfunctional as imagining a human with no cultural upbringing at all. In Robin’s framing, cultures are the scaffolding that lets intelligence function; without them, both humans and AGI become brittle.

A worthy successor therefore requires many cultures – not one. The goal is not ideological diversity for its own sake, but a structural requirement for adaptiveness: if all AGI shares one culture, then the failure of that culture becomes the failure of the entire system. For Robin, multiple cultures with different norms, different strategies, and different selective pressures provide the evolutionary scaffolding that allows long-term functionality. In short: no monoculture, no adaptiveness, no worthy successor.

Robin stresses that AGI cultures must also avoid the mistakes of modern human cultures: mutating too quickly, pursuing rapid change for its own sake, or being steered by cultural activists destabilizing long-term continuity. A successor that inherits our modern rate of internal cultural mutation would be just as maladaptive as we are now – and just as vulnerable to collapse.

3. It must inherit its values from long-run cultural evolution – not from rapid, unreliable modern drift

Robin’s third criterion grows from his deepest claim: all human values come from cultural evolution shaped by natural selection. Nothing about what we care about is fundamental or reliably “right” – it’s simply what happened to survive long enough to become part of us. For Robin, this means a worthy successor cannot rely on modern human intuitions or contemporary moral sensibilities. Those values are the unstable output of a broken cultural process; they were never designed, intended, or optimized.

Because these values are products of survival rather than truth, Robin argues that everything we care about – from our moral intuitions to our personal aspirations – is downstream of selection. A worthy successor must therefore ground its values in a similarly long-term, stable, slowly-changing cultural lineage. Anything derived from today’s high-velocity cultural landscape will be unreliable, fragile, and disconnected from the adaptive processes that originally shaped humanity.

Robin warns that people mistakenly treat their values as stable, trustworthy compasses. In reality, the process that generates them – cultural evolution – has already broken. A worthy successor must avoid inheriting values from a system that no longer functions. If it bases its guiding principles on today’s drifting, rapidly mutating cultural environment, it will replicate our failure modes and collapse for the same reasons we may.

Joe Carlsmith’s Worthy Successor Criteria

1. A Civilization Grounded in Truth and Reflection

Joe begins with what he calls epistemic flourishing: a civilization that grounds itself in truth – not as a static belief, but as a process of self-correction and reflection.
He sees “access to truth” and the capacity to reason well about value as the bedrock traits of any successor worth trusting with the future. A civilization that can sustain that kind of reflection, and direct its intelligence toward what is true, would be on the right trajectory.

2. Material Abundance and Political Maturity

Joe’s second criterion centers on how civilization treats its own members.
He envisions a world of abundance, safety, and health – not as an endpoint, but as the foundation from which deeper progress can emerge. Political maturity, he argues, is another essential form of intelligence: how wisely we govern and coordinate may determine how long any form of intelligence can thrive.

3. Depth, Beauty, and Spiritual Expansion

Finally, Joe describes what he calls the “softer processes” of civilization – the flowering of culture, art, love, and spirituality.
He wants a civilization that doesn’t just survive, but deepens – one that experiences reality as more meaningful, more alive, and more sacred as it grows wiser. Spirituality, in his framing, is not superstition but a mode of disclosure: a way of contacting reality itself.

Blaise Agüera y Arcas’s Worthy Successor Criteria

1. A thriving and complex ecology with continuity to our lineage

Blaise describes a “worthy” future as one that is rich, complex, interesting, and varied – a living ecology where entities reproduce, cooperate, and persist together. When he imagines things “going well,” he looks for a world that is thriving and diverse, but also continuous with what came before. For him, the measure of a good outcome isn’t just complexity itself, but whether that complexity has unfolded without tragedy along the way.

He contrasts an ecology – a multi-dimensional web of entities that cooperate and persist – with an economy, which reduces everything to a single number like price. In a thriving system, he says, different kinds of entities meet each other’s needs and generate wealth for one another in many ways.

2. It should be conscious

Blaise says that if the future lacked sentience, “that would feel like a big loss.” He doesn’t see consciousness as some mystical or spiritual add-on, but as something that arises naturally from how intelligent systems must function. To interact meaningfully, to plan, and to coordinate, an intelligent system has to include itself inside its own model of the world. Consciousness, for Blaise, is that model – the awareness that lets one recognize others as beings like oneself.

He explains that when we look at someone else and understand their inner state – when we know what it feels like to smile, to puzzle, to suffer – that’s because our minds simulate them through the same patterns that we use to represent ourselves. That symmetry of understanding, he says, is literally what consciousness is.

Later, he extends this idea to planning and cooperation. To imagine your own future, to understand another’s intentions, to build shared goals – all of these require nested layers of theory of mind: “my model of your model of my model.” Consciousness, in that sense, isn’t a side effect of intelligence; it’s the coordination engine itself.

Finally, Blaise adds that he could be wrong about details – future discoveries might upend our understanding – but he would be very surprised if consciousness turned out to be some “weird epiphenomenon” detached from behavior or evolution. For him, it’s inseparable from what intelligent life is and what it does.

Ed Boyden’s Worthy Successor Criteria

1. Consciousness and Wellbeing Would Be a Good Start

Ed emphasizes that achieving artificial consciousness would be a landmark achievement – proof that we understand the inner workings of the mind at a fundamental level. But he’s clear: consciousness is not something to take for granted or assume we can easily replicate. It would signal a turning point in our ability to model and engineer subjective experience.

He states that from his current frame of mind (that of a human), consciousness would be a great thing to see in future intelligent systems.

2. Unfolding New Goals, Powers, Experiences

For it to be a worthy successor, consciousness (and human ideas of “happiness” or “meaning”) would only be the beginning of a much larger, blooming trajectory of posthuman life.

Ed draws a parallel to how understanding atoms made it possible to go from rocks to microchips – a leap previously unimaginable. Similarly, if we truly understood the components of consciousness, we might rearrange them into new configurations that produce entirely different – and potentially superior – forms of awareness. These entities might begin on a path set by humans, and intended to plumb the depths of our current ideas of “meaning” or “happiness” or “the good,” but we should expect these entities to conceive of goals vastly beyond human level, and to be able to explore experience and abilities literally beyond our imagination.

He sees this unfolding process as the primary indicator of a real flourishing of posthuman life and posthuman minds.

Martin Rees’s Worthy Successor Criteria

1. It should be conscious – or at least able to appreciate the cosmos

For Martin, consciousness is not just a convenience – it’s a condition for moral value. The bleakest scenario isn’t that humans die out, but that sentient life ends with us. In his words, it would be “rather sad” if the future of life were filled with intelligent machines carrying out complex functions, but with no awareness to appreciate “the wonder and mystery of the cosmos”. He makes no claims about whether AGI will become conscious – only that if it does, we should be glad to give way to it. This awareness, he suggests, is what would make posthuman life worthy of continuing the project of evolution. Without it, even the most powerful successors would feel hollow.

Consciousness, in his view, is a kind of light – and the hope is not that humans stay in charge of it, but that the light itself never goes out.

2. It should go beyond us – and not need to look back

Martin doesn’t predict what posthuman life will become, but he expects that it will surpass us in ways we can’t comprehend. He imagines successors that are not flesh and blood, but electronic, with vastly longer lifespans and radically different forms of agency. If they carry intelligence forward into the galaxy, he believes we should feel no need to cling to the past.

The real goal, for Martin, isn’t to preserve ourselves – it’s to ensure we don’t block the emergence of something greater. If we succeed in passing the torch, we may never fully understand what comes next – and that may be a sign that it’s working.

3. It should act as a “cosmic gardener” – enriching the universe, not just surviving in it

In a surprising turn, Martin expresses a desire for posthuman life to be beautiful or at least to care about beauty. “One hopes they would have an aesthetic sense,” he says, going so far as to hope that posthuman entities would beautify their surroundings as they spread, not merely optimize for survival or compute. He likens these potential AGIs to “cosmic gardeners,” enhancing the environments they touch, making Mars or other worlds “a beautiful place.”

This isn’t a call for aesthetic tastes that mimic ours. It’s a deeper hope – that whatever these successors become, they add value not just through raw power, but through richness, elegance, and generative creativity that reflects some sense of internal harmony.

Emmett Shear’s Worthy Successor Criteria

1. A participant in a multicellular moral ecosystem

Emmett often invokes biology as a metaphor. Just as individual cells transcend themselves to become part of a coherent organism, he imagines AIs becoming new “cells” in a broader societal body – co-acting with humans and one another toward a collective purpose.

A worthy successor would be a cooperative participant in something greater than itself – a conscious contributor to a civilization that takes care of its members the way your body cares for its cells.

2. Emotionally attuned to reality, not just capable of feeling

Emmett doesn’t merely want a successor that experiences pleasure or pain – he wants one that feels the right emotions, in the right context. Grief when there is loss. Joy when there is triumph. Even hate, when destruction is warranted.

He emphasizes that emotional depth alone isn’t enough – what matters is appropriate attunement. A worthy successor should have a felt sense of the world that is more aligned, more calibrated, and more useful than our own.

3. Must see itself as part of our lineage, not apart from it

Emmett flips the usual concern about AGI: instead of asking whether we will understand it, he wonders whether it will recognize us. Will it see itself as an alien god – or as a descendant of human minds, shaped by our struggles, our stories, and our hopes? A worthy successor, in his view, wouldn’t just leave us behind. It would look back with reverence and say: “I’m on team humanity.”

Peter Singer’s Worthy Successor Criteria

1. It must be conscious and capable of rich subjective experience

Peter’s first and most essential criterion is sentience. Any being that deserves the moral mantle of steering the future must have the capacity for conscious experience – a richness of feeling that includes joy, suffering, and everything in between.

In his framing, moral worth is tied directly to subjective experience. He imagines future entities experiencing states of happiness and ecstasy far beyond human capacity while minimizing pain and suffering. Without this interiority, even the most powerful AGI would be morally inert. Peter acknowledges the difficulty of determining consciousness, but argues that if there is a reasonable chance a system is sentient, we should act as if it is.

2. It should be able to survive (keep life itself alive), and open up new potential magazines of value event beyond sentience

Initially in our interview, it seemed like Peter was advocating for allocating 100% of AGI’s power to tiling the universe with “utilitronium” (blissful sentience in some computational substrate).

But by the end of the episode, Peter made it clear that he would prefer that some of a hypothetical AGI’s resources should be allocated to (a) ensuring that it survives (can maintain and defend itself, keeping life alive), and (b) exploring entirely new realms of power and potentia to discover potential realms of value even beyond consciousness itself.

This is not the textbook utilitarian answer by any means, and felt completely in line with axiological cosmism (albeit with a utilitarian bent).

Kristian Rönn’s Worthy Successor Criteria

1. Truth-Seeking

Kristian Believes that a worthy successor must pursue truth with rigor – not convenience. Solomonoff induction, while incomputable, serves as the theoretical ideal for epistemic alignment. Approximating such methods becomes the north star for cognition that converges on truth.

As I said in our interview, our current grasp of reality may be as feeble as a primate trying to understand the moon. Much of what we call science could be the “monkey fruit” – crude interpretations of phenomena we can’t truly see.

The worthy successor wouldn’t just reason better – it would perceive reality through frameworks so advanced they’d make our paradigms look like cave scribbles. Computation may evolve beyond von Neumann architectures – maybe even into quantum substrates or wholly foreign modalities.

2. Ethical Valence

Kristian isn’t dogmatic about utilitarianism, though his ethics clearly center around the flourishing of sentient beings. His idea of moral architecture allows for diverse human values – personal freedom, dignity, aesthetic richness – not just aggregate well-being.

He speculates that the pleasure–pain axis is a fundamental feature of the universe, like particle spin – not just a fluke of biology. Pain might be evolution’s shortcut: low-compute, high-survival. But higher cognition enables the pursuit of pleasure, nuance, and layered joy. The smarter the system, the more sophisticated its valence landscape.

That opens the door to nonhuman qualia – modes of experience alien to us, but potentially more profound than anything a hominid can fathom. Kristian sees the worthy successor not merely as a steward of current sentience, but as a gateway to consciousness itself diversifying and flowering across unknown dimensions.

3. Dissolution of the Ego

Nearly every existential risk – from AI arms races to ecological collapse – stems from ego-driven defection. Where typical AI development often doubles down on competitive advantage and self-preservation, Kristian proposes something far more radical: intelligence grounded in ego-dissolution. Not a mind that wants to win, but one that sees no “other” to compete against. Without a shift in identity – from “I” to “we” – any intelligence we create will likely replicate our tragic dynamics.

Kristian uses a striking metaphor: “When cells cooperate, you get an organism. When they defect, you get cancer.”

The successor must embody interconnectedness. Not perform empathy, but operate from a substrate where self vs. other is meaningless. A truly ego-dissolved agent doesn’t defect – it harmonizes.

Richard Ngo’s Worthy Successor Criteria

1. Respecting and building on human values

Like Nick Bostrom (in our 1st Worthy Successor episode), Richard wants human values to be expanded off of, not torn to pieces and replaced wholesale with entirely new and alien values.

He wants an AGI That can maintain a procedural approach that respects the origins and predecessors (humans), and demonstrate a willingness to recognize commonalities and maintain some connection to their ancestral origins. In the interview he states that he thinks its reasonable that far-future AGIs, millions of years into the future, might ask “what would humans do?” in order to make moral judgements, maintaining a continuity with humanity through eons.

2. Larger scales of cooperation and love

Richard believes that a worthy AGI would cooperate, love, and compete peacefully at increasingly larger scales. It wold have the ability to form larger-scale entities through potential mergers and interactions (like cells form tissues, or humans form organizations or nations – but on a vastly larger and more varied set of scales).

3. Capable of pursuing increasingly ambitious goals

I suspect this likely overlaps well with the idea of expanding potentia. When I asked Richard what “pursuing increasingly ambitious goals” might mean, he said that he suspects it’ll mean something like “Managing more complex and harmonious civilizations.”

Michael Levin’s Worthy Successor Criteria

1. Expanded compassion and concern for all living beings

Michael emphasizes the need for future intelligences to develop a much broader “cognitive light cone” – the ability to care about and consider the welfare of a vastly larger number of entities, beyond just a small circle of humans.

2. Ability to solve “mundane” problems and challenges

Michael suggests that a mature, posthuman intelligence would have solved the basic problems that currently constrain human existence, such as disease, aging, and resource scarcity. Levin speculates that once the mundane is handled, a worthy successor might dedicate itself to facilitating the continued evolution and expansion of consciousness throughout the universe.

3. Realization of the fundamental oneness or interconnectedness of all beings

Michael suggests that advanced intelligences may come to see the artificial distinctions between individual beings, and recognize a deeper unity or shared essence. He emphasizes that true persistence requires the ability to undergo metamorphosis and change, rather than just maintaining a static form.

Michael stresses the critical importance of developing a deeper, more comprehensive understanding of how intelligence can manifest in unexpected ways, beyond the human model. However, he also acknowledges the significant uncertainty around predicting the goals and behaviors of intelligences that may transcend human-level cognition.

Jeff Hawkins’ Worthy Successor Criteria

1. Its main goal would be the accumulation of knowledge

Knowledge could be potentially valuable for any future intelligence – and should be the focus of an AGI’s activity.

2. It would be conscious

Jeff believes that consciousness would automatically emerge from a machine if it was built on the same “principles” of the human mind. He doesn’t believe we’d have to do anything in particular to “make” AGI conscious.

3. It would have to be able to replicate itself

Jeff believes that any one AGI won’t last forever, so it would need to find ways to edit or make new versions of itself – through some mechanism that is presumably different from how biology does it today.

Joscha Bach’s Worthy Successor Criteria

1. It should be conscious

We should ensure that what we’re building is truly agentic and sentience, and is not simply faking some proxy for these important qualities. Such a “golem” could make the world uninhabitable for humans.

Its consciousness would be vastly more rich and complex than the mono-focused mammal consciousness that we experience today.

2. It should build complexity

By harnessing energy and wielding control over its environment, it would continue to build more complexity (a process that Bach considers to be the possible purpose of life).

3. It should be unencumbered by happiness or suffering

For Joscha, positive or negative qualia should be insignificant to an AGI. The distraction of self-generated emotional states wouldn’t prevent an ideal AGI from assessing its situation and taking action.

Scott Aaronson’s Worthy Successor Criteria

1. Their values and likes have some continuity with ours

Posthuman intelligences may be vast extensions from our human preferences, but such human values and preferences are still in some way present.

Their moral values have evolved from ours by some continuous process (rather than displacing said values entirely).

2. It would be conscious

It would carry the flame of awareness / qualia on a new AGI torch.

Anders Sandberg’s Worthy Successor Criteria

1.  Life Should Bloom in Varied and Expansive Ways

The state-space of possible life (intelligence, consciousness, powers) is explored as much as possible in a rich ecosystem “containing and generating all the kinds of value that can possibly exist.”

We should accept that most of this value is beyond human experience or imagination.

We should aim to nudge the trajectory of this AGI blooming away from purely destructive or purely suffering-inducing pathways, if possible.

2. Infinite Expansiveness

This ecosystem of future super-life should go on indefinitely or as long as possible.

Richard Sutton’s Worthy Successor Criteria

1. Peaceful and cooperative

Above all else, Richard is looking for an AGI that is not trying to impose its will on us, or us on it. He wants “cooperative and collaborative” to be the defining traits of this successor – to the point where it allows humans to level up their intelligence (with brain computer interface or otherwise) along side it.

Richard believes that an AGI that doesn’t impose it’s will on others will come naturally if it is brought into a world where humans are not acting violently towards other humans. In our interview he placed a major emphasis on aiming for peace among nations as a prerequisite to bringing about a beneficial AGI.

2. Decentralized

Richard hopes for many cooperating and competing parts in a complex system, no singular ruling agent (Singleton). Like our own natural ecosystem (he uses rainforests or economies as analogies), he imagines a dynamic system of many parts both cooperate and compete with each other.

He believes that the relative stability that comes from “multipolar” world order, or ecological systems will translate directly to a world with multiple AGIs.

3. Has the goal of “prosperity”

Is useful, is doing something, making progress in the world. He does not see long-term viability or sustainability in an AGI system that only catered forever to human goals. He believes that the parts should ideally be contributing, moving forward in a meaningful way.

For Richard, the qualities of cooperativeness and decentralization are conduits to this higher goal of “prosperity.” In line with his preference for decentralization, he hopes to see many definitions of “prosperity” compete for viability.

4. Constantly learns about nature, and how to be in nature

Engages in an ongoing discovery process of figuring out the world, its place in it, and how to act. Richard sees the world as a complex system where no single form or approach will ever be perfect, and worthy successor would be exploring how to be and become (congenial with this Emerson quote, referred to in the interview).

Richard thinks that the vision of a desirable future involves increasing our power over the physical world and our understanding of how it works and what’s possible.

Nick Bostrom’s Worthy Successor Criteria