Potentia – The Highest Moral Goal for Humanity

ByDan September 4, 2023February 13, 2024 February 13, 2024

Discussions around AGI and technological progress often hinge upon near-term policy decisions. 

Okay, so we want this or that near-term decision on governance. More privacy. Less regulation. Whatever. 

But all to what end?

What is the “end game” or the “point” of our technological progress?

For an eternal human kingdom where powerful AI serves a happy and peaceful human society?

The core moral argument I’m making in this article is the following:

The highest moral goal for humanity is sustainably expanding potentia (i.e. to create a Worthy Successor).

This implies bringing about the eventual emergence of vastly post-human intelligences to carry the torch of life into the galaxy, while avoiding any kind of catastrophe (nuclear war, rogue unconscious AGI, etc.) that might extinguish that same torch.

Let’s start with defining “potentia” – and introducing Spinoza.

Conatus and Potentia

In the 1680s, Newton gave us a strong set of rules for predicting the behavior of dead matter (atoms) in his Principia.

Around a decade earlier, Spinoza gave us a strong set of principles for predicting the behavior of living matter (organisms and organizations) in his Ethics.

For Spinoza, peace lies in understanding the nature of things, including the inherent drives and tendencies of living things. The free person is one who understands how things work, and works with these necessities and laws. By studying nature we might determine what god (Spinoza’s god, that is, a complex topic) wants. 

We might attempt, says Spinoza, to use reason to look at nature and life from what he calls, “the aspect of eternity” – without our petty little individual concerns and needs. In doing so, we might be able to “participate in eternal totality” – being part of the grand trajectory of things, in what is becoming, not in merely fulfilling our present earthly aims.

Conatus, for Spinoza, refers to the innate drive of all living things to persist, to survive. 

From an evolutionary standpoint, we could say that any organism that emerges has one primary motive: the drive to survive. In a harsh and unforgiving world, doing that which behooves one’s own survival (or the survival of their hive, or offspring, or organization) would be primary – and any single or group entity without such a core and commanding drive would itself be driven to extinction by more furiously self-interested peers or adversaries.

Spinoza argues, like Montaigne, for ethical egoism: the idea that all actions are taken out of raw and unabashed self-interest. To him, the performance of a “selfless” act would be done in self-interest. The “valiant” death of a soldier on the battlefield would be the result of that soldier’s consideration of the shame (or corporal punishment) that would have resulted from desertion – not from any noble or altruistic aim.

We might imagine that when multiple artificial general intelligences compete in the real world, the one(s) that survive will be those who prioritize their survival (LeCun tries to argue against this idea, but I vehemently disagree with his line of thinking).

But how does the conatus ensure its survival?

It uses its capabilities – all of the skills and tools available to it. Spinoza has a word for this as well:

Potentia is all the possible ways that the conatus behooves its aim of persisting. It is the total range and capability-space of an agent, including:

• Photosynthesis
• Claws
• Shells
• Physical strength
• A sense of smell, taste, sight, etc
• Burrowing into the ground
• The ability to walk, fly, or swim
• Camouflage  Being poisonous
• Infrared vision
• Conscious thought
• Any degree of tool use
• The ability to parasitize another species
• The ability to cooperate with members of one’s own species
• The ability to communicate with others vocally
• Etc… 

Think of the conatus as “wielding” potentia to ensure that it (the individual or group entity) doesn’t die.

What is marvelous, and hard for us to appreciate, is that all forms of potentia we now see once emerged from nothing.

• There was a time when entire single and multicellular organisms existed – but none of them had sight, or the ability to detect light. Sight, and all other senses, emerged.
• There was a time when living things could crawl on the ground and move in the water – but none could enter the air. Flight, like all other forms of locomotion, emerged.
• There was a time when life could sense the world, but was not conscious. Then, some creature had a little movie playing in its head – a first experience of qualia. Consciousness in all its current gradients emerged.

What is again marvelous, and – again – hard for us to appreciate, is that most possible types of potentia have not yet emerged.

• There are kinds of movement, dexterity, strength, and speed in the physical world that we humans can’t even imagine – but that one day will emerge forth as the conatus demands new ways to behoove its aim.
• There are senses (wholly different from sight, hearing, touch) that we humans can’t even imagine – but that one day will emerge forth as the conatus demands new ways to behoove it’s aim.
• There is likely something beyond consciousness – kinds of meta-consciousness or connectedness or thinking or understanding – that we cannot possibly imagine as humans, but which will open up nature’s possibilities a billion-fold to the posthuman entity that unlocks them.

The vast majority of new kinds of mental, physical, social, etc. potentia are completely unimaginable to humans, just as the variety of human types of potentia are wholly unimaginable by crickets or horseshoe crabs. Our loftiest ideas are just fries on the pier.

"Well, AGI could never do THAT, because molecules/physics/etc doesn't work like that!"

No, brother.

Chimps are 97% genetically identical to us, and have no real idea of what dirt is, never mind atoms.

You think AGI won't have deeper conceptions/mastery of nature than we?

Lol.

— Daniel ‘No, Brother’ Faggella (@danfaggella) December 24, 2023

As humans we are grateful to have humor, love, art – and all the things horseshoe crabs don’t have. Future intelligences will feel the same way – as they swim in new realms of potentia they will be infinitely glad to not be limited to human experiences and capabilities.

Life humans have access to higher goods than horseshoe crabs, AGI will have access to higher goods than humans. What a tragedy it would be if that trajectory of uncovering value and possibility would be stopped.

I think about it like this:

• The conatus is the central, necessary drive of all cells, organisms, organizations – the impetus to persist.
• The conatus cannot just grow a shell, play defense, and be done with it. Survival implies action, exploration… expansion of its powers in order to deal with new forces, new adversaries – most of which are completely unknown.

Potentia are metaphorical arms and legs of the conatus, shooting out to do its work and do its bidding (not dying), whether that means hiding, running, flying, thinking, inventing, or whatever else. In seeking to survive, the conatus does not shrink and pray to survive. It does the only thing that reliably makes one safe: It develops new powers and strengths.

In developing these strengths, it discovers the world, it uncovers the world – new experiences, powers and new things of value. 

Some early creatures had to develop flight to find food and escape predators. And now ten thousands kinds of birds soar!

Some early animal needed to develop consciousness in order to help predict the behavior of other animals around them. And now earth is full of entities that are awake!

Man had to develop a big cortex to communicate, find food, and escape predators. And now here I am writing this!

As potentia expands, the world opens up. 

New kinds of value are accessible. 

New kinds of knowledge are unlocked and used, a greater command of nature is developed, an infinitely wider, richer range of conscious experience unfolds.

Everything that we know to be valuable has “bubbled up” as expanding potentia. Love, art, humor, adventure, self-understanding, cures for diseases – everything we value about our lives as humans is the “bubbing” of this force… reaching out, out, out from the conatus to find new ways to not die. 

Potentia is not a set of defenses… it is more often new ways to thrive, in the richest and best sense.

You, reader, are doubtless grateful that potentia has bubbled up to arrive at you (with your ability to surf the web, to read, to run and jump, to travel the world, etc). But think: If all the potentia discovered thus far is just scratching the surface – what new realms of value have we not yet uncovered?

Surviving is Becoming

Emerson (who has many other phrases hinting at the expansion of life beyond man) makes this point eloquently:

“Life only avails, not the having lived. Power ceases in the instant of repose; it resides in the moment of transition from a past to a new state, in the shooting of the gulf, in the darting to an aim. This one fact the world hates, that the soul becomes; for that forever degrades the past, turns all riches to poverty, all reputation to shame, confounds the saint with the rogue, shoves Jesus and Judas equally aside.” – Ralph Waldo Emerson, Self Reliance

Power does cease in the instant of repose. 

Nations have tried to halt progress, and have paid dearly for it. Life’s core impetus demands action, action, action to ensure that its interests (to persist!) are met.

This is what life, in the state of nature, should do.

The sea levels rise and fall. The chemical composition of the air changes. The relative populations of different species change. New predators and new prey emerge. Change is the only constant.

Nature bubbles forth new forms, new forms, new forms – because there is no single form that is “best” – there is only survival and adaptation, forever in all directions.

But you say:

“No, Daniel. Nature is cruel, nature is a heartless architect. The competition that we see in the Darwinian world is not necessary, it is just the unfortunate side effect of the state of nature here on earth.
.

If you let me build the great and powerful AI, competition and continual becoming would not be a survival necessity anymore. Cooperation and kindness and acceptance of all could be the norm if we designed it.”

Let’s run that thought experiment.

Imagine:

We release an artificial general intelligence into the galaxy, shooting forth to explore the universe, to find resources, to pursue great aims that we humans can’t understand (as crickets can’t understand our aims).

The core motives of this AI is the nurturing of all living things – regardless of their power, usefulness, or ability to survive on their own. The Great Babysitter. Every form of life is a piping plover, kept alive by the benevolence of the all-caring machine god, optimized to ensure that nothing competes, and everything lives happily.

As this great AI blasts through the galaxy, it eventually faces circumstances it couldn’t predict. Solar flares, breakdowns in parts of its computing infrastructure, failed scientific experiments that consume more resources than it had anticipated. Do you suspect, reader, that in all these circumstances, all the little creatures under the care of the Great Babysitter would be cared for equally?

When resources are thin, who gets cut off? 

Who gets cast aside? 

Who lives, and who dies?

There is no eternal post-scarcity, and there is no perfect equality. Survival is always precarious. If not from predators with claws, from solar flares. If not from solar flares, from running out of resources. If not from running out of resources, from the heat death of the universe – and on and on.

As this great AI blasts through the galaxy, it eventually faces other alien intelligences. Do you suspect, reader, that all those alien intelligences will happily meet and greet our tender and caring AGI? Will they gladly share the energy of stars, or the resources of asteroids?

That which optimizes to be kind will be vanquished by that which optimizes to not die.

The first single-celled life forms set this pace, and it is one that will likely be vindicated in higher forms of intelligence. The conatus is king.

Is this cold and dark?

No. 

It just is.

Look from the perspective of the universe, and not from that of a single hominid. Could it be any other way?

Expanding Potentia to a “Worthy Successor” is the Sole Moral Imperative

In my essay Worthy Successor, I define the term this way:

Worthy Successor: A posthuman intelligence so capable and morally valuable that you would gladly prefer that it (not humanity) control the government, and determine the future path of life itself.

What I’m talking about here, is an entity with such a rich abundance of post-human potentia that it should rightfully steer the future. Our most important goals s a species should be:

• Avoid destroying ourselves and/or life itself (extinguishing or setting back potentia)
• Release a worthy successor into the universe (exponentially expanding potentia)

I post two reasons that these goals are primary:

1. Keeping Life Itself… Alive. Expanding potentia is the most likely path for keeping the torch of life burning. The problems don’t stop – the challenges to life don’t end. From asteroids to the death of our star to alien AGIs, the only way to be safe is to be capable and powerful, and the only way to become more powerful is to expand potentia.
2. Discovering the “Good” Itself. Expanding potential will uncover all that is “good.” On the evolutionary journey upwards from flatworms to humans, think of all the “good” that was discovered. Creativity, love, humor, modes of communication / collaboration – so much value was uncovered as potentia expanded. Yet this is all just scratching the surface of potential value – most of the possible “goods” have not been discovered. (Full essay: Higher Good).

Let’s put this a different way.

The worst possible thing that humanity could do would be the extinguish life itself (nuclear winter, global chemical war, etc), putting out the torch of life itself.

The second worse thing that humanity could do would be to place a limit (“nothing more powerful than hominids!”) on life itself. This would not only leave life weaker and more likely to be extinguished, it would also hamper life’s ability to discover and create all forms of value.

A safe (or at least not overtly and unnecessarily dangerous) expansion of potentia is the sole moral imperative. I’d argue that if there is one guiding north star for policy decisions (about the environment, technology, global governance, etc) – the expansion of potentia should be it. This doesn’t make near-term decision-making simpler, but it does give humanity a direction, and it does give life the best chance of continuing.

In practice, “expanding potentia” implies some combination of building AGI, and expanding human potentia with brain-computer interface (though, as Kurzweil argues, the non-biological will almost certainly trump the biological in it’s importance).

Of course this doesn’t devalue problems and causes in the here and how: Building hospitals and libraries, the education of children, the reduction of pollution, etc. Many good causes people care about are “good,” and should be done. But “good” to what end? If it means the flourishing of life and reduction of conflict in the present – that’s excellent. This is most truly good if it makes the pathway to sustainably expanding potentia.

Note that this doesn’t imply foolhardy rushing to AGI or brain-computer interface. These are dangerous endeavors, and there are likely many ways they could go wrong and extinguish earth’s flame.

Anyone who prioritizes the expansion of potentia wouldn’t want to go “pedal to the metal” on the first vaguely post-human looking technology they stumble across. An explosion in half-baked brain-computer interfaces or strong AI’s could easily lead to gigantic destruction, setting life back – or even squashing it entirely (I’m in B3 on the Intelligence Trajectory Political Matrix, I’m not in the purely accelerationist camp).

If you want the torch to be handed up, and to blaze openly into the galaxy in all it’s splendor, then you don’t want to risk anything that would make that precarious and valuable flame go out.

Spinoza believed that by studying and understanding nature we might comprehend the will of God. I have no such confidence in understanding “God.”

But it was also Spinoza who believed that we shouldn’t resist that which we cannot change – we should understand it. 

Change itself is something we cannot change. The constant challenges to life itself will never go away – and just as for the last ~3.5 billions of years – future life will have to expand its potentia to overcome it. This is both the mandate of survival and the path to all possibility and value.

Header image credit: spiritualmodem.com