The Ambition Singularity – Controlling AGI as the Only Power Game Worth Playing

ByDan February 5, 2022February 13, 2024 February 13, 2024

The hypothesis of the Ambition Singularity:

At some point in the 21st century, over 90% of the most ambitious persons on Earth will be working directly or indirectly to either create or control AGI/post-human intelligence.

(I’ve referred to this in jest as The Final Flex, the ultimate aim for status and power.)

There was a time when the channels for achieving status or power were few.

One could be a poet, maybe a blacksmith, maybe a warrior – and a few maybe could strive to be king.

Today, fortunately, the pathways are vastly more nuanced and varied. One can become an opera singer, a skateboarder, a Twitter intellectual, an astronaut. Even those who strive for power and influence alone can do so ways other than war and conquest.

Today’s powerful persons come in the form of politicians, media moguls, and businesspeople – and philanthropy can become its own kind of status competition between the powerful. We should be grateful that Musk and Gates could build companies or compete philanthropically. If only past generations had such outlets, we might know the name of Bonaparte or Cromwell for something other than what we know them for today.

Structuring incentives that bend the will of these super-achievers into productive (or at least non-destructive) ends is an important consideration for government – Lincoln lays this out well in his Lyceum Address:

The question then, is, can that gratification be found in supporting and maintaining an edifice that has been erected by others? Most certainly it cannot. Many great and good men sufficiently qualified for any task they should undertake, may ever be found, whose ambition would inspire to nothing beyond a seat in Congress, a gubernatorial or a presidential chair; but such belong not to the family of the lion, or the tribe of the eagle. What! think you these places would satisfy an Alexander, a Caesar, or a Napoleon?–Never! Towering genius disdains a beaten path. It seeks regions hitherto unexplored.–It sees no distinction in adding story to story, upon the monuments of fame, erected to the memory of others. It denies that it is glory enough to serve under any chief. It scorns to tread in the footsteps of any predecessor, however illustrious.

Replace “the 1% most ambitious persons” in the Ambition Singularity hypothesis above with “persons of the tribe of the eagle.” “Ambition” itself might rightly be considered to be what Lincoln refers to above – the hunger for status and power.

Why Control AGI?

The highest conceivable power game is birthing or (temporarily) wielding the deity-like AGI itself, is being part of the creation or control of the entity that will be astronomically more powerful, knowledgeable, and important than humanity itself.

Billionaires, philanthropists, politicians, artists – will be as ants and beetles to a drastically post-human intelligence, and the persons of the strongest ambition will not settle for striving to be king of the ants.

For those truly of the stamp of “1% most ambitious” are not only unlikely to be satisfied adding luster to another person’s institution, they are unlikely to even be capable of standing happily on their little molehill, knowing that anyone stands instead on a mountain – wielding a level of power and influence that completely shadows their aims.

Status is now, always was, and always will be, relative. In terms of the satisfaction of our basic needs, even poor people in the first world often live like Gods compared to common people just a few generations ago – and yet we are riddled with jealousies and desires to outdo (or at least outshine – the perception of outdoing) our peers.

The Atlantic’s Matthew Hudson ran into some predictable findings in the research for his article Why We Compete:

No wonder we constantly measure ourselves against our peers. In a survey of faculty, students, and staff at the Harvard School of Public Health, nearly half of the respondents said they’d prefer to live in a world where the average salary was $25,000 and they earned $50,000 than one where they earned $100,000 but the average was $200,000. Similarly, a majority favored relative over absolute advantage when it came to their own intelligence and attractiveness, their child’s intelligence and attractiveness, or praise from a superior. Apparently the survey respondents would rather the planet be filled with stupid, ugly children than have their own child left behind.

For humanity in general – never mind for those outliers in the tribe of the eagle – achievement and attainment are benchmarked by their peers.

From Plutarch’s Life of Themistocles:

It is said, indeed, that Themistocles was so carried away by his desire for reputation, and such an ambitious lover of great deeds, that though he was still a young man when the battle with the Barbarians at Marathon​ was fought and the generalship of Miltiades was in everybody’s mouth, he was seen thereafter to be wrapped in his own thoughts for the most part, and was sleepless o’ nights, 4 and refused invitations to his customary drinking parties, and said to those who put wondering questions to him concerning his change of life that the trophy of Miltiades would not let him sleep.

What lunacy to think that any toweringly ambitious person today does not have their sleep snatched from them for exactly these same reasons.

Dr. Nayef Al-Rodhan, Head of the Geopolitics and Global Futures Programme at the Geneva Center for Security Policy, refers to five drives, or the five P’s, that will likely drive humanity towards a post-human transition:

• Power
• Pride
• Profit
• Pleasure
• Permanence (longevity)

What could be more permanent, or more powerful that creating or influencing the post-human intelligence that populates the galaxy, understands vastly more of the mysteries of nature, and extends intelligence itself?

The expressed aim to Steve Jobs to “dent the universe” speaks volumes. It is neither good nor bad, it is about size, it is about impact. The tribe of the eagle lusts for nothing more than impact, for distinction by any means.

Influencing the trajectory of intelligence directly is the loftiest reach for the five P’s imaginable – and could be argued to be the only aim that might actually “dent the universe” in a literal sense.

We intuitively know where power is – and those hungriest for it are most keenly aware. Social pressure will likely tip the scale towards AGI – as it did for Themistocles in his pursuit of military glory. I’m happy to go on record for this one:

When it is widely known that even a handful of the world’s most powerful people are focused on influencing/controlling AGI or post-human intelligence, a huge majority of the world’s ambitious people – seeing a new and ultimate peak to strive towards, and recognizing their power as being petty compared to the power of those who wield post-human intelligence – will pivot their efforts to pursuing that same trophy.

As of the time of this writing (February 2022) only Musk seems to be pursuing this aim, but it isn’t something most people understand. I’d guess that Zuckerberg, Larry Page, Jensen Huang, and others are already on this page – but for good reason aren’t openly saying as much.

How Will AGI Control be Attempted?

In the long-term, I argue that AGI won’t be controllable – or not for long. The best humans can hope for is to imbue a bit of our values or preferences, and then let it rip – allowing the trajectory of intelligence to take off beyond humanity, with some hope that it won’t treat us too terribly in the process.

Even those who agree with these very poor odds of controlling AI should nonetheless be expected to aim to controlling it in some way. Bonaparte knew well that his empire wouldn’t survive after after he was gone, and grandiose self-destructive aspirations litter the biographies of outlandishly ambitious people.

Some people probably do believe that AGI will be controllable, and that it might be possible for an organization to wield it to their own aims.

In either case, the ambitious will pursue two main channels to AGI control and influence:

• Innovation: Some people will fund, found, or lead organizations with the express purpose of bringing about post-human intelligence. OpenAI is the most obvious example of this today, but it’s undoubtably the first of many.
• Regulation: Some people will create technology policy organizations, or attain positions in government that wield influence over said technology. There are too many of these to count already, with the United Nations, OECD, and other IGOs establishing AI wings, and with myriad new organizations cropping up – Partnership on AI, AI Now Institute, Center for the Governance of AI, etc.

That said, these two main paths will be approached from an infinite number of angles:

• Some people will build adjacent technologies (hardware, software) that will profit from that AGI competition boom, and permit them to enter into the fray at a later time, from a position where they have strong existing connections and resources.
• Some people will become public intellectuals, weilding followers and fans to influence policy, rather than starting or leading a think tank or policy organization themselves.
• Some wealthy people will donate generously to all of the innovation or regulation groups that they believe to be most likely to obtain power – thereby holding some sway and influence regardless of who wins the commercial or political battles.
• Etc.

(At some point I’ll devote an entire article to the “how” of influencing AGI development/control, but that’ll have to wait for another day.)

Regardless of which “how” the aspirants take – whether through innovation or regulation – whether through money or social influence – we can expect they’ll do so while bearing the following tenets in mind:

• Aspirations for winning power over AI should be hidden. It behooves Zuckerberg to talk about “connection people” and the fun and possibilities of VR. It does not behoove Zuckerberg to discuss the fact that – should everyone live and work in his VR ecosystem – he’d have the ability to train the most powerful AGI.
• Openly recognized power should feign benevolence. Any organization openly in the runnings to influence/control AGI will claim to do so for the benefit of others.

We can expect the smart and capable actors to act in accordance with those general tenets.

I suspect that if the tech publications truly knew, and were able to publish, the grand AGI ambitions of tech leaders today, the Ambition Singularity would already be in full effect. The first tenet (hiding one’s power ambitions) means that the Ambition Singularity will be staved off for a while – but it’s time is coming.

In Conclusion

While the “how” may vary, I will posit that at some point in the coming 10-50 years, directly or indirectly, the truly ambitious will ultimately be vying for influence and power over post-human intelligence.

There will be no other goals. Ruling the molehill of running a tech unicorn or becoming President will mean nothing if they don’t offer a leverage point to influence the control of AGI – the wielding of power that will be vastly beyond anything that any human could achieve.

So what do we do about this?

I don’t really know.

We should expect an explosion of AGI organizations, and interest groups aiming to influence or prevent AGI. We should expect intergovernmental organizations to explode with activity about AGI (as they already have, in a mostly hype-y and ineffectual way, with AI), and you should expect the most tech savvy and ambitious of your friends and colleagues to move their careers and aspirations somewhere that they believe is their best position upstream of AGI.

Header image credit: Hiphop DX