The ancient and imperial China, rulers were legitimized by an ideology of the Mandate of Heaven – the idea that heaven selects a rightful virtuous ruler.
Capable and benevolent rulers held this Mandate of Heaven until they lost in battle, or their people fell victim to famine or plague, or some other unfortunate event that seemed to tarnish their divine right to rule.
Like ancient Chinese emperors, AGI company leaders occupy a precarious position of power. AGI company leaders know that – in order to be the one to birth AGI first (the final flex) – they must defeat their rivals not only in technical capability and resources, but also in the perception of their own goodness.
Smart AGI lab leaders who want the highest likelihood of birthing AGI and maintaining power will aim to do everything they can to maintain the AGI Mandate of Heaven – by occupying the space closest to the top right of this quadrant:
Before creating your own version of where the main AGI labs (i.e. the “players”) fit on the quadrant, let’s explore the axes and how the plotting works:
The AGI lab that “builds god” needs more than capability.
Bands of chimpanzees will kill an alpha male if he is aggregately detrimental to the tribe, ancient Chinese would revolt against an incapable or vicious king.
Similarly, super powerful AGI labs are a few moves away from being commandeered by the military, or torn down by an angry populace who believes that they are not acting in the interest of the public (read: Political Singularity).
Both elements of the AGIMoH must exist in order for the top player to have the best chance of weidling power and influence in the future.
For the sake of using the matrix to analyze the AGI power landscape, I do the following:
I’ll anticipate some of your immediate responses to the image I posted above:
Bear in mind also that my plotting above isn’t “right” or permanent, it’s an example.
I developed this matrix in order to open an amoral dialogue about incentives and power.
Power struggles quickly devolve into tribal squabbles – where one side is “good” and the other side is “bad,” and one leader is “virtuous,” another is “selfish.”
There is no progress in tribal enemizing, but there is progress in squarely understanding the positions and respective self-interest of the parties involved. Only by doing this can the “other” be treated as something other than “bad” or “evil”, and only by doing this can we establish arenas (as we do in sports, in science, and the best versions of fair markets / justice systems) where competition can be harnessed for a kind of net good.
The Peace of Westphalia, the founding of the United Nations, and the end of many lengthy and complex conflicts have been solved by squarely understanding the motives of respective parties, and orchestrating ways to navigate incentives rather than label parties as good or bad.
As AGI becomes the singular, defining political topic of the century (and likely the last such issue for humanity), there will be many emerging “tribes,” some political, others based around specific AGI initiatives and labs. By understanding the kockeying of power between major players on the AGIMoH, I hope to be able to make their jockeying less of a “good vs evil” story, and more of a reasoned understanding that…
We must deal squarely with human nature and accept that the conatus is king, and trade fruitless tribal conflict for a measured approach to aligning and allying incentives.
Use this Google Slides link to create your own version of the AGIMoH.
Remember that if you plot the the players correctly, you’ll always have one player touching the top, bottom, right, and left boundaries. The rest of the players sit in between the extremes, relative to their public support or capability.
Ping me on Twitter with what you come up with, I’m interested to see other perspectives here.
Much of what we’ll see pan out in the AGI race will be more-or-less identical to how we’ve seen other power struggles – because humans haven’t fundamentally changed very much.
OpenAI was founded as the “good little guy,” and quickly changed their tune when power consolidation was possible.
Anthropic spun out of OpenAI, waving the same banner of virtue, only to jump into the same ravenous AGI race, aligning with the same Big Tech allies.
Now OpenAI is in the “powerful” category, no longer an underdog – and predictably they’re focusing on owning the regularity narrative.
Neither of them are “bad,” neither of them were “corrupted,” they are merely operating on the principles of self-interest, just as you and I do. We would do likely exactly what they would do if in their shoes. But it behooves us nonetheless to understand how the powerful and the less powerful will behave.
This graphic from the first-ever full length essay in our AI Power series at Emerj sums up the dynamics we should expect in the years ahead:
It’s easy to be the good guy (the far right of “public support”) when you’re small.
The little guy gets to say “Great masses! I am one of you!”, and mean it. Their inherent self interest (against current incumbents in power) is in fact in line with more of the populace.
But once in power, like Robespierre, they again do what behooves when: Which generally involves the consolidation of power.
As players ascend in capability, it will become harder and harder to stay on the far right of them quadrant – as new upstarts rally under the “virtuous little guy” banner – and new waves of people believe the “virtuous little guys” are somehow not driven by the same ravenous self-interest that defines the modus operandi of individuals and organizations.
Header image credit: Study.com
ByDan
During the Second Opium War, in 1860, a combined force of French and British troops trampled Old Summer Palace (known in Chinese as Yuanming Yuan 圆明园; 圓明園) on their way…
