The Death of Sardanapalus and the Birth of AGI

Locked away in his palace, fated to death after losing a war against rebelling factions, Sardanapalus orders a funeral pyre constructed, and piles up his precious metals, royal clothing, and his concubines and eunuchs – and ascending the pyre himself – is burned alive along with it all.

Sardanapalus is painted as a singularly flawed, spoiled, and evil man, but his actions echo the striving of men everywhere: To go out on ones own terms – to carry one’s pride beyond their own death.

This prideful urge has lead men* to some dastardly things:

• To commit suicide when their pride is hurt sufficiently.
• To take out their perceived sense of being disrespected (hurt pride) by killing as many people as possible in a blaze of glory. Think Elliot Rodger, the Columbine killers, etc.

…but it has other manifestations as well:

• Over 290,000 American men died in combat in WW2 – many of whom presumably did so openly for the honor (pride) of their comrades, nation, or wives. Could they imagine saying “no” to battle, and facing their leaders or families as a coward?
• Houdini risked death on many occasions for the sake of his fame, and to push the boundaries of showmanship – ultimately dying from the after effects of one of his dangerous demonstrations. World famous tightrope walkers have died in the pursuit of setting records. What if someone else had bested their stunts, or if they became nothing but a has-been?
• The Wright brothers risked death a hundred times over the course of their experiments. Plenty of other inventors of flying machines or submarines died risking their lives to create something dangerous and new. Could they resist the urge to create? Or worse: live to see someone else claim the prize?
• Explorers from the South Pole to Mount Everest to the Amazon jungle have died in pursuit of a new place, a new discovery, achieving a world first – or simply doing something brave and bold.

For reasons good, bad, or otherwise – for some men, physical death is often preferable to an ego death. Winning the race is worth all the risks that winning entails.

I’m not saying that I like this element of human nature – or that you should like it. It just is.

The Sardanapalus Urge: When faced with being vanquished by a foe, many men would prefer to go out by their own hand, in a blaze of glory that showcases their power and might – and seems to them to turn their defeat into a kind of manful, final victory. An imposition of their own power, not a succumbing to anyone else’s.

If racing towards a flying machine or a scientific discovery is worth risking death – imagine what’s it worth to bring a post-human intelligence to life. The incentives are too great to resist.

Whoever creates AGI will wield massive economic power (a machine that can do anything is pretty valuable), and massive physical power (the military with AGI is almost certainly secured the position of the strongest).

But it seems somewhat obvious that AGI won’t be controllable for long – if it’s controllable at all.

This might involve the machines:

• Taking over the world
• Pursuing their own goals and neglecting human needs (like we treat ants)
• Keeping humans as pets
• Relegating us to little powerless reservations, or
• [Gasp] the outright extermination of humanity

Would these risks halt the founders from progressing with AGI?

Surely the threat of their own deaths, and the end of humanity itself, would deter these ambitious men from putting the petal to the metal on AGI innovation?

Of course not.

AGI founders have the incentive to go petal to the metal. Pride above safety. As it has always been with ambitious men. Call it a vice if you want.

The only true efforts towards “safety” will come from two groups:

• Cunning rival groups who would like to slow down AGI creators long enough to either beat them to it (winning power through innovation), or find a way to control the AGI company’s creation through policy or governance (winning power through regulation).
• Groups who believe that their incentives lie in preventing post-human intelligence (such as: some religious groups, luddite political parties, etc).

Both these groups act by the same force driving today’s AGI company founders: Self-interest.

Incentives rule the world.

From AGI company founders – expect lip service to safety, a safe preening of virtue and benevolence. Expect the delicate tactful art of making one’s power more palatable or less threatening to the powerless. Do not blame them for it. Power is a hard game.

If you were in their shoes, you’d have to do the same. Your own selfishness wouldn’t let you admit it publicly, but you would.

If your pride and power were tied to your technical prowess… if you faced a meaningless existence, born to die (the condition we all share)… and you had the opportunity to birth a deity that might bloom beyond humanity and populate the galaxy… maybe discover an escape from the eventual heat death of the known universe, you would do the same.

If there is a kind of poetic and prideful appeal in dying to create an invention, or defending a nation, or reaching a new mountain top – that appeal is a million times greater for creating a superintelligence. It would be the highest human achievement – arguably the last human achievement.

Even if humanity came to an end from such an event – think of that final poetic victory for the founder – the final act of power, of dealing with their existential condition in a way that is on their terms. Imagine the motive to race to that finish line when the alternative might mean living their final moment being devoured by the AGI of their rival.

Like Sandanapalus, but a thousand times loftier.

If you were in their shoes, you would do the same.

But you’re probably not smart or ambitious enough to be in their shoes (it’s okay, neither am I). Don’t go to bed thinking you are, and thinking that their position comes from mere luck or vice. Those are whining pleas to protect your own pride. You have no time for that, and I have no patience for it.

Remember only this:

• The incentives of the founders of these firms will drive them to rush towards AGI, whether you like it or not. If you want to prevent the creation of AGI (either permanently, temporarily, or long enough for to gain enough influence to control it yourself), you’ll have to join one of those rival forces – the tech-controlling regulators, a luddite political group, or something else.

Just don’t pretend you’re doing it because you’re selfless.

The Death of Sardanapalus – T.J. Wilkins, 1875

.

Think I would hang my head, ye fools?
Think open I this golden gate?
Conquered, my will could never be,
think ye I would not sneer at fate?

.

See you this gilded castle high?
See you these soft and slender maids?
You think the glory I have won,
you’d deign touch with hands or blades?

.

My gaze commands the fire and steel.
My port, an emperor’s remains.
My soul does tower over thine,
my will, of this defeat disdains.

.

Ye call this evil, trifling fools?
Ye call this spite, and weakened will?
Or do thee cry to see now how,
my lofty soul conquers ye still?

.

‘Tis I who plunder, capture all.
‘Tis I make pretty heads to roll.
This final, crimson work of art,
makes greater still my lofty soul.

.

Now look – and tremble at my strength!
Now look – before I’m choked in flame!
By this here mound of gore and gold,
I force the world to know my name!