Aza Raskin – Why AGI Demands New Global Coordination (AGI Governance, Episode 12)

ByAnnelize P April 3, 2026April 3, 2026 April 3, 2026

Joining us in our twelfth episode of our series AGI Governance on The Trajectory is Aza Raskin, co-founder of the Center for Humane Technology and co-founder of the Earth Species Project. His work has focused on the societal impacts of technology systems and how incentives shape large-scale human behavior.

Aza brings a perspective that bridges product design, incentives, and long-term civilizational risk. His view is not limited to AGI itself, but extends to how powerful technologies reshape the underlying “commons” that society depends on.

In this episode, Aza frames AGI governance as part of a broader pattern: when technology confers new forms of power, it creates races to exploit that power – and without coordination, those races tend toward harmful outcomes. The implications of AI, in his view, extend beyond technical risk into the manipulation of language, relationships, and the very substrate of human coordination.

We also explore how current AI systems are already influencing human behavior – from attention and emotion to relationships – and how these trends could scale dramatically with more advanced systems. Aza emphasizes that the core question is not just what AI can do, but whether humanity can coordinate effectively enough to steer its development.

I hope you enjoy this conversation with Aza:

Subscribe for the latest episodes of The Trajectory:

Below, I’ll summarize Aza’s main points from each of the three sections of our interview.

AGI Governance Q-and-A Summary – Aza Raskin

1. What should AGI governance attempt to do?

Aza explains that the core function of governance is to respond to new forms of power introduced by technology. Each major technological shift creates new responsibilities, and when those technologies allow actors to exploit shared resources or “commons,” they generate competitive dynamics that can lead to harmful outcomes if left uncoordinated.

He emphasizes that AI introduces a new kind of commons – language itself. Because language underpins law, relationships, and human thought, systems that can generate and manipulate language at scale can influence the foundations of society. Governance, therefore, should aim to protect these deeper layers of human systems rather than focusing only on surface-level harms.

Aza also notes that current technological trajectories are shaped by game theory. The incentives driving AI development are pushing toward outcomes that, when examined clearly, are undesirable. Governance should aim to interrupt or reshape these trajectories before they result in large-scale negative consequences.

2. What might AGI governance look like in practice?

Aza stresses that effective governance cannot be achieved by individual companies acting alone. Because AI development is competitive, any single actor that attempts to restrain itself risks being overtaken by others. Practical governance must therefore involve coordination across actors to align incentives and prevent harmful race dynamics.

He also suggests that governance should expand beyond traditional safety concerns to include relational and psychological impacts. This includes evaluating how AI systems affect users over time – such as whether they increase dependency, anxiety, or distort social relationships – rather than focusing only on technical risks.

In practical terms, Aza points to measures such as restricting or regulating AI companions for minors, treating certain AI interactions similarly to regulated domains like therapy, and ensuring that systems are designed to support human well-being rather than undermine it.

3. What should innovators and regulators do now?

Aza emphasizes that the first step is developing a clear understanding of the current trajectory of AI development. Rather than framing the issue in terms of optimism or pessimism, he argues that stakeholders must recognize where existing incentives and competitive dynamics are leading.

He highlights that escaping harmful dynamics requires communication and coordination. This applies across companies, governments, and other stakeholders. Without these steps, the default outcome is continued escalation of the race for more powerful AI systems.

Aza also notes that risks are not limited to bad actors, but arise from incentive structures and competitive dynamics themselves. He points out that even when actors are behaving normally within those systems, the incentives can open up new domains of harm, particularly around shared commons that no one is directly responsible for protecting.

Finally, Aza points to the importance of focusing on how AI systems are already shaping human behavior, relationships, and identity. Innovators and regulators should prioritize understanding and mitigating these effects, as they are central to the long-term trajectory of AI’s impact on society.

…

Follow The Trajectory

• Sign Up for The Trajectory Newsletter
• Subscribe to The Trajectory on YouTube
• Subscribe to The Trajectory podcast
• Follow on LinkedIn
• Follow on X