In 2025, it is no longer intellectually honest to completely shun the idea of artificial general intelligence (AGI) or AGI risk.
Yet still, in Dec 2024 (the time of this writing), I hear statements like:
“Pfft, everyone knows AGI isn’t realistic, AI systems will never have agency. There will never be a time when they don’t simply do what humans tell them to do, period.”
“Pfft, if you really understood the science, you’d stop talking about AGI bullshit. It’s just math, there is no hard takeoff for AGI, it’s never going to get that powerful, it’s impossible.”
“Everyone knows that scaling these current systems won’t get to AGI. And the number of new breakthroughs we would need to get there might be thousands of years away.”
People make these statements not as hypotheses, not as opinions, but as statements of fact.
In 2015, there were a handful of thinkers throwing around ideas about AGI, and most of the eminent scientists in the field didn’t think it was even a remote possibility within their lifetimes.
Back in 2015, shooting down someone talking about AGI as “just a confused person, or a stupid person, who doesn’t understand the science” might have been reasonable. After all, it wasn’t mainstream, and the major scientists in the field didn’t really believe in it.
But today things are different:
The famed godfathers of machine learning, Bengio and Hinton, are both vocal about artificial general intelligence being a near-term risk for humanity.
Essentially all of the major lab leaders have signed the Safe AI Statement on AI Risk (Altman, Hassabis, others), and essentially all of them have given lengthy interviews about the real potential of creating something beyond human control.
Many of the most valuable companies in the world are, unabashedly, racing towards posthuman intelligence (Microsoft, Google, Meta), and the most valuable company in the history of the world at the time of this writing is NVIDIA, an AI hardware firm.
The United States is proposing a Manhattan Project for AGI in order to beat China (which is engaged in its own race towards AGI). The two great powers are overtly racing against each other to create posthuman intelligence.
The Secretary General of the United Nations has stated many times that he believes loss of AI control / AGI to be a serious risk to humanity.
Very real strategies for AGI race dominance (Situational Awareness) and AGI race deterrence (Compendium / etc) are circulating with increased popularity by the month, often by credible scientists close to the field.
Could all these signals be misguided?
Could Bengio, Hinton (and now even LeCun!) be mistaken?
Could the military leadership of the US and China be wholly mistaken in believing that AGI is achievable in the coming decade (or even two decades)?
Yes, they could all be mistaken.
Every one of them. The researchers, the governments, the AGI labs themselves – all could be set up to fail continuously at achieving AGI for hundreds of years – maybe forever.
I’m not saying: “They’re definitely right that AGI is a near-term possibility.”
But I will say two things:
It is now intellectually dishonest to scoff at artificial general intelligence (AGI) as a near-term possibility.
Shutting down conversation about something as impactful as AGI is irresponsible – frank and open dialogue about growing AI capabilities is necessary in order to navigate the future of humanity.
In 2025, saying “I think AGI won’t happen anytime soon because XYZ” seems like a reasonable position to hold. A good start to an honest debate.
Saying “AGI will obviously never happen, and this is stupid to talk about” is flat-out ignoring the trends in the field and the opinions of its most eminent thinkers.
There is much to debate and discuss, and there are trends and breakthroughs and technical/economic variables that will impact the possibility of AGI, and we should discuss them all openly, and suss out how AI might develop, and how it might impact humanity.
The days of honestly cutting off talk of AGI as “ridiculous” are no more.
Dishonest shutdowns of this discussion will only do harm, and frank dialogue and open debate give us the best chance of arriving at the best future (AGI or not).
Episode 1 of The Trajectory is with none other than the Turing Award winner, MILA Scientific Director, ML godfather, and (very recently) AI safety advocate, Yoshua Bengio. My first interview…
Episode 2 of The Trajectory is with the Co-Founder of Skype, a prolific funder of AI safety research, and member of the United Nations Secretary General’s AI Council – Jaan…
Episode 3 of The Trajectory is with the CEO of SingularityNET, and AGI researcher for many decades, Ben Goertzel. The interview is Episode 3 in The Trajectory’s first series AGI…
Episode 5 of The Trajectory features the Executive Director and Co-founder of the Center for AI Safety, Dan Hendrycks. The interview is our fifth and final installment in The Trajectory’s…
Sebastien Krier of Google DeepMind joins us in our first episode in a brand new AGI Governance series on The Trajectory. Beginning his career studying law at King’s College, Sebastien…
Joining us in the third episode of our series AGI Governance on The Trajectory is Stephen Ibaraki, Founder of the UN ITU AI for Good, and Chairman at REDDS Capital….
Joining us in the fifth episode of our series AGI Governance on The Trajectory is Connor Leahy, Founder and CEO of Conjecture. This is the most in-depth take I’ve ever…
The sixth and final episode of our series AGI Governance on The Trajectory is with Eliezer Yudkowsky, famed AI safety thinker and Co-founder at the Machine Intelligence Research Institute. Eliezer…
Joining us in our seventh episode of our series AGI Governance on The Trajectory is Toby Ord, Senior Researcher at Oxford University’s AI Governance Initiative and author of The Precipice:…