To make AI safe, we must develop it as fast as possible without safeguards
Ia Magenius explains why we need to make AI as powerful as possible to ensure it can't have power over us
We hear a lot in the media about how AI companies should stop and think before forging ahead with new and powerful models. According to this argument, we should move cautiously and develop regulatory safeguards to manage the potential risks.
I get it. I myself once endorsed a six-month moratorium on AI research (remember that?) to give us time to think through the implications of what we were doing.
But I've now realised that attempting to develop AI safely is the most dangerous thing we could do.
History shows us why.
Take nuclear weapons. Once it was clear that an atomic bomb was possible, the US government rushed to develop one. If their scientists had waited to develop it safely, they might never have been able to peacefully end WW2 by killing over 150,000 innocent Japanese people in Hiroshima and Nagasaki.
By pursuing aggressive technological expansion at any cost, the Manhattan Project led to a safer world. Not only that: the subsequent nuclear arms race meant that, within just a few years, humanity possessed thousands of nuclear weapons that could blow us all up many times over. But once you discount the many near-misses that have almost led to planetary-scale nuclear war and the end of all human life, it's clear that nuclear weapons are actually one of the safest technologies around.
The lesson is clear. To their critics nuclear weapons were a dangerous technology, but in practice pursuing their development made the world significantly safer, apart from all the wars that still seem to happen for complicated reasons I don't have space to discuss here.
The risk with AGI is much the same. As the leader of an AI company which stands to benefit enormously if I convince enough investors that AGI is inevitable, it’s clear to me that AGI is inevitable. But developing superintelligence safely is a complex process. It would take time and require difficult discussions — discussions that everyone in society should have a say in, not just the small number of researchers working on it. If we pursue that path, there's a real risk that somebody else will make AGI first and destroy all human life before we have a chance to ourselves. That would be unacceptable.
To stop bad actors developing AGI that could kill us all, we need good actors to develop AGI that could also kill us all.
I've come to realise that our best hope is to race at breakneck speed towards this terrifying, thrilling goal, removing any safeguards that risk slowing our progress. Once we've unleashed the technology's full destructive power, we can then adopt a "stable door" approach to its regulation and control — after all, that approach has worked beautifully for previous technologies, from fossil fuels to microplastics.
We stand at a precipice. If we get this right, the lesson of nuclear weapons suggests that we'll be able to create a sufficient number of different AGIs such that they'll hold each other in check and prevent any one of them from destroying humanity.
If not, we'll all die and AGIs will take our place. But how cool would that be?
Sharing this article might be better using this link from the CAAAC site. This article was written by CAAAC’s anonymous co-founder (no, not actually by Ia Magenius).
👋 (RE?)INTRODUCING ATTENTION!
Hi! Chances are you signed up to the mailing list for the Center for the Alignment of AI Alignment Centers and are now wondering what the heck this Attention! thing is.
Simply put, Attention! is the publication behind CAAAC. Here’s more about us, here’s our website, and here’s a previous project of ours, The Box, which we’re excited to announce will come to life later this year at MozFest in Barcelona.
You’re currently reading Attention’s newsletter, which we’ve temporarily rebranded as CAAAC’s newsletter for a few weeks as we release critically important updates and reports from the CAAAC universe.
After that, this newsletter will go back to being regular old Attention. We won’t be offended if you don’t want to stick around — but if you do, you’ll be rewarded by our attempts to Make Tech Fun Again through satire and creative coding.
✏️ WRITE/CODE WITH US
Want to write/code something fun about AI, or tech in general? We’d love to chat. Email hi [at] louis [dot] work.
🎥 DIRECTING ATTENTION
This is Directing Attention, where we go behind-the-scenes at Attention HQ.
It’s official! CAAAC is Attention’s biggest hit so far, eclipsing The Box. The Center struck a chord on LinkedIn, Bluesky and X, leading to nearly 40k pageviews (compared with 7k for The Box), hundreds of subscribers — probably including you — and a very cool write-up from The Verge.
CAAAC was reposted by folks like Timnit Gebru, Emily Bender and Yann LeCun, one of the three so-called ‘godfathers of AI’.
As far as I can tell, that all happened in large part because of this post by Kendra Albert which led to CAAAC going viral. Thank you so much, Kendra!
It’s hard to describe quite how creatively fulfilling it was seeing people sharing their favourite parts of CAAAC on social media — spotting easter eggs like the word spelled out by the animation on the front page, or giving their Freudian interpretations of the diagram here…get a shrink, people! Dozens of others proudly posted or emailed in the business cards they made with our cutting-edge CenterGen-4o tool (still plenty of time to become an Executive Director yourself — scroll down here to ‘Start your own AI center in 60 seconds’).
And apparently CAAAC was sent around practically all real AI alignment organizations too, so for those folks reading this: hi, and thank you for taking the joke well! At last inbox refresh I still hadn’t been threatened with a defamation lawsuit but hey, these things take time to pull together. There’s still hope.
Back soon with more cutting-edge research and updates from CAAAC…and finally, an extra thank you to the anonymous co-founder of CAAAC, who wrote the Ia Magenius article above and without whom CAAAC wouldn’t have been possible.