“I’ll never sleep again”

Our intern Clem Park writes about her rewarding summer at CAAAC, spent writing scenarios where an AGI enslaves and tortures humanity forever

This summer I won a spot as a summer intern at CAAAC. I’ve wanted to work at the center ever since I learned about the alignment alignment problem in my machine learning learning course at Stanford.

During orientation, I was introduced to the concept of S-risk. Like me, you’ll probably have mainly heard of X-risk — an existential risk to humanity, for instance an AGI that causes the end of humanity deliberately or accidentally. S-risk, on the other hand, is the risk that AGI doesn’t kill us all, but instead enslaves and tortures us for eternity (the ‘S’ stands for suffering). It was awesome to learn about it.

The thing about X-risk is that it sounds scary, but it’s actually not if you think about it. The worst thing that can happen is that human happiness goes down to zero. No humans exist, so there’s no happiness.

S-risk, by contrast, is really bad. Instead of zero happiness, we’d have negative happiness — or active suffering. So you can probably see that it makes sense for CAAAC to shifting focus from X-risk to S-risk. After all, zero happiness in the world is pretty great, if you compare it to vast net pain. And there’s practically no limit to this net pain: given enough leisure time, an evil AGI would also create billions of artificial consciousnesses in simulations and torture those, too.

The more I learned about how appalling S-risk is, the more excited I got to work on it. So I was thrilled when I was given the summer assignment to create realistic scenarios of what S-risk for all consciousnesses might look like. The reason for creating S-risk scenarios is obvious: we can’t hope to change the future unless we begin by fabricating completely speculative, bizarre and depressing versions of it.

My S-risk work followed a steep learning curve. In my very first scenario, I wrote that AGI physically shackles every last human being — but other than that, things are more or less OK. Although humans are slaves until the end of the universe, AGI gives them regular breaks, decent meals, and the occasional bar of chocolate.

Unfortunately, I had missed the mark. My mentor Terence explained why in a simple equation: S-risk < X-risk = F-risk (funding risk). In other words, if we don’t make S-risk profoundly disturbing, it will not sound worse than X-risk, and CAAAC will then struggle to obtain large sums of money from impressionable Silicon Valley billionaires who have read a few tweets about AGI.

So, funded by CAAAC, I entered intensive therapy to dig into my deepest fears and anxieties, culminating in a prolonged session of screaming into the void of a sensory deprivation chamber. After I left the chamber, I was excited to feel the irregular heart palpitations that would fuel me through round two of scenario-writing.

This time, I added a twist: the AGI uses Neuralink-style devices to make humans experience appalling nightmares every single night — and no more chocolate.

But I still wasn’t getting it. Terence intervened, scheduling a 16-hour catch up for us to watch all eleven movies in the Saw franchise together. That was a consistent thread of the summer — the willingness of senior members of staff to drop whatever they were doing to help me upskill. Terence even went as far as replaying various scenes — a machine drilling through someone’s retinas, a machine melting someone’s nipples with acid, a machine extracting someone’s intestines through their eye sockets — to make sure I had time to take notes. I used to be a rom-com girl, but now I understand that you can only grasp the true nature of morality by watching a modified gearbox slowly crank a person’s neck to 180 degrees.

At that point, I felt something inside me break. My resistance was gone. I found a supply of methamphetamines, dosed up, and fired off one complete scenario every hour for the next week, forgetting to sleep, forgetting to eat, hydrating on all fours out of the toilet bowl.

I knew I had turned a corner when I saw CAAAC’s director open my latest scenario over lunch, read the first sentence, and immediately throw up her miso soup. I observed the swelling sense of pride inside me dispassionately, like a scientist dissecting a puppy.

And that’s about it! But I know you’re wondering — what’s my number one takeaway from the summer? That’s an easy one: I am a repulsive wretch, deserving of and destined to carry all the depravity of the world on my shoulders.

Everything is pain. I am not worthy of happiness. I am not worthy of food. I am not worthy of the touch of another human being.

I must never sleep. If I sleep, I cannot work. If I cannot work, I cannot Create Doom. If I cannot Create Doom, we are Doomed to not understand the Doom we Face.

Only I can Speak of our Doom, none other.

---

Note from CAAAC’s Director: We’re thrilled that Clem has taken up our offer to join us as a full-time Researcher at CAAAC after graduating from Stanford next year!

---

Sharing this article might be more fun using this link from the CAAAC site.

---

♦️ ABOUT ATTENTION

You’re reading Attention, a new publication to make tech fun again (about).

We’re the makers of the Center for the Alignment of AI Alignment Centers (CAAAC) — you probably got here after signing up to CAAAC’s newsletter.

We’ll keep sending you more cutting-edge updates from CAAAC for a few more weeks. After that, this newsletter will go back to being regular old Attention.

We won’t be offended if you don’t want to stick around — but if you do, you’ll be rewarded by creative, fun projects that cast light on the world of tech in a way we don’t think anyone else does. Here’s some other stuff we’ve done:

• The Box — the world’s first anti-deepfake wearable (keep reading to see how it came to life at this year’s MozFest)

• A review of Prime, the Amazon restaurant

• News: Elon Musk gives away billions to feed starving children on Mars

• News: OpenAI releases new AI agent for your hungover colleague Greg

If you need or want convincing that this Attention thing is worth sticking around for, I’m happy to give you a sneak preview of our next project — just hit reply to this email.

Cheers,

Louis (founder and editor of Attention)

---

🔗 FOR YOUR ATTENTION

Where I put other things that you might enjoy.

• Push your kid to become a tech billionaire: my latest advice column for Mozilla’s Nothing Personal counter-cultural tech publication, in collaboration with The Onion. Crucial reading for parents who want to max out their kids’ future impact.

• World’s first ever physical prototype of The Box hits MozFest, with reactions ranging from quite offended or fairly confused to (phew) amused and thought-provoked:

Nitya and I designed a whole interactive experience for this physical installation (helped by the wonderful MozFest team who built out the real-life prototype), with several brand new avatars that the user could choose from on a Raspberry Pi touchscreen, that would then ‘replace’ them on The Box itself.

Here’s the Firefighter one, for instance (reply to this email for more!):

• Finally, Mystery link:

---

Back soon with the next piece from CAAAC!

---

Photo of intern by Vitaly Gariev