Why Orwell would hate AI

The idea this week: totalitarian regimes killed over 100 million people in less than 100 years — and in the future they could be far worse.

That’s because advanced artificial intelligence may prove very useful for dictators. They could use it to surveil their population, secure their grip on power, and entrench their rule, perhaps indefinitely.

I explore this possibility in my new article for 80,000 Hours on the risk of stable totalitarianism.

This is a serious risk. Many of the worst crimes in history, from the Holocaust to the Cambodian Genocide, have been perpetrated by totalitarian regimes. When megalomaniacal dictators decide massive sacrifices are justified to pursue national or personal glory, the results are often catastrophic.

However, even the most successful totalitarian regimes rarely survive more than a few decades. They tend to be brought down by internal resistance, war, or the succession problem — the possibility for sociopolitical change, including liberalisation, after a dictator’s death.

But that could all be upended if technological advancements help dictators overcome these challenges.

In the new article, I address:

• Whether totalitarianism constitutes an existential risk
• How a totalitarian regime could come to dominate the world
• What it would take for such a regime to last forever
• If any of this is even remotely plausible

To be sure, stable totalitarianism doesn’t seem to be the most likely course for the future. An aspiring permanent dictator would face formidable barriers, and — while AI could help them — other actors may use AI to the world’s benefit. I think that the chance of stable totalitarianism really coming to pass is much less than 1%. (Others have estimated as high as 5%.)

But I think it’s less far-fetched than it might seem at first. Totalitarian regimes have been common throughout history, they often expand and entrench their influence, and new technology may well centralise power in ways that help dictators.

Despite the risk being low, it seems worrying enough that 80,000 Hourswe include stable totalitarianism in their list of the world’s most pressing problems.

The good news, I think, is that there are things we can do to help make this outcome even less likely — and these things also have the benefit of helping with other issues. At the end of the article, I discuss four ways you might work on this problem:

• Working on AI governance to ensure no country can use advanced AI systems to dominate the rest of the world
• Researching how states can coordinate to address global risks without centralising power in ways that are easy for dictators to subvert
• Working on defensive technologies that could prevent dictators’ attempts to control people
• Working to protect democratic institutions to reduce the chance that more democracies become autocracies in coming decades

There’s much more to be found in the full article, including the bizarre story behind why the leader of Kazakhstan invested millions of his nation’s tax dollars to develop a probiotic yogurt drink.

And if any of the options I mentioned for working on this problem sound interesting to you, check out our articles on political skills, research skills, and AI governance career paths.

Learn more:

• Our problem profile on preventing an AI-related catastrophe
• Our problem profile on great power war