#259 – Julian Gewirtz on the AI risks that actually scare Beijing

What would it take for Beijing to take AI risk seriously, and what could a US–China deal on AI realistically look like?

Julian Gewirtz, historian and former senior director for China and Taiwan affairs at the National Security Council under President Biden, argues that any answer has to “start with the reality of what China is and what the Chinese Communist Party cares about.”

The Party’s top concern is regime security, so when Chinese discourse warns of “extreme loss-of-control risks,” it means losing political control as well as losing control of AI systems. In Julian’s view, what is most likely to shift Beijing’s approach to AI safety is China’s “own domestic sense of the risks.” Next is seeing the US regulate AI at home: a “costly signal” that Washington takes the risks seriously.

The US looms large in that calculus. Julian says Anthropic’s Claude Mythos has been the most significant event shaping Beijing’s thinking on AI risk over the past year or so, sharpening its longstanding concern about American cyber capabilities.

On a deal itself, Julian is a realist. Designing an ideal agreement in the abstract is where “the impossibility confronts you at every turn.” He would rather start where “self-interest can lead to action in parallel” — such as neither country wanting to lose control of AI systems — and build incrementally from there.

He is also candid about how little outsiders can see. Many Chinese AI companies are AGI-pilled; the government has been “much less oriented that way,” though Julian thinks “even that is changing.” He expects the clearest evidence of a shift to be material rather than rhetorical, such as centralising compute or buying more of the Nvidia chips Beijing has so far rationed.

Julian and host Tom Reed also discuss why terminology would be an early sticking point in US–China AI talks, whether China would disclose its own Hugging Face incident, whether Beijing is reconsidering open-weight models, which Chinese thinkers are worth reading even when you disagree with them, and how to judge China expertise.

This episode was recorded on October 2, 2026.

Julian Gewirtz, a senior research scholar at Columbia University’s School of International and Public Affairs who served as senior director for China and Taiwan affairs on President Biden’s National Security Council, argues that the most important thing that could change Beijing’s approach to AI safety is China’s own domestic sense of the risks.

What could change Beijing’s approach to AI safety?

  • The US actually regulating AI at home: a costly signal that Washington takes the risks seriously. (Julian thinks this is second only to China’s own risk perception.)
  • Messaging from any one CEO matters far less, in his view. Beijing treats US AI companies as one project of American power, and Chinese state media has called Anthropic “a monster bred by the American path.”
  • Domestic regulation need not mean losing the race: China already regulates AI-generated content labelling and chatbots for minors, and the US still worries about it catching up.

For China’s leaders, AI loss of control includes losing political control

Regime security, which Beijing calls political security and social stability, is the Chinese Communist Party’s top concern, Julian argues. When Chinese discourse warns of extreme loss-of-control risks, it means losing political control linked to AI, not only losing human control over AI systems. A very close second, on his reading, is the US dimension: Anthropic’s Claude Mythos has been the most significant event shaping Beijing’s thinking on AI risk over the past year or so, sharpening its fear of American cyber capabilities.

Would China disclose a serious AI incident to the public?

Public disclosure would be very unlikely, according to Julian. Beijing would treat a Chinese equivalent of the OpenAI Hugging Face incident as a cybersecurity incident first, and at every layer of the system the incentives run against disclosure. Though Julian is cynical about American companies too, he says some of them have disclosed incidents: a very low bar, but one Chinese companies have not cleared.

Which signs would show that Beijing’s thinking on AI is shifting?

  • Many Chinese companies are AGI-pilled, Julian says; the government has been much less so, though he thinks that is changing. He expects the leading indicators to be material rather than speeches: centralising compute, and buying more of the Nvidia chips the Trump administration has approved but Beijing has been rationing.
  • A discourse on the risks of open-weight models is emerging in China, Julian noted when the episode was recorded on 2 October 2026. Beijing’s new AI Safety Governance Framework 3.0 warns that open-source releases lower the barriers to sophisticated cyberattacks.

Where US–China AI cooperation could realistically start

  • The deal the Chinese Communist Party would find most appealing is unilateral US restraint, Julian argues. Even his low expectations for the Trump–Xi meeting were not met: it did not produce a technical working group.
  • If talks begin, he expects the first sticking point to be terminology. In Chinese, safety and security are the same word, anquan.
  • Rather than designing an ideal grand bargain, he proposes starting where self-interest leads both countries to act in parallel, such as neither wanting to lose control of AI systems.

This summary was drafted with AI from the episode transcript and reviewed by the 80,000 Hours team.

Our production team includes:

  • Video editors: Josh Alward, Dominic Armstrong, Jasper Luithlen, Milo McGuire, Luke Monsour, and Simon Monsour
  • Producers: Elizabeth Cox and Nick Stockton
  • Coordination and support: Katy Moore and Lou Moran
  • Music: CORBIT

Articles, books, and other media discussed in the show

Julian’s work:

Recent developments:

History and background:

80,000 Hours career reviews:

Other 80,000 Hours podcast episodes:

Related episodes

About the show

The 80,000 Hours Podcast features unusually in-depth conversations about the world's most pressing problems and how you can use your career to solve them. We invite guests pursuing a wide range of career paths — from academics and activists to entrepreneurs and policymakers — to analyse the case for and against working on different issues and which approaches are best for solving them.

Get in touch with feedback or guest suggestions by emailing [email protected].

Our crash course on transformative AI

We've carefully selected 10 key episodes to help listeners get to grips with the potential upsides and downsides of powerful, transformative AI.

Check out 'The 80,000 Hours Podcast on AI'

Listen here, or anywhere you get podcasts:

If you're new, see the podcast homepage for ideas on where to start, or browse our full episode archive.