#259 – Julian Gewirtz on the AI risks that actually scare Beijing
- Introduction
- 1 Articles, books, and other media discussed in the show
- 2 Transcript
- 2.1 Julian Gewirtz on the realities of AI risk in China [00:00:00]
- 2.2 What does Xi think of Dario? [00:00:41]
- 2.3 Would we ever know about a Chinese Hugging Face hack? [00:08:27]
- 2.4 How Beijing views distillation [00:18:56]
- 2.5 How do Chinese labs really view AI risk? [00:23:18]
- 2.6 Could Xi Jinping ever be AGI-pilled? [00:28:12]
- 2.7 What's really shaping the CCP's views on AI? [00:33:25]
- 2.8 Does anyone in Beijing believe in RSI? [00:38:32]
- 2.9 Chinese labs hate red tape, too [00:45:47]
- 2.10 Would China invade Taiwan over AI? [00:51:04]
- 2.11 The prisoner's dilemma of a US–China AI deal [00:56:51]
- 2.12 How China stumbled into open source [01:07:12]
- 2.13 How to build a realistic deal, starting now [01:13:04]
- 2.14 How to spot a fake China expert [01:20:48]
- 2.15 Julian's China and AI reading list [01:28:43]
- 2.16 The Chinese thinkers worth reading, even when you disagree [01:33:34]
- 2.17 Which AI model is the best China watcher? [01:39:27]
- 2.18 A world tour of China's influence [01:42:59]
- 3 Learn more
- 4 Related episodes
What would it take for Beijing to take AI risk seriously, and what could a US–China deal on AI realistically look like?
Julian Gewirtz, historian and former senior director for China and Taiwan affairs at the National Security Council under President Biden, argues that any answer has to “start with the reality of what China is and what the Chinese Communist Party cares about.”
The Party’s top concern is regime security, so when Chinese discourse warns of “extreme loss-of-control risks,” it means losing political control as well as losing control of AI systems. In Julian’s view, what is most likely to shift Beijing’s approach to AI safety is China’s “own domestic sense of the risks.” Next is seeing the US regulate AI at home: a “costly signal” that Washington takes the risks seriously.
The US looms large in that calculus. Julian says Anthropic’s Claude Mythos has been the most significant event shaping Beijing’s thinking on AI risk over the past year or so, sharpening its longstanding concern about American cyber capabilities.
On a deal itself, Julian is a realist. Designing an ideal agreement in the abstract is where “the impossibility confronts you at every turn.” He would rather start where “self-interest can lead to action in parallel” — such as neither country wanting to lose control of AI systems — and build incrementally from there.
He is also candid about how little outsiders can see. Many Chinese AI companies are AGI-pilled; the government has been “much less oriented that way,” though Julian thinks “even that is changing.” He expects the clearest evidence of a shift to be material rather than rhetorical, such as centralising compute or buying more of the Nvidia chips Beijing has so far rationed.
Julian and host Tom Reed also discuss why terminology would be an early sticking point in US–China AI talks, whether China would disclose its own Hugging Face incident, whether Beijing is reconsidering open-weight models, which Chinese thinkers are worth reading even when you disagree with them, and how to judge China expertise.
This episode was recorded on October 2, 2026.
Julian Gewirtz, a senior research scholar at Columbia University’s School of International and Public Affairs who served as senior director for China and Taiwan affairs on President Biden’s National Security Council, argues that the most important thing that could change Beijing’s approach to AI safety is China’s own domestic sense of the risks.
What could change Beijing’s approach to AI safety?
- The US actually regulating AI at home: a costly signal that Washington takes the risks seriously. (Julian thinks this is second only to China’s own risk perception.)
- Messaging from any one CEO matters far less, in his view. Beijing treats US AI companies as one project of American power, and Chinese state media has called Anthropic “a monster bred by the American path.”
- Domestic regulation need not mean losing the race: China already regulates AI-generated content labelling and chatbots for minors, and the US still worries about it catching up.
For China’s leaders, AI loss of control includes losing political control
Regime security, which Beijing calls political security and social stability, is the Chinese Communist Party’s top concern, Julian argues. When Chinese discourse warns of extreme loss-of-control risks, it means losing political control linked to AI, not only losing human control over AI systems. A very close second, on his reading, is the US dimension: Anthropic’s Claude Mythos has been the most significant event shaping Beijing’s thinking on AI risk over the past year or so, sharpening its fear of American cyber capabilities.
Would China disclose a serious AI incident to the public?
Public disclosure would be very unlikely, according to Julian. Beijing would treat a Chinese equivalent of the OpenAI Hugging Face incident as a cybersecurity incident first, and at every layer of the system the incentives run against disclosure. Though Julian is cynical about American companies too, he says some of them have disclosed incidents: a very low bar, but one Chinese companies have not cleared.
Which signs would show that Beijing’s thinking on AI is shifting?
- Many Chinese companies are AGI-pilled, Julian says; the government has been much less so, though he thinks that is changing. He expects the leading indicators to be material rather than speeches: centralising compute, and buying more of the Nvidia chips the Trump administration has approved but Beijing has been rationing.
- A discourse on the risks of open-weight models is emerging in China, Julian noted when the episode was recorded on 2 October 2026. Beijing’s new AI Safety Governance Framework 3.0 warns that open-source releases lower the barriers to sophisticated cyberattacks.
Where US–China AI cooperation could realistically start
- The deal the Chinese Communist Party would find most appealing is unilateral US restraint, Julian argues. Even his low expectations for the Trump–Xi meeting were not met: it did not produce a technical working group.
- If talks begin, he expects the first sticking point to be terminology. In Chinese, safety and security are the same word, anquan.
- Rather than designing an ideal grand bargain, he proposes starting where self-interest leads both countries to act in parallel, such as neither wanting to lose control of AI systems.
This summary was drafted with AI from the episode transcript and reviewed by the 80,000 Hours team.
Our production team includes:
- Video editors: Josh Alward, Dominic Armstrong, Jasper Luithlen, Milo McGuire, Luke Monsour, and Simon Monsour
- Producers: Elizabeth Cox and Nick Stockton
- Coordination and support: Katy Moore and Lou Moran
- Music: CORBIT
Articles, books, and other media discussed in the show
Julian’s work:
- Paper Tiger — Julian’s Substack, including recent posts:
- How to make the US–China AI race less dangerous (with Matt Sheehan)
- The dream of AI primacy
- Can America avoid a Jack Ma moment? (with Dan Wang)
- Never turn back: China and the forbidden history of the 1980s
- The futurists of Beijing: Alvin Toffler, Zhao Ziyang, and China’s “New Technological Revolution,” 1979–1991
- Follow Julian on X
Recent developments:
- Trump’s AI order won’t stymie US competition with China by Matt Sheehan
- Yuyuan Tantian: Anthropic has caught America’s disease
- For China, a mock AI attack on WeChat signals a dangerous new era by Lily Kuo and Pei-Lin Wu
- What China’s new AI safety framework tells us before Xi meets Trump by Gabriel Wagner
- When Trump and Xi meet they will discuss AI. ‘Track Two’ talks are already buzzing by John Ruwitch
- How Trump should approach AI talks with China: Targeted dialogue, maximum pressure by Chris McGuire
- Inside China’s all-out push to catch up with American AI chips by Josh Chin and Raffaele Huang
- Opinions of the State Council on deepening the implementation of the “Artificial Intelligence+” initiative — translation by the Center for Security and Emerging Technology
- Full text: Keynote speech by Chinese President Xi Jinping at opening ceremony of 2026 World AI Conference
- Dario Amodei: We must pace the frontier and On DeepSeek and export controls
- Anthropic: Detecting and countering misuse of AI: September 2026
- Political censorship in large language models originating from China by Jennifer Pan and Xu Xu
- The questions the Chinese government doesn’t want DeepSeek AI to answer by Kyle Orland
History and background:
- AI 2041: Ten visions for our future by Kai-Fu Lee and Chen Qiufan
- The gilded cage: Technology, development, and state capitalism in China by Ya-Wen Lei
- The Party’s interests come first: The life of Xi Zhongxun, father of Xi Jinping by Joseph Torigian
- Lu Nanfeng and Wu Jing on the Industrial Party
80,000 Hours career reviews:
- Improving China-Western coordination on global catastrophic risks
- China-related AI safety and governance
Other 80,000 Hours podcast episodes:
Transcript
Table of Contents
- 1 Julian Gewirtz on the realities of AI risk in China [00:00:00]
- 2 What does Xi think of Dario? [00:00:41]
- 3 Would we ever know about a Chinese Hugging Face hack? [00:08:27]
- 4 How Beijing views distillation [00:18:56]
- 5 How do Chinese labs really view AI risk? [00:23:18]
- 6 Could Xi Jinping ever be AGI-pilled? [00:28:12]
- 7 What’s really shaping the CCP’s views on AI? [00:33:25]
- 8 Does anyone in Beijing believe in RSI? [00:38:32]
- 9 Chinese labs hate red tape, too [00:45:47]
- 10 Would China invade Taiwan over AI? [00:51:04]
- 11 The prisoner’s dilemma of a US–China AI deal [00:56:51]
- 12 How China stumbled into open source [01:07:12]
- 13 How to build a realistic deal, starting now [01:13:04]
- 14 How to spot a fake China expert [01:20:48]
- 15 Julian’s China and AI reading list [01:28:43]
- 16 The Chinese thinkers worth reading, even when you disagree [01:33:34]
- 17 Which AI model is the best China watcher? [01:39:27]
- 18 A world tour of China’s influence [01:42:59]
Julian Gewirtz on the realities of AI risk in China [00:00:00]
Julian Gewirtz: We have to start with the reality of what China is and what the Chinese Communist Party cares about, as opposed to kind of working backwards from the place we think they need to get in order to accomplish our broader goals.
The number one concern is regime security, regime stability. As I’ve seen the Chinese AI discourse evolve, there is increasing attention to what they call extreme loss-of-control risks.
The most significant event in shaping the evolution of Beijing’s thinking about AI risks over the course of the past year or so is Mythos.
What does Xi think of Dario? [00:00:41]
Tom Reed: Today I’m speaking with Julian Gewirtz. Julian is a historian, poet, former director for China affairs in the NSC [National Security Council] under Biden, and more recently also a blogger. Of all the things that I’ve learned from reading Julian’s work, perhaps my favourite is that much of the credit we all usually give to Deng Xiaoping for the reform of China’s economy should instead be redirected towards the comparatively uncelebrated former Chinese Premier Zhao Ziyang. Julian, thank you for coming on the show.
Julian Gewirtz: Thanks, Tom. It’s great to be with you.
Tom Reed: So, Julian, I have a bunch of not-so-hidden agendas for speaking to you. Above all, I want to understand: what on Earth do you think Xi Jinping’s model of Dario Amodei is? What do you think he thinks this guy wants?
Julian Gewirtz: Well, honestly, I don’t think we have to speculate, because to a stunning degree, Beijing has been on the record going after Anthropic. In the history of sort of US–China relations, it’s hard for me to think of another example of an American company that Beijing has taken on so repeatedly and so directly. There have been a series of state media commentaries about it. There was one ahead of Xi Jinping coming to the United States that I found really striking, and I wrote a little bit about it, but the headline was “Anthropic has contracted the American disease.”
Now, what does that mean? The core argument was basically that this is a company that says whatever, but is actually all about preserving so-called technological hegemony for the United States. And crucially, we often think about these different AI companies, the different model companies, as very differentiated actors in our system. We know that many of them don’t like each other and that they’re competing intensely, and all of those sort of things that define what it feels like to be in this space.
Beijing said Anthropic is literally “a monster bred by the American path” — saying, like, all of this, all of these different firms, this is all part of one big thing. And that big thing is the United States trying to dominate globally and trying to keep China behind. And they have been going at Anthropic hardest, but they don’t, I think, see it as somehow separate or outside of a mainstream in the United States, but very much a part of it.
And it’s just maybe worth driving home the upshot of that, which is that for Beijing, it appears that a lot of the distinctions or differences in views among American tech titans and American AI companies matter less than what they represent to China, which is the project of American power, in their eyes.
Tom Reed: Sometimes I get confused by Dario’s comms on China, because sometimes it sounds like he’s calling for we really need to cooperate with them; sometimes it sounds like what he’s calling for looks a little bit less like cooperation.
Dario, famously, I think he only has one direct report, which is his chief of staff. But let’s imagine you were his second-ever direct report, and he’d hired you specifically as his China comms person. What would you tell him to say? How should he be talking about China differently in a way that actually increases the chance of them taking his calls seriously, rather than as a call for slowing China down compared to the US?
Julian Gewirtz: Well, I guess, honestly, I don’t think it’s very likely that Beijing can somehow be disabused of the notion that the United States wants a significant lead in AI over China. And I don’t think that messaging from any particular executive is going to be the key thing there.
I mean, let’s just step back for a second and say we’re in a moment where the president of the United States has said, “No slowdown. AI risk is not something I want to take seriously.”
Tom Reed: “It’s a hoax.”
Julian Gewirtz: Sure. And where the secretary of the Treasury has said, “There’s no day after tomorrow if China wins at this.” To the point I was just making, I actually don’t think that they see the rhetoric or comms of one CEO, Dario or otherwise, as the thing that could shift their view.
My own strong view is that the most important thing that could cause Beijing to shift its current approach to AI safety is their own domestic sense of the risks. The second most important thing is if they were to see the United States actually doing AI regulation and taking AI safety seriously — doing basically what we would call a “costly signal,” the kind of thing that we’re doing because we care about these risks, and that also has the benefit of signalling to Beijing, like, “Hey, these guys are serious about this.” Those two things are far more important, I think, than a shift in comms.
I will say I do think that there is a difference between the United States wanting to compete and have a lead, and a narrative of total supremacy, permanent dominance. So the language, I think, that is least helpful toward getting Beijing there is the language that suggests that what the United States wants here is not just a lead, but total kind of what we would call “primacy.” And there is a distinction there, and I think it is relevant, but all part of a puzzle.
Tom Reed: So the idea being that a precondition for China taking any of the US’s apparent AI safety concerns seriously would be: actually regulate AI domestically. Otherwise it just looks like it’s all talk.
Julian Gewirtz: Yeah. And obviously that gets us right into the core question of tradeoffs around regulating at home to preserve our edge, and regulating at home with the risk of losing our edge over China. But there, my friend and collaborator Matt Sheehan has a really good point, which is that Beijing has itself done some AI regulation at home — not sort of frontier AI risk, but they have deepfake and AI-generated content labelling requirements. They have regulations on chatbots, particularly for minors. They’ve done some stuff there.
And of course we don’t see that as condemning them to be in second place necessarily. In fact, they’re doing all that and we’re still very worried about them catching up to us.
So I think there’s a bit of a fallacy embedded in the idea that any AI regulation is at odds—
Tom Reed: Going to lose us the whole race.
Julian Gewirtz: Exactly. Now, pacing is a separate question, because that is explicitly about the speed of frontier development. But regulation is a lot more than just pacing.
Would we ever know about a Chinese Hugging Face hack? [00:08:27]
Tom Reed: How do you think the Chinese government would react to a homegrown version of the Hugging Face incident?
Julian Gewirtz: Now, in China, I think an incident like that would be seen as a cybersecurity incident first and foremost. And the thing to remember in China is that while the AI regulatory space, and particularly the frontier AI regulatory space, is very much still coming together — and we’ll talk about that, I’m sure — the cybersecurity structures have been in place for a while, and this is a top kind of regime security issue for Beijing.
So I think they’d treat it that way. They do have certain kind of notification requirements where I think the central government could potentially become aware of it pretty quickly. But the most important thing that I think we have to stress is that at every layer, the incentives in that system are not to disclose more information, and certainly not to disclose it publicly.
So the most remarkable thing, as we imagine what a kind of Hugging Face, OpenAI-style incident in China would look like, is the public, I don’t think, would know about it. We might never know about it.
And I think that’s part of the reason why there’s so much concern as we think about these issues. Because in the Chinese political system, whistleblower protections, shall we say, are not something that gets treated with much seriousness. The famous Chinese whistleblowers are people like the doctor in Wuhan who warned about the spread of COVID and was punished for that years ago.
So I think anyone at a firm — whether an executive wanting to report up the chain or a whistleblower kind of considering whether to make their concerns known — would be weighing the risk of serious kind of smackdown by the government against the safety risks. And so that’s a pretty challenging place to be.
Tom Reed: Interesting. Do you think the government would be fully informed of what happened, or do you think the labs themselves would also be incentivised and capable of hiding such an incident if they became aware of it? I’m just trying to understand exactly who would know if something like this happened.
Julian Gewirtz: Well, in an incident like that, where what happened fully escaped the lab’s control and was a third-party website, I think presumably the government would find out that something had occurred, at least via the third-party website. And if there was an investigation, I think some of the labs might be more forthcoming than others.
There is a small chance you could imagine that if they felt they really kind of handled it — and/or if it became publicly known that there had been some sort of hack of this third-party website for other reasons that were visible — I think you could then imagine them doing a very limited kind of disclosure, saying, “There was a problem, we found out about it, we solved it, everything is good, proceed accordingly.”
I think the biggest policy question in Beijing in that kind of moment would be: would something like that trigger what they would call a “broader rectification campaign,” meaning an effort to bring greater control from the central authorities over this area? And that, I think, is possible. But again, it would likely be in a context without the kind of public disclosure that eventually marked the OpenAI Hugging Face incident. And crucially, that METR report — the things that have brought a bit more public confidence back to thinking that at least third-party evaluators have the capacity to inform the public about this — I think highly unlikely in the Chinese case that that would happen.
Tom Reed: Is there a particular flavour of near-term AI incident that you think would be more likely to cause something like a broader rectification campaign? What I’m trying to think of is: are there some kinds of ways that AI could go rogue that would be more worrying to the CCP [Chinese Communist Party] than others, or that might encourage them to view things as not just any other familiar cyber incident?
Julian Gewirtz: We have to start with the reality of what China is and what the Chinese Communist Party cares about, as opposed to kind of working backwards from the place we think they need to get in order to accomplish our broader goals.
And I say that because the number one concern is regime security, regime stability — what they call political security and social stability. So the kinds of risks that I think are most likely to cause them to recalibrate some of their current approach to the AI sector are really in that domain. They do want to avoid risks to regime security, also sort of major economic disruptions. So from that perspective, I think as we consider the kinds of scenarios that might unfold, it’s really the ones that are in that core political and regime security space.
And as your listeners well know, there are those risks — and in fact, control over information, particularly information about the government, about the Communist Party, access to a very carefully controlled cyber and information ecosystem: that is the kind of thing that we can imagine these new tools, new technologies disrupting, either inadvertently via loss of control by a company, or deliberately. And I think the deliberate piece of this is a big element to just bring in, because candidly, as I have seen the Chinese AI discourse evolve, there is increasing attention to what they call “extreme loss-of-control risks.”
But we have to remember that when they use that phrase, “loss of control,” they don’t simply mean it in the way that a lot of AI safety folks here mean it — loss of human control over AI systems — but also as the loss of political control linked to AI more generally. And so when we think of deliberate hacking, deliberate dissemination of information that the government doesn’t want spread, those are all the kind of risks, I think, that are likely in the first instance to make them really start to reconsider.
And, sorry, maybe just one last thing, but there was recently this vulnerability exposed—
Tom Reed: This is the hack of WeChat?
Julian Gewirtz: Yeah, exactly. The vulnerability in WeChat, which is the sort of One App to Rule Them All in China.
Tom Reed: So just to clarify for our audience: this was a hack of WeChat that was disclosed by a group that had used AI to find the vulnerability. They’d found a vulnerability, but they disclosed it responsibly. But we have no idea exactly how the government sort of processed this information. We just know that it happened and they would be aware of it.
Julian Gewirtz: Right. And sort of in line with what we were just saying a second ago, the response of Tencent, the company that created WeChat, was to say, “We patched the vulnerability.” But the shoe that never dropped publicly was how the government saw that and what kind of conversations about the vulnerability there were.
But there was just one other thing which I should have said in the prior set of comments, which is: the US factor here is major. And I think in a way, the most significant event in shaping the evolution of Beijing’s thinking about AI risks over the course of the past year or so is Mythos — and the kind of sense that the United States is gaining extraordinary capabilities in cyber. And that that is motivating a set of counter-responses that we have very little visibility into, but that they see as part of the AI risk conversation centrally. And whenever they talk about AI risk at the government level or the party level, they talk about the United States as one of the threats.
Tom Reed: It’s like the main risk for them is US primacy through AI advantages or something like that?
Julian Gewirtz: I still think loss of political control, political security, is number one. But a very close second is the US dimension, exactly as you say. And crucially, they’re connected.
Tom Reed: And how do we know that Mythos was this sort of wake-up call for them? I’ve heard this before, but how is it that we know this is the case? They’ve not said, “Oh, this was a wakeup call for us.” Are there signs that Mythos shifted things for the CCP?
Julian Gewirtz: No, I don’t think any government would call a press conference and say, “We’re really freaked out about this.” But no, there are a few things.
First, there are indications in official statements where they have quite directly invoked it. So they have spoken about it a little bit. They don’t say, “We’ve turned up the alarm to the highest level,” but they acknowledge the reality of it, and they acknowledge it as something that could pose a kind of security set of challenges for China. And then second, in track II dialogues and other settings where there’s communication, it’s been quite clear that this is something that has reverberated profoundly in their system.
And in general, to go back to that initial point of starting from what China is and what the party cares about, their concern about American cyber capabilities and all the rest has been a huge issue for well over a decade. Xi Jinping has spoken about this frequently over the years. So they are primed to see that as a threat vector of the greatest significance.
How Beijing views distillation [00:18:56]
Tom Reed: Can I force you to make a guess about something?
Julian Gewirtz: Sure. You’re not going to necessarily be able to force me, but you can try.
Tom Reed: I could try, I could try. And yeah, to clarify for our audience, I should say, if you hadn’t heard about this: there was a report released by Anthropic where they discussed that requests from users to Kimi models or models from DeepSeek were being rerouted to Claude. And some of these even included requests from, I think, a PLA [People’s Liberation Army] official, even in one instance. So really sensitive things that people thought they were using a Chinese model for, but was in fact being sent to Claude.
What I’d love to know is: what do you think the CCP officials who spoke to the CEOs of Moonshot and DeepSeek about the data leaks to Claude, what do you think that conversation was about?
Julian Gewirtz: So the first thing I’d say is: I wonder at what level of technical sophistication that conversation happens, because there’s a version of that conversation — in our idealised version of, generally, globally, how governments and private sector should interact — that could be pretty technically sophisticated. But I think it’s fairly unlikely that that was the only kind of conversation that happened.
I think it’s more likely that there was a conversation about this — maybe that did occur at a technical level, or they sent in people from the Cyberspace Administration or one of the real AI technical bodies that does exist to understand how this happened. But the political dimensions of that — the idea that it embarrassed China and embarrassed these companies — had to have been front and centre. And that’s the kind of thing that would probably happen somewhat distinct from the more technical conversation about what was actually revealed and how did that actually happen.
I think we all had the reaction reading the Anthropic report that, like, first, this is crazy. And it is a reminder of how the incredibly fast-moving Chinese efforts to catch up have led to a degree, in some areas, of sloppiness that is certainly not what a perfectly centrally planned version of an AI race from Beijing’s perspective would look like. And it is embarrassing. I mean, it’s deeply embarrassing for them.
And the broader point about distillation and dependence on American models, though: we were sort of talking just a second ago about some political dimensions of that conversation. It’s important to underscore that Beijing’s line on distillation is not, “It doesn’t happen, and how dare you.” It is precisely, actually, “This is normal practice –”
Tom Reed: Fair game. Yeah.
Julian Gewirtz: “– the United States is trying to twist this into something it’s not,” et cetera, et cetera.
And I was also quite struck that they connected that report to when Dario came out with the call to pace the frontier, and some of the China language there.
One of the core points I always make about China is that they connect dots in our system that we may not connect ourselves. And candidly, I think we do the same sometimes in their system; we can come back to that. But they connected, in very official, authoritative state media, that report that we were just talking about — including those really embarrassing disclosures — with the calls to pace the frontier, which included language about the need for the US to stay ahead of China. They said, “See, these are all part of the same initiative, the same effort to make Chinese AI look bad and keep the United States number one.”
And candidly, I don’t think too many people in the United States connected those. I’m not even sure that Anthropic necessarily meant them as part of some kind of coordinated thing. I don’t know. But they saw it that way, and it’s just a reminder of that core dynamic.
Tom Reed: Yeah, that makes sense.
How do Chinese labs really view AI risk? [00:23:18]
Tom Reed: So it’s still deeply unclear how either the US government or the Chinese government might relate to a very rapid increase in AI capabilities. How do you think about the difference between how the American labs versus the Chinese AI labs think about AI risk?
Julian Gewirtz: It’s such an important question, because that distinction in each country between the governments and the labs is actually something that can sometimes get overlooked by either player. China often talks about the United States ecosystem as just one monolithic thing. Obviously, Americans often sometimes talk about the Chinese ecosystem in the same way.
And over the summer, my friend Dan Wang and I wrote an essay in The New York Times that was arguing essentially that we needed to focus more on the competition for influence and bargaining and dynamics within the United States and China respectively than we do currently, where the focus is largely on the race between the two. And obviously that’s become pretty salient in the months since we wrote about that, which was right around the time of the US government and Anthropic dispute.
I guess what I would say there is, to my mind, the key difference is that the American labs — some of them, at least — have a demonstrated track record of taking safety somewhat seriously and doing certain things to disclose to the public, and to even in some cases constrain their own racing forward. I think it’s nowhere near what many of us would like to see, but there is something of a track record there. And candidly, we don’t have that on the Chinese side.
I think when people talk about these questions of where is the risk greater these days, the US government certainly doesn’t seem like it’s being particularly responsible. But I think there is reason to say that some of the American labs, at least, have behaved in a way that is a little bit more attuned to the safety concerns than some of the Chinese labs.
Tom Reed: Do you never entertain cynicism — like, cynicism toward the American labs? Like, you take this—
Julian Gewirtz: Do I? Absolutely.
Tom Reed: Yeah. But why doesn’t that make you more bearish? Or what gives you the confidence that it’s not just show? I have this opinion too, for what it’s worth.
Julian Gewirtz: Yeah. I mean, I guess what I’d say is I’m not terribly optimistic here. I don’t want to convey the wrong impression that I think somehow the American labs are highly responsible stewards of the fate of human civilisation. But I do think that the incentives that they have had, at least so far, to disclose incidents — very low bar, not a bar that the Chinese labs have cleared. So actually, I would frame this less as all the things that we know they need to be doing and are not doing, and just look at the lowest possible bar of disclosure — and that’s one fundamental difference.
And so I think in both countries, the question of how the relationship between government regulation and lab self-regulation plays out is a huge part of everything that we’re discussing today. It’s going to play out differently in these two countries. This is not going to be symmetrical, for very obvious reasons. But if we’re just looking at the labs, some of those asymmetries, I think, aren’t all a bad story for America.
Tom Reed: Have you ever had the chance to speak to people at Chinese AI labs?
Julian Gewirtz: Yeah, to some. I mean, look, I’m a former US government official. I’m not somebody who I think Chinese labs have a—
Tom Reed: Speak candidly to.
Julian Gewirtz: Well, I don’t know about that. I think the most striking conversations that I’ve had with folks at Chinese labs were around the kind of early wave of discussion of pacing, sort of when the lab employees’ letter came out in the United States.
And I asked a bunch of these folks how they interpreted that. And one person said to me, in a comment that I found very striking, “The United States has been pacing the frontier for China for years. So we don’t see this as really anything new about us. And we don’t think the United States is going to slow down.”
So they have a healthy degree of cynicism there. Now, obviously there’s a degree of performance in that — they’re performing for a certain kind of American audience. But on that point at least, I think that’s a credible position that likely many of them do believe.
Could Xi Jinping ever be AGI-pilled? [00:28:12]
Tom Reed: I also want to understand a bit more how the CCP might respond to blisteringly fast AI progress, if it does happen. I’d like to know: is there anything that exists within Xi Jinping Thought that would make sense of a potential tenfold acceleration of scientific and technological progress? What concepts does he have available to make sense of this?
Julian Gewirtz: It’s a really interesting way of putting the question. When we think about speed of progress and development, there are few societies on earth that have dealt with that dynamic as recently as China itself. Meaning if you think of China in 1980, and if you think of China today in 2026, that is a stunning speed of transformation. And that’s not a tenfold increase in tech progress in the span of a year — we can talk about those kind of hard takeoff scenarios — but it is, by the standards of pretty much all human history, pretty darn fast.
So I say that because when we compare it to societies that have been wealthy and experienced sort of development in a more gradual process or longer ago, I just think there is something interesting there in that fact that China has experienced such rapid development in the recent past.
But I don’t think any of that prepares any of us, in either country or anywhere around the world, for that kind of speed. And I think in the minds of Chinese leaders, the risk of losing control over these systems, over society, over technology, is a risk that kind of has a much deeper and more fundamental status than it does even in the United States.
Because if you think about the way in which, in Xi Jinping’s kind of more ideological thinking, technology is positioned, it is, as he says, the main battlefield of international competition. But it is also kind of a physical instantiation of political power. And the Chinese Communist Party’s relationship to technology over a long time has been that this is something we need to harness to drive the development of China — the so-called great rejuvenation of China — and our political power more generally.
In the 1980s, when the information revolution started to happen, Chinese leaders were thinking about very rapid — by those standards — shifts, and wanting not to miss out on them, as they had missed out, in their telling, on the prior—
Tom Reed: The Industrial Revolution.
Julian Gewirtz: Exactly. Obviously missed out on the Industrial Revolution, and missed out on several prior waves. Xi Jinping has said this revolution, we will not just not miss out on, but we will help lead.
So those are some of the concerns and considerations that would be a part of this. But I also will just maybe end this answer by saying the question of kind of exactly how this plays out, and exactly how much control by that point the Communist Party thinks it has, is going to shape so much of this. Because, for instance, right now Beijing is still significantly constrained on compute. They see themselves as behind, relentlessly trying to catch up. But if, by whatever point we’re talking about in this hypothetical, they feel less constrained on compute — which I don’t think is terribly likely, but let’s say — or they feel that they have reached closer to parity at the frontier and suddenly it might be slipping out of their grasp, I think those are just profound uncertainties that could introduce even more volatility into this moment.
Tom Reed: And we just don’t know.
Julian Gewirtz: I hate to kind of have a fancy four-part answer, the upshot of which is that we don’t know — but it’s the truth. And maybe just more generally to say, as we think about why this is so challenging for the Chinese Communist Party: this is generally a system that moves pretty slowly.
There are exceptions to that, and I think AI governance could end up being one of the exceptions to that. But they move pretty slowly. They like to have these really laborious deliberative processes, paper and endless meetings and all of that. And they, like pretty much any government — but to a pretty intense extreme, given the authoritarian nature of the system — want certainty. And so in their system, one of the hardest things—
Tom Reed: Good luck with that.
Julian Gewirtz: Exactly. One of the hardest things about frontier risk has been that the upshot of every conversation is, “We don’t know, but the scale of outcomes is near infinite.”
What’s really shaping the CCP’s views on AI? [00:33:25]
Tom Reed: Who actually has shaped the CCP’s views on AI the most? I remember you have this great article from a couple of years ago about the influence of the American futurist Alvin Toffler on China’s sort of national R&D programme in the sort of late 1980s. They read a lot of Alvin Toffler, and apparently this inspired a lot of their approach to using new information technologies. Who might be the equivalents for AI? Who are they reading to understand what this technology is about and what opportunities it offers?
Julian Gewirtz: Well, there are a few ways to get at that question. One is that, for good or for ill, a lot of people in that system, I think, are reading a lot of the same stuff that Americans read — meaning the core people who are working in the labs, in the companies, generally have had exposure, even if they haven’t studied in the United States. It’s a pretty global research ecosystem — increasingly, I think, less that way, but there is that there.
And we do see people in the Chinese ecosystem respond to everybody from Dario to Leopold Aschenbrenner and whoever else.
Tom Reed: This is at the labs?
Julian Gewirtz: Yeah, this is at the labs. But I think that’s important, because one of the really interesting dynamics inside China is that the companies — or some of the companies — I think have very different visions of the AI future than the Chinese Communist Party. And so that was where I was kind of going with this, which is that there has been a debate, for instance, about whether China — air quotes around “China” — is AGI-pilled. And I think the candid way to answer that is many of the labs are. The Chinese government has been much less oriented that way, but I think even that is changing.
And I think the upshot is that now, basically, the views that are shaping the Chinese Communist Party on this are increasingly internal to the Chinese Communist Party. They are not publishing essays on microsites. They are reporting up through the system. And one of the reasons that I think more engagement, technical exchange, and dialogue is a good thing between these two countries is we need to be really thinking about how we’re shaping those reports that go up to the top Chinese leaders that they pick up and read. Because what’s in there is likely very different than what Liang Wenfeng, the DeepSeek CEO, might say on an investor call.
Tom Reed: In a world where Xi does end up taking the idea seriously that AI could lead to something like a tenfold acceleration in scientific progress, how would that have happened? Who would be the sort of important precursors to Xi also believing this? Do we know what the information flow could plausibly look like?
Julian Gewirtz: It’s a really interesting question to unpack, and I don’t want to get way too in the weeds on this, but basically, we—
Tom Reed: Get an org chart out.
Julian Gewirtz: No, exactly. You can put it up on the screen. But basically, their responsibility on AI is somewhat diffused across different parts of the Chinese government, because the fundamental approach that they have taken to AI policy — not AI safety, but just AI policy more generally — the shorthand is AI Plus: essentially integrating AI into everything.
Of course, they have made tremendous progress integrating it into manufacturing and robotics. They are integrating it into the military. They’re integrating it into the political and state security apparatus, and so on. So that means that responsibility is somewhat differentiated.
I think the least likely way that Xi Jinping comes to that impression is that he reads a bunch of essays from American lab leaders.
Tom Reed: What would be a more likely route? Maybe if it comes from the industry themselves, perhaps, or it’s unclear?
Julian Gewirtz: I mean, I think the likeliest route, honestly, is that it starts to happen in the United States first, and it doesn’t feel like it’s happening in China. And the effect is that, in connection to their becoming more concerned about the risks domestically, they also see the possibility of the United States really pulling ahead very quickly.
And I think that’s a moment when you can have very divergent paths. But there’s a nonzero chance of a path — if the United States is smart about how it approaches it, which I think is a big if — where you could imagine Beijing coming to some sort of internal reckoning that pacing is clearly in their interest. Small probability, but nonzero.
Does anyone in Beijing believe in RSI? [00:38:32]
Tom Reed: Is there anyone important in China who might have what we would call the Silicon Valley view of AI? Do you think there’s anyone other than perhaps the lab leaders themselves? Is there any other person who is significant who you think—
Julian Gewirtz: And when you say the “Silicon Valley view,” you just mean about speed of progress and scale of impact?
Tom Reed: Yeah, exactly. Possibility of something like RSI [recursive self-improvement], possibility of vast technological progress.
Julian Gewirtz: That is the kind of question that I wish people outside of China could answer authoritatively. And one of the challenges is that I can give you my best answer, but I want to start it by just saying we have very limited insight into those kinds of questions. There is an emerging discourse on RSI inside China, for sure. I mean, you can see that; you see increasing discussion about it. But senior people, particularly people inside the system who have these views, we have very little indication.
There are obviously people who advise the government, who are outside of the government itself — not necessarily at companies, but who are sort of the experts on the governance of this technology — who brief the Politburo and other senior bodies. Some of those people, very kind of cosmopolitan, go to Silicon Valley or have thick relationships there. So it’s not like there’s nobody who’s briefing the government with those views. But does Xi Jinping’s chief ideological adviser, Wang Huning, believe that? Or does He Lifeng, who’s the sort of senior economic planner, believe that?
Tom Reed: He met with Scott Bessent?
Julian Gewirtz: Exactly. He’s the guy who meets with Bessent — who’s Bessent’s counterpart. We simply don’t know.
The way that I have been saying we can evaluate that most effectively is that there are a set of decisions that the government would make if they came to believe that. Centralising compute. Buying suddenly a lot more Nvidia, which they are approved to do by the Trump administration, but have been rationing.
Tom Reed: They’ve been rationing it.
Julian Gewirtz: Right. No, there’s this bizarre situation where several years ago we would have said the core dynamic is China wants more of the world’s best chips to import than they can get. And now the dynamic is the United States under President Trump has greenlit the sale of some chips that were previously not permitted, but Beijing is buying them in quite limited quantities.
Why? Well, there are a couple reasons. First reason is that in general, Xi Jinping is trying to get the Chinese AI ecosystem to be more reliant on domestic sources of compute, particularly from Huawei. But second, crucially, they are still able to get access to the best servers and the best compute in third countries through remote access to data centres.
Tom Reed: Malaysia.
Tom Reed: Exactly. And there’s also a bunch of smuggling activity that’s going on. It feels like every week we get some new report of large-scale smuggling activity, which is just wild.
So anyway, but that is all a preamble to saying: if they thought, “OK, we have two to three years in which the world is going to change fundamentally, and we simply cannot afford to be behind,” you’d see them centralise compute, you’d see them more aggressively buy American chips. And you’d see a set of policy changes like that.
So it’s a long way of saying, to evaluate the answer to your question, I think it’s less likely that we’re going to get a speech from Wang Huning as a leading indicator, saying, “I have decided that RSI is about to happen,” and it’s more likely that we begin to see some of these physical, material movements that undergird actually permitting that to occur in China on a technical basis.
Tom Reed: Yeah, just on the chips thing, was it Ding Xuexiang who recently said that if you’re buying non-domestic chips, you’re a traitor or something like this?
Julian Gewirtz: That was reported. Pretty interesting. And to the point about kind of diffusion of responsibilities at the senior ranks: Ding is an extremely senior policymaker, one of Xi Jinping’s top advisers, who has been put in charge, by all accounts, of the internal process of kind of coordinating compute and AI policy in some key dimensions.
What we don’t know is how the national security dimensions that we were talking about a minute ago — military and espionage and cyber and all of that — are integrated, and there’s also other people who are playing in this space at quite a senior level. But yeah, it was pretty striking to see those reports from Ding making clear to the Chinese AI sector that if there’s a tradeoff between going a little bit slower and making sure that you’re not dependent on foreign compute, go a little bit slower.
Tom Reed: Is there a parsimonious explanation of how the CCP takes radical policy action in response to perceived crises? I’m thinking of things like zero-COVID or the one-child policy.
Julian Gewirtz: I don’t think zero-COVID and the one-child policy are apples and apples. I think that’s an apples and oranges comparison, because in one case you have a near-term response to a crisis, and then this kind of upswell of public discontent around COVID that leads to initially doubling down and then eventually suddenly loosening up.
And in the case of the one-child policy, you have a decades-long effort that the party comes to see as having led them into a demographic crisis. And so they then are scrambling to figure out how to deal with that, and to incentivise people to have children and all the rest of it. So they’re pretty different.
In the current environment, I don’t know if it’s parsimonious per se, but the way to think about this is: when does the ledger begin to shift the core interests of the Communist Party that we’ve been discussing to see a different approach as necessary?
And Xi Jinping has left the door open. In his speech at the World AI Conference, he did lay out a set of principles, but he also said, using a kind of Chinese proverb, that flexibility and changing and updating as new information comes in is going to be a part of this. And to put it mildly, that is not a signal that he always sends. Normally it’s like, “We will resolutely uphold X, Y, or Z until the bitter end.”
So the door is open there. And I think therefore, just to go back to kind of where we started, the most important thing is their own perception of AI safety and AI risk evolving inside China. And I do think that will happen, and I think it could happen sooner than many of us expect.
Chinese labs hate red tape, too [00:45:47]
Tom Reed: Other than asking for more chips, what sorts of things do you think the Chinese AI labs are lobbying the CCP for?
Julian Gewirtz: A lot of things. And I think a really interesting dynamic that is super underreported, because it’s really hard to report on, is we know a lot about the bargaining and all the rest between the labs and the government here in the United States. We know much less about it in the Chinese ecosystem. But if you talk to people who work at Chinese AI labs, they complain constantly about government overreach, and all this stuff they have to go through, and government getting in the way of X, Y, or Z. So there are these dynamics there.
Now, compute is one. The government has been trying to channel much more capital into this sector, but obviously that’s another. A third, crucially, is these really startling new regulations on people being able to leave China.
So there have been a series of reports about initially employees of Chinese model companies, and then even their family members, no longer being able to travel abroad without permission. And this is the kind of restraint on human mobility, ordinary life, that I think will affect… Somebody’s a PRC national working in the United States, and they’re thinking about going back to China to take some sweet deal they’ve been offered, or to stay. That’s a pretty big difference that goes beyond whatever your compensation number is.
And then one other thing to just say is I think for a number of these companies, they look at what happened to the Meta–Manus deal — meaning a Chinese startup that was super buzzy, got a lot of attention, moved to Singapore, and then Meta wanted to buy it; it almost happened. And then the Chinese regulators came and yanked the chain and shut it down. I mean, that is the kind of thing that, as you know, is any startup founder’s nightmare, wherever they live: that the dream deal comes through and the government stops it.
So there are all of these pain points. Now, there’s a lot of stuff that they’re getting: I want to be really clear that the government is pushing resources and prestige and support in other ways into this sector. But it is complicated.
Tom Reed: It’s making their life difficult in other ways.
Julian Gewirtz: That’s right.
Tom Reed: What are the other exogenous factors, so non-AI factors that are most likely to influence Chinese AI policy? I’m thinking of things like, I don’t know, an ageing population. Is there anything else we should be tracking to understand exactly what’s happening?
Julian Gewirtz: Stepping back from the kind of frontier risk conversation into how AI policy has evolved in China: I do think the ageing population is a big point here. Because kind of similar to the way that some model companies in the United States talk about, like, “We may not need all the engineering talent in the future.”
I think in China’s case, one of the hopes has been that increasing automation, robotics, embodied AI will enable them to maintain or even increase their manufacturing dominance, even as their low-cost labour and young population advantages go away.
I also think their perception of the United States right now is that — remember, they see us always as these kind of capitalists who have a system that has a lot of irrationalities in it — and I think if you really were to press them, my guess is they would say there’s an AI bubble, there’s a lot of speculation. But I would say the way that that plays out — the way that sort of valuations, financial flows, all the rest of it play out — is AI-related, but is a little bit sort of exogenous in some sense, because it’s not about their own perception of their core AI capabilities.
Those, I think, are two big ones. Maybe the third is just broader perceptions of US–China relations, which I do think are really important here. I think in this period where President Trump has been trying to clearly put out a very welcoming image to China, and emphasise the strength of his relationship with Xi Jinping and all the rest of it, there’s a lot of mistrust in the Chinese system toward that, and I don’t think they see it as much more than buying time.
But I do think if US–China relations were to deteriorate markedly — kind of back to Trump in 2020, rather than Trump today — that would be a pretty significant factor for them in further intensifying the zero-sum, national-security-centred frame for their approach to AI. But that would be exogenous.
Would China invade Taiwan over AI? [00:51:04]
Tom Reed: What are you most worried could deteriorate the US–China relationship further? Is it another trade war? Is it something like Iran? What should we be tracking?
Julian Gewirtz: Oh, there’s a lot. Well, I’ve described the kind of current state of the US–China relationship as K-shaped. And by that I mean, in the way that we think of a K-shaped economy, where there’s an upward leg, like AI, that’s going gangbusters, and then there’s a lower leg that is doing much worse, you have this upward leg of the US–China relationship that is leader-level palling around, these big optics —
Tom Reed: They’re vibing.
Julian Gewirtz: Right. And then you have all the realities of competition — military, technological, economic, and so on — that are continuing. It’s not as if that stuff has stopped, even if it is less salient or visible to many people. So I think there’s a long list of things that could disrupt these dynamics.
But I will say — not to get us into a kind of conceptual doom loop — AI is a huge factor here. And I think there are scenarios where China appears to have advanced much more rapidly than people expect that could really shake up this current dynamic. And I also think there are scenarios in which the United States appears to have pulled ahead so dramatically that Beijing’s calculus changes, that could be really disruptive.
What worries me the most day to day is… There are two things. One is these leader-level interactions: they look kind of predictable or stable, but actually there’s so much uncertainty to them. I mean, you just never know when something’s going to go wrong. And so I never feel like the fact of a summit is itself a sure thing, that it’s going to lead to things going better, because it’s easy to imagine President Trump changing his mind.
And then the second thing is just our militaries continue to operate in very close proximity in the Western Pacific every day, and the risk of an accident is always there.
Tom Reed: Is it magical thinking from Western AI folks to think that China would invade Taiwan if they thought the US were on the cusp of RSI? Is this a silly idea?
Julian Gewirtz: For Beijing, Taiwan is not about AI, right? This is a core regime longstanding priority. This is what they view as the unfinished business of the 1949 coming to power. And Beijing has wanted to control Taiwan long before anybody was talking about AGI. So we gotta remember that, because that is the defining fact. The fact that TSMC [Taiwan Semiconductor Manufacturing Company] sprouted up in Taiwan and now has become totally central to all of this—
Tom Reed: It’s a strange accident of history.
Julian Gewirtz: But to me, actually, one of the most remarkable dimensions to this is the flip side of the way you pose the question, which is how long it has taken Beijing to reckon with that fact — meaning that Taiwan’s global salience and global kind of role have changed because of this newfound economic centrality.
But I don’t think when they’re thinking about this, they’re first and foremost thinking about AI. That said, it’s not something where you can completely dismiss it. But I think it’s pretty important for people to just remember that Beijing has wanted control over Taiwan before AI, and they want it for reasons that have nothing to do with AI in the first instance.
And just on political timelines in Taiwan — without wanting to get too in the weeds — Taiwan has a presidential election in January 2028 that Beijing is really hoping will go in a kind of direction that is with the leaders who they prefer.
Tom Reed: The KMT [Kuomintang]?
Julian Gewirtz: Precisely. And so from their perspective, this is a window where the political path feels, I think, more open than it has in a long time. So in general, the approach is, yeah, keep the coercion up, but see if this works, because the military options for them are not good. It’s a very costly thing, and not something I think that they want to do if they can avoid it.
Tom Reed: Do we have any idea what the significance of a KMT victory in early 2028 would be? Or is it like that’s completely open?
Julian Gewirtz: Well, there are a couple different permutations for what that would look like. We don’t know who the KMT candidate is going to be yet. The current KMT party leader is someone who has gone to the mainland, who met with Xi Jinping. So they’ve clearly had quite a positive view of her, but it’s certainly not a sure thing that she or her view would be the view of the eventual KMT nominee. There’s local elections coming up soon, this year, in Taiwan that may shape some of the way in which the KMT is configured.
So it’s not that we don’t know in an abstract sense; it’s that, like any process, it’s just like asking, who’s going to be the Democratic nominee, and what’s that going to mean for US–China relations?
Tom Reed: It has yet to unfold.
Julian Gewirtz: It’s like, well, you could have very different scenarios there.
The prisoner’s dilemma of a US–China AI deal [00:56:51]
Tom Reed: Let’s go back to a potential deal between the US and China. So imagine if we get some very early-stage cooperation. So maybe it looks something like simple information sharing, or maybe a shared working group between US CAISI [Center for AI Standards and Innovation] and I think Matt Sheehan said that China’s working group number nine is the one that is tasked with AI safety standards.
Let’s imagine we get something like this. What would you expect to be the early sticking points of this kind of cooperation?
Julian Gewirtz: OK, so first I would say getting something like that is itself going to take some work. I think it was really striking to me, where I had pretty low expectations for what the Bessent–He Lifeng meeting and then the Trump–Xi meeting would produce on this score — pretty low expectations, and they were not met.
I thought we would get at least some sort of technical working group commitment, because that seems so obviously necessary, such a no-brainer, and not hard to do: you’re not committing them to do anything; you’re just creating that kind of connectivity. And we didn’t even get that.
And I thought we would get some kind of commitment to a sort of more intense cadence of interaction and crisis communication, incident response communication. And it appears that we basically just have a kind of verbal commitment to call each other under a pretty vague set of circumstances — which, let’s be honest, was the status quo ante: Bessent and his counterpart can call each other when they want.
But let’s say we were to get that kind of connectivity established. I think the first sticking point would actually be terminology, language. Because if we’re talking about a fairly professionalised approach from CAISI here, and the way that these bodies in China approach it, there are some basic language questions that need to be worked through. What are we talking about when we talk about loss of control? What are we talking about when we talk about biorisk?
And I think actually there’s a lot of work to be done in just nailing those pieces. But I don’t think that would happen naturally. I mean, for your listeners, even at the most fundamental level: I was in a meeting not long ago where a very senior American AI expert gave a long spiel to a bunch of Chinese AI safety experts where he was making a fundamental distinction between safety and security — the safety of an AI system and its alignment and other aspects like that, and then the security in a more sort of cybersecurity, model security sense.
Now, in Chinese, safety and security are the same word, anquan. So the people on the Chinese side who knew English were following just fine, and the people who didn’t were deeply confused.
So that kind of thing is actually–
Tom Reed: It’s a real blocker.
Julian Gewirtz: Yeah, it’s a sticking point. It’s not an ideological disagreement, but it needs to be worked through. It’s one of the reasons why what track II dialogues do is actually important, even if it sometimes can feel kind of painful, because it’s helping deal with that.
Second, let’s say this kind of technical conversation happened: I think there would be a big set of debates over whether they were talking principally about near-term, immediate risks, or frontier risks that may emerge in a shift toward RSI.
And assuming that the Americans wanted to talk a lot about the latter, I think there might be some resistance from the Chinese side to fully getting into that, particularly if they felt that it was kind of being presented to them as something the United States was going to get to first. And again, just this trope of, “They’re ahead, they’re trying to keep us behind.”
Tom Reed: What kind of a deal do you think would be most appealing to the CCP on AI?
Julian Gewirtz: Well, I mean, not to be too cynical, but the most appealing thing would be unilateral restraint from the United States. To them, everything flows from their conviction, as I was saying earlier, that they’re behind, we’re ahead, and they need to catch up.
So one of the hard questions, which we talked about, is that the United States does have to show that we’re serious about these risks and serious about regulating this sector. And from Beijing’s perspective, that part of it’s very appealing. What’s not appealing is constraints on China’s own development. And that’s the core kind of prisoner’s-dilemma-like situation that we’re in.
And we haven’t really talked about verification either, but we need the verification science to catch up to the frontier. But there is then a whole other question about Beijing accepting — and, candidly, about the United States accepting — those kinds of what would be pretty intrusive external monitoring or evaluation.
Tom Reed: Yeah, this would be something like having on-site auditors in labs. Actually, this was going to be my next question, which is: if you imagine we’re in a fairly radical world, and the US and China have in fact agreed to temporarily pause AI training, if you put your NSC hat on, what would give you confidence that China had in fact paused?
Julian Gewirtz: Some of this starts with, well, what is the verification technology? Not just third-party auditors walking around, but when I talk about that, I mean the kind of true, robust technical ability to understand — again, it can be framed in many different ways; people are trying to work on many different dimensions of this — but the size of runs, where–
Tom Reed: Where the data centres are located.
Julian Gewirtz: Where the data centres are located, the amount of energy use, actually testing model capabilities — these more built-in or integrated technical solutions to these questions, I think there is not going to be a high level of trust. I just don’t think we can premise any of this on that. So instead, what we’re looking for is the ability to have some degree of confidence via verification in this kind of speculative scenario.
And I would just say we could be having one version of this conversation that is about — again, we’re in the good place here; a lot of things have gone really right in this hypothetical — but we were talking about, OK, there are agreed limits to what kind of RSI will be permitted. Meaning, we’re not going to just hand over everything; we’ve agreed to limits on that.
OK, well, that is something that I certainly would be supportive of. But the question then is: given that you’re dealing with systems where humans will struggle to understand what they are doing, period — forget about the US–China dimension — there too, you need to kind of solve the alignment science and the verification science piece of it to have confidence, not simply in the US–China dimension, but in the human–machine dimension of this.
And so I think as we get to those more sort of frontier curve scenarios for the US–China piece, a lot of the sticking points that I think of — which also are reason for some degree of optimism — are actually not simply US–China, but are actually how much visibility, understanding, control over the systems that we’ve created we still have.
And so, but to my mind, that problem is going to have to get solved in each country separately. And so one of the framework ideas that Matt Sheehan and I have been kind of pushing — and this is really driven by his work — is that we really can think about this as: there are just some areas where self-interest can lead to action in parallel, that can be coordinated and can be the building blocks for more ambition that happens later. But we gotta begin in each case from that self-interest question as the driver of action.
Tom Reed: Yeah, that makes sense to me. I guess I was also wondering how far you’d expect unilateral surveillance to go. So probably there’s some amount that you can do even without having a lot of cooperation between the two sides if you just use espionage to try and figure out how we can have some idea of what the largest training run looks like in China right now, purely through our spies and things like this, or satellite imagery, something like this.
Julian Gewirtz: Yeah, I think that question is one worth both sides reckoning with at some level, but when we think about an agreement, and the history of agreements more generally, of course that activity continues from both parties in the agreement, and it’s part of how they are assessing each other’s capabilities and intentions and all the rest of it.
But the reason that we talk about diplomacy and an agreement that’s more constructive and proactive is that that is insufficient to rise to the scale of the challenge in front of us. If you could just kind of surveil or espionage your way out of this problem, we wouldn’t be talking about it in the same way. But I don’t believe that you can.
And I think what we are ultimately talking about are two countries that are extremely powerful making their own decisions to constrain some of their own capabilities, and showing one another in some form or fashion that they are doing that, actually. And that is different than just kind of snooping on each other.
How China stumbled into open source [01:07:12]
Tom Reed: What are some of those other areas where you think they have overlapping shared incentives to build regulation in parallel? What else comes to mind?
Julian Gewirtz: Well, to me, the core thing that I always think about on this is: neither government, neither country should have an interest in losing control of these systems. And that is a very high level of abstraction, but it is actually quite a commonsense principle. And what that means in practice, I mean, by some metrics we’re already losing control of these systems, to be clear, but we do still have the capacity to rein them back in. And so that is at least one of the core interests.
Another is that there are very powerful capabilities that are created that should not be in the hands of bad actors, third parties, terrorist groups, and the like. And we haven’t actually, I think, said the words “open source” or “open weight” in a long conversation, but there we have to be just very blunt that the proliferation of capabilities that we’re seeing in real time — where guardrails can be removed, models can be downloaded and used without the model company having any idea what’s going on, or anybody having any idea what’s going on except the person doing it — that is a really concerning place to be.
We’re recording this on October 2. We are starting to see an emerging discourse in China raising concerns about the risks of open weight and open source. And this is very new, but it is indelible, and I’ll be writing about it this weekend in my newsletter, but it’s starting to happen. What we don’t know is how those emergent risks get weighed against the fact that there’s a very strong set of political and geopolitical incentives to keep this going.
Tom Reed: It’s good to undercut American companies, is that what you’re thinking?
Julian Gewirtz: Great to undercut American companies. Great to spread Chinese influence around the world. Great to do both of those things on a very limited compute budget, because obviously, it’s not like Z.ai has to be building the data centres with chips they don’t have to serve all these things.
And it also is a great political narrative that Beijing has been pushing very hard, to say the United States has these closed, “exclusionary” — air quotes — systems, and China stands for so-called openness. And what I always say about this is, actually, though, that does not need to be an absolute principle. They’ve given themselves a lot of manoeuvring room. They just need to be more so-called “open” than the United States is, and they have a lot of running room on that. They don’t need to somehow be absolute open-source fanatics to keep this political narrative going.
Tom Reed: Where is this new discourse happening, by the way? There’s a sudden discourse about whether open-weight models are worth the risks. Why is this happening?
Julian Gewirtz: Yeah, so there are a few really striking things, and it’s literally happening in real time. And so by the time your listeners are hearing this, it may be that there have been a tonne of New York Times articles about it or whatever.
But what we’re seeing now is: first, there was some really interesting language in the new kind of AI safety governance framework that Beijing released recently. This is sort of from the body that advises the government on AI safety technical matters.
Tom Reed: This is TC260 3.0?
Julian Gewirtz: Yeah, exactly, precisely. And in there, there is some really striking language about open-source releases lowering the barriers to conducting sophisticated cyberattacks, and accelerating and expanding the scope of those attacks. So the warning is already there. The question now is what safeguards Chinese authorities and developers are going to put in place.
And then you’ve also seen, in the kind of scholar and commentator ecosystem, emerging language basically saying, “We can’t have unrestricted openness. Constructive forces can use it; destructive forces can also use it. We’ve got to find the balance” — which is strikingly different from where they were a few months ago, when the narrative was “open source is synonymous with China,” and all the political narrative that we were just talking about.
Tom Reed: Yeah, I mean, prima facie, it is surprising to me that China is still open-weighting as many models as they are, at the level of capability that they are. I feel like this is not something I would have predicted in, say, 2023.
Julian Gewirtz: I don’t think they necessarily predicted it either. Remember, initially it was like Meta had amazing open models, and China was kind of figuring it out. But I think because of the desire to undercut competitors and spread globally, and the compute constraints and everything else we’re talking about, this happened — with obviously the big DeepSeek release as the most powerful symbol of it — and it just kind of gathered momentum.
And the party, candidly, I think, was happy to go along for the ride, and then to figure out, “Wow, we can actually spin this geopolitically to our benefit.” But again, as soon as they begin to assess that the security and stability risks are getting out of hand, they have the tools to change this quite rapidly.
How to build a realistic deal, starting now [01:13:04]
Tom Reed: If you think about the US sitting down at the table with China to discuss a potential deal, discuss further cooperation, what do you think the US’s priorities should be for strengthening its bargaining position? Should it be something like indigenising the rare-earth supply chain? Is that the number one? What else should they be focusing on?
Julian Gewirtz: OK, so there are two categories. One category is things specific to AI, and the second is things beyond the scope of AI, but that, like rare earths, affect the balance of leverage.
So maybe start with the first category. The way that I think about this — which is borrowed from my former colleague Chris McGuire — is you’ve got kind of a three-legged stool, which is: you have your domestic regulation, meaning the kind of costly signals that we’ve been describing. You have the export controls that are in place and could be tightened, and that are part of the puzzle here. And then you have the actual — both bilateral and global — diplomatic outreach you’re doing to create communities of interest and momentum toward that kind of bargaining.
So those are the three dimensions specific to AI itself. Then you have the much broader questions related to the balance of leverage in US–China relations. And I absolutely think that many of the things we need to be doing there — you talked about dealing with the rare earths dependency, which is a huge priority — that those things will all help the US at the negotiating table with Beijing.
So we kind of have to have both strategies firing on all cylinders at once.
And then just again, the third thing that I’d say is we also have to be tracking so closely how Beijing sees these risks, because as their own risk perception evolves, there will be windows of opportunity. And I don’t know that necessarily the current team is thinking about it this way, but certainly if I were in government, or if others who have this approach were, I think you’d be looking for these windows, these moments when Beijing starts to take the risk more seriously, and seeing those as each creating an opening — not for a sweeping grand bargain that solves all of our problems overnight, but for incremental progress that gets us to a better place on a timescale that’s relevant, but also realistic.
Tom Reed: What is the right level of pessimism to have about the prospect of China as an international partner? As I understand, in 2023 there was a US offer of something like a nuclear crisis hotline, which they stonewalled or rejected. They also didn’t really exchange timely information with the rest of the world on COVID. What’s the right way to view how cooperative they’re likely to be or not?
Julian Gewirtz: I think probably it’s come across through this conversation that I think we have to have a pretty limited sense of what they’re going to see as in their interest — to return to that core idea. It’s never going to be a binary switch where they go from seeing this as this intense domain of competition to suddenly seeing this as a no-brainer: “Any rational person would understand, we gotta make a deal.”
So I think that initial point that I made — about we gotta begin from China as it is, and how they think about these challenges, rather than how we think they ought to — is core there.
I will say that in some of those areas you alluded to, like nuclear arms control, there is a similarity, in the fact that they’re behind, or they have fewer nuclear weapons. And they therefore always say, “Why would you talk about constraints with us when you have so many more than we do?”
The other thing I would say, which I probably should have said earlier, but better late than never, is they often, I think, over the past couple decades, have looked at the United States and felt that we get certain fundamental risk factors wrong. They looked at the global financial crisis in 2008, and they said the United States fundamentally got wrong the financial risk in its own system, on the side of way too much risk tolerance.
They look at the predictions that were made 25 years ago about how the internet was going to change China. Remember, 25 years ago, Bill Clinton said that trying to control the internet was like trying to nail Jell-O to the wall. He literally said “good luck” to China. And they were staring down those predictions, and they said, “No, the United States is wrong. We are going to nail Jell-O to the wall.” And they did.
Tom Reed: And how do you keep your ambition? Because when I think about, sometimes, all of the things that a real full deal on AI might require, we’re talking about potentially many years of not just monitoring each other’s compute supply chains, but also monitoring each other’s robotics supply chains. We’re talking about what might need to be a completely unprecedented level of cooperation between these two countries, on a technology for which there is less consensus about the risks, on a technology that is moving faster than any previous technology. Sometimes I just think it seems completely impossible. How do you view our chances here?
Julian Gewirtz: Well, I guess the way that I think about it is, rather than designing in the abstract a mental model of what the ideal deal would look like — I think if you’re doing that, if you’re doing the kind of AI 2040 thing, the impossibility confronts you at every turn. But in fact, I don’t think right now that kind of ideal mechanism design is the most urgent question. I think figuring out how we can start immediately, wasting no time, to take incremental steps in the right direction, based on an understanding of each of our self-interests, is the question before us — the most urgent problem.
Tom Reed: And that’s tractable.
Julian Gewirtz: And that is tractable. So I think beginning there, and then also continually saying: what is actually happening in the technological development? What are the scope of predictions or models for how this goes? Where is the discourse in China? Where is the discourse in the United States? And trying to just, again, be continually ambitious and opportunistic about it. I think that, as a mental model, shall we say — in addition to just a praxis — that is actually not such a dark place to be, even though it’s still an alarming and challenging moment.
the hardest version of the question right now is if someone were to ask me, “What does success look like in a year?” And I say that’s the hardest version because, as you know better than I do, you have RSI timelines that are shorter than that. You have just such fundamental uncertainty about the technology itself, in addition to the geopolitics and everything else. So to me, it’s the kind of classic one foot in front of the other, walk before you run.
Tom Reed: What’s the phrase about stepping, making your way over the river, over the stones?
Julian Gewirtz: Right, exactly. There’s a famous Chinese saying, which Deng Xiaoping used about economic reform: crossing the river by feeling for the stones. I don’t see another alternative, because we’re not gonna sit down at a table and design a perfect mechanism and then implement it tomorrow.
How to spot a fake China expert [01:20:48]
Tom Reed: I now have a series of very selfish questions I would like to ask you, in that I’m essentially interested in how I could build or recognise real China expertise. In particular, I’m wondering: how do you recognise a China specialist as the real deal when the whole field is really quite opaque?
Julian Gewirtz: So the number one thing is humility. If a China expert comes to me, or comes to anybody, and says they have all the answers and are presenting them with total confidence, the bullshit metre should go off right away. And from my perspective, the central challenge of doing China policy and building China expertise really is dealing with uncertainty and having nonetheless to make educated decisions.
That’s square one. Some of that has come across in this conversation, I hope, where we’re sort of saying there are some things we know, there are some things we don’t know, and everything we’re trying to do is in that space between them.
And then second, it’s not easy to go spend time in China, necessarily, for everybody. But people who have spent time there, who have a long track record, who read the source material and have the context to make sense of it — that’s also just essential, and it’s hard.
It’s this funny thing I feel at this point in my career: I am doing that almost every day, and I was doing that almost every day a decade ago. And it has not gotten more fun, because Chinese Communist Party documents are not beautiful New Yorker articles written to keep the reader interested, and certainly aren’t in short-form video. But candidly, there’s no other way. And it’s how that system works. And so if you want to understand it, you gotta go there.
Tom Reed: Where in China should I hang out? I’ve been to Beijing, I’ve been to Shanghai, I’ve been to Hangzhou, I went to WAIC [World Artificial Intelligence Conference] this summer. But other than that— I mean, I also don’t speak Mandarin. But if you were to recommend to me, “You should really go to these cities”—
Julian Gewirtz: Well, hold on. I was giving you an answer to how a hardcore China expert should operate. I think there is a different answer for somebody who wants to be educated and informed on China, but is not going to make their professional life being the China expert in the room, so to speak.
So for that category — which, unless you’re planning a major life renovation, I feel like is probably the sweet spot — I don’t think actually you need to be going and… There’s a famous Sinologist who described reading these Chinese Communist Party documents as eating sawdust. So I don’t think you need to be doing that.
But yeah, I think getting over there, getting exposure directly, and then also finding a staple of things you read and people you talk with who are really reliable — and who are not making BS calls, but are kind of really doing the hard work as intermediaries, almost — that’s sort of the most essential thing.
And then I also think remembering that China is not simply the Chinese Communist Party, and there is incredible cultural production, and there’s actually a really diverse kind of ecosystem.
And some of the most interesting moments over the past few years for me have been when friends of mine… For instance, a lot of people who are on TikTok migrated briefly to a Chinese app when TikTok went offline here in the United States, and were coming to me, sending me videos of people in China singing and doing all kinds of random stuff. And it was sort of like, wow, this is actually a degree of exposure to China that friends of mine who have nothing to do with China are suddenly getting.
In terms of where to go in China, for most foreigners who travel there, you hit the kind of main tech-focused hubs — Shanghai and Hangzhou — and then Beijing, which is one of the old imperial capitals and today the place where the Chinese Communist Party is headquartered. It’s a huge country with a lot of diversity, geographically and all that. So I do think it’s always good to kind of get out of the main coastal hubs, where you see all the development and all the modernity, and get to places that are a little bit more off the beaten path.
But I’ll also just be candid and say that’s hard to do as a foreigner, particularly somebody who doesn’t speak Chinese. So I think you don’t have to set too high a bar for yourself. If you make it to Beijing and Shanghai, you are already in the very top percentage of people in the West who have exposure directly to China.
Tom Reed: I will say, I was surprised at how much heavy lifting Google Translate could do for me, though. It’s a very well-oiled machine, where people know to pull out their translation apps. You can usually get across the things you need to.
Julian Gewirtz: Yeah, not to tie this back to the substance of our conversation, but that does actually reflect, I think, a really interesting difference that we observe between the United States and China, which is just a much higher degree of comfort with and optimism about AI over there than here in the United States. The polling is remarkable on this. Basically, the United States is at the very bottom of rankings for optimism about the benefits of AI for people, and China is at the very top.
And what you just said made me think of it, because it’s exactly that thing of: you’re walking around Beijing or Shanghai, and someone in China whose life is already largely integrated on WeChat and into other virtual platforms is very comfortable doing a bit of real-time speech translation, and having that be kind of seamless, in a way that I think is still pretty unusual here.
Tom Reed: Yeah. OK, you’re not going to like this next question, because I’m going to ask you not to be humble, and you just told me that the mark of a China expert is humility. But what is the nature of your edge? What do you think is the thing you understand about China better than almost anyone else in the US? Is there a thing you would identify there?
Julian Gewirtz: OK, that is making me uncomfortable. So, I mean, I don’t think this is a unique-to-me thing, but I do think the most important kind of skill that I have tried to acquire is not simply to treat China policy as some sequestered, distant, esoteric thing that happens over there — and then you bring back the right answer to people who care about domestic policy or broader US foreign policy or tech policy — but to see it as totally integrated and inextricable.
And that’s actually, I find, very challenging intellectually, because you have to master the China side of it. You have to do all that deep work — getting your PhD in modern Chinese history or politics or whatever the case may be, spending time there, learning how the system works. And then you have to somehow then also figure out how to make that integrated into the tradeoffs that policymakers and business leaders have to make when it comes to China policy.
Because one of the themes, in a way, of what we’ve been discussing is there might be an ideal set of solutions vis-à-vis China, but at every turn reality presses against them in some way. And I think in the field of people who work on this stuff, that skill, when I have seen it, is the most impressive thing, and I’m trying to bring it to bear in my own career.
Julian’s China and AI reading list [01:28:43]
Tom Reed: Who else should I be reading, other than your blog, Paper Tiger? Are there other things which you would really recommend to me?
Julian Gewirtz: A lot of things. When I was in government I did a lot of work on US–China tech competition, and before I went into government I was writing about Chinese technology policy a lot. But what I saw over the four years I worked in the Biden administration made me convinced that the AI story that we’ve been talking about today was on a scale of importance so much greater than almost anything else I could imagine doing.
So when I came out, I was like, “I’m going to use the time that I now find myself with to go as deep as I can.” And I talked to a few friends who were those people, and were much deeper, and basically got a reading list from them.
So I learned so much from Matt Sheehan, who we mentioned; from Graham Webster, who’s out at Stanford and does incredible work; from colleagues at CSET, the Center for Security and Emerging Technology — people I’m sure are well known to your listeners, like Helen Toner; and from Chris McGuire, who’s at the Council on Foreign Relations now, and many others on the AI account.
But I would just say, to me, one of the most striking things is these are not all people who agree on the policy solutions, but they are all people who are doing the really hard work analytically, and who sort of share that with the public in a really meaningful way. So that’s sort of on the AI account.
And then on China, I mean, we are in a little-noticed sort of renaissance or flowering of amazing China studies and China scholarship. And I say little-noticed because I think there’s a meme out there that, because it has gotten harder to go to China and do research, the quality of China studies must therefore have suffered.
And one of the most extraordinary things to me is that, despite how hard it is, there is incredible work happening in the academy and think tanks and elsewhere, and journalism — even though for the journalists it’s incredibly hard; many of them can’t even go to China anymore. So I think my diet of China-related content feels as rich as it’s ever been.
And just there, I would say, maybe for your listeners — they can like to nerd out; obviously they’re certainly doing that if they’ve reached this point in our conversation. And so there, I would say it’s worth actually reading some of the stuff that China academics are doing — not just the people who are sort of in the hot-take industry, but–
Tom Reed: My industry. [laughs]
Julian Gewirtz: Well, no, I mean, I don’t think that’s fair. But there’s a book that I always give as an example of this for me, which is an academic book called The Gilded Cage by a Harvard sociologist named Ya-Wen Lei. She has a whole framework for the development of China’s technology regulatory regime. And it’s a classic example of extraordinary scholarly work that actually has direct relevance to the kinds of things we have been talking about every day. But you’re not going to know about it unless you are in this field.
Tom Reed: What’s it about? The Gilded Cage? What’s it about?
Julian Gewirtz: It’s basically about the development of China’s technology regulatory regime for the internet and for fintech. And she did an incredible amount of fieldwork embedded in tech companies in China, and she also comes up with a series of kind of frameworks for analysing this. And she’s playing for 2big stakes: she’s playing for fundamental understandings about development and state capitalism in China, relations of firms and the state. But it’s just a good example of this kind of book.
And there are some books that cross over, like Joseph Torigian‘s biography of Xi Jinping’s father, which is a deep scholarly work crossed over, got broader attention.
Tom Reed: This is The Party Comes First?
Julian Gewirtz: Exactly. And I always think about the fact that, for each of those books, there are two or three others that maybe don’t, but have a huge amount to offer as well. And so one of the things I’m trying to do, actually, in my own writing is, as much as I can, give a little lift to some of those works — because they’re out there; it’s happening. China studies is vibrant in all of these ways, and yeah, it’s a cool moment to be in the field.
The Chinese thinkers worth reading, even when you disagree [01:33:34]
Tom Reed: Great. Do you have a favourite non-dissident Chinese intellectual?
Julian Gewirtz: The people who I find most interesting are largely not people I agree with. It’s not like I’m reading Chinese academics and nodding along saying, “Preach,” or whatever.
But for instance, there’s a guy named Jiang Shigong — and I can send you some stuff of his — but he is very far from somebody who I think shares any views with me, but I find him to be an enormously revealing analyst of contemporary China, of Xi Jinping. And he’s a very ambitious intellectual. He is very supportive of many of the things Xi Jinping does, and he’s definitely comfortably operating, by all accounts, in that system.
But when I think about who has been the most sort of precise framework generator that has helped me generate my own frameworks for understanding that system, his work is quite useful, and a bunch of it has been translated — even if I’m like, “I do not like the upshot of that for America.” But it’s important to actually reckon with those people.
And I would say, more generally, there is a set of government-aligned thinkers who are serious thinkers. These are not simply people who are doing propaganda work, but they are very supportive of what China is doing. And I think it’s actually really important for people in my position to read them, to reckon with them, to take their ideas seriously.
The best example of this, by the way, is Wang Huning, who we mentioned earlier, who’s one of Xi Jinping’s top advisers. He’s on the Politburo Standing Committee. He started off as [an international politics] professor at Fudan University, and a number of his early writings — all the way back from the 1980s and early 1990s — are still incredibly interesting, layered reads, and sometimes they get caricatured.
His most famous book is called America Against America, and it sometimes is described as just an anti-America polemic. And there’s a lot of criticism of the United States in there, for sure, but there’s also all this really interesting language about how technologically advanced the United States is and how innovative the United States is. And when you think about how he has approached governing China, actually both sides of that are really important.
And a closer reading of that text — a closer understanding of this isn’t only the parts that are about what he sees in America and criticises, but also what he sees in America and envies — both of those impulses shape what he and Xi Jinping and other Chinese leaders have done. So yeah, it’s important to reckon with.
Tom Reed: What Chinese intellectual currents that are particularly relevant to AI policy do you think we should be aware of? So I’m vaguely familiar with the Industrial Party as a movement. I’ve read Lu Nanfeng and Wu Jing’s essay on this. Can you carve them into blocks that I should reckon with, or is that difficult to do?
Julian Gewirtz: So I’d say a couple things. First, there is a longstanding sort of futurist strand of writing in China. You mentioned earlier in our conversation the Alvin Toffler influence in the 1980s that I wrote about. And I think understanding these kind of techno-futurist visions in China, and the way in which they see industrial development and party leadership, is one really important strand of thinking.
A second, which used to be much buzzier a few years ago and has certainly diminished since then, is Chinese science fiction. Jordan Schneider, who does the ChinaTalk podcast and is a friend, likes to talk about how influential sci-fi has been in shaping American and Western visions of the AGI future.
And there was this incredible flowering of science fiction writing in China over the course of the 1980s, 1990s, and 2000s. And it was one of those areas where you saw people doing politically sensitive writing, but somehow getting away with it under the sort of sci-fi banner. And I think it’s important for people who are trying to understand the kind of currents in China to look at that writing.
And obviously Liu Cixin is the most famous — The Three-Body Problem is the most famous — but there are a range of others. There’s a guy named Chen Qiufan, who ended up actually cowriting a book with Kai-Fu Lee on AI futures, but is also just a really interesting sci-fi writer; a woman named Hao Jingfang, who wrote an amazing sort of novella — long short story — about kind of inequality in Beijing. And so there’s a number of these things which are really interesting, even if the space for that writing is smaller now. But I think it reflects this kind of set of possibilities.
And then, I guess, third and finally, just to kind of foreground, is to say: reading these regime-aligned or regime-supportive intellectuals in China, who position AI at the absolute heart of the way in which power will be distributed in the future, and give that kind of deeper intellectual foundation to Xi Jinping saying something like “technology is the main battlefield of international competition” — that’s really important too, in addition to reading the people who are actually working on AI issues.
Which AI model is the best China watcher? [01:39:27]
Tom Reed: And what about using the AI models themselves? Which AI model is the best China watcher?
Julian Gewirtz: Of the Chinese models?
Tom Reed: Across the whole range. If you’re doing research with a model, which one are you using? I’ve found, for instance, ChatGPT seems to be better at finding WeChat links than Claude, for whatever reason.
Julian Gewirtz: Yeah, it’s interesting you said that. I was about to say something similar, which is, I’m not terribly original on this, but yeah, I find for the kind of research that I’m doing — to the extent that I’m using AI to help me do it — ChatGPT somehow has seemingly greater facility with the Chinese parts of the internet than Claude. And I’ve heard some people say I should be checking out Gemini more on this front. I confess that I haven’t really. To the extent that I’m using this, I’m pretty satisfied with what I’m getting. But yeah, it is interesting.
And I will say, for me, what I have not done, but others have done — and there’s some really interesting work on this — is kind of comparing what you find when you research China on Chinese models and American models. Obviously we all know about certain censorship guardrails that are built in. It’s not surprising that you find much less about the Tiananmen crackdown on a Chinese model, but that, actually, because of the priority given to the training corpus being in Chinese, there are sometimes very different answers you get when you ask the question in Chinese versus in English — which is itself a really interesting thing, and I suspect is also probably true in some of the American models.
But that connection between the corpus that’s being used — the language of the corpus that’s being used to train — and the kind of epistemic consequences of that is something that I think is just going to become more and more important.
By the way, we are beginning to see a Chinese discourse on this too: people in China writing about the fact that what they choose to feed in is what others will be processing for centuries to come — and hoping to shape international discourse through shaping the kind of corpus of Chinese and China-relevant material — is a super interesting area that there’s some work happening on, and we need more.
Tom Reed: Do we have any idea what they’d be prioritising other than censorship? What narratives would they be actively trying to push?
Julian Gewirtz: Absolutely. So the simplest version is essentially propaganda, where the goal is to push a positive set of narratives about China and downplay criticisms of China; a positive set of narratives about Chinese leaders, downplay criticism of them. But then there’s a whole range of other topics, like Taiwan — where their goal is to maximally shape a version of understanding of Taiwan issues that comports with their official line.
And the idea that we could have models that would be unwittingly built in such a way that they reinforce that, I think, is the kind of concern that is really reasonable to have — particularly now that we begin to see corners of the sort of official and semi-official discourse in China talking about this as an opportunity for them.
A world tour of China’s influence [01:42:59]
Tom Reed: A final question I have: where do you think are the best places on Earth to visit to understand China’s international strategy and its investment in the Global South?
Julian Gewirtz: I spent a bit of time when I was at the State Department working on this issue set and doing a lot of travel around it.
To focus on the Global South questions specifically, my own view is that it’s good to get a kind of spectrum of different places that have found different ways, and are finding different ways, of striking the balance between getting more benefit out of their ties with China and maintaining their own sovereignty and autonomy on the world stage.
So at one end of that spectrum you have India — a country that, because of the border conflict and other issues, decided to block a bunch of Chinese apps, and really try to push back against Chinese influence. And I think it’s super interesting and important to hear their perspective, which is not one-for-one with the Washington perspective, even though it is quite concerned about increasing Chinese influence.
Then I think Central Asia is a super interesting place, because of the dynamic of competition between China and Russia there. Obviously China and Russia are close strategic partners, but they also have some areas where they’re competing for influence, and Central Asia is one of them.
Another super interesting place, which I have not been to, but I think would love to sort of see and talk to people on this score, is Egypt — where Xi Jinping just went on a state visit — and the kind of question there of a country that has deep security ties with the United States, but is clearly making a major play to upgrade its relationship with China. Super interesting.
And then in the Western Hemisphere, I also think — whether it’s Mexico, where there has been a tonne of scale-up of Chinese companies manufacturing, or Peru, where China has been building this megaport on the Pacific coast — these are really significant stories.
And then finally, I would just say: every year, the first trip abroad that the Chinese foreign minister takes, for decades, has been to Africa. And it is a place where China, I think, has a much more complex and differentiated and layered strategy than some accounts would have you think. But whether you’re going to countries that have really gone all in and see the Chinese development model as something that they want to ride for their own development, or to countries that feel like they’ve gotten bad deals out of China but still know that they have to have a good relationship with Beijing — super worthwhile. I realise I have just essentially told you to go to every place in the world, but the upshot—
Tom Reed: I like an excuse to go everywhere.
Julian Gewirtz: Exactly, exactly. You should get the podcast to pay for it. But no, I guess to my mind, the upshot of all of that is: China has a global strategy. It is pursuing it in many different ways around the world. It wants global influence. It wants that influence to look different than American global influence has looked. But there’s no one place where you can go to see it all, because Beijing is doing so many different things in so many different places.
Tom Reed: Great. Well, it’s been a real pleasure, Julian. Thank you for coming on the show, and talk to you next time.
Related episodes
About the show
The 80,000 Hours Podcast features unusually in-depth conversations about the world's most pressing problems and how you can use your career to solve them. We invite guests pursuing a wide range of career paths — from academics and activists to entrepreneurs and policymakers — to analyse the case for and against working on different issues and which approaches are best for solving them.
Get in touch with feedback or guest suggestions by emailing [email protected].
Our crash course on transformative AI
We've carefully selected 10 key episodes to help listeners get to grips with the potential upsides and downsides of powerful, transformative AI.
Check out 'The 80,000 Hours Podcast on AI'
Listen here, or anywhere you get podcasts:
If you're new, see the podcast homepage for ideas on where to start, or browse our full episode archive.







