Enjoyed the episode? Want to listen later? Subscribe here, or anywhere you get podcasts:

Technology is just going to be an amplifier of human intention, this human innate desire…to deceive, to manipulate. The visual medium is a very powerful way of doing that.

Nina Schick

You might have heard fears like this in the last few years: What if Donald Trump was woken up in the middle of the night and shown a fake video — indistinguishable from a real one — in which Kim Jong Un announced an imminent nuclear strike on the U.S.?

Today’s guest Nina Schick, author of Deepfakes: The Coming Infocalypse, thinks these concerns were the result of hysterical reporting, and that the barriers to entry in terms of making a very sophisticated ‘deepfake’ video today are a lot higher than people think.

But she also says that by the end of the decade, YouTubers will be able to produce the kind of content that’s currently only accessible to Hollywood studios. So is it just a matter of time until we’ll be right to be terrified of this stuff?

Nina thinks the problem of misinformation and disinformation might be roughly as important as climate change, because as she says: “Everything exists within this information ecosystem, it encompasses everything.” We haven’t done enough research to properly weigh in on that ourselves, but Rob did present Nina with some early objections, such as:

  • Won’t people quickly learn that audio and video can be faked, and so will only take them seriously if they come from a trusted source?
  • If photoshop didn’t lead to total chaos, why should this be any different?

But the grim reality is that if you wrote “I believe that the world will end on April 6, 2022” and pasted it next to a photo of Albert Einstein — a lot of people would believe it was a genuine quote. And Nina thinks that flawless synthetic videos will represent a significant jump in our ability to deceive.

She also points out that the direct impact of fake videos is just one side of the issue. In a world where all media can be faked, everything can be denied.

Consider Trump’s infamous Access Hollywood tape. If that happened in 2020 instead of 2016, he would have almost certainly claimed it was fake — and that claim wouldn’t be obviously ridiculous. Malignant politicians everywhere could plausibly deny footage of them receiving a bribe, or ordering a massacre. What happens if in every criminal trial, a suspect caught on camera can just look at the jury and say “that video is fake”?

Nina says that undeniably, this technology is going to give bad actors a lot of scope for not having accountability for their actions.

As we try to inoculate people against being tricked by synthetic media, we risk corroding their trust in all authentic media too. And Nina asks: If you can’t agree on any set of objective facts or norms on which to start your debate, how on earth do you even run a society?

Nina and Rob also talk about a bunch of other topics, including:

  • The history of disinformation, and groups who sow disinformation professionally
  • How deepfake pornography is used to attack and silence women activitists
  • The key differences between how this technology interacts with liberal democracies vs. authoritarian regimes
  • Whether we should make it illegal to make a deepfake of someone without their permission
  • And the coolest positive uses of this technology

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: Type 80,000 Hours into your podcasting app. Or read the transcript below.

Producer: Keiran Harris
Audio mastering: Ben Cordell
Transcriptions: Sofia Davis-Fogel

Highlights

Deepfakes

Nina Schick: A deepfake is essentially a piece of media, a piece of synthetic media. That’s to say, a piece of fake media that’s either manipulated by AI — or, increasingly, as the technology improves — entirely generated by AI. The first thing to say is that the ability of AI to do this is nascent. A huge breakthrough came only in 2014 when somebody named Ian Goodfellow — who is now actually one of the lead AI scientists at Apple — when he was a graduate student, he published a paper on this challenge of how to actually get a machine learning system to create content that didn’t exist before. It was a really difficult one.

Nina Schick: Thanks to the revolution in deep learning over the past decade, AI was very good at categorizing things. That’s why we can get things like autonomous cars. But actually getting it to create something new is a difficult challenge. What Goodfellow did was that he decided he could pit two machine learning systems or two neural networks against each other in an adversarial game. He found that once you did that, it could actually generate content — which was a huge breakthrough in the sense that we hadn’t been able to see a machine learning system do this before.

Nina Schick: It’s only in 2014 that this first paper was emerging at the cutting edge of artificial intelligence. And a few years down the line, in 2017, that’s when, as this field was developing, we started to see the emergence of the first deepfakes online. I’m sure we’ll get into that in the form of nonconsensual pornography. Again, the first thing to point out is that the unbelievable ability of AI to actually manipulate and generate entirely synthetic media is new, we’re at the very, very beginning of this journey. This can come in many different forms.

Nina Schick: It can come in audio format. It can come in video format. It can come as an image. It can even come as synthetic text. One of the unique things about synthetic media — which is very relevant to our conversation today — is its ability to recreate humans. This is now manifesting in two ways. One, AI is starting to be used to create entirely synthetic people. You already see that with thispersondoesnotexist.com, for example, where you go to that website, you click the page, and you see AI generating autonomously an image of a person who does not exist — who to your eye looks absolutely real, photorealistic. We would not be able to tell that that’s a person that doesn’t exist. Increasingly, AI will be able to do that also in audio visual format, so voice synthesis, video synthesis.

The influence of synthetic media today

Nina Schick: I don’t think it’s right to think of deepfakes and existing mis- and disinformation as two separate issues. Deepfakes are merely the more sophisticated form of visual disinformation that is going to become increasingly ubiquitous in already what is a corroding information ecosystem. When people suggest that, oh, deepfakes haven’t been as harmful as the existing modes of mis- and disinformation, I don’t see why those are two different issues.

Nina Schick: I think it is absolutely vital when you talk about mis- and disinformation to underline that way before deepfakes even started doing damage, old forms of mis- and disinformation were already doing a significant amount of real-world harm. Then getting to the issue of deepfakes more specifically, the reason why perhaps they haven’t been seen to do as much damage as sometimes has been predicted — at least in hysterical media reporting — this is particularly relevant to the realm of politics. A lot of the fears around deepfakes was that somehow an election would be swung or what happens if Kim Jong-un releases a deepfake saying he’s nuking America and then we’re in the nuclear armageddon scenario. The reason why we haven’t had that yet is because existing forms of mis- and disinformation when it comes to politics are already devastatingly effective. There’s no need to make a deepfake right now. Because, again, unlike what is sometimes perceived as being true from the hysterical reporting, the barriers to entry in terms of making a very sophisticated video deepfake are a lot higher than people think.

Nina Schick: That’s not to say that those barriers are going to exist forever, because we’ve already touched upon how this field of technologies is evolving so quickly that any kind of restrictions you see right now are not because of ethical concerns. It’s purely to do with technical limitations.

The history of disinformation

Nina Schick: One of the brilliant early examples is a photograph of Abraham Lincoln, who — although he was lionized after his death as this iconic president, during his lifetime he was beset by rumors of ugliness. After he was assassinated, a portrait painter needed to find photographs of him looking heroic, and he couldn’t find any. What he did was take an engraving of a southern politician, John C. Calhoun, who ironically was a bitter rival of Lincoln’s during his lifetime, because they were opposed on the abolition of slavery…

Robert Wiblin: On slavery, right?

Nina Schick: Yeah, exactly. He took a photograph of Lincoln’s head and superimposed it onto Calhoun’s body, because Calhoun was the kind of politician, he had the gravitas, he had the posture that this portrait painter was looking for. That was only discovered to be a manipulation in the 1980s, I think. So 100 years after the fact. The thing that’s different now is accessibility, scale, fidelity, and what type of media we’re talking about. We’re not talking about editing images with Photoshop. It’s far more sophisticated than that.

Nina Schick: Before, we weren’t able to say, okay, we’re going to take AI and take five seconds of your voice, and now I can clone your voice. It’s not, for me, comparable at all to what has been possible in the past. But I also absolutely agree that if you look at the history of visual or media manipulation, it’s been something that’s been around since the birth of modern media. Again, to me, that just is more of an interesting point about the nature of humanity. Technology is just going to be an amplifier of human intention, this human innate desire. It’s always going to exist to deceive, to manipulate. The visual medium is a very powerful way of doing that.

Text vs. video

Nina Schick: I’m not at all saying that text cannot be compelling or convincing. Again, when it comes to synthetic text generation, if you look at what GPT-3 is capable of right now, I can see it being an extremely powerful tool of persuasion and coercion or manipulation. Because at scale, you could create human conversations or interactions in a way that is just mind blowing. There’s a great paper you should read by the Middlebury Institute of International Studies Center on Terrorism, Extremeism, and Counterterrorism, where they tested GPT-3’s capability to radicalize people online, and it makes for very scary reading.

Nina Schick: Again, going back to the visual side here, the reason why I focus on this — and this is by no means saying that synthetic texts should be discounted — is because the most important medium of human communication right now is audiovisual media. People read less. People interact with text less. The majority of the world who’s going to join the information ecosystem in the next 10 years is going to be, well, the literacy levels might be lower than some in western countries. It’s already two-thirds of humanity.

Nina Schick: That’s 67% of people who go to video as their first source of information. You know this, like from the attention economy we’ve built over the past 30 years. Is it easier for you to scroll through your phone on Instagram or Twitter and get information that way? Or is it easier for you to sit down and read a textbook? It’s not to say that texts cannot be compelling or a written lie cannot be compelling, and I’m sure, again, there’s a whole area of study to be done into AI-generated synthetic texts and how that could be used as a way to convince people. But I think to say that somehow visual media is not the most important medium of communication, when video seems to be becoming the first source of information for most people in the world, I think is probably intellectually dishonest.

Positive uses

Nina Schick: Every industry that uses media (and what industry doesn’t?) is going to be touched by the rise of synthetic media. And that’s because AI is going to democratize content creation, it’s going to make it so much cheaper. By the end of the decade, a YouTuber or a TikToker will be able to produce the same kind of content that’s only accessible right now to a Hollywood studio. So, that is going to mean so many opportunities for the creative industries. I mean, for one, entertainment and film are just going to get very good. And you won’t need to be a Hollywood studio to produce some really amazing creative content.

Nina Schick: Another real-world legitimate application of synthetic media is a startup that I really think is doing fantastic work. They’re based in London, they’re called Synthesia. And they basically use their synthetic media platform to generate corporate communications videos, training videos, educational videos for their Fortune 500 clients. You don’t need to go into a studio anymore and hire actors and get a green screen, you can basically create your communications video as easily as though you’re writing an email. And you can then, on their backend, choose to put that out in like 16 different languages with the click of a button, right? So it’s going to transform every industry imaginable. I think by the end of the decade, some experts who I was talking to — and it’s a really punchy stat — but I think the direction of travel is clear. They think that up to 90% of audiovisual content online will be synthetically generated.

Robert Wiblin: It’s a big forecast within 10 years.

Nina Schick: Punchy stat, yeah. But I think that is the direction of travel. And for a real social good example, here’s one. There’s a company called VocaliD, which is working on synthetic voice generation, to give those who have lost the ability to speak through stroke, cancer, neurodegenerative disease, etc. their voice back. Or those who never had the ability to speak at all can have a synthetic voice. Again, this technology is just an amplifier of human intention. It will be weaponized by bad actors and used for mis- and disinformation, but it’s also going to be commercially very relevant, transform entire industries, and also be used for good.

Articles, books, and other media discussed in the show

By and about Nina

AI companies doing interesting work

Mis- and disinformation programs and resources

Other links

Related episodes

About the show

The 80,000 Hours Podcast features unusually in-depth conversations about the world's most pressing problems and how you can use your career to solve them. We invite guests pursuing a wide range of career paths — from academics and activists to entrepreneurs and policymakers — to analyse the case for and against working on different issues and which approaches are best for solving them.

The 80,000 Hours Podcast is produced and edited by Keiran Harris. Get in touch with feedback or guest suggestions by emailing [email protected].

What should I listen to first?

We've carefully selected 10 episodes we think it could make sense to listen to first, on a separate podcast feed:

Check out 'Effective Altruism: An Introduction'

Subscribe here, or anywhere you get podcasts:

If you're new, see the podcast homepage for ideas on where to start, or browse our full episode archive.