#121 – Matthew Yglesias on avoiding the pundit’s fallacy and how much military intervention can be used for good

If you read polls saying that the public supports a carbon tax, should you believe them? According to today’s guest — journalist and blogger Matthew Yglesias — it’s complicated, but probably not.

Interpreting opinion polls about specific policies can be a challenge, and it’s easy to trick yourself into believing what you want to believe. Matthew invented a term for a particular type of self-delusion called the ‘pundit’s fallacy’: “the belief that what a politician needs to do to improve his or her political standing is do what the pundit wants substantively.”

If we want to advocate not just for ideas that would be good if implemented, but ideas that have a real shot at getting implemented, we should do our best to understand public opinion as it really is.

The least trustworthy polls are published by think tanks and advocacy campaigns that would love to make their preferred policy seem popular. These surveys can be designed to nudge respondents toward the desired result — for example, by tinkering with question wording and order or shifting how participants are sampled. And if a poll produces the ‘wrong answer’, there’s no need to publish it at all, so the ‘publication bias’ with these sorts of surveys is large.

Matthew says polling run by firms or researchers without any particular desired outcome can be taken more seriously. But the results that we ought to give by far the most weight are those from professional political campaigns trying to win votes and get their candidate elected because they have both the expertise to do polling properly, and a very strong incentive to understand what the public really thinks.

First and foremost, that means representing issues as they would be in a hotly contested campaign. If someone says that they sure like the idea of taxing carbon, how much do they still like it when they find out it means their electricity bills would be $100 higher, and gas will cost 20 cents more a gallon? And do they still like it when they know one of the candidates is against it and says it will cost local jobs? This sort of progressive ‘stress testing’ is more work, but can lead researchers to very different conclusions than just asking people favour ‘policy X’.

The problem is, campaigns run these expensive surveys because they think that having exclusive access to reliable information will give them a competitive advantage. As a result, they often don’t publish the findings, and instead use them to shape what their candidate says and does.

Journalists like Matthew can call up their contacts within campaigns and get a summary from people they trust. But being unable to publish the polling itself, they’re unlikely to be able to persuade sceptics.

That’s a pain and a legitimately hard problem to get around. But when assessing what ideas are winners, one thing Matthew would like everyone to keep in mind is that politics is competitive, and politicians aren’t (all) stupid. If advocating for your pet idea were a great way to win elections, someone would try it and win, and others would copy. If none of the pros are talking about your hobby horse, it might be because they know something you don’t.

One other thing to check that’s more reliable than polling is real-world experience. For example, voters may say they like a carbon tax on the phone — but the very liberal Washington State roundly rejected one in ballot initiatives in 2016 and 2018.

Of course you may want to advocate for what you think is best, even if it wouldn’t pass a popular vote in the face of organised opposition. The public’s ideas can shift, sometimes dramatically and unexpectedly. But at least you’ll be going into the debate with your eyes wide open.

In this extensive conversation, host Rob Wiblin and Matthew also cover:

  • How should a humanitarian think about US military interventions overseas?
  • From an ‘effective altruist’ perspective, was the US wrong to withdraw from Afghanistan?
  • Has NATO ultimately screwed over Ukrainians by misrepresenting the extent of its commitment to their independence?
  • What philosopher does Matthew think is underrated?
  • How big a risk is ubiquitous surveillance?
  • What does Matthew think about wild animal suffering, anti-ageing research, and autonomous weapons?
  • And much more

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

Producer: Keiran Harris
Audio mastering: Ben Cordell
Transcriptions: Katy Moore

Highlights

How Matt thinks about public opinion research

Matthew Yglesias: Something that I have found is that the world of public opinion research is kind of bifurcated — or trifurcated, if that’s a word. On the one hand, you have people doing polls to go to the media, for public consumption. Then you have people doing polls to issue advocates, who want polls that say their positions are popular. And then you have people doing polls because they’re trying to win campaigns. Survey methods are the same across these things, but what your incentives are and what you actually care about are pretty different.

Matthew Yglesias: Unfortunately, for the mass public, to get a good read on public opinion — like a rigorous look at how things are without putting a lot of English on it with question wording — you do have to talk to the people who are doing private work for political campaigns. They are the ones who have the strongest incentive to get the question right — not in terms of the survey sample, but in terms of what questions you ask people.

Matthew Yglesias: I know there’s a couple guys on Twitter who are very angry that I say carbon taxes are unpopular. They’ve got their polls that say it’s popular, but we know in America that when this has been put to ballot initiative in Washington State, it’s lost badly, twice. We also know that in Europe, where the macropolitics is greener than in the United States, the European elected officials act as if stiff carbon pricing is going to be very unpopular. In Australia, when the government tried to put a carbon tax in place, there was a big backlash to it.

Matthew Yglesias: So people say that if you do the best kind of issue surveys, where you do partisan frames, you say, “Some Democrats are proposing a blah, blah, blah, price on carbon, which they say will reduce climate change in the most cost-effective way possible. Republicans say it’ll raise the price of gas by…” — then you give them a real number, not a crazy lie: “It’ll raise the price of gas by so many cents, raise the price of electricity by so much.” It becomes incredibly unpopular in that framework — like, less popular than commercial legalization of heroin.

Matthew Yglesias: When the price of gasoline spiked in the United States a few months ago, it was a huge deal politically. People were losing their shit about it. Probably the best media poll on this is a Reuters survey, where they asked people, “Should we take drastic action to stop climate change?” Most people said yes. Then they asked, “Would you be willing to pay $100 a year more in taxes to stop climate change?” People said no. A hundred dollars a year is not that drastic, you know? I spend $100 a year on things that I don’t think are important at all.

Matthew Yglesias: So I think that it’s tough. As a professional journalist, the kind of thing that I can do is ask people to tell me things that they cannot say in public, and then I can try to triangulate the things that they have told me against publicly available information — like this Reuters thing, like the Washington polls, like the fact that practical politicians don’t campaign heavily on green tax shifts — and try to make people sort of see the truth. But it’s hard. I never know, because I’m not a huge scoops guy, and if you want to believe that I’m lying to you or that my sources are lying to me, it’s challenging to prove otherwise.

Matthew Yglesias: I have stumbled, I would say, a little bit ass-backwards into being a guy who writes a lot about polls and public opinion surveys. And it was not my intention, exactly — I don’t know that it’s super important for the typical person to be deeply, deeply invested in this. The main thing that I would like the person on the street to take away from it is that workaday politicians running for office are probably better informed about the state of public opinion than you are. And if you are finding yourself baffled as to why someone won’t say something or embrace something that you think they should, it’s probably because their surveys indicate that it’s not that popular, and maybe try to be a little less mad.

Matthew Yglesias: Now, what should you advocate for? Whether it’s you, or the listener, or whoever else, you probably should advocate for the right thing to do. And you should probably advocate for the right thing to do as if you were trying to be persuasive, which I think people oftentimes don’t do on the internet. I would say heavy consumers of political punditry spend a lot of time pounding the table on behalf of what they think is the right thing to do, being very impatient that other people in positions of greater responsibility aren’t saying exactly what they want them to say, and searching for bias-confirming information that indicates that they are right about everything.

Loss aversion and long-term interests

Matthew Yglesias: People dwell on the downside and loss: people are more worried about losing what they have than about gaining something that they don’t currently have. So it’s hard for people on the left to persuade people that some new program is going to be amazing and it’s going to be worth paying the taxes, but it’s also hard for people on the right to win arguments that are like, “Well, without this program, we can cut taxes and that’s going to create better incentives to save and invest, and that’s going to increase growth by 0.002%.” When you compound that out for X number of years, people get a lot of like, “Eh, I don’t know about that.”

Matthew Yglesias: People are also just not great at thinking about the long term. If you ask somebody who’s 40, “Do you care about what’s going to happen to you when you’re 70?” They’d be like, “Of course. I’m not a crazy person.” Now you ask them, do they care what’s going to happen hundreds of generations in the future, they might say no. We might say, “OK, you discount more than you should.” But even when they say that they’re not discounting the future of their own life, I think they just pretty clearly are when they’re making policy decisions.

Matthew Yglesias: People are not in the habit of really thinking about their long-term interests, and we see that in all kinds of behaviors. It’s why there’s people who smoke. It’s why lots of people struggle with all kinds of personal health issues. And in their political thinking, it’s even worse. To really get people to think, “How’s this going to play out?” — it’s tough.

Rob Wiblin: Yeah. People talk a lot about means testing making policies more or less popular. So whether you allow everyone to access a program, or only people who are below some income threshold.

Matthew Yglesias: Yeah. People argue about this a lot. I actually think if you pay attention to what everyone’s saying, they are not in that much of a tension really. If a program exists and everybody is using it, that makes it much, much harder to get rid of, versus a very narrow type of program — so there is a political durability benefit to some universalistic designs. At the same time, the broader you make the program, the more tax revenue you need to get.

Matthew Yglesias: And so, if we do comparative welfare state design, typically the really big welfare states have a broad tax base: they use a lot of payroll tax, they use a lot of value-added tax. Those taxes tend to be unpopular, so it’s harder to create a really broad universal program, but also harder to get rid of a really broad universal program.

Matthew Yglesias: What should you actually try to do? I don’t know that there is a general answer to that kind of question. I think the most important thing about means testing is how it actually makes the programs function — and that making everyone fill out 17 forms to verify their income is itself quite costly.

How military intervention looks as a humanitarian intervention

Matthew Yglesias: This was my original entry point into the world of effective altruism, because I was very taken in the late ’90s by the idea of humanitarian military intervention, by the idea that we’re doing incredible good in the world in Kosovo. And that this was a big thing, that post–Cold War, the American military could end genocide around the world, and do all kinds of good things.

Matthew Yglesias: I think in Iraq, that line of thinking turned out poorly on a sort of first-order practical basis. But that also got me thinking about the question of, “How do you help people around the world?” If you want to take a non-national interest view of the world, and say, “Who do we help?”, it turns out that there are radically cheaper ways of helping people in need around the world. If you want to do something that is selfless, there’s so much that can be done. We’re not close to exhausting the possibilities for doing that. I try to do some of it with my personal giving. I try to encourage others to do it. I try to encourage the government to do it.

Matthew Yglesias: But it began to seem to me that another area where we had to be more honest with ourselves about, “What are we actually doing here? Is the military apparatus that exists really a tool for human betterment?” Now, it can be. When there was the tsunami in Indonesia, the fact that we have a navy. The navy is full of boats — they’re really good boats, they have really skilled sailors. They were able to deliver a lot of humanitarian relief aid. It was great that they did that.

Matthew Yglesias: I’m not gonna go full paranoia, like it was just a propaganda game to help the CIA dominate the universe. But it is not really helpful to think about military force as a primary tool for humanitarianism. The cost benefit is not there. The incentives aren’t aligned. It’s not what the people are trained to do. And you see how quickly it turns. As long as we were fighting in Afghanistan, we would say we were doing all these good humanitarian things. The moment the Taliban takes over Afghanistan, we have them under severe sanctions that are causing a famine over there.

Matthew Yglesias: So I think we should not do that. Famine is very bad. But it’s a warmaking tool: their mission is to beat the Taliban. When beating the Taliban involves doing a certain amount of humanitarian propaganda — which involves a certain amount of humanitarian real stuff — the military goes and does it. But the moment the way to beat the Taliban is to starve the country into submission, that’s what that exact same kind of apparatus flips to.

Matthew Yglesias: It’s challenging because the policymakers don’t speak honestly about their own commitment to humanitarian values. So we are very eager to back opposition figures against authoritarian regimes sometimes, but not against Kuwait. There’s not going to be a colour revolution there. We can say casually, “Well, that has something to do with oil” or something.

Matthew Yglesias: But nobody is going to write down on a piece of paper, actually, what are the criteria in which the United States or other Western governments will back democratic opposition? When will we stop people from massacring? In part because we don’t want to give a clear green light to authoritarian regimes — we always want the Saudis to be guessing like, “How much can we get away with?” But it makes it a very hazy landscape, and I think it’s not closely comparable to global public health interventions — if we could convince people to care a little bit more about distributing insecticide-treated bed nets, we could stop malaria cases.

What governments should do about comets and supervolcanoes

Matthew Yglesias: NASA poked around on the supervolcano issue a few years ago, and they seem to have the idea that you could try to cool down the magma underneath Yellowstone with essentially injecting water. That’s related to the idea of advanced geothermal as an electricity-generating concept.

Matthew Yglesias: So there are these regulatory sensitivities around geothermal drilling on federal lands — a ton of sensitivity about doing it in an actual national park, which is where Yellowstone is. But I think that sacrificing a portion of the park in order to not have literally the entire planet explode would probably be a win-win. And we could offset that with some more parkland someplace else. Again, as you would expect, it’s not totally clear that that would work, but it’s worth investing some money in the exploration of whether or not it works. There’s a few other supervolcanoes that are out there.

Matthew Yglesias: As I understand it, after Deep Impact and Armageddon came out, we actually got the government to track asteroids better. And comets are just a little bit harder to track: they come in at a sharper angle and they go further away. But we should put some more telescopes up there and try to find them.

Matthew Yglesias: There’s a lot of nostalgia for the kind of heroic age of NASA and space exploration, which was very motivated in its heyday as a kind of national defense imperative against the Soviet Union. Today I don’t think that you can very plausibly claim that we need to go to Mars to stick it to China. But there’s a very clear national defense rationale for comprehensively tracking objects in outer space as far away as we can.

Matthew Yglesias: The amount of money that’s spent on defense programs is very, very, very large, and tracking those comets is good — especially as we have more private sector interest in some of the sort of sexy low-hanging fruit of, “Let’s get some human beings and have them go around the Earth in a circle.” You know, good for the entrepreneurs. And there’s not a huge ROI financially in tracking comets, but socially it’s very valuable.

How Matt thinks about advances in AI

Rob Wiblin: Do you think we should do anything more now to prepare domestic society or the international order for the possibility of major advances in AI over the coming decades?

Matthew Yglesias: We definitely should. The question is always what, right? As the person who asks the questions on podcasts, I’ve had people from the effective altruism and longtermist world on the show. Or just talked to them informally, to be like, “What should I do? Say I decide that I take AI risk extremely seriously. What do you want me to say about this?” And their answers always strike me as fairly fuzzy, so I’m left at a bit of an impasse.

Matthew Yglesias: Now again, to the young people out there, if you are technically literate and you’re in your early 20s or late teens and wondering what to do with your life, there is a lot of demand for good ideas about incentive-compatible artificial intelligence — how to make this maximally beneficial rather than threatening to humanity, on a policy level. I’m in the market for hot takes.

Rob Wiblin: Yeah, I think that there are policy ideas coming up through the academic conversation, but it does still feel like the conversation is pretty preliminary. And they seem fairly reluctant to go out and publicly advocate for anything in particular, which is interesting. I suppose they just worry that their ideas could do more harm than good a lot of the time.

Matthew Yglesias: That’s definitely a risk that’s out there. And I worry sometimes about how you have communities, or you have schools of thought — and that’s good. But sometimes something can become a scene. It’s like to be “in” in part of the tribe, you’d be like, “Oh, I’m really worried about AI existential risk.” OK, but “being worried” about things doesn’t accomplish anything in life.

Matthew Yglesias: And you can just kind of dismiss other problems. Say, like, “Well, don’t worry about that — worry about the AI risk.” But is there a tradeoff? Is there something that a typical person — a typical elected official even — should be doing differently? I’m not sure. I’m not saying no, but I haven’t been super convinced, in the way that I have on pandemics or even supervolcanoes. There are these kind of fiscal tradeoffs in the NASA budget, in NIH — what they care about that really addresses existential risk on those topics.

Matt's views on the effective altruism community

Matthew Yglesias: I’m for it, I think. I want to be effective; I want to be altruistic. I think it’s good — I think that these lines of thought are very good. I should have said this when we were talking about foreign policy before, but actually one of my EA origins is when I was in college, I saw Peter Singer lecture about Iraq. And I was an Iraq War–supporting student at that time, and I thought that what he had to say was very impressive and challenging to my preconceptions. Part of why I went to see him in the first place is that I had no idea what he thought about foreign policy, but I’d just read some of his general books about normative ethics and stuff like that — so I was bought in and therefore open to it.

Matthew Yglesias: So I think it’s great. Something I worry about is that as you start to have a community, all communities can become a little bit of an extreme version of themselves — when that isn’t necessarily where intellectual movements actually do good in the world at the margin.

Rob Wiblin: So your concern would be that if people are all talking to one another — or they’re trying to impress other people who agree with their broad worldview — then you tend to get this extremizing effect over time, and people maybe solidifying views that have gone too far in some direction and that’s not the most accurate?

Matthew Yglesias: The views can be wrong, but also they can become a… OK, what is “effective altruism”? Effective altruism is the idea that we should be more critical and more rigorous about our altruistic impulses, right? And that we should push people to ask the questions, “Is this really a good way to help people? Are you too focused on things that are close to you in your community, too focused on things that get praised by other people? Or should we cast a wider net? Should we deploy more rigorous empirical methods when we evaluate our programs?” And that’s good, those are a good set of questions, a good set of pressures in the world. So we say, “Is military intervention humanitarian? Should we be clapping for the guy giving money to the art museum? Here’s this great study about bed nets and malaria.”

Matthew Yglesias: Then you have a community that buys those premises, and they start talking to each other a lot. They eventually develop the idea that we all should just think about the long term and we shouldn’t discount at all. And if you don’t discount at all, the only thing that really matters is human extinction. And a bunch of smart people have thought about this, and the biggest threat to human extinction is rogue artificial intelligence.

Matthew Yglesias: So now we’ve all read that book, and we’ve all put it around. And now what it means to be an effective altruist is, somebody is like, “What’s on your mind?” and you’re just like, “The threat of human extinction due to artificial intelligence.” And a large share of people are going to hear that and they’re going to be like, “What? That’s weird.” And an even larger share of people are going to be like, “Well, what does that even have to do with me?” Right? So now you’re not actually asking people to be more self-critical about their charitable giving or to give more money at the margin. You’re not pushing —

Matthew Yglesias: You’re asking people to take this Kierkegaard-like leap and join the community — as opposed to being in your community and living your life and caring about roughly the things you care about, but shifting your orientation a little bit.

Matthew Yglesias: I think that you become less effective, in a meta sense, when you become this set of doctrines that are quite odd and esoteric to most people — and that also don’t have a lot to do with people’s lives and the decisions that they’re actually making — versus one that’s trying to say to people, “For your end-of-year charitable contributions, consider the GiveWell Maximum Impact Fund as something to do. Think about giving a little bit more than you usually do. Think more critically about the doing/allowing distinction in your life.”

Matthew Yglesias: These are really important real-world messages, I think. As I’ve seen this evolve over the years, I both am myself somewhat bought in and try to get people to think more about x-risk and other stuff like that. But I worry also that the most broadly relevant stuff can get lost a little bit.

The impact of Matt's work

Matthew Yglesias: When I’m trying to be self-conscious and reflective about what I do — this is what I try to do: I take ideas that exist in the academic-y realm and that are very neglected in politics, and bring them to more people’s attention, until the point where someone has done enough with it that it becomes a conventional political conflict that people then cover as, “Well, the YIMBYs are fighting against the NIMBYs.” And at that point, I don’t think that political punditry actually does a lot. That’s the most common kind of punditry: “I’m going to write an article about why the Freedom to Vote Act is good.” That strikes me as incredibly low efficacy.

Rob Wiblin: Because it’s so saturated?

Matthew Yglesias: It’s incredibly saturated, but also it’s on the docket, right?

Rob Wiblin: I see. It’s already on the agenda. People are going to think about it regardless.

Matthew Yglesias: Right. For actual members of the United States Senate to be taking a vote on an issue is so far down the chain, and the idea that Kyrsten Sinema is going to read my blog posts —

Rob Wiblin: Crack open the newspaper.

Matthew Yglesias: — and be like, “Shit, I’ve got this all wrong.” That’s an insane level of hubris, and yet it seems to be what most people think is the humble way to do columnist work. I think it makes so much more sense to just be like, “Hey, here’s a thing,” and then maybe one person somewhere is like, “Oh, maybe I could work on that.” Or one guy who’s rich is like, “Sure, I could cut a check to that,” and you see where it goes.

Matthew Yglesias: It’s not that hard to believe that you can convince one of the world’s many, many, many wealthy people to, one time, deliver some financial support to something that nobody is doing. That, to me, is a much more realistic aspiration in life. And yet I think in my field, it sounds more egomaniacal to be trying to take credit for some early-stage grants than to be like, “I’m waging the war of ideas. I’m going to convince people that Donald Trump is bad.” Like, how would I do that?

Articles, books, and other media discussed in the show

Matthew’s work:

Polling, public opinion, and voting:

Philosophical views:

Other 80,000 Hours Podcast episodes:

Everything else:

Related episodes

About the show

The 80,000 Hours Podcast features unusually in-depth conversations about the world's most pressing problems and how you can use your career to solve them. We invite guests pursuing a wide range of career paths — from academics and activists to entrepreneurs and policymakers — to analyse the case for and against working on different issues and which approaches are best for solving them.

The 80,000 Hours Podcast is produced and edited by Keiran Harris. Get in touch with feedback or guest suggestions by emailing [email protected].

What should I listen to first?

We've carefully selected 10 episodes we think it could make sense to listen to first, on a separate podcast feed:

Check out 'Effective Altruism: An Introduction'

Subscribe here, or anywhere you get podcasts:

If you're new, see the podcast homepage for ideas on where to start, or browse our full episode archive.