Transcript

Cold open [00:00:00]

Rachel Glennerster: So, during COVID when we were working on how to accelerate a COVID vaccine, we did estimates of how valuable it was to bring forward a COVID vaccine, to try and get people to put more money into it and explain the benefits. So, Operation Warp Speed spent $12 billion trying to accelerate a COVID vaccine. We estimate that that would have been cost effective if it accelerated a vaccine by 12 hours.

Luisa Rodriguez: Oh my god. What?

Rachel Glennerster: We were doing the number crunching from kind of March, after COVID hit, and we were just saying, you know, we are losing trillions of dollars — and it is worth throwing absolutely everything you have at this and looking at multiple vaccines, because if you accelerate a vaccine by a week, a few days, it’s worth billions of dollars.

Luisa’s intro [00:01:03]

Luisa Rodriguez: Hi listeners, this is Luisa Rodriguez, one of the hosts of The 80,000 Hours Podcast.

In today’s episode, I was honoured to speak with a long-time hero of mine, Rachel Glennerster. She’s a University of Chicago economist, the recently appointed President of the Center for Global Development, and one of the founders of the “randomista” movement, which made randomised controlled trials a staple of development economics. She was last on the show in 2018 to talk about how to get year’s worth of education for under a dollar and other ‘best buys’ in development.

In this episode, we talk less about specific cost-effective interventions to improve global health and development, and more about how to “shape” markets to incentivise different institutions to come up with and implement those interventions themselves. Specifically, we discuss:

• What “market shaping” means, and what causes current markets to under-produce valuable things like vaccines.
• Examples of mechanisms that can shape markets for social good, like “advance market commitments” (or AMCs).
• How speeding up vaccine development by even just a little bit can have massive benefits in both lives saved and impacts on economies.

Without further ado, I bring you Rachel Glennerster.

The interview begins [00:02:31]

Luisa Rodriguez: Today I’m speaking with Rachel Glennerster. Rachel’s an associate professor of economics at the University of Chicago, and helped launch the University of Chicago’s Market Shaping Accelerator — which aims to use insights from economics to solve market failures that lead to too little investment in innovation to solve problems important to society; they’re focusing initially on climate change and pandemic risks.

She previously served as chief economist for the UK’s Foreign, Commonwealth & Development Office, and was the executive director of the Abdul Latif Jameel Poverty Action Lab, or J-PAL. She also helped establish the Deworm the World initiative, a programme that’s helped deworm millions of children worldwide, and which we’ve talked about on our show before.

Thanks so much for coming on the podcast, Rachel. It’s really an honour to have you on.

Rachel Glennerster: It’s great to be on. I have to say, my kids listen to the podcast, so since I last came on, it’s even higher stakes for me.

Luisa Rodriguez: Well, that’s also an honour. That’s really cool to hear.

The Market Shaping Accelerator [00:03:33]

Luisa Rodriguez: OK, so let’s dive right in. You’ve recently set up what you’re calling a Market Shaping Accelerator, alongside fellow economists Michael Kremer and Christopher Snyder. What exactly do you mean by “Market Shaping Accelerator”?

Rachel Glennerster: Market shaping is the idea of leveraging the power of markets to incentivise innovators to generate the innovations we need to solve some of the world’s biggest problems.

Now, markets have a lot of benefits: they help incentivise innovation, they develop products that people actually want to use, they aggregate information. But they also get things wrong. They’re not very good at solving things where there are externalities, like climate change. So the idea is to get the best of both worlds, and incentivise markets to respond to the things where we really need them to respond to.

Luisa Rodriguez: Learning about this accelerator actually made me think of this blog post that a friend shared with me recently. The blog post was by Jacob Trefethen, and it’s called “10 technologies that won’t exist in five years.” So Jacob’s the science policy grantmaker at Open Philanthropy, so he has kind of a sense of where science funding goes, and how hard it is to make progress in some areas.

And I found the blog post really moving and really tragic, and it feels relevant, so I wanted to share a couple of the technologies that he thinks won’t exist. One relates to tuberculosis, which kills 1.5 million people a year, which is twice as many people as die of malaria each year. And according to Jacob, a tuberculosis vaccine that worked in adults is totally achievable. But he thinks, for basically no good reasons, we won’t have one in five years. Like, nothing related to the science will prevent us from getting there. Similarly, strep A kills 500,000 people a year, and again, a vaccine is achievable. Hepatitis C kills 300,000 people a year — again, a vaccine is achievable.

And yeah, it’s really horrible. I definitely had the intuition that diseases like this, that are this deadly, we’re not creating the vaccines or treatments we need because the science is too hard. But when I learn that’s not the case, it just feels really unacceptable.

Are these the kinds of market failures that you’re talking about? And if so, can you talk me through why they happen?

Rachel Glennerster: Yeah, those are exactly the kinds of targets we need to do market shaping for. Actually, tuberculosis was one of the diseases that brought me into thinking about this many, many years ago, when I first started working on trying to accelerate vaccines. And it took way too long for us to get a malaria vaccine compared to the huge death toll.

There’s a number of market failures going on. There’s a number of reasons people don’t invest in doing innovations for these kinds of problems. One of them is that there’s a lot of pressure to keep prices down once you invent something, especially if it’s really important for poor people. So it’s a time-inconsistency problem, we call it: you would be willing to pay for something in advance, but once it comes, there’s a big pressure to keep prices down.

And the patent system that we have to incentivise people for innovation is really, really inefficient. It works by allowing companies to keep prices high, to get a reward and a return on their investment. But if you’re designing something for really poor people, nobody’s going to buy it when the price is high. So you have a small quantity at a high price, and you make your money back, and then later you reduce your price and get it to lots of people. So it doesn’t work very well for diseases that affect a lot of… Or a vaccine: people are willing to pay a lot to get a drug that you’re about to die of a disease. But a vaccine, you need a low price to get to lots of people. That’s not how our patent system is designed to work. It works by charging a high price to a small number of people. And that’s just really inefficient.

So part of market shaping is solving the patent problem, which has not been successful in getting vaccines in particular, and vaccines and drugs for low-income countries. It’s also inefficient in high-income countries too, but it’s not such a massive, gaping hole.

Luisa Rodriguez: Right. And so the thing happening is, once a company has a tuberculosis vaccine, everybody’s mad if they charge really high prices, because people in poor countries, it feels really egregious and uncharitable and kind of morally outrageous to charge some high price that would help them recoup the cost that made it worth creating that vaccine or that drug. And because there’s this kind of moral outrage, rather than create the vaccine at that low price, they just don’t at all, because it doesn’t make sense to. And so then we have this kind of lose-lose. We just don’t get these things.

Rachel Glennerster: Yeah, it’s not just that it’s a moral outrage. If it’s a tuberculosis vaccine for poor countries, then it’s going to be bought by often a single purchaser. So the UN buys most vaccines for poor countries. So you know, you have a lot of purchasing power, so governments are able to kind of drive down the price. So it’s not just moral outrage; it’s just that they’re going to drive the price down. Once you’ve invented it, you sunk your cost, they might as well drive the price down. There’s a lot of reason to drive the price down. You want to get it out to lots of people.

Luisa Rodriguez: I see.

Rachel Glennerster: But because they know that’s going to happen, they’re not going to be able to recoup their R&D costs, so they don’t do the investment in the first place.

Luisa Rodriguez: So that’s why we don’t get vaccines, very specifically. But there are a whole bunch of market failures that have really big costs in other areas. So climate change you’ve already mentioned, but also pandemic preparedness and biosecurity. What are the other kinds of market failures that are happening there, and what are the impacts of those?

Rachel Glennerster: I think there’s a number of market failures that mean that we don’t get innovations in general, and in particular some specific innovations. So one reason is, for biosecurity and pandemic preparedness and climate change, those are all cases where the benefits to society are very different from for an individual. So the person who buys the gas-guzzling car is not paying the cost, right? Or someone who takes a vaccine helps themselves a bit, but they also help lots of other people. So that’s an externality problem. And so we don’t get enough innovations when there’s an externality — either a negative externality from me driving a gas-guzzling car, or a positive externality from me taking a vaccine. So that’s one reason that this kind of innovation we don’t get enough of.

We also don’t get innovation in general enough, because when I develop something, a lot of people learn from my innovation. So the first people who worked on solar panels aren’t actually the ones who have the market now. They learned a lot about how to make solar cheaper, and a lot of other firms benefited from that. So we don’t get innovation enough in general.

And then finally, the way we incentivise innovation in the world is that we use patents. And patents are really good at some things, and really bad at other things. So they do reward people for doing innovations, but they reward people in a way that encourages them to produce a low quantity at a high price. And particularly for vaccines and climate change, we need to reward people for doing things, but we need to reward them to produce a large quantity at a low price. So that’s another thing that’s screwed up about our way that we reward innovation. And market shaping can help solve that age-old problem that patents are a terrible way of rewarding innovation.

Luisa Rodriguez: OK, great.

Pull mechanisms for innovation [00:13:10]

Luisa Rodriguez: So one of the big ways that you’re trying to shape the market to address these kinds of market failures — and we’ve only talked about a few so far, but we’ll talk about more — is through pull mechanisms. Can you explain what a pull mechanism is?

Rachel Glennerster: A pull mechanism is where you pay based on the outputs: so whether you succeed in coming up with a vaccine, how many people you get it in the arms of. And it’s in comparison to push, which is how we often fund innovation, which is just giving a grant to this research group to work on a vaccine.

Luisa Rodriguez: And just to make it concrete, can you give a quick example?

Rachel Glennerster: Yeah, so there’s lots of examples of pull mechanisms. You could commit to buy something if it was developed. You could give a prize if it was developed. But something like a subsidy for an electric car is also a pull mechanism, because it’s tied to the number of people who buy electric cars. So there’s a whole range of ideas that are based on this.

Luisa Rodriguez: And then, what is the basic case for using these mechanisms as opposed to what seems like the more traditional approach to me, which is just that there’s a good idea that might help society, and let’s fund this firm or this nonprofit to develop that idea?

Rachel Glennerster: One of the problems with push funding is you have to pick which company or which research group you think is going to do the best job. And you, at the centre, making that decision, don’t have all the information. There are all these research groups and all these firms out there who could work on it, and they tend to have information about whether they think they’re going to succeed or not.

The benefit of doing a pull is the ones who have that private information, they think they’re going to succeed, and they’re the ones who try. Whereas otherwise, I’m choosing from the centre, and I don’t have the information. So it’s better at incentivising the people who have private information who know more about success rates.

The other benefit of a pull mechanism is it helps with scaling up. So if you just pay someone to develop something, it doesn’t incentivise getting it out to people. Whereas a pull mechanism is linked to how many people you get it out to. Or it can be. Not all of them are, like prizes aren’t. Prizes are just based on the outcome, just, did you invent something? But you can design pull mechanisms so that they reward people who get it out to a large number of people at a low price, which is exactly what we were saying is a huge problem in vaccines in lower-income country diseases. So you basically cover their R&D costs, but only on condition that they get it out to a lot of people.

Luisa Rodriguez: Right. So what that can look like is saying that we will actually pay for parts of the vaccine up to a certain amount, and that makes it so that a supplier can make back enough to recoup their development costs, but also so that the people who are benefiting don’t have to pay that level of high price. It’s basically just a way to kind of bridge the gap between how costly this thing would be and the fact that people actually don’t have the ability to spend that money — and for various psychological reasons, might not think it’s worth it for them to.

Are there any other reasons that pull funding can work better than push funding?

Rachel Glennerster: One of the benefits of pull funding is you can design it so there is a market test, so you only end up subsidising something if people want to buy it. And an example of where this went wrong is we put a lot of money into R&D on cookstoves — these stoves that people use in low-income countries, and produce a lot of indoor air pollution, and a lot of women in particular die from and get respiratory disease from cooking over these fires. So we put money into research teams working on better stoves, but women didn’t like cooking on them, so they didn’t use them very much. And it’s been a huge problem that a lot of that R&D didn’t lead to products that women wanted to cook with.

Now, if you tied your reward to whether women were actually cooking with them, then the innovators at the very start of the process would have been trying to innovate not for the thing that reduced air pollution the most, but the thing that women most wanted to cook on, and also produce less air pollution. So I think there’s a lot of technologies where if we try and just pay people from the centre, you just get rewarded for coming up with a cool idea and you don’t tie it at all to whether people actually want to use it, you get a bad innovation.

And that’s why markets are so good: they reward people who invent things that people actually want to use. So that’s what we’re trying to mimic here. That’s why it’s market shaping, and getting the benefit of the market: you really want to tie the reward to whether people use it.

Luisa Rodriguez: Right.

Accelerating the pneumococcal and COVID vaccines [00:19:05]

Luisa Rodriguez: To what extent do you feel like you have thought about and have empirical evidence about how big the benefits of pull funding relative to push funding can be? Is there a way to think about that quantitatively?

Rachel Glennerster: I think the best way to look at this is to have an example. So we worked on an advance market commitment for a pull mechanism for accelerating a pneumococcal vaccine that was appropriate for low-income countries. And we evaluated that it would be really useful to do this at the outset. This was one that there was a vaccine that was relevant for high-income countries, and it needed to be adapted to something that covered the strains that were common in low-income countries. So it wasn’t that hard, but there was a worry that people wouldn’t get around to it for a while, and also that they wouldn’t produce a lot of it early on and get it out to people fast.

So a lot of our analysis was about the benefits of getting it to more people faster, and then we did an analysis afterwards of what actually was the benefit of having this mechanism. So it was a $1.5 billion advance market commitment that committed to buy a pneumococcal vaccine that was relevant for the strains that were common in low-income countries. And because of the way it was structured, there was an incentive for firms to produce a lot — to build big factories, produce a lot, and get it out quickly — because they wanted to use up the subsidy. They wanted to grab as much of the subsidy, and there was only a set amount, and there were several firms producing it.

So colleagues of mine did an analysis that looked at that, and compared how much faster the vaccine got out for pneumococcal than rotavirus. Rotavirus was a vaccine that was produced at about the same time, and it was also bought by Gavi. So both of these vaccines were bought by the same organisation, they were bought by governments, but subsidised by Gavi, the Global Alliance on Vaccines Initiative. And the pneumococcal one reached most people five years faster than rotavirus.

Luisa Rodriguez: Wow. That is really striking.

Rachel Glennerster: Yeah. And a lot of kids are dying every year, so the faster you get it out, the more lives you save. So that’s an example of how, we’ve been talking about how pull mechanisms accelerate innovation, but there’s this other part of it: they incentivise getting it out to people fast, because you tie the incentive to producing a lot and charging a low price. Basically, for the economists, it solves a monopoly pricing problem: that normally firms want to produce a small quantity and charge a high price. This is, no, we give you a subsidy if you produce a lot and charge a low price. So you get it out faster and that’s worth a lot. Even if it doesn’t speed up the innovation, it still saves a lot of lives.

Luisa Rodriguez: Right. OK, so in advance you already did some analysis to estimate roughly how much acceleration you expected by using this kind of commitment, rather than just push funding. Do you have a vague memory of what kind of acceleration you expected?

Rachel Glennerster: I can’t remember exactly how much we were projecting at the time, I’m afraid. But I think we were all pretty excited by the five-year comparison. I mean, that’s a lot.

Luisa Rodriguez: Sure, fair enough. My understanding is that you and your colleagues have estimated that 700,000 lives were saved because of the vaccines the AMC incentivised. How exactly do we know that? Where does that number come from?

Rachel Glennerster: So during the period, 150 million children were immunised, and you can then look at the efficacy of the vaccine and mortality rates, and that suggests that the equivalent of 150 million children being immunised is 700,000 lives.

Luisa Rodriguez: Got it.

Luisa Rodriguez: It does feel like it’s easy to not kind of intuitively grasp that a five-year acceleration is really meaningful and important. Are there other examples where you’ve kind of tried to quantify the value of accelerating something like a social programme like this?

Rachel Glennerster: Yeah. So, during COVID when we were working on how to accelerate a COVID vaccine, we did estimates of how valuable it was to bring forward a COVID vaccine, to try and get people to put more money into it and explain the benefits. So, Operation Warp Speed spent $12 billion trying to accelerate a COVID vaccine. We estimate that that would have been cost effective if it accelerated a vaccine by 12 hours.

Luisa Rodriguez: Oh my god. What?

Rachel Glennerster: We were doing the number crunching from kind of March, after COVID hit, and we were just saying, you know, we are losing trillions of dollars — and it is worth throwing absolutely everything you have at this and looking at multiple vaccines, because if you accelerate a vaccine by a week, a few days, it’s worth billions of dollars.

Luisa Rodriguez: Right.

Rachel Glennerster: So that just highlights how it really matters when you crunch the numbers, it really matters to bring these innovations a bit faster. And the same is true in climate change. People have done these analyses about if we curb the emissions slightly faster, because it’s a stock and it builds up, it really matters to get it out. Not just innovation faster, but scale up faster.

Luisa Rodriguez: Yeah. That actually preempts one of my questions, which is just, it seems like some of these innovations might have happened eventually, and so do we really need these pull mechanisms? And it sounds like, at least in many cases, the answer will be that there is clearly a lot of social value on the table by speeding things up. Sometimes you don’t even have to speed it up by very much for it to be really meaningful.

Rachel Glennerster: Yeah. As I say, 12 hours. And it’s not just speeding up the innovation; it’s speeding up getting it out there. The two combined. Again, for COVID, the thing that took the longest was speeding up the manufacturing. We invented the vaccine in a month, we took nine months to get regulatory approval, and it took two years to manufacture enough to get enough to vaccinate the world.

Luisa Rodriguez: Wow.

Rachel Glennerster: So our work on pandemics is all about how we’ve got to do innovations, we’ve got to do investments that mean that we can produce more faster. And the market failures are such that that is not necessarily in the interests of the firms, or at least it’s not sufficiently in the interest to do the investments to do that speeding up.

Luisa Rodriguez: Yeah. Can you actually talk about why that is? Why that’s not kind of taken care of?

Rachel Glennerster: So one of the real problems in pandemics, why the markets don’t work very well in pandemics, is you can’t charge what something is worth during a pandemic. So we estimated that the value of one course of COVID vaccine in January 2021 was over $5,000. They were selling for between $6 and $40. So nothing like their social value.

Now, don’t get me wrong. I don’t think that they should have charged $5,000 or $6,000. That’s not ethical. It’s also not economically efficient, because they didn’t cost $5,000 at the marginal cost. So you actually want low price, getting out to lots of people.

But it shows you that the market is not going to reward people who do the investment in preparation for a pandemic — because when a pandemic hits, they’re not going to get the reward in line with the social value. They may even have to charge less than they would in a non-pandemic time. So prepping for a pandemic is not an efficient market strategy if I’m a firm, but it’s a very efficient strategy for society, and so we’ve got to bridge that gap.

Luisa Rodriguez: Yeah, it makes sense to me why that means the incentives aren’t there for firms to create pandemic-ready vaccines. Does that explain why specifically the rollout ends up being really inefficient?

Rachel Glennerster: Yes. Because in COVID, what we wanted to do — “we” as in what was good for society — is to start pouring money into making sure we can manufacture these vaccines even before we know whether they worked.

Luisa Rodriguez: Right, OK.

Rachel Glennerster: So in March we were saying, “Build factories, get this manufacturing scaled up. We have no idea if this vaccine is going to work. It doesn’t matter.” We had a very low percentage rate for the vaccines working. It was still worth putting billions of dollars into accelerating manufacturing. If I’m a firm, I’m not going to put billions of dollars into expanding manufacturing capacity until I know whether the vaccine works. Because I’m only going to get $6 on it — $6 was the AstraZeneca; $40 was the Pfizer type. So it’s not worth risking putting billions of dollars in building a manufacturing plant if the vaccine might not work.

But that’s exactly what the US and the UK did. They poured money into expanding factories and producing the stuff so that as soon as it was approved, they had vaccine ready to go. And that’s why… Sorry, I’m going to get emotional. That’s why my parents could get a vaccine in January 2021. And other people couldn’t. It’s because of that investment. And if only we’d managed to persuade more people to do this, I think of the lives that could have been saved.

Luisa Rodriguez: What do you think it would have looked like to really have nailed that from a funding perspective, to actually get things out as quickly as we could have?

Rachel Glennerster: Oh, so many things we could have done differently. So many things. One of the things that we really wanted middle-income countries to do was to put money into expanding their capacity before we knew whether they were going to work — and multilateral development banks wouldn’t lend to them until they knew whether the vaccine would work.

The international mechanisms for buying the vaccines for low- and middle-income countries didn’t buy them until they knew whether they worked. And we spent months arguing about how to fund these vaccines. Initially, people were arguing that no low- and middle-income countries should pay for them; it should all be donor money. We’re just not going to get enough donor grant money to get it out to enough people. So they traded off waiting for donor money and saying nobody should ever have to borrow to fund vaccines.

Well, that’s crazy. If they’d been able to borrow and get more vaccines faster, they would have saved billions, trillions of dollars. And so absolutely, it was an economic return to Brazil and other places, but they thought the rich countries are going to pay for it, so we’ll just wait. Well, that was a really bad miscalculation. And holding out the prospect that rich countries were going to just pay for that when they were coping with their economies collapsing, it just wasn’t realistic.

I mean, even the EU was arguing over exactly what price, and trying to drive the price down. It made no sense. Like, get the money out there. Don’t argue whether you’re getting it for $40 or $45 — it’s worth $5,000! Don’t nickel and dime. And the hours you spent negotiating, a lot of people are dying. It just didn’t make any sense.

Luisa Rodriguez: It’s really, really sad. Coming back to pull mechanisms in particular, this wasn’t a case of using a pull mechanism, it was actually just using push funding to accelerate this development. Why did push funding make more sense than pull funding in this particular case?

Rachel Glennerster: Yeah, it’s a good question. And it’s at the heart of what the Market Shaping Accelerator is about: to use economics to figure out what is the right mechanism in a particular situation. So this was a group of academics who had all worked on pull funding, we’d worked on accelerating vaccines. We were thrown this question by a lot of people. We did the analysis. We said, don’t do pull in this case; just pay for manufacturing capacity.

And the reason was, you wanted people working on a very large number of vaccines — because, remember, we didn’t know whether one would work. So we needed to back many, many different candidates, and the probability of success was very different for different candidates. So the ones that were more likely to succeed, you didn’t have to pay them that much to keep going. But you wanted to pull in the really marginal ones, right? You wanted to pull in even the candidates that had a 1% chance of success — it was still worth it.

So if you’d set a market price for all of them, if you’d pulled this by saying, we’re willing to pay this price, it would have to be a very high price to pull that marginal person in, and way above what you needed to pull the highest likely candidate, the “sure bet.” Nothing was the sure bet, as they say. We did our analysis assuming everything had only a 10% chance of success — and it was still worth doing many, many candidates, but it was still the case that you had to pay less for the ones that had a higher chance of success. So you wanted to pay different amounts to different companies.

And then, as they say, the biggest market failure is they weren’t going to build big enough factories. “Build” is not quite the right word, but expand capacity enough before they knew whether it worked. So that was the targeted thing that you could put your money into. And if you look at Operation Warp Speed, you look at what the UK were doing, that’s where they put a lot of their money. Now, they were also committing to buy it, and they were paying for things even if they didn’t succeed. But a big part of it was this targeted funding for expanded capacity.

Luisa Rodriguez: OK, to make sure I understand, part of what was going on was the fact that different firms had different probabilities of success meant that a single price wouldn’t have created the right incentives. It might not have made it sufficiently profitable or exciting to the companies with low probabilities of success, and it might have just made no sense to pay the firms with higher probabilities of success. So there’s something there about when you have wildly different probabilities of success, and when some are reasonably high and some are reasonably low, advance market commitments make less sense?

Rachel Glennerster: Well, I’d put it slightly differently. So it’s very different in a pandemic, when you want 25 different companies to all be working on a COVID vaccine, and it’s worth it. If you’re not in a pandemic, you don’t want the guy who’s only got a 1% chance of success working on it.

Luisa Rodriguez: Sure. That makes sense.

Rachel Glennerster: That’s the whole benefit of the pull mechanism, is companies self-select: they know, they have the information about whether their thing is going to succeed — in a way that, if you’re sitting there looking at grant applications, you probably don’t know; it’s private information. They have private information about whether they’re going to succeed. So the benefit of a pull mechanism is only the ones who think that they have a high probability of success keep going and keep working on it.

We didn’t want that in the pandemic. We wanted everyone to try. But normally you don’t want everyone to try. You don’t want someone with a really small chance of success to try. So that’s why we went for a different mechanism.

Luisa Rodriguez: That makes tonnes of sense. So I guess we’ll come back to the different mechanisms and the factors that determine when you want to use the different kinds. But I’m curious: if this work with this accelerator went extremely well, what would that look like long term? For example, is part of the hope that pull mechanisms just become much more widespread as the types of policies used in the next 10, 15, 20 years?

Rachel Glennerster: So a couple of different measures of success that we’re going for. One is just to have a couple more examples of successful pull mechanisms — including, we hope, another advance market commitment of the kind that we did for pneumococcal, so there’s commitment to buy a vaccine if it’s developed and scaled up. There are, as you were noting earlier, some complications in terms of how do you actually make that happen? And there’s legal contracting issues, there’s budgeting complications. We want to take some more things through that process, so that we have a real template for doing this so that it’s easier for the next ones to do it.

Luisa Rodriguez: Yeah. Cool.

Rachel Glennerster: So obviously we want to pick a couple of big ones that actually happen, and we actually get a new climate technology and we actually get a new vaccine or pandemic preparedness innovation. The ones that we’re working on would be really big wins if we got them.

So get a couple of successes. But in getting a couple of successes, work through more of the kinks of, how do you actually get this implemented? So success for us is taking it all the way through to actually implementation. As I said, there are mechanisms out there that governments use that are effectively pull, but we think that they could be designed better. So working through the details of how you would actually do it in the way that we think would be the best way to do it, I think would be really useful for then producing a template that then could be used for lots of other things.

And in this process, we’re hoping to train a lot of other people about how to do this work, and how to evaluate the benefits, and how to design these things — so that this is a whole kind of movement of people doing this. And if you think back to my previous life of helping build a movement that does randomised trials for anti-poverty programmes, we’re trying to build a movement here again.

Luisa Rodriguez: Yeah. As you were saying that, I was already thinking, you’ve done this before. You’ve built a movement around using RCTs in development. And I think I’d have been a little sceptical if someone else had said something like, “I’d like to build a movement where there’s a whole new kind of policy mechanism used in policy for social good” — but you’ve got a track record that speaks for itself, which I find very inspiring and exciting.

Rachel Glennerster: That’s very kind. I did not do it on my own. It was part of a team. But I’m part of a team now too.

Advance market commitments [00:41:46]

Luisa Rodriguez: Exactly. OK, let’s talk through a few specific examples of pull funding mechanisms. So we’ve already mentioned a couple, but I want to kind of talk about them in a bit more detail. One example that’s come up already is an advance market commitment, or an AMC. How do AMCs work?

Rachel Glennerster: So an advance market commitment is something done in advance. You commit in advance to either buy or subsidise a set quantity of a product at a set price. So you’re defining the market, because the market is price times quantity; you’re committing in advance; and you’re defining what it is that you want to buy. You’re setting out, “I will buy it if you meet these criteria.” And it’s legally binding, which is really important, because there are all sorts of statements about, “I will do this green thing or that green thing,” but this is a legally binding commitment.

Luisa Rodriguez: Yeah, that does seem really important. You’ve talked about a few examples, but can you give another one to make this super concrete?

Rachel Glennerster: Yes. So we’ve talked about the pneumococcal, and I think that is the preeminent example of where we think it was done well. We were very involved in it. So there are other things that are called AMCs, there are other things that are very close to AMCs, there are other pull things.

But I think if we want to define the prototypical AMC, I think the pneumococcal is the example to talk about: a $1.5 billion commitment to buy a pneumococcal vaccine if it’s produced. And this process of, it was a subsidy, so governments had to buy, be willing to distribute the vaccine, but then it was a subsidy on top of that that firms would get, and it was a set amount, and it would run out. Therefore, there was an incentive to kind of get through it quickly. And it was tied to a commitment to then keep the price low afterwards. So there was a price and you get a subsidy on top of it, but you don’t get the subsidy forever, but you have to keep the reasonable price after the subsidy goes away.

Luisa Rodriguez: Yeah, that also seems great and important. And I want to come back to some examples of cases that aren’t actually AMCs, that are maybe similar, and what distinguishes those. But first, are there very rough heuristics for when these are useful? I know broadly, we’re talking about places where the incentives just aren’t aligned with socially good outcomes. But AMCs in particular, what do those cases look like?

Rachel Glennerster: So the benefit of an AMC is it is firm agnostic, by which we mean it’s open to any firm. So it’s particularly useful when you don’t know who’s going to be the people who are going to come up with this technology. Also, you have to be able to define what the technology is. So there’s all sorts of innovations that we don’t predict in advance, but it’s useful when you can say, we want an 80% effective vaccine. Like, we know how to judge whether it’s successful. And it’s also useful when getting it out is part of the challenge. Normally with drugs and vaccines, somebody gets a monopoly on the patent, and then they do a high price and a low quantity. So it’s helpful when you’re solving a monopoly pricing problem too.

Luisa Rodriguez: OK, so the big success story is this pneumococcal vaccine story, which we’ve talked about. Are there other examples worth highlighting?

Rachel Glennerster: Yeah. So we worked with Frontier, which is a group that has committed to spend $1 billion on buying carbon removal as a way to stimulate innovation into removing carbon from the atmosphere and putting it away. And that’s not exactly the prototypical AMC, because they actually do pay different prices to different firms — because you’ve got very, very different technologies, and they believe, and I think they’re right, that some of the ones that currently have high prices are going to have declining costs over time.

Luisa Rodriguez: I see.

Rachel Glennerster: But I do think the fact that they committed to spend $1 billion on this brought people out of the woodwork who had ideas and were able to go to banks and say, “I can get money from Frontier, because they’re going to be spending money on this.” So it’s not exactly an AMC. They call it an AMC, but we can quibble because the price is different for different firms.

Luisa Rodriguez: Right. And have they started paying any of that money out yet?

Rachel Glennerster: Oh yeah.

Luisa Rodriguez: Wow. Cool. So it’s just happening and working.

Rachel Glennerster: It’s happening. They’re paying for people to sequester carbon and remove carbon.

There’s another example, which is the government has an advisory committee on vaccines in the US which just says if you come up with a cost-effective vaccine, these are the rules under which we will advise the government to buy it. They probably don’t think of themselves as an AMC, but they kind of are. It’s not a legally binding commitment, but it is a commitment. And it’s happened many times, and firms can see that if they develop a vaccine that is cost effective, they know that they have a market for it.

It’s not that they have a committed price for a whole schedule of different vaccines, but the fact that they will buy it if it’s cost effective means that a firm can figure out, can back out from kind of knowing how many people have the thing and what the DALY cost of it is. They can figure out basically what price they can charge. They know that, “If I charge this price, it will be cost effective and then the government will buy it.” So it’s sort of like an AMC. They might not think of it as an AMC, but it sort of is.

Luisa Rodriguez: Yeah. Right. There’s a credible commitment, though. It’s not legally binding. It’s credible because it’s been done a bunch, and they’re able to figure out how to make it worth their while.

Rachel Glennerster: Yeah. They can figure out the price and the quantity that the government’s going to buy it at.

Luisa Rodriguez: Neat. Are there clear examples of cases where firms have developed vaccines knowing there was that market, or is it too hard to establish that causality?

Rachel Glennerster: Yeah, I can’t sort of say this vaccine wouldn’t have been produced without this advisory committee.

Luisa Rodriguez: Sure. But it seems totally plausible that’s happened. Maybe even likely.

Rachel Glennerster: Exactly.

Is this uncertainty hard for funders to plan around? [00:49:17]

Luisa Rodriguez: Cool. I guess, trying to be sceptical, it seems really hard for an institution to make funding plans when making advance market commitments. You don’t really know… Well, maybe you do know how much you’ll end up having to pay, but you don’t really know when. And I guess you don’t know in the sense that it’s possible no one produces the output. So maybe you’ll pay zero or maybe you’ll pay some capped amount.

Is that not a dealbreaker for some funders? If I imagine government bureaucrats sitting and thinking about their budget, I wouldn’t expect them to be able to say, like, “Maybe $2 billion in two to five years.” Can they do that?

Rachel Glennerster: So it certainly makes life harder. That is absolutely clear. And it’s funny because one of the benefits is that you don’t have to pay if it doesn’t get invented. But actually that turns out to be quite complicated for governments to cope with.

So one thing is, philanthropists often have a bunch of assets of stock in the company that they founded or whatever, and then they’re paying out money in their foundation every year. But they can actually use that stock of assets and put it to work now by stimulating innovation, by saying, “I’m willing to sell some of that stock and spend more in any given year if somebody comes up with this thing.” So I think philanthropists, actually, it’s not hard to do it because they have a stock of assets that they’re going to run down slowly. But if there’s a brilliant opportunity —

Luisa Rodriguez: And they’re looking for good opportunities…

Rachel Glennerster: — they should be willing to pay more that year, right?

And now let’s turn to governments, because governments are important, and I think will probably do more of this. Let’s start with the logical economic argument, which is: if this brilliant, cost-effective opportunity comes along suddenly, you should be willing to borrow for it, right? You should just say, “Wow, we suddenly have a way to reduce climate change that is incredibly cost effective and way more cost effective than other things we’ve been doing. You know what? We should suddenly just do it, and we should be willing to borrow for it” — because you set the price so that you will only be buying it if it is cost effective. So there’s no risk. You know it’s going to be cost effective. So you should just borrow for something that has a high benefit whenever it hits, and you can just borrow whenever it hits.

That’s a logical economic argument. Is that actually how budgets work? Well, if it’s a big enough thing then maybe, yes. And with pneumococcal, the Ministry of Finance in the UK was really behind this, and they kind of understood it and they realised they can just do this. But you’re right, that normally the way bureaucracies work is, “I’m the group that has got $X million to spend this year on this particular thing, and my budget is capped over many years, and so I don’t want to put money aside for something that might not happen.”

I actually think — and now we’re going to get a little bit into the weeds — but I actually think accrual accounting helps a lot with it. So one way that governments budget is if they make a commitment, it’s scored: it counts as expenditure depending on the probability of that thing happening.

Luisa Rodriguez: Great, that’s so sensible.

Rachel Glennerster: So as you get closer to something happening, the probability goes up, and so you actually start budgeting that expenditure. And then it’s kind of there when the thing happens. Not all governments. The US government doesn’t do budgeting that way, is my understanding, although I am really not the expert on it.

So part of it is, yes, it makes things complicated. I think the solution is you sort of have to get people at a higher level who understand that this is worth it and kind of say, we’ll just increase your budget if this hits, because it’s worth it. Part of it is working through some of these details, like when we got so close to doing an AMC for malaria — way back, way back — and actually Larry Summers figured out a way to put it in the tax code, and it was in the Clinton budget, which then never passed. If the budget passed, I can’t remember exactly which year it was, it would have happened. So, you put sufficiently clever people…

Luisa Rodriguez: They figure it out.

Rachel Glennerster: They’ll figure out a way to make it work in the government structure. So, yes, I have spent many hours talking to bureaucrats about their challenges of how do they make this work. And some of the solutions are, “I’m just going to put some of my budget every year into a fund, and it’ll just accumulate.” I don’t think that’s the most efficient thing; that’s sometimes the only way that they’re going to do it.

But it’s part of the challenge of what we’re doing is to work through these mechanisms. Again, we did it with pneumococcal: it’s possible to do it, governments did it. But we want to work through it again, because these things do come up. But there’s billions of gain on the table. It’s worth figuring out the financing mechanisms.

Luisa Rodriguez: Right. It does just seem absurd that if the opportunity were there, or if you knew that it were going to be there in two years, it would be obvious that you would spend that money. But people are unwilling to risk not spending the money. It’s not like they’d lose money. They’re willing to put that money there either way. It’s just the fact that maybe they’ll actually get to keep all that money and use it for something else that is too much for them to deal with. But yeah, it sounds like there are workarounds.

Rachel Glennerster: Let me just say something which you weren’t quite asking, but the way you were setting it out actually does raise another question. You just said that if this was there, people would want to pay for it. So that might make people think, hang on, if it was there and it was willing to pay for it, then why are people not doing the innovation? Because they’d know that governments are willing to pay for it.

Luisa Rodriguez: Right.

Rachel Glennerster: Well, actually, there’s another market failure that we haven’t talked about, which is once you’ve invented it, governments have an incentive to drive down the price. Once it’s there, it’s like, I’m buying this thing. I’ve got to get the lowest possible price.

Luisa Rodriguez: Right. And you’ve already incurred the cost of creating it.

Rachel Glennerster: So knowing that, people don’t invent it in the first place. So that’s why it has to be an advance market commitment, because I have to commit now when I know it’s valuable and I know it’s worth it to pay for it. But if I wait until afterwards, I will drive a hard bargain, and then you won’t recover your R&D costs.

Luisa Rodriguez: Yes, that makes sense. And is tragic. And yeah, it does seem important to have brought up.

The story of the malaria vaccine that wasn’t [00:57:15]

Luisa Rodriguez: I’m actually curious to go back to the case of the malaria vaccine. So what is the basic story? It sounds like something like the AMC was designed, and then people got on board, and it was literally just the failure of not getting the budget through?

Rachel Glennerster: So the basic story is, Michael Kremer, my husband, was working on different ways to promote innovation, and then got into the neglected tropical diseases as an important case study. And we worked together on designing something for doing the work of what the benefits were of accelerating — again, it wasn’t that you come up with a malaria vaccine; it was that you would accelerate getting a malaria vaccine. We did malaria, HIV, and tuberculosis. We actually wrote a book on how you would design this, called Strong Medicine, and it went through a lot of the details of how you would design it.

There was an expert working group that was set up with the Center for Global Development, and that was the one that ended up saying let’s start with pneumococcal. We’d been hoping that it would be a far-off challenge rather than a kind of near-case challenge. But the idea is, well, let’s try it out with pneumococcal. If it works, we’ll then go on to malaria or something. And it worked, but kind of things had moved on. We never got to the malaria vaccine. I mean, we were arguing for it to be malaria or HIV from the beginning, but it was kind of like, we haven’t done one of these before, let’s do something and get a win. And of course, there were companies who thought that they were close on the pneumococcal, so they were lobbying that it should be pneumococcal.

Interestingly, and annoyingly, the pharmaceutical companies were all saying, “We’re not putting much resources into a malaria vaccine. Not because we couldn’t make money from it. No, god forbid we would be influenced by money or returns or anything — we’re not evil people. We would work on it if it was scientifically possible, but it’s too hard a science problem.”

Luisa Rodriguez: Wow.

Rachel Glennerster: And I can see why they do that for PR purposes, but no, you are meant to take into account your shareholder value. Of course you shouldn’t be working on something, of course you wouldn’t be working on something that wouldn’t make you money. And of course, if there was a malaria vaccine, there would be huge pressure on you to keep the price down. This was in an era when there was huge pressure on people to keep the price of HIV drugs down — which saved a lot of lives, but also taught pharmaceutical companies, for god’s sake, don’t invent anything that’s useful for poor people, because you will have your price driven down. So in a sense, it was the right thing to do for HIV, but it had these negative consequences.

And our response to this, and remember, we were doing this in the midst of all of this HIV battle, and we were saying, great, I’m glad that HIV drugs are getting out to people, but you are sending a terrible signal to pharmaceutical companies about how to do innovation for diseases of relevance to low-income countries. So can we please do this other thing that combines an incentive and gets it out at low price to people? Like, you don’t have to have this zero or one argument about patents or not patents — which, again, we saw in COVID. That’s just the wrong argument to be having. We can have both. We can have the incentive for innovation, we can reward the innovation and charge a low price and get it out to people.

Sorry to be so passionate, but if we had had a malaria vaccine earlier, so many people’s lives would be so different.

Luisa Rodriguez: Yeah. I think passion is the appropriate reaction.

Challenges with designing and implementing AMCs and other pull mechanisms [01:01:40]

Luisa Rodriguez: OK, so you’ve personally been involved in implementing AMCs before, and you’ve already described a couple of challenges. Are there any big challenges with either designing or implementing these that we haven’t talked about yet that are worth mentioning?

Rachel Glennerster: I mean, there’s 1,000 details that you have to get right. We talked about the price, which I think is a big one: how do you set the right price?

I think another one that we haven’t talked about, that was a big debate for the pneumococcal vaccine, was: how much do you want to try and promote more entrants into the market? So you can design this so that you just get it out as fast as possible, which is obviously a big part of the reason for doing this. But Gavi and some other people were really keen to draw into the process a low-cost producer. In the end, there was an Indian low-cost producer of the pneumococcal vaccine, and they thought that was important for sustaining the market over a longer period. But that meant that some of the market was kind of held back, and some kids didn’t get the vaccine as a result.

Luisa Rodriguez: Right.

Rachel Glennerster: That was a really difficult one that we spent a long time arguing about, and I still don’t know exactly what is the right answer there.

But another thing that we worry a lot about — and there’s ways to design it, but it’s important to get the details right — is people can copy an innovation. And you don’t want the copiers to get too much of the benefit, because that undermines the reward going to the person who did all the work — but you also don’t want to get stuck with the first person if other people are coming along with a better thing, and it improves and improves. So there’s a tradeoff there between making sure that a lot of the reward goes to the person who did the heavy lifting at the beginning, but also that there’s constant reward for innovation. And in our book, we talk a lot about the ways to do that, but it just shows that that’s one of the implementing things that you’ve got to get right.

Luisa Rodriguez: Yeah. Can you give an example of how you can deal with this?

Rachel Glennerster: One of the things that we talk about is you can enter in and get some of the benefit of the subsidy if you have a sufficiently improved vaccine. We talk about, in pharmaceuticals, “me-too” drugs or vaccines, which are just kind of a pure copy. And then there’s an innovation that’s sufficiently different or sufficiently improved, and then you give a reward based on the improvement. And you could even keep some of the original subsidy going to the initial innovator, even if you end up using the better vaccine that somebody else has come up with that is based on theirs, but was even better — so that the kids actually get the better vaccine, some of the reward goes to the original innovator, and the new innovator gets kind of the marginal benefit.

And that’s logically very easy to do. To figure out what’s the right thing to do is harder. You’re always in this tradeoff of, you don’t want to make a mechanism too complicated. Like, you can design something very complicated that figures out all of this.

Luisa Rodriguez: Exactly how to optimise.

Rachel Glennerster: Optimised, exactly. But you’ve got to implement it, and you don’t want it too confusing. So again, you’ve got to trade off how big a problem you think this is. Like in vaccines, it’s very hard to copy — again, something people got completely wrong in COVID — but anyway, it’s not like you just make the patent available and anyone can copy it. It’s all about the cooking. It’s not in the recipe; it’s in the expertise of the cook in vaccines. Whereas drugs are very easy to copy. So this is an example of where you’ve got to really understand the market to design things like this.

Luisa Rodriguez: Which, again, makes it just sound incredibly difficult. But yeah, that makes me feel excited about the goal in particular that you have: just training loads of people to know what to look for, and which things are actually kind of the details that are going to make or break it.

OK, so there are a bunch of other pull mechanisms that we’re not going to cover right now, but we’ll link to a really nice explainer doc about them in case people want to learn more.

Luisa Rodriguez: I guess my understanding is that a big part of the benefit of many of these pull mechanisms — and we’re going to talk about several of them in more detail — but my understanding is that part of the benefit is that you don’t pay out until an output is delivered. Which seems great; it means that you don’t end up spending money on inputs that don’t end up generating anything valuable, and it means that you don’t have to figure out for yourself who the most promising groups to fund are.

But I guess in some cases it seems possible that no one develops an output, in which case you might think that the pull mechanism was totally free. But in fact it sounds like, at least to make a good pull mechanism, you need planning, you need probably political will. And all of that does sound pretty costly. How do you think about those costs for pull mechanisms that don’t end up generating the thing?

Rachel Glennerster: Let me start with, “If you don’t succeed, it’s free”: there’s two costs. One is the setup cost of organising the legal framework. I actually think that’s pretty small compared to the huge benefits of most of the things we’re talking about — I mean, billions of dollars in benefit, if you spend $100,000 or a few hundred thousand dollars setting it up, I’m not worried about that.

A bigger cost is the firms who try and do it and fail. That’s not free to the world. People will actually be trying and failing, and that you have to factor in when you design how big the incentive is at the end. When we decide how much to pay for these things, we look partly at how valuable they are, but we also look at the failure rate of firms that tried to get there and failed. So that’s a cost to society, and we think about that.

Then there’s the political cost you’re talking about, which is, I think, very real. Politicians and bureaucrats and philanthropists like to have a win. They’ve put all this effort in, and they’d like to have a win. And I think that means that there’s a temptation — and we’ve seen this in the world — to do pull mechanisms for things that are what we call “near term”: they are solving problems that we’re almost about to solve. And those are good, because it helps speed up solving them.

Luisa Rodriguez: Sure.

Rachel Glennerster: And another thing that pull mechanisms do, which is really important, is they not just help to solve the innovation, but they help get the innovation out fast. And boy, do we need that for climate and pandemics, right? Which is partly why we picked those topics. So it’s not just that it creates the innovation; it is designed in a way that the innovation will get out to a lot of people fast, at low marginal cost and low price. So we can get into how it does that, but that’s a key important factor.

So these near-term ones are good, but there is a real challenge, because we would like to be able to use them for also the things that we don’t know whether we can do. We want to just hold out the incentive that we don’t know if you could do this innovation, but we would pay for it if you could.

And when we first started working on this topic, we were really hoping to do this for a malaria vaccine. That was our first key thing, an HIV vaccine and a malaria vaccine. And everyone said these are really hard technologically, so we’ll put some push funding in, but we don’t know if it’s going to happen. And our argument was, these are unbelievably valuable things. We should say that we are willing to pay quite a lot for them if someone came up with them.

And we actually just published an article, because we do now have a malaria vaccine. And that’s extremely exciting, but I think we would have had it faster if we’d done this. I think part of the reason we didn’t is because people picked a near-term task, the pneumococcal vaccine, because they thought they could solve it in a few years, whereas they thought the malaria one was far off.

So I think this is a real problem: people are a bit reluctant to make a big furor about something that will then happen once they are no longer in parliament, no longer in government. But, having thought about it a lot, I think the answer is to have a portfolio, and have some near term, some far. We’ll have a set of things. We might get on to climate-resilient crops; you could say we want to do a whole set of climate-resilient crops. Some are very easy, some are hard. We’ll commit to all of them. We’ll get some quick wins, we’ll hold out for some really big wins. And I’m hoping that is the way to solve the political issue. But I agree it’s a political issue.

Luisa Rodriguez: And do you have a sense, if you were looking back at the kinds of pull mechanisms that have actually been used, or making a guess based on your work on this, about how many pull mechanisms you might expect not to produce anything? I guess it just really depends on which kind of good things people are trying to bring about, and some will be much harder than others?

Rachel Glennerster: Yeah, exactly. It’s going to really vary by the targets you pick. I think we’ve tended to pick relatively easy targets, and therefore we’ve tended to get solutions. It also depends on how long you wait. When we started, you weren’t going to get a malaria vaccine in five years. But if we’d waited long enough, we would have got one.

But when we do the estimating how much you should reward people for these innovations, we do take into account the probability of failure, say, of vaccines. And we have a lot of data on the probability that a given vaccine will fail at stage one and stage two and stage three trials, and we know that quite well — which is different from, will anyone ever be able to do this?

Luisa Rodriguez: Right. Cool. OK, so part of the design is really thinking quite empirically about what kinds of results you can expect, and the total amount really comes from that. And is it basically just what is the amount that that, in theory, should cover, maybe plus a little bit more or something, the expected profit or whatever, that whoever comes up with the solution should expect to make?

Rachel Glennerster: Yeah. Coming up with the right amount of incentive is, I think, one of the toughest challenges in designing these pull mechanisms. And the way you tackle it is from the top down or the bottom up.

So one thing you do is you say: how valuable is this to society? That’s the maximum that you should be willing to pay. And of that, how much will the private public be willing to pay? And how much is the externality? How much is the market failure? So the subsidy needs to be linked to how much the market failure is. That’s kind of the top down: what’s the social value of this innovation?

Luisa Rodriguez: Right: what should we be willing to pay?

Rachel Glennerster: What should we be willing to pay, exactly. And then, from the bottom up, you think about the costs. Because you don’t want to reward people the full social value if it costs them only a hundredth of that to come up with it; you’d like to use that money to stimulate some other innovation.

So then you look at the costs, what you think are the costs, and how much do we think it would cost to innovate? Or how big an incentive does the private sector normally need to come up with this kind of innovation? We can use that a lot in drugs and vaccines, like: how big a market stimulates the pharmaceutical industry to come up with these kinds of breakthroughs? We look at the chance of failure: how often do companies put a lot of money in and then fail?

And there’s normally a huge gap between those, with the costs at one level, and the social benefit is way, way, way higher. You don’t want to put the incentive exactly where the minimum cost is because you might have got it wrong, you might end up with no vaccine or no climate solution. And that’s really damaging to the world if you put the price too low. So you probably want to put it a bit above that, but exactly how much above it? Again, you can start getting empirical about that. You can say, what’s the chance that we underestimated, and what would be the social loss if we underestimated?

It’s basically a lot of kind of simulations of different scenarios, and then a certain amount of judgement, and an assessment of what we think donors are willing to pay for and what’s realistic. It’s a really interesting mix of quite a lot of technical number crunching, on the benefits side and on the cost side, and some political judgement and understanding in a sense the political market of who might be willing to pay for this. It brings together all the things I love about my work, which is some interesting economics and some tough number crunching and some political judgement.

Luisa Rodriguez: Well that’s great, because I was just thinking, “Wow, that sounds really hard. I don’t know who’s going to design these” — but it sounds like you and your colleagues are excited to design them. It does sound really difficult to get these right. The idea sounds really compelling, and then as soon as you start talking about how difficult it is to even just pick a number… In my head, I hadn’t even considered that. It really makes more salient to me what a different challenge this is to push funding, where you’re just kind of evaluating grant applications where someone said what their budget is.

Rachel Glennerster: Right. But we might be giving up huge benefits to the world if we don’t do this. So yeah, it’s hard, but the reward is incredible if we get this right.

Universal COVID vaccine [01:18:14]

Luisa Rodriguez: I hope we do! I would love to talk about a case where you’d like to see an advance market commitment used: the universal COVID-19 vaccine.

So I think, if I understand correctly, the default approach is keeping the world inoculated against COVID-19 in the same way that we keep the world kind of inoculated against flu — where every year we kind of try to predict which flu strains will be prevalent, and then develop a vaccine to target those. But you’d like to see a universal vaccine which isn’t strain specific. Can you walk me through the case with that? I mean, it does sound good, but I think I’m asking because I’m like, why wouldn’t we do that? If we’re not doing that by default, maybe there’s a reason?

Rachel Glennerster: So the reason you want a universal COVID vaccine is because it takes quite a long time to wait until a strain arrives, and then develop a vaccine that’s appropriate for that, and then get it out to people. And we have data in our paper showing this, but by the time the new booster is out, you’re kind of over the peak of the wave anyway, so you’ve missed a lot of the benefit. And that’s partly because COVID spreads really fast, mutates quite fast. So if you could do a universal COVID-19 vaccine, you could even get it in arms, but certainly have it stockpiled and be ready to go. But ideally, a lot of people would just be inoculated against it, and then the new wave hit and it wouldn’t kill as many people.

Now, why is that not the default? Well, again, it’s more expensive, it’s harder to do. And one of the main benefits of having it be universal is in a pandemic situation, right? The next wave comes and it’s a Delta, right, when millions of people died, and because we couldn’t get the vaccine to them fast enough. It has this huge pandemic value. And as we talked about earlier, companies don’t get that pandemic value, because they can’t charge a lot in the midst of a pandemic when suddenly everybody wants this thing. So that’s why you don’t get it. Now, people are working on it, but they’re just not working on it as much as the social value would suggest they should work on it.

And for flu: again, we don’t have a universal flu vaccine. I mean, you said it’s like flu and we do flu every year. Well, it’s also really bad that we don’t have a universal flu vaccine — because, again, our flu vaccines aren’t incredibly effective, because they’re based on the strains that were around six months ago. And if there’s been mutations since then, it doesn’t work very well. But more importantly, we need it because it very well may be that the next pandemic is flu. The 1918 flu was worse than COVID. And economists were saying to get ready for the next flu pandemic, even before COVID hit. So if we had a universal flu vaccine, it could protect us from the next COVID-like hit. But then you won’t be able to charge for it in the way that is socially valuable. So that’s why we need an AMC for it.

Luisa Rodriguez: OK, so the universal COVID vaccine is harder. Obviously you’re not a biologist, but do you have any sense of how they work, and why they’re so much harder?

Rachel Glennerster: So again, I’m not a scientist, but I will give you my nontechnical views or thoughts. My understanding is there’s at least two different ways you could do this. One is you just target many more parts of the virus. So with a vaccine, you’re getting the body to respond to characteristics of the virus. Well, you can pick one bit of it — and we all got to know about the spike protein on COVID, right? — well, there’s many different bits of the virus, so you can have it trigger alarm bells in the body if you see any of those things. So that if there’s a mutation and the next round doesn’t have one part of it, well, it’ll have the others. It’s very unlikely you’ll get a mutation that changes all of it.

So that’s kind of one direction to go, and that has challenges associated with it. You might be less prone to respond well if it is similar to what you currently have, because you’re kind of diffusing the response across many different targets. OK, now my science is getting really bad, but there’s some downsides to having many different targets.

The other approach — which would be amazing if we got there, because it would have many wider benefits — is you can start tackling the mechanisms behind which viruses attack a body. Then it’s much more universal. But that’s obviously really hard science to do.

Luisa Rodriguez: Have you done much work to quantify how much better a universal COVID-19 vaccine would be relative to the boosters for specific variants?

Rachel Glennerster: Yeah, we have. We’ve just published a paper on this which people could go look up. So we said that a universal COVID-19 vaccine could be worth $1.5–2.6 trillion more than variant vaccine boosters. So this is an example of how the rewards of doing this stuff is big.

Luisa Rodriguez: And where do those come from? I don’t have the intuition that a universal vaccine is trillions of dollars more valuable than boosters. Can you explain how that works?

Rachel Glennerster: The way we do this is we do Monte Carlo simulations of what the world would look like under different scenarios. And what are the scenarios we plug in? Well, that we get different waves. We look at the arrival rates of waves that we’ve seen, and the arrival of Omicron, the arrival of Delta. You can make different assumptions about whether new harmful waves are getting less and less common, but you make some assumptions — like the high level is kind of assuming that they come at the average rate that we’ve seen so far, and the lower numbers are, well, let’s assume that it is tailing off very fast, and the probability of getting a new dangerous wave is going down quite fast.

So you just have a probability of a next wave hitting based on past data. And then you put in how long would it take to get a variant-specific booster based on how long it took to do that in the past, and then an assumption about how many people would get a universal COVID-19 vaccine in their arms before the wave hits, and how quickly, if it was all approved, we could get it out once the wave hits. So we make those assumptions, and we base them on there’s a lot of data from COVID about how many people are willing to take vaccines. So we base it on how many people took the booster? Well, if they took the booster, we think they’d probably be likely to be willing to take a universal COVID, one which is even better and protects you more.

So you make those assumptions and then you run these Monte Carlo simulations with the probabilities of things hitting. And then you look at the deaths that would happen if a bad wave hit and we were late in getting a vaccine out that helped. And that’s how you come up with the calculations. I should say that those numbers I gave you are just for the US. So the $1.5–2.6 trillion benefit is for the US.

Luisa Rodriguez: Wow, that’s really mind blowing.

Rachel Glennerster: I mean, I think this is the thing: we lose so much money from curbing the economy when these pandemics hit, when even kind of moderate things hit, when people die from waves — which they are still doing. And just putting numbers on these things is really important. I think probably one of the most important things I’ve done in my life is just put numbers on the value of vaccines during COVID, to just make it real and understandable how much it is worth investing in these things.

Luisa Rodriguez: That makes sense, and does seem like an incredibly good buy. I guess we haven’t yet talked about how much the AMC would cost, but I’m guessing it is not on the order of trillions of dollars. How much would it cost?

Rachel Glennerster: So we know how much benefit there is to the world of getting this. Then we say it would be worth, therefore, the US doing an advance market commitment to cover everyone in the US who wanted to receive it — roughly half the US population. So that’s 165 million people, so the AMC would require something like $6.4 billion.

We think it would be reasonable to spend $6.4 billion on an AMC. That would be an incredible deal, versus $1.5 trillion in losses, or 1.5 trillion in benefits from having it. I don’t know exactly what it’s going to cost a firm to produce this, but we talked before about how do you set the price: the top-down, the bottom-up approach. And our estimate is we should be willing to pay $6.4 billion in order to get the benefit of at least $1.4 trillion.

Luisa Rodriguez: Cool. Why use an advance market commitment in this case, rather than just funding groups doing promising work? Because you’ve already said there are firms working on a universal COVID-19 vaccine. If we know anything about which firms seem to be doing the most promising work, should we just give them money to do that?

Rachel Glennerster: So what there are are research groups working on this. What we need is big firms coming in and thinking that it’s going to be profitable to work on this. Because it’s one thing having research groups coming in, but you need a significant market out there for the big guys to come in, who know how to run trials, who put the work into making these things effective at scale. That’s what we’re missing at the moment.

Luisa Rodriguez: OK. I’m tempted to ask whether the AMC is really necessary, but given that we only have research groups and not firms, I guess it’s just totally plausible that an AMC would meaningfully accelerate the kind of development of a universal COVID-19 vaccine, even if it maybe was going to eventually be produced. It just seems like we’re really early on, and things aren’t necessarily going to steam ahead.

Rachel Glennerster: Yeah. Just remember, the main issue is not, “Is anyone working on it?” There may be some firms who are working on it, but there’s not sufficient attention to it, given the benefits. And the more work you put on it, the faster it will come, and the more likely it is to come. So you don’t have to have this hard cutoff of it wouldn’t happen without this. It’s: is there a probability that that will increase the chance of it happening? And what’s the probability that it will speed it up? And I think we can be pretty confident it would speed it up and increase the probability of it being actually produced.

And we see this too often, that research groups will be able to take things to a certain point, but you need the really big guys in there to kind of pull it into effective-at-scale, gone through clinical trials, et cetera — and they will come in earlier and they will push it harder if there’s a bigger market for it. Of that, I’m pretty confident.

Luisa Rodriguez: Yeah, that makes complete sense. So I have one more question about this, but I’m realising it actually kind of depends on something I don’t understand about what the universal COVID-19 vaccine would be like. Is it something that you’d have to get every year, in the way that you do for flu? Or would you only get it less frequently, the way you do some other vaccines?

Rachel Glennerster: So this is something that you’d want to work out in setting the criteria for this. I think the hope is that it would last for several years. One of the reasons that we need new flu vaccines every year is because the flu is mutated, and the whole point of this is that it copes with mutations. There are other mechanisms by which things need to be boosted, but you could imagine requiring a booster, like many vaccines that we have where you need two shots to be fully vaccinated, and there are other vaccines that you need to get boosted after 30 years. So all of that is not going to make that much difference in terms of our calculations. I think the idea is not to have to do it every year. And as I say, the main reason we have to have it every year is because it’s not universal.

Luisa Rodriguez: Right. So to the extent that existing booster manufacturers might be the groups who are best suited to developing the universal vaccine, is there an actively bad incentive? Where currently they might get to make money on annual boosters for new variants, but with a universal vaccine, they might only get paid for a few times. Is that a problem or a worry here?

Rachel Glennerster: Yeah, we certainly took that into account about how much the incentive had to be.

Luisa Rodriguez: Got it.

Rachel Glennerster: I mean, it depends how much competition there is. If you think someone else is going to come in and do it, that’s not that much of a deterrent for you working on it. But if it’s a small market, there aren’t that many people doing boosters, which is the case, then yes, you do worry about the fact that they don’t have much of an incentive to come up with a universal one, because they’re kind of doing fine selling it every year.

Luisa Rodriguez: Right.

Rachel Glennerster: So yeah, if there isn’t perfect competition, you do have to worry about that. And we did take that into account when we came up with how much it should be.

Luisa Rodriguez: OK, so that just is another factor that comes up, and then that you deal with when you design it.

Climate-resilient crops [01:34:09]

Luisa Rodriguez: OK, another area you’d like to see a pull mechanism is to incentivise the development of climate-resilient crops. Can you walk me through the specifics of that problem?

Rachel Glennerster: A huge problem with climate change is that crops are going to collapse in their yields. So for most crops, the productivity or the yield goes up gradually as you get more warm days, but then you get to a certain point where suddenly they just collapse — they’re just not going to produce much, because it’s too hot.

And this is happening already, and we don’t have enough innovation happening of development of climate-resilient crops. You can actually do selective breeding to get crops that are more resilient to climate, so they don’t collapse when it gets really hot, or they’re more resilient to drought, or they’re more resilient to floods. These are all things that we know how to do, but we’ve got to do them on an absolutely unprecedented scale.

And if you look at the climate models, and you look at these tipping points where a particular crop is going to suddenly collapse, we’re getting really close to some really important crops collapsing — particularly in Africa, which is where we’ve looked at it. So sorghum is a big staple in West Africa, and it’s been quite resilient up to now, but we’re going to get to the point quite soon where suddenly the yields are going to collapse, and we’re just not seeing enough innovation to make it climate resilient.

Luisa Rodriguez: Yeah. It kind of surprises me that you even need a pull mechanism here, because with new vaccines, it makes sense to me that it’s not always at least obviously profitable — especially for tropical diseases, where people aren’t going to pay super high prices for them. But it seems like heat stress is already reducing yields in some areas, including sub-Saharan Africa, and that there should be an immediate willingness to pay for heat-resistant crops. Is that just not the case because it’s going to affect kind of poorer areas of the world? Or why is it that this is a case that needs a pull mechanism?

Rachel Glennerster: Yeah, so there are a couple of market failures here. One is due to humans’ biases, and we’re not very good at paying for things that are hard to see, and hard to see the benefits of immediately. So this is a trait in a crop, a trait characteristic of the crop which you can’t see, and you can only see when you get hit by a really bad year. So we’re not very good at paying for things that are basically insurance. There’s a reason that we’re required by law to have insurance when we drive a car in the US, and that’s because we’re not very good at paying for insurance. So that’s part of the reason.

It’s also the case that what we really need is crops that spread themselves. So we want a heat-resilient sorghum, which, if you plant it one year, you can reuse the seed and share it with your neighbour. That’s a much more productive way of doing things than, you know, in the US, you have to buy the seed every year and then the big commercial firms make money from that. But that’s not the way that seed works, particularly for these traditional crops in Africa. People share the seed, and they reuse the seed from last year.

So you’ve got two problems with that. One is you don’t have the market systems working very well in a lot of West Africa, because you can’t really sell much seed to people, because they’re just buying it from their neighbour. But the other problem is that a lot of the people who benefit are kind of far down the chain of “my cousin’s cousin gave me the seed,” right?

I think those are the reasons why we think it’s not going to happen unless we have some kind of mechanism. And it’s not happening. We’re just not seeing much innovation in Sorghum. I mean, hats off to the few people who are working on it, but compared to what’s going to happen and the millions of people who rely on this crop, it’s a huge problem.

Luisa Rodriguez: Hi listeners, Luisa here, recording after the original with Rachel. I wanted to give just a few more facts about the scale of this problem before letting you continue listening…

Over 100 million people rely on sorghum in West Africa and the Sahel, and a 2022 paper found that there’ll be something like a 28% fall in crop yields in sub-Saharan Africa by 2098, even accounting for business-as-usual adaptation and development.

OK, back to the interview.

Luisa Rodriguez: And how do you think about what kind of pull mechanism you want in this case? What are the considerations, and where do they lead?

Rachel Glennerster: So one of the things that we want to do is incentivise people to do the actual R&D. But as we’ve talked about before, a big part of pull is also getting it out. And there is a challenge in the markets for some of these crops, so the pull mechanism would have to incentivise people to actually sell this.

So what we would do is tie the payment to the innovator to how many people actually use the sorghum. You can go out and, you do a representative sample, and then you take clippings from people’s sorghum and you run it through — you take it back to the lab, and you see if there is a gene in there which comes from the innovative new approach. So even if it’s kind of crossbred or it’s been passed on to your cousin or whatever, you can still tell that it’s due to that innovation.

And then you reward the innovator by how many people use it — which gets all the right incentives for them to develop a crop that’s easy to use, that people want to use, and also incentivises them to do lots of advertising and distribution. Because there are a lot of really good research groups out there working on trying to improve crops, but they’re not always as good at getting it out there and doing this distribution, and we really want to pull in the private sector, who thinks a lot about how to do this, to work on it too.

So we see likely a collaboration between researchers and research organisations — of which there are a lot in Africa, a lot of agricultural research stations based in a country who know how to develop things that work in the climate of that country — but you need them to be collaborating with seed developers, and multipliers, and the people who get it out to farmers. And I think if we tie the reward to how many farmers actually use your product, that will get the right incentives.

Luisa Rodriguez: That’s so, so cool. I was either expecting the incentive to be tied to sales — which would have seemed pretty good, but then maybe the bags sit somewhere because they’re not actually that easy to use; they’re slightly different and the instructions aren’t super clear. Or even worse, I could have imagined just having the seeds available on the market but not tied to sales, and that obviously would have been even worse. But you can actually check that the seeds are being planted.

Rachel Glennerster: Yup. Planted and grown. And if you do that several years in, farmers are only going to keep replanting it if it works well. I’ve done work in the past about some of these new agricultural innovations, and they work great on research sites, but then when actual real farmers use them, they find them hard to use. And we really need incentives for things that are really useful and easy for farmers to use.

Luisa Rodriguez: Cool. Yeah, I’m very inspired by that. I can also imagine it going wrong, because it’s a weird new seed that they might not otherwise pay for. They might just, as you said, get it from a neighbour. And so having marketing that actually sells this seems really difficult to me, but that’s just part of it. You only get the incentive if you’ve thought all of that through.

Rachel Glennerster: You’ve got to figure it out. Yeah, we’ve done scenarios about the benefits of under different scenarios in how much people use it. I mean, it seems obvious, but that’s where a lot of the benefit comes from, is getting it just out to more people.

Luisa Rodriguez: Yeah. Cool. And so this is a case where you’d like to see this mechanism used. It’s not implemented yet. How optimistic are you that this will go forward?

Rachel Glennerster: I am always an optimist. I think there’s a lot of debate about exactly how to structure this. And there may be some markets and some seeds where there isn’t enough private sector involvement that you can’t really use pull, because the only actors in the field are public agencies.

So what we’re doing is going through all the different crops where we really need this, and figuring out where you should just pay for the research and where you should use a pull mechanism. I think there are definitely crops where it’s worth doing pull, but some we may decide — and we’ve already decided for some — that it just doesn’t make sense to do pull. We should just pay people, the research agencies to do that. And yes, you might not get as much takeup, but there’s no one to respond to an incentive.

But in other crops, a lot of the work of this is going and understanding the markets for sorghum in Nigeria, and who are the private companies who are doing this. And that’s the nitty-gritty of this work. I can tell you the broad outlines, but the real work is in getting all of those details right, and really understanding how markets work in these countries.

Luisa Rodriguez: Yeah, I think that’s the bit that is surprising me most about this conversation, is both how much the nitty-gritty matters, but then also how there are solutions to the nitty-gritty obstacles that come up.

The Market Shaping Accelerator’s Innovation Challenge [01:45:40]

Luisa Rodriguez: OK, let’s move to another topic. You launched a $2 million Innovation Challenge as part of your work on the Market Shaping Accelerator. And the goal is to basically identify new opportunities for pull mechanisms to address market failures in the fields of biosecurity, pandemic preparedness, and climate change. Just broadly, how exactly does the challenge work?

Rachel Glennerster: Yes, so the idea was to generate ideas about where these pull mechanisms might be useful. We did a very broad call for ideas, and we started with quite a simple application of just tell us your idea, why it’s an important idea, and why you think pull is the right way to solve it. And we got over 180 submissions. It wasn’t that easy. You had to understand a little bit about it to fill in the form.

Then we awarded prizes to 39 ideas, and then we selected seven of those to really work on — work with, provide some funding, but also provide advice and input to the teams who’d put forward those ideas. And we’re working on them now to think through, like, what is the cost effectiveness? Is pull really the right response? Is it practical? Is there potentially someone who would be interested in paying for this? So starting on that process of the nitty-gritty. Then the idea is we’ll narrow that even further, and pick a few to try and push all the way to the end, and actually get someone to take them up.

So we had ideas when we started off, but we really wanted to generate all the ideas out there that we might be missing, and also crowd in a lot of other people. So a lot of the work on these teams is not being done by us — it’s being done by people who really know about the specifics, and we provide some input on how you do the cost-benefit analysis in these cases, et cetera.

Luisa Rodriguez: Right. Yeah. Correct me if I’m wrong, but I like that it sounds like at least Phase I of this was a prize — it is basically that you didn’t know what all of the possible ideas were, so you let people select themselves in and offer ideas if they had good ones, and then you got to choose from those and kind of move them forward.

Rachel Glennerster: Yeah, exactly. We set the criteria on which we judged, and we offered a prize, and we got a response. We also thought that this process would be good because there’s an element of competition. If you just pick one, everybody wants to make the amount that they should get as big as possible. But if you’re competing as to which of these is going to go forward, there’s a little bit of healthy competition, I think, in there. And that was deliberate.

Luisa Rodriguez: Yeah, that makes sense. Because you thought through how to get the best results. Cool. So after Phase II, which is where these projects currently are, there’ll be just a couple. And the idea at that point is to potentially pitch funders on putting up the funding that these pull mechanisms would need to be implemented?

Rachel Glennerster: Absolutely.

Indoor air quality to reduce respiratory infections [01:49:09]

Luisa Rodriguez: Amazing. OK, let’s talk through some of those seven ideas that have made it to Phase II. One is led by Gavriel Kleinwaks at 1Day Sooner, and her team wants to focus on indoor air quality. Can you explain the basic idea?

Rachel Glennerster: Yes. So infectious diseases cause a lot of problems, a lot of deaths. Just regularly, we get about 400,000 deaths per year from respiratory infections in high-income countries. But also they are the thing that we most worry about for pandemics. And there are ideas about how to stop them spreading in the air; the reason they’re so dangerous is that they spread very easily through the air.

And people may be familiar with putting filters in during COVID, and the importance of ventilation — just like opening a window was useful — but you can go further than that, and actually try and zap the air and kill the bugs in the air. And that would be really exciting if we could do that. There’s some basic ideas about how to do that using ultraviolet light, which there’s a lot of work that’s needed to make them test whether they’re safe and effective. But that’s a really exciting idea.

Luisa Rodriguez: My impression is that there are lots of for-profit businesses that would just benefit enormously from having indoor air quality. Employees would get less sick less often, and customers would get less sick, which they might want in various contexts. Why isn’t that incentive enough to solve this?

Rachel Glennerster: So there’s two problems. One problem is that, yes, firms benefit a bit, but they also generate benefits for other people. So your workers don’t get as sick, but then they also don’t get the person sitting next to them on the metro sick. So there’s positive spillovers for other people for you doing this. So you’re willing to pay something, but not quite enough. And actually, in a pandemic, that’s really important, because a little bit of benefit can sometimes tip you below the level where the disease isn’t spreading as fast. So it is particularly a failure in pandemics, that it’s really beneficial to have it not spreading in the subway. So you don’t capture all the benefit as a firm.

The other problem is that there is a lot of research and development that needs to happen, like testing the safety and efficacy of this thing. And whoever does that, whichever firm does that testing, is not going to capture the benefits of it at all. All the other firms are going to learn immediately whether it was safe and effective. So nobody wants to be the one who puts in the millions of dollars to test whether this thing works. So that’s a huge market failure.

Luisa Rodriguez: And this isn’t just solved by them then patenting it, partly because that would mean that there would be a monopoly provider and that creates problems. But also, are there other things going on that mean that the typical reasons that it’s OK for one firm to do a bunch of research and then get the benefits that those aren’t happening?

Rachel Glennerster: Partly it’s really inefficient. Patents are really inefficient, because then only one firm does get the ability to do it, and they may be then expensive. So that’s one problem. Then the other problem is, usually with patents, people find ways to kind of copy. So you learn that this is a safe thing to do. And you can’t do exactly what they did, but you’ve still learned that it’s safe, and so you might find some other way of doing it. We call it a “me-too”: it’s far enough away that you get around the patent system, but it’s close enough that you’ve learned a lot. So those are two of the problems with just doing a patent.

I think actually probably the best way to solve this R&D thing is just to pay for it, just to do kind of classic push. We’ll just pay for the trial. But I think the externality — the fact that firms, particularly small firms, don’t capture a lot of the benefit of this, so it might be that big firms want to, but small firms might not want to — so pull is more appropriate for the scaling up in this case.

Luisa Rodriguez: OK, yeah. And then how does the pull work in the scaling-up part?

Rachel Glennerster: What we’re looking into is a buyers’ club where some of the organisations that are willing to pay something for it — like cruise ship lines — might get together and agree what they would need, what it would look like, the product they would like to buy. And then if they commit to buy quite a lot of it, then somebody else would come in with a subsidy to basically cover that externality — the fact that the whole of society benefits from some firms doing this. And that provides certainty to people who are thinking of working on this, and it also gets the firms to commit in advance to give that certainty by providing that subsidy.

Luisa Rodriguez: Yeah, that makes sense and feels very ingenious. Do you feel reasonably optimistic that a bunch of cruise ship executives are going to be willing to commit in advance to buy this technology if it meets their criteria? It just feels almost too good to be true, because it’s such a good social outcome. So some part of me feels sceptical that they’re going to be like, “Yes, we’ll do this thing that’s good for the world.”

Rachel Glennerster: Yes. So we don’t expect people to do anything that’s just for the sake of the world. We’re trying to take a hard-nosed economic look at this, and say, what’s in their self-interest to do? And it builds on that.

Do I know that this is going to work? No, I think that the benefits are worth investigating. So we’re going to take a good hard look at this in the coming months, and test it and critique it and see if we think that it makes sense.

Repurposing generic drugs [01:55:50]

Luisa Rodriguez: Cool. OK, so another one of these ideas that’s made it through to Phase II is the repurposing of generic drugs. I’ve never really heard of this as a potential solution to any particular problem. What is the use case here?

Rachel Glennerster: So there are a lot of drugs which are approved for one purpose, one disease, and end up actually being quite effective for another disease. We actually saw this during COVID. There was a fantastic trial in the UK where they just tried a lot of existing low-cost, off-patent drugs and to see whether they worked, and some of them did. The problem is firms don’t have the incentive to spend millions of dollars testing a drug that’s already off-patent to see if it works on another disease.

Luisa Rodriguez: Just to make sure I understand, is this kind of like the fact that people take aspirin for various specific reasons, but it also seems to reduce some types of heart disease? And so there are just things like that in other places, and we could learn about them, but we don’t really, because it’s not that profitable to sell generic drugs?

Rachel Glennerster: Yeah, it’s exactly that. And it’s profitable to sell them if you’re not doing any R&D, but if you do a big clinical trial, you’re not going to get the returns to that, because all your competitors can sell it for the other disease too. So you can’t make any money from it.

Luisa Rodriguez: This almost sounds like related to the case where you are going to use push funding — because as soon as we know that a generic drug is helpful for some other disease, other firms can sell it for that other disease. Why is it that pull mechanisms apply here?

Rachel Glennerster: So the key thing that we want to generate with the pull mechanism is firms to think about all the millions of drugs out there. If you’re sitting in the centre, you can’t really tell what possible drugs are there that could be repurposed for what possible disease. So what you want to say is, we’ll pay a higher price for anything that you come up with that is cost effective. That is decentralising the search process. So I’m not in the centre saying that I think this drug could be repurposed for that; I’m just saying, “If anyone out there has a really good idea for a particular drug that could be repurposed for something else, we want to give an incentive for all of that.”

A key thing about pull is that it harnesses the information of all millions of people out there who might have ideas that you don’t know anything about. So that’s why we need pull here, because there’s millions of drugs, there’s millions of different uses, we don’t know what they are. We need to create an incentive for people to come forward with ideas.

Luisa Rodriguez: Got it. So unlike the indoor air quality thing, there’s no specific idea where we could just be like, “Test this thing — here’s the money to test it.” There are a million ideas, and so you need different people to come through it and say, here’s an idea. And then maybe at some point, someone has to figure out how to fund that testing.

Rachel Glennerster: Well, yes, but I think actually we can do this all by pull. We don’t want people to come forward and just say, “This is an idea, can you pay for it?” Because they have information about whether it’s likely to succeed. The purpose of market shaping is to put the risk on the people who have the most information about something.

So I think there’s two different ways that you could do a pull mechanism for repurposing drugs, and they solve different problems. One is in high-income countries and one is in low-income countries.

So in high-income countries, you can tie the reward to whether people use the new repurposed drug for its new purpose. So you can say, if you test and find that an existing drug works for a new disease, and people use it for that new disease, we’ll pay you based on how many people use it for the new disease. And that works in high-income countries. In the UK, there’s a pharmaceutical price regulation system, which sets prices for all drugs. So you can say, “This is the price that we’re going to repay you, if you use the drug for this purpose” — and it’s all tracked, and we know why doctors are using it, so we can tell whether doctors are using it for the new purpose or the old purpose.

In a low-income context, we don’t have those kind of centralised data systems to know whether someone is using that drug for the old reason or the new reason. But there’s a lot of things where probably repurposed drugs might really help in low-income countries. So then I think the best thing to do is a prize, where you just pay someone, but you pay someone if it works, right? You tie it to an outcome. So you say, “We will give you money if you manage to show that this generic drug that’s super cheap actually works for some other reason, and you get regulatory approval for it” — and then you get a prize for having done all of that work.

Luisa Rodriguez: Cool. So are you interested in exploring both of these for this challenge, or do you think one is more promising than the other?

Rachel Glennerster: Yeah, we’re working with the Duke Margolis Institute for Health Policy. So I want to give a shout-out to our partners here who are doing a lot of the work. We’re still in the stages of kind of figuring out which is the right use case. I think they’re particularly interested in the low-income countries; I think there’s a lot of benefit in the high-income countries. We’ll work through it. I think there’s a lot of promise in both. They’re slightly different. At some point, we’re going to have to focus our energy on just a few of these. There are so many exciting ideas, and we’re going to have to focus — but we need a little bit more time working through how we think these things would work.

Clean air conditioning units [02:02:41]

Luisa Rodriguez: OK, let’s talk about another one. The Clinton Health Access Initiative proposed using a pull mechanism to incentivise the development of clean air conditioning units. Why is clean AC a top priority?

Rachel Glennerster: It’s a top priority because we’re going to see an explosion in the use of AC units in low- and middle-income countries with climate change. As they get richer, they’re going to use more. It’s already hot in India, right? If you’ve ever worked there, you really like the AC in parts in India. But it’s also going to get hotter, and it’s a real health threat. The temperatures are getting to the point where you will die if you’re outside in the heat. So it’s a real need, but it’s going to use a huge amount of electricity. And in somewhere like India, a lot of that electricity comes from coal, so it’s really exacerbating the climate crisis.

So to give you a sense of the extent of this, 4% of global emissions come from just from cooling.

Luisa Rodriguez: That is wild.

Rachel Glennerster: Yeah. And we’re going to see really dramatic increases in the amount of cooling in low- and middle-income countries.

Luisa Rodriguez: Right, because you have this double problem of the world getting hotter and low-income countries getting richer. That seems like a really big problem. What kind of pull mechanism might work in this case?

Rachel Glennerster: One thing to do is to provide subsidies to people who buy more efficient air conditioning, but we might want to tie that subsidy to incentivise people to do more work on generating more efficient air cooling.

So there’s a problem here. Again, this is something that consumers buy. And again, we face this problem of behavioural biases: people don’t like paying a lot up front for savings in the future. And that might be because they’re credit constrained, but it is also likely to be because they just look at the sticker price and don’t figure out how much they’re going to save in energy later.

Luisa Rodriguez: Yep, sounds relatable.

Rachel Glennerster: It’s also a problem, frankly, that they’re not paying the full cost in India of electricity. It’s worth it for society, because you save a lot of energy. But electricity is massively subsidised in India, so there’s a whole tonne of problems going wrong in why you don’t get people buying efficient air conditioners in India. And it’s a huge market, and it’s a huge need. So you could do a commitment to subsidise new kinds of air conditioners that are a lot more cost effective.

So there’s really a choice of: you could just do a subsidy to get people to use the existing more efficient air conditioners, or you could design it so that it’s tied to somebody inventing a more efficient air conditioner. Now part of what we’re working through at the moment is which of those to focus on.

Luisa Rodriguez: What are the considerations there?

Rachel Glennerster: One of the things that we’re trying to think through, and you hear this a lot in climate, is: if we get more production and get the scaleup, the costs will come down. And that is true of solar, and everyone got really excited about the fact that the more we subsidised solar, the more people used, and then the cost came down. It’s not always the case. If a more efficient air conditioner just uses more materials, it just is more expensive because you need a bigger compressor. Or, again, I’m not an engineer, but my understanding is that part of more efficient air conditioners are just that they have more stuff inside them, and the stuff is expensive. And if you produce more of them, the costs may come down a bit, but they might not come down all the way.

So what we’re trying to figure out at the moment is: how much are we just trying to get more scale — and the price will come down, and then we can stop subsidising, but the price will be lower, and then people can just keep using it — or how much is it that we just need a whole new technology which is just going to produce cheap air conditioners, but they’re more efficient, but they’re also cheaper and therefore we’re not going to have to subsidise them forever? We may well have to subsidise these forever, and then how much would you have to subsidise them? So when we do the cost effectiveness of these two alternative options, those are some of the things that we have to take into account.

I think the other issue is that a lot of people are going to be buying air conditioners in India in the next five years. So can we afford to, is it worth it to wait for an even better one? Or do we just subsidise the ones that are more efficient now, because so many are going to get bought? So maybe you try and do both. We’re still working it through.

Luisa Rodriguez: Yeah. Well, thanks for indulging me, because I know it feels like those are a bunch of, again, nitty-gritty details, but I’m finding those really fascinating. I think it’s just giving me so much hope that these ideas don’t get roadblocked, that you just think really hard about them.

Rachel Glennerster: One reason to tell you about the nitty-gritty is this is 80,000 Hours, so people are thinking about what to do with their careers. So this gives you a sense of what you actually spend your time working on. It’s rarely the kind of highfalutin idea. It’s like the nitty-gritty of, how much will costs come down? And how many more air conditioners will be bought in the next five years, and what’s the climate impact of that? So that’s what you actually spend your time doing if you work on this kind of thing.

Luisa Rodriguez: Yeah, and I agree. That is exactly what people should hear if they’re at all interested in doing this kind of work.

Broad-spectrum antivirals for pandemic prevention [02:09:11]

Luisa Rodriguez: OK, so the last idea we have time to talk about is incentivising work on broad-spectrum antivirals. Can you give me some background on antivirals? I feel like I’ve never really understood why we’ve had broad-spectrum antibiotics for so long, but not broad-spectrum antivirals.

Rachel Glennerster: So you should probably, again, talk to some scientists about why one is harder than the other.

Luisa Rodriguez: Fair enough.

Rachel Glennerster: But let me tell you why it’s a good thing to have. Maybe it seems obvious, but if we want to prepare for a pandemic, we’re really behind the curve if we develop an antiviral that works for that virus once it’s already hit. We’re just going to lose a lot of lives by the time we figure it out. So the real benefit is to develop something that could cope with many different viruses in advance, and then we would have it stockpiled and ready to go.

Now, it’s a hard problem, and I think my very basic science is that viruses are hell of a lot more complicated than bacteria. So I think doing something that copes with lots of different viruses is just a hell of a lot more complicated than something that deals with a whole range of bacteria. So it’s a hard challenge, it’s expensive to do, and a lot of the benefit comes in a pandemic — when, as we’ve talked about, you can’t charge really high prices at that point, because there’s a lot of appropriate social pressure to keep the prices down.

But this is something that we could work on in the meantime. And unfortunately, it’s sometimes hard to get people to work on things when it’s not right in front of their faces. So making a commitment. And it’s probably going to be governments or philanthropists who think longer term to say, yeah, it’s worth it for the risk that this happens 10 years from now or 15 years from now or something.

Luisa Rodriguez: Yeah. I find it so depressing that you’d think this kind of thing actually would be super salient, but already it just seems like lots of people have kind of forgotten how valuable this kind of thing would have been three or four years ago.

Rachel Glennerster: I think one of the reasons why we’re not focusing enough on pandemics is there’s this weird psychological thing where because it’s just happened, we think it’s less likely to happen. So if you throw a dice and you get a six, you kind of think, well, I’m not going to get a six next time. Actually, it’s completely independent — whether you throw a six next time has nothing to do with whether you throw a six last time — but somehow the brain doesn’t think that way. A six is rare; it’s only one in six. I got it last time, so I’m not going to get it next time.

And I think there’s a bit of that in pandemics. It’s like, COVID was a one-in-50-years event, so it’s going to be 50 years.

Luisa Rodriguez: We’ve got at least 50 years.

Rachel Glennerster: Yeah. So there’s two problems with that. One is, no, you’ve got a one-in-50 chance every year of having a new COVID. And also, there’s a one-in-50 chance of COVID, but there’s also a one-in-100 years of it being twice as bad as COVID. I’m not getting my probability. But add to that the probability that there’s something half as big as COVID, and actually the probabilities start stacking up.

Luisa Rodriguez: Yeah, I think that’s a great point. And I feel very sympathetic, because my brain is also like, “We just had our pandemic. I’m good.”

Rachel Glennerster: Yeah, we’re done. No, it doesn’t work like that.

Luisa Rodriguez: Yeah, my grandkids can have a pandemic, but we’re set for now. And I get it, and it’s just completely wrong. So what would the pull mechanism here look like? What does it have to solve?

Rachel Glennerster: I think in some senses, the pull mechanism here is relatively straightforward, because governments are probably going to be the people who want to stockpile this stuff. So you could offer to buy this and stockpile it. Now, this is a little bit more complicated because you don’t want to just have a stockpile, because it might go out of date. You want them to be able to keep manufacturing it. So you want to tie your payments not just to sell me the stockpile, but also keep production going. You want the production to keep ticking over and being able to expand, show that they can expand when the pandemic hits.

So you have a purchase agreement with a government, or hopefully a set of governments. And we’ve been working with people who work with the German government who are quite interested in this idea, which is great. So ideally, a set of governments come together and say, “We would buy this. We would buy a stockpile, but part of the purchase contract is that you keep making this, and you have the ability to make more in a pandemic.”

Luisa Rodriguez: That makes sense. So what is the timeline for potentially a couple of these moving to Phase III? When will we find out which of these end up looking really promising?

Rachel Glennerster: We’re working at the moment to see who goes on to the next stage. I think in the next couple of months we will be figuring out the next step. It’ll be many more months then before we say that we have these two or three or one that we’re going to take all the way, or we’re going to try and take all the way. But yeah, we’re making rapid progress, I think, in honing down, adapting these. So in the next few months, we should have some news about next steps.

Luisa Rodriguez: OK. Listeners can keep an eye out to see if some of the ones we’ve just talked about end up seeming…

Rachel Glennerster: Follow us on the Market Shaping Accelerator website. We talk about all the different ideas and lots of content, and we’ll be putting more training materials up about how to think about this.

Luisa Rodriguez: Great. We’ll link to all of that.

Improving education in low- and middle-income countries [02:15:53]

Luisa Rodriguez: Let’s turn to another topic. So you are part of an advisory panel convened by the World Bank, UNICEF, USAID, and the UK’s Foreign, Commonwealth & Development Office that searched through over 13,000 studies to identify over 400 high-quality studies on education in low- and middle-income countries to basically find the most and least cost-effective education interventions that we know of. And we’re going to talk through some of these. The report is also just really interesting, and we can’t talk through all of the findings, so we’re going to link to it. I really recommend anyone remotely interested in education look through it. It’s very readable.

But we’re going to start with the “Great Buys.” These are interventions that have a strong body of evidence suggesting that they’re highly cost effective — an order of magnitude, in some cases, more cost effective than the next-best category, which is “Good Buys.” So just really good, promising education solutions. What is one Great Buy for improving education in low- and middle-income countries?

Rachel Glennerster: One is providing people with information about the benefits of education. And exactly what information you provide depends on the context. What is it that people don’t know? In some cases, it’s the benefits of staying on in school — like how much more would you earn if you stayed on in school? In other cases, it’s that there’s financial support for if you wanted to go to university. And therefore people actually, when they know that, end up working harder.

But because providing information is so cheap, across many different kinds of information we found you can get big impacts: kids work harder, they stay in school longer because they have this information.

Luisa Rodriguez: I love this one, because it sounds too simple to be good, but I guess it is just very cheap. Is it cost effective because it’s really cheap, but the impact is a little bit small, and that’s how you get the big cost effectiveness? Or are you confident it’s a meaningful impact, if that makes sense?

Rachel Glennerster: It is a relatively small impact at a really small cost. And when we talk about “meaningful” impact, if you impact a very large number of people and make their lives a little bit better off, then I think that’s meaningful. I do think there’s a bias in the system. This is a kind of different point, that the people like to see things that they can actually see — like they can see the difference in a child’s life. And you probably can’t see this difference, because it’s a small impact on millions and millions of people at a very low cost. So I think we actually don’t do enough of those things.

Luisa Rodriguez: Yeah, I see what you mean. It would feel much more satisfying to me as a donor to give 10 people scholarships that mean they can go to college, relative to tell 1,000 kids that there’s some scholarship out there that can help them go to college, and maybe two of them then work hard enough that they get that scholarship and it’s good, but I never know that.

Rachel Glennerster: Yeah. Or dozens, hundreds of them work harder.

Luisa Rodriguez: Right. But it’s hard to know for sure that that’s why. And they’re small enough benefits that they’re hard to measure, but they’re there and they have some impacts on people’s lives even if they’re not.

Rachel Glennerster: Yeah. We can measure them in big RCTs, which is how they’re measured, but you as an individual can’t literally see it. And again, this is sort of a cognitive bias that we have, and I think we should try and overcome it and do the things that are very low cost but you can do for very large numbers of people.

Luisa Rodriguez: Makes sense. So I think there have been studies in lots of different places measuring the impact of this intervention. Is there anywhere where it hasn’t been cost effective or does it look really good across many different contexts?

Rachel Glennerster: It’s cost effective in a lot of different contexts. I think here the trick is you’ve got to provide information that’s correct, and you’ve got to understand what is needed — like, what’s the information that people don’t have? So there have been some cases of providing information about which is the best school to apply to in Ghana, and people weren’t able to respond.

But when you’re looking at this body of work, and there’s a very large number of these studies, you want to not just try and learn from the studies that don’t work, but you also want to base policy on what’s most likely to work, and where you’ve got a very large number of studies that seem to be pointing in the right direction.

And I should say, on this panel, we do a metaanalysis, we do the number crunching — but we also do quite a bit of judgement calls. And one of the things that we’re looking for is not just that there are X number of studies that show this, but that people can actually implement it, that governments can implement it at scale. So we don’t recommend anything that nobody’s actually been able to pull off at scale, and ideally test it at scale. And I think this is a really nice thing about the panel: there’s a lot of people on there who have experience implementing at scale as well as researchers, so we bring a lot of different perspectives when we look at the data and the evidence.

Luisa Rodriguez: So it sounds like you think this has been and could continue to be implemented at scale. What seems like the hardest thing about implementing it somewhere it hasn’t been implemented before? Is it that you just have to know what information is missing and how to get it to people?

Rachel Glennerster: Yeah, and those have to be designed in a particular context. What’s the information that you think might be needed and how do you get it out is going to be different in different countries.

And again, this is something that we urge and talk about in the report: how do you take the data from what kind of context do you think this will work in? So you’re not starting from scratch when you start in a new country; you know the kinds of places where people don’t know about the returns to education, and it’s quite easy to diagnose, for example, am I in a situation where kids aren’t going to primary school? Or is it the case that they’re dropping out between primary and secondary school? Or is it that low-income kids aren’t trying for university?

Those are very different kinds of countries that face those different problems. So that can be relatively simple to figure out: what level am I working at? What kind of information am I working out? In other places, it’s really complicated to know how to get funding to go on; in other places, it isn’t. So it is something you have to work out.

I think how you deliver information is also very different. Do you send text messages? Do you do presentations at school? Do you have kids look at videos? Again, though, it’s pretty obvious that you can do the video thing in Peru, but it’s harder to do it in Sierra Leone where there are less tablets around. So some of it is kind of common sense.

Luisa Rodriguez: Yeah. OK, cool. Well, on a very personal side note, it really tickled me to see this one as a Great Buy, because I was an RA in Peru when they were evaluating the impacts of this intervention on tablets.

Rachel Glennerster: Fantastic.

Luisa Rodriguez: So that was really fun for me. So that’s one. What’s another Great Buy?

Rachel Glennerster: Structured pedagogy is another great buy, and this was moved up because it’s had many more studies come out since the panel last met; we revise our recommendations every couple of years based on the new studies coming out.

So structured pedagogy is where you provide teachers with some structure about how to teach. You’re not scripting them, you’re not telling them exactly what to say, but you’re doing things like lesson plans, and then you match that with appropriate textbooks. And all of that is based on the science of learning, so that they are introducing concepts in the right order. We now know quite a lot about that science. So it’s guiding teachers through their teaching to introduce things in the right order and at the right pace, and prompting them to think about, have the kids reached this benchmark and are they ready to move on? It’s a package of things to support teachers to do that. So the textbooks are linked, there’s mentoring to go with it, and it’s all based on, as I say, the science of how you should teach.

Luisa Rodriguez: And what are the impacts and costs like here? Is this kind of diffuse impacts at low costs, or is it more in the middle?

Rachel Glennerster: It’s much higher cost than the information, but you can get some really big gains from it.

Luisa Rodriguez: Cool. Is it hard to get teacher buy-in for this?

Rachel Glennerster: It’s interesting, because some people get very worried about telling teachers how to teach, and we’re undermining the profession of teachers. But actually, most teachers like to have a bit more structure. They like to have help about how to do their lesson plans.

So I think you’ve got to introduce it in the right way. You’ve got to talk to the teachers’ unions. This is the kind of thing where you really need to do it at a pretty big scale. You’re often changing the curriculum and the approved government textbooks. So it’s often done at quite a big scale. So then you are getting all the stakeholders on board before you introduce these big changes, and you make sure that the teachers understand that you’re not undermining them, and that you’re just trying to support them. So working in school systems is quite complicated, and difficult politics as well as science. But if you do that right, then it’s actually gone down quite well.

Luisa Rodriguez: OK, so another one is targeting teaching instruction by learning level and not grade. Can you explain that in a bit more detail? What exactly does that mean?

Rachel Glennerster: Let me explain it by starting with the problem, which is very, very prevalent in low- and middle-income countries, which is that you often have ambitious curriculum that is designed for the top 10% of the population.

This used to be very true in the UK, for example, when I was growing up, and you had exam systems that were designed for only 10% of the population to pass, and all the curriculum was based around that. And then you have a lot of kids who are struggling to keep up with the curriculum, and you also have teachers facing the problems that you might have in a class with a huge range of learning levels.

So there’s good data from India, which shows that in a ninth grade class you will have some of the kids learning at third grade level, fourth grade level. In this particular study, nobody was at ninth grade level. So the teacher is teaching at ninth grade level, but the kids range from third grade to eighth grade. That’s a hugely difficult task for a teacher to teach to that range of levels of learning.

A better approach is to take children and divide them up, at least for part of the day, where you clump them into groups of kids who are still learning their letters, and kids who know their letters but are still putting together words, or kids who can read quite fluently actually, and you should be introducing them to more advanced material. So you divide kids by their learning level, and then the teacher can just focus on that particular thing.

And that has proved extremely effective. So there’s a particular programme that the NGO Pratham developed, which is called Teaching at the Right Level, and it’s a very specific programme for how to do that. It’s a couple of hours a day, and you rearrange people by where they are, not their age. And then there’s other ways of doing it, which might involve computer-assisted learning, where the computer program adjusts to the level of the child; or streaming, where you just take all the kids coming in the first grade, and if you’re teaching in English, let’s divide kids by whether they actually know any English when they come in or whether they don’t. So there’s many different ways of doing teaching linked to the level of learning, some of which is this very specific programme, TaRL, and some of which are other ways of getting at a similar approach.

Luisa Rodriguez: I found this one interesting because it sounded a lot like the kind of tracking that I experienced in my junior and high schools. My high school was about half lower-income Hispanic kids and half higher-income white kids, and it ended up feeling to me like almost exclusively the students at the higher level doing the harder classes were the white kids, and the students at the lower level were the Hispanic kids. Is there any evidence that a programme like this could exacerbate inequalities like that, or does it actually just boost everyone up, and that just seems good?

Rachel Glennerster: Let me just first say that what we’re talking about is quite different from the standard tracking that people in high-income countries are used to thinking about. It may apply to tracking, too, but that hasn’t actually mainly been the thing that’s been studied. But that is exactly what people think about and worry about.

So part of what is different is a lot of these approaches just are for only part of the day. So you’re in your main classroom and you’re doing other things, but part of the day you’ve just rearranged things so that, “For the next hour, we’re going to just focus on reading skills, and we’re going to divide up into our little groups on our reading.” So this is much more like when you’re in primary school and you had different reading tables, and you got the books that were the right level for you. That’s more what’s going on. And actually the kids who are falling behind do really well, so you catch them up right to the rest of the class.

Luisa Rodriguez: As opposed to just letting them fall further and further behind.

Rachel Glennerster: Yeah, otherwise they get further and further behind, and they drop out. There’s evidence that this is actually particularly good for the people who’ve fallen behind. And in most of these programmes, it’s specifically designed to not cause the kind of social divisions that people worry about with tracking. I mean, earlier in the UK, we had grammar schools — and people that were just put into a separate school for the rest of their life, and they were told you’re never going to be academic. If you went to a secondary modern, you’re never going to be introduced to these subjects. So that’s very different from two hours a day, you’re meant to be reading by now, but you don’t know your letters, so let’s just give you really focused attention and get you up to speed. And that’s mainly how these programmes work.

Luisa Rodriguez: So then there was a level below that, which you called “Good Buys.” Were there any there that you were particularly excited about?

Rachel Glennerster: Yeah. I’ve learned a lot in being in this panel about early childhood intervention — pre-primary and also early childhood development programmes.

And just as an aside, this is one of the areas where I think it’s really amazing that we have on this panel people from economics and people from psychology, and what they pointed out to us is that in some of this early childhood work, they have outcomes that we’re not used to in economics. And through their help, we ended up having early childhood programmes become more cost effective, because we understood that they were more cost effective because we’re able to look at a whole wider range of outcomes than just school test scores. I think that those programmes are really exciting, because they can have huge impacts on the long-term trajectory of children.

So some of these studies, some of the early studies, they’ve now been able to follow kids up and they’re earning more as adults, and all sorts of long-term benefits many, many decades after they went through these programmes. But there is still work to do about how to make them work at scale. And again, there is more promising evidence. So there was recently a new evaluation of a programme in Bangladesh where you were able to do these early childhood programmes, and integrate them into an existing healthcare programme where people visited young mothers at relatively low cost, and still managed to get a sufficiently high-quality intervention that it had impact.

So this is an example of what I was talking about before, about how we only recommend things where people have been able to pull them off at quality, at scale. And these early childhood programmes are ones where we’re still learning about that, but we’ve now got sufficient evidence that it is possible. Again, it’s an example of something that’s moved up with more evidence, but I think there’s huge potential there to correct kids falling behind at a very young age. But there’s still work to do to make it low cost and easy to implement, and that’s why it’s not quite in the top category.

Luisa Rodriguez: Not in the Great Buys yet. OK, that’s really exciting. I guess the Great Buys are great because you can just roll them out, but this also seems really great for people to at least consider working on, because there’s this huge upside if you can help make the difference between it not being that scalable and it becoming very scalable.

I’m also curious about the Bad Buys. What’s one bad buy that seemed like it was particularly un-cost effective? I think it might have even been negatively cost effective.

Rachel Glennerster: Most of the Bad Buys are just, again and again, they’ve proved to have no impact. So computers: just introducing computers does not help. There were a few computer-assisted programmes that do work, but just providing computers, there was this great optimism that you give kids laptops and they’ll generate all sorts of innovations and learn more. That just doesn’t happen. In fact, it can sometimes even detract from learning, because it takes time away from other things in the classroom.

In general, just more inputs — just more textbooks, more teachers, more computers: none of these things help. Even just more money that is flexibly given doesn’t help, because what matters is how teachers teach. And none of those things on their own change how teachers teach, and that’s the big thing that we know improves learning. And there was a lot of pressure for us not to come out and say things are Bad Buys, but we really thought that this was important, because a lot of money is spent on, for example, computers — a lot of pressure to put money into these, because they’re very visible, but they really don’t help.

Luisa Rodriguez: And where was the pressure coming from not to talk about these?

Rachel Glennerster: Oh, there’s always some interest group that’s like, “Well, there’s this programme with computers that works,” or, “Do we really know enough so we’re saying that nothing ever works?” It’s like, yes, maybe there’s a risk that there is some good computer programme that gets shut down because of this. But you know what? There’s a bigger risk that people are spending a lot of money on it, and they’re going to waste a lot of money, and there’s lots of good things they could spend that money on.

Luisa Rodriguez: Yes. This one I did find really counterintuitive, especially that just giving money wouldn’t help. I would have guessed that lots of schools are resource constrained and that they could figure out how to use some money. But it just sounds like there are very concrete things that we know help, and money alone doesn’t get you those things.

So it seems plausible that focusing on specific interventions is meaningfully worse than encouraging governments to make systemic reforms to their education systems. Is that something that you worry about?

Rachel Glennerster: It’s certainly the case that there are big-picture systemic issues that cause problems in education. It’s a big deal. So we were talking about the benefits of teaching at the right level, and the fact that curricula is sometimes very overly ambitious. That’s a problem, and it’d be great if people fix that problem. There’s a lot of RCT evidence that basically individual programmes that help you get around that fact, like Teaching at the Right Level, are really beneficial.

And you might think the right thing to do is, let’s go fix the underlying problem. I’m all for that, but it’s hard, and there’s all sorts of political economy reasons why things are designed for the top of the class. So if you can do that, great. If you can get an education system to respond to the needs of low-income children — that’s more flexible, that’s evidence-based — great on you.

But I’m not going to wait for that to do the other things. Because let’s be clear: every single thing that we’re recommending here has been tested in isolation and worked in isolation. So it is not the case, it’s empirically not the case that you need systems reform to do these things. They have been tested and they have proved to work when they were the only thing that was done. Because a lot of these come from RCTs. Not all of them, but a lot of them do. And that’s what RCTs do: they just move one thing, they don’t move everything else.

So systems change is important. Having systems that actually care about low-income kids is really important. But you can’t just magic that, you can’t put in that as a recommendation: “Care about low-income kids.” And there’s a political economy of that. It is true that there’s probably some benefits of doing a bunch of things. There may be benefits of doing some things together. If you had better tech… Like the structured pedagogy is an example of that: they actually tie together a whole bunch of things that move together. But don’t wait until you have it perfect. Get on with doing the things that you can do.

Luisa Rodriguez: I am curious: why is it that low- and middle-income countries are more likely to have this political economic structure that means that low-income kids are basically underserved by their education systems?

Rachel Glennerster: Oh, low-income kids are underserved everywhere. I don’t think it’s just a problem in low- and middle-income countries. In this GEEAP panel, we’re focusing on low- and middle-income countries. We actually have an expert who’s amazing, Susan Dynarski, who’s an expert in high-income countries, and we have her on the panel because she will come in and say, “Actually, nobody’s managed to do that at scale even in a high-income country. So I really think you shouldn’t be advising it if we can’t even do it in the US.” So that’s fantastic. But yeah, I think the political economy problems are there all over.

Luisa Rodriguez: Just everywhere. Just kind of, the children of elite people who make these systems are just going to be better served because that’s…

Rachel Glennerster: I mean, that’s a big problem. Going back to the UK example of where we had grammar schools that were designed for the top 20% of the population, and people still say we ought to go back to grammar schools because they were better. And they’re comparing grammar schools with the average school now, as opposed to what they should be doing, which is most people are in secondary moderns in the non-grammar schools.

And actually, my parents were very involved in the move to push for the ending of grammar schools in the UK, because you had an education system that was entirely designed around the top of the population. And my dad was one of those kids who failed the eleven-plus [exam]. And he ended up being a university professor — he is an academic’s academic. And yet, because he came from a low-income family, he failed the eleven-plus and got pushed into a school that basically said, “You’re going to do factory work. You’re not good enough to do anything academic.” But he ends up a very successful academic. So it’s really bad.

We have all sorts of really bad systems, and this is why it is good to reform systems. But it’s also the case that education systems are often designed for the elite. And as you were the elite, your kid was going to get into the grammar school, and they were going to get more resources, and they were going to do well. And that was great for you. I mean, democracy, I hope, helps democratise education systems — but we know that democracy doesn’t completely address power issues.

What’s still weird for Rachel about living in the US? [02:45:06]

Luisa Rodriguez: Yeah. I would love to ask you more questions about that, but we’ve only got time for one more question. So you grew up in the UK, but you’ve now lived in the US for longer than you ever lived in the UK. Is there anything that you still find weird about living in the US after all this time?

Rachel Glennerster: Yeah, it’s very interesting. As you say, I’ve lived longer in the US, and there’s still things I just can’t understand about the US. It ranges from the deep politics of the fact that I need to get a different driving licence when I move state — and I lived in DC, and you can move across a few streets and you have to completely retake your driving licence; that makes no sense to me — to cinnamon. Like, I’m sorry guys, you should not be putting cinnamon in absolutely everything. I am now, like, allergic to cinnamon. I can’t eat anything with apples in it in America.

Luisa Rodriguez: We do put cinnamon in everything related to apples!

Rachel Glennerster: You can have an apple without having cinnamon. And you can’t taste the damn apple, because there’s so much cinnamon in it. It drives me insane. That and trees. There are too many trees.

Luisa Rodriguez: What? I was with you until “too many trees.”

Rachel Glennerster: I know it’s really shocking to Americans when I say this, but actually, a lot of British people have the same reaction. If you grow up in the UK, hiking in the countryside, you see these amazing vistas and moorlands. You know, hiking on a moorland, these open spaces. Now, it’s totally artificial. It’s all because of the sheep. The UK used to be covered in trees. But for me, countryside is these vast, open vistas. And then you go hiking in America, and all the hiking trails are in forests. You can’t see anything. You can’t see a view. And there are mosquitoes! It’s miserable. So take me back to the moorlands.

Luisa Rodriguez: Well, you’re welcome back. I have grown to love the UK, and we’d love to have you back. My guest today has been Rachel Glennerster. Thank you so much for coming on. This has been a pleasure.

Rachel Glennerster: Thanks.

Luisa’s outro [02:47:52]

Luisa Rodriguez: Hi listeners. If you enjoyed this episode, you might like our 2018 interview with Rachel: #49 – Rachel Glennerster on a year’s worth of education for under a dollar and other ‘best buys’ in development.

All right, The 80,000 Hours Podcast is produced and edited by Keiran Harris.

The audio engineering team is led by Ben Cordell, with mastering and technical editing by Milo McGuire, Simon Monsour, and Dominic Armstrong.

Full transcripts and an extensive collection of links to learn more are available on our site, and put together as always by Katy Moore.

Thanks for joining, talk to you again soon.