Transcript

Hey listeners, this is the 80,000 Hours podcast, the show about the world’s most pressing problems and how you can use your career to solve them.

I’m Rob Wiblin, Director of Research at 80,000 Hours.

If you care about global development, you’ve probably heard of J-PAL. They’ve been pioneers in the movement to conduct experiments to figure out what actually helps people in poverty. You can read about the in the book Poor Economics.

Claire Walsh leads their work to increase the use of evidence in policymaking, and managed to be given that senior role at the age of 29. I wanted to learn more about her work and how she advanced her career so quickly.

As always there’s a blog post with a full transcript, summary and links to articles discussed in the show.

And now I bring you Claire Walsh.

Robert Wiblin: Today, I’m an EA Global San Francisco, speaking with Claire Walsh. Claire is a senior policy manager at the MIT Abdul Latif Jameel Poverty Action Lab, most popularly known as J-PAL. Claire manages J-PAL’s government partnership initiative, environment and energy sector and oversees J-PAL’s policy publications. Claire has an MA from the Fletcher School of Law and Diplomacy at Tufts University where she specialized in development economics and international business relations. Prior to joining J-PAL in 2012, she worked for nonprofit organizations in East Africa working to improve the quality of education and employment opportunities for youth in the region. Thanks for coming on the podcast, Claire.

Claire Walsh: Thanks so much for having me, Rob.

Robert Wiblin: We’ll get to lots of concrete career advice later in the episode, but first off, what is the J-PAL’s government partnership initiative?

Claire Walsh: The government partnership initiative is a multi-million-dollar fund that supports three things. One, policy-relevant randomized control trials and partnership with governments, so governments who have an important policy problem that they want to try to test a solution to will apply in partnership with our researchers and our offices for funds to do that research. Say they have a cash transfer program, and the cash transfers aren’t actually reaching the poor people they’re intended to, and they want to test a solution, and the researchers that want to help them work on this problem. That’s the first thing we fund.

The second thing we fund is that there are a lot of completed randomized control trials out there about the different types of interventions that are effective in reducing poverty in a particular context. Sometimes, those RCTs haven’t been used, so we provide governments with technical assistance, either mostly human resources to help them build a political will and technical knowledge to be able to scale it up.

The third thing we fund has a much broader goal of trying to help governments make evidence-informed decision-making the norm rather than the exception. It funds governments who want to set up evaluation labs or innovation funds or nudge units that have the mandate of systematically testing new programs before rolling them out. To-date, we’ve funded 21 partnerships in 12 countries around the world and in lots of different regions across Southeast Asia, South Asia, Sub-Saharan Africa and Latin America.

Robert Wiblin: Maybe let’s take those in turn. The first fund, what kinds of research projects are you involved with or funding?

Claire Walsh: GPI is completely sector-agnostic and region-agnostics. What we’re really targeting is governments who want to use evidence and decision and a program that’s meant to do social good. It could be, for instance, a program to reduce crime in Bogota in partnership with the security office and the mayor’s office. It could be RCT in partnership with the industrial training council in Egypt that’s trying to look at the impact of government job fairs, but the thing that unites them is that the governments are committed upfront to using the results from the study to inform a particular policy decision, which is really important to us.

Robert Wiblin: The second class of activities is policy advocacy, is that right, for policies that you think are evidence-based?

Claire Walsh: It’s not necessarily advocacy. It’s a government who is aware of a particular randomized control trial, and an intervention that has been shown to work and wants to either pilot in their context and scale it up or just scale it up.

Robert Wiblin: The third class of activities is trained to change the culture in governments around the world, so that they’re more empirical, more evidence-based. How do you go about doing that?

Claire Walsh: Luckily, there’s a growing movement among governments that actually lies outside of effective altruism. It just happens to be going on simultaneously where governments are more and more interested in things like big data and data science, and using evidence from RCTs to inform policy decisions. Often, the way they do that is by creating new institutions within government that have the mandate and the requirement to use evidence. Take the example of MINEDU lab, which is an innovation lab within the Ministry of Education in Peru that does low-cost randomized control trials using administrative data to test interventions that are supposed to improve education quality in the country. Yes.

Robert Wiblin: There’s a whole lot of different policy ideas that I’m sure you’re aware of and that your division might work on testing or on helping people to scale up. Are there any particular ones that stand out as having a particularly large impact on human welfare, relative to the cost?

Claire Walsh: Yes, so it’s probably good at this point to mention that all the views I express on this podcast are my own, and not J-PAL’s. Yes, so I think it won’t be surprising to hear that a lot of the GiveWell recommended top charities and the causes that GiveWell recommends are also the causes that we think are most cost-effective. Obviously, unconditional cash transfers, things that help people take up health services that are proven to have an impact and save lives like vaccinations. We’re really excited about things like incentives for immunizations. We’re also really excited about a lot of education interventions. In developing countries, there’s this huge problem of kids in school not actually learning to the level of their grade. A lot of randomized control trials have shown that an approach that was pioneered by this Indian NGO called Pratham, the teaching at the right level approach, which is really basic remedial education targeted at the level of the child, rather than at the level of curriculum can drastically improve basic numeracy, literacy, math skills. That’s something we’re excited about in education.

Robert Wiblin: J-PAL has a really attractive and clear website where it tries to layout your findings in a way that people can easily digest and actually apply.

Claire Walsh: Well, thanks.

Robert Wiblin: Is it fair to say that quite a lot of your work is perhaps more on the communication side than on research?

Claire Walsh: For me in particular, yes, a lot of what I do is communication. I am distilling these 60-page papers from Econometrica into one-page descriptions of their key take-aways for policy makers.

Robert Wiblin: Interesting. We might put up a few links to some of the most interesting pages on the site that people can go and take a look at. For example, I think you have a great page where you describe a whole lot of different interventions to try to make education better in the developing world. You rate them based on how strong is the evidence that it works at all, and how large is the impact? How much does it cost per student, so people can take a look at that. Are there any other things that you think it would be particularly interesting for people into policy reform to check out?

Claire Walsh: Yes, so two things. Well, three things. The first is one of the goals of our website is to be an evaluation clearing house for randomized control trials done by our entire network of over 140 professors. If you go to povertyactionlab.org/evaluations, you can search our library of over 800 studies. All of them have short, two-page summaries. You can filter it by country, by sector. You can also filter it by which ones have publicly available data, so that you can go do your own analysis with the data if you’re a data wonk. Okay, so that’s the first one.

Second one is our policy lessons page, so if you go to povertyactionlab.org/policy-lessons, that’s the list of some of the areas where we think there’s the strongest evidence that particular interventions are effective, so that’s great for some of the main take-aways from this big body of research. The third thing we have and recently launched is called Research Resources. This is for anyone who is an aspiring researcher themselves like state a code for particular aspects of running randomized control trials, guides on how to write survey questions well, how to code in different programming languages and free and available to the public to download.

Robert Wiblin: What are some of the challenges that you face working with governments in the developing world either to set up trials or encouraging them to scale up things that have been demonstrated to work?

Claire Walsh: I think a lot of people’s assumptions about what it’s like to work with governments are true, not unfounded. In general, evidence is not the top factor that policy-makers consider when making decisions. It might be eighth or ninth on the list. What I’ve come to learn over time is that there’s often good reason for that. They have to work on the issues that their constituents care about. It may not always be the policy that’s backed by evidence that’s a priority for their constituents at that time. It does tend to be slower working with bureaucracies. Governments have a lot of restrictions in how they can hire researchers or work with researchers or partner with outside organizations. It can take longer to develop partnerships, but I think we’ve overall been really surprised that there are particular ministries or champions within ministries or mayors who are really excited about using data and evidence that are eager to partner with NGOs or researchers in order to get things done more effectively.

Robert Wiblin: It sounds like you have really, a very senior role in J-PAL, but you’re only 29, right?

Claire Walsh: Yes, that’s true.

Robert Wiblin: Is it fair to say that your career has gone pretty quickly, maybe more quickly than you thought, or is this just typical?

Claire Walsh: Definitely more quickly than I thought.

Robert Wiblin: Yes. How do you think you advanced up the ladder so fast?

Claire Walsh: Not sure I have the exact answer to your question, but in my experience, looking at my peers who are involved in similar international development policy work, I think we owe a lot to our master’s degree programs. There are a lot of public policy schools that tend to launch people into rapidly rising careers in governments and international policy in research organizations. They’re called the APSIA schools.

Robert Wiblin: Okay, yes, well, we’ll stick up a link to those.

Claire Walsh: Yes.

Robert Wiblin: It’s something we found in general that across the hall, policy sector that people can really end up in positions of serious responsibility late in their 20’s or early in their 30’s. We would just have to be a little bit opportunistic and hope to be in the right place at the right time that they can end up really jumping up a couple of rungs, if they’re lucky enough to get that situation.

Claire Walsh: Yes, I think there’s a lot of truth to that.

Robert Wiblin: Could the work you’re doing unintentionally cause harm, perhaps? Is there anything that keeps you up at night?

Claire Walsh: Oh, I don’t think there’s a huge risk that we’re doing harm, but I could think of a couple different potential pathways. It could be the case that we’re evaluating a program that ends up being ineffective or has an unintended impact and unintended negative effect, but in the sense, that’s why we’re doing the evaluation so we can hopefully catch that at a pilot stage. If it’s found to not work, make a recommendation based on the evidence that the government shouldn’t scale it up. I guess there’s some risk that by partnering with governments on research, we’re taking away time and personnel resources that could be spent on implementing programs or using monitoring data to improve programs, but I think that there’s so much low-hanging fruit in terms of how government services are implemented that the likelihood of doing massive good is much bigger than the small probability of doing harm.

Robert Wiblin: Fair enough. Let’s change track now to talking about J-PAL as a whole. What other work does J-PAL do?

Claire Walsh: J-PAL does three main things: research, capacity building and policy work. In the research vertical, we have hundreds of research managers and research associates based on countries all around the world, working with J-PAL affiliated professors who are all faculty at universities to run randomized control trials of anti-poverty programs in the field. This is always in partnership with an actual organization like an NGO or government or a company, et cetera. They’re actually producing the high quality research. Second thing we do is policies, so this is about 100 staff around the world. We have seven offices in total. We are responsible for making sure the research gets used, so in every sector where we work, agriculture, education, environment and energy, governance, health, et cetera, we synthesize the results from the larger body of RCTs that are already completed and distill into concrete policy lessons for decision makers. We also, in addition to that more broadcasting approach, work directly with particular governments to help them scale up or NGOs or foundations to help them scale up something that they’re interested in scaling up, after seeing in RCT, found it to be effective.

The last thing we do is capacity building, and this is really important because we’re just one small organization, and the number of organizations doing impact evaluations and the number of governments and private companies doing impact evaluations is much larger. We want to help grow it so that this tool can be used to help decision makers all around the world, regardless of whether they get to work with IPA or J-PAL or CEGA, et cetera. We offer training courses both in person and online to help anyone who’s interested learn how to do, learn some skills and impact evaluation themselves.

Robert Wiblin: Does J-PAL have a good sense of what kind of social impacts it’s having, and whether it’s delivering bang-for-buck for the people who fund J-PAL itself?

Claire Walsh: Yes, that’s a great question. We like to track, since we started in 2003, how many people have been reached by programs that were originally evaluated by J-PAL-affiliated professors, and then scaled up by one of our partners, either in NGO or government, et cetera. Since 2003, over 300 million people around the world have been reached as a result of these scale-ups. We’d like to think that, that is a pretty good indicator that this research can change people’s lives. Yes.

Robert Wiblin: Do you know what those programs are? Is there any one of them that’s really taken off in a big way?

Claire Walsh: Oh, of course. Deworming, antimalarial bed nets, free distribution of antimalarial bed nets, some reforms in Indonesia related to ID cards for national social protection programs, curing dispensers for safe water, et cetera.

Robert Wiblin: In the past, you were working in nonprofits. Maybe can you give us a description of the path you’ve taken in your career to end up where you are now?

Claire Walsh: Sure, so in college, I was actually an Anthropology major. I had studied abroad in Uganda, and knew that I wanted to continue to work in East Africa, and so I started working part-time for two small NGOs that have now since grown a lot bigger, Educate! and Africade, working on employment opportunities for youth who face the labor market that’s just dismal in Uganda and Tanzania where they graduate high school with all those great credentials, and find that they’re faced with a lot of really sorry job options, and so trying to find ways that they can start their own businesses or become entrepreneurs. I was excited to work for them for about a year and quickly realized that I didn’t have enough skills to really make it in the international development sector. That’s why I went back to grad school so quickly just after a year of work.

Robert Wiblin: What kinds of skills were you lacking?

Claire Walsh: Oh, gosh.

Robert Wiblin: Too many to name?

Claire Walsh: So many to name, but I’ll never criticize my Liberal Arts education. I owe everything to it, so yes, I don’t knock that either, but I didn’t have one, management skills, accounting, basic business, basic finance, how to fundraise. Then, on the more technical side, I was supposed to be working on monitoring and evaluation, but I had no idea about statistics, econometrics, impact evaluation methods, and so I made it a priority to get those skills in grad school.

Robert Wiblin: Then, what did you do after that?

Claire Walsh: Immediately upon finishing grad school, I applied to J-PAL’s policy wing.

Robert Wiblin: Okay, wow, right. Got helicoptered in to, yes, do some really important work. The two charities that you worked for earlier, do you think they were high-impact?

Claire Walsh: It’s so interesting. I’m so impressed, in particular, by Educate! because they’ve taken their monitoring and evaluation extremely seriously. They have, I think, three ongoing RCTs of their model. They’ve already been able to scale up to hundreds of thousands of kids in Uganda and are piloting, making their curriculum part of their national curriculum in Rwanda, and all the while doing high quality evaluations to inform their model going forward. I’ve been really excited to see what they’ve done since I’ve left. None of it’s due to me, but yes, they’re a really great organization. Check them out.

Robert Wiblin: Excellent. We’ll stick up a link. If you were going to move on with your career after leaving J-PAL, where else could you go to where you might have an even larger impact or just be able to advance your skills to the next level?

Claire Walsh: I would love to work in institutions within government agencies, probably within an implementing agency in the executive branch that’s tasked with doing research about how to make government programs more effective and programs that reach the poor. Organizations that I know that do this are the Office of Evaluation Services, the General Services Administration, the Consumer of Financial Protection Bureau. I know that countries around the world, these are popping up more and more frequently. I know that D.C., Washington D.C. has just opened a lab like this. They’re popping up everywhere. I think why that could potentially be higher impact is that I could be doing research that would potentially affect actual spending decision.

Robert Wiblin: Perhaps the place that’s most similar to J-PAL is the Center for Effective Global Action. I spoke with your friend, [I feel like 00:17:37], just this earlier, this weekend about that. It’s a group at UC Berkeley. How do you feel that J-PAL and CEGA compare?

Claire Walsh: We’re close partners and very much part of the same family.

Robert Wiblin: It’s all very friendly.

Claire Walsh: Yes.

Robert Wiblin: You’re not competing for funding too hard.

Claire Walsh: No. Well, and they do really cool things that we don’t. In particular, they have this great program where they bring developing country researchers to Berkeley for a semester to learn how to do impact evaluation and public policy research, and then, take that back to their countries. That’s huge for building capacity among developing country researchers. They also do a lot more related to innovative measurement like satellites and sensor networks and data science and data visualization. We do complimentary things.

Robert Wiblin: That’s good. Do you know of any particularly promising job opportunities that J-PAL has available at the moment?

Claire Walsh: Yes, people should check it year-round at povertyactionlab.org/careers. We usually have between 80 and 100 positions available on research projects, but also in our headquarters and regional offices. We don’t just post the jobs at J-PAL but all the other similar organizations as well, so IPA, Innovations for Poverty Action for those who don’t know, CEGA, Evidence For Policy Design at Harvard and many other similar organizations as well. We recruit on an annual cycle. Applications are usually due in December. We make a hiring push in the first quarter of every year, but then, we also have jobs that are up year-round that people can apply to on a rolling basis.

Robert Wiblin: What are the differences you’ve found between working in academia versus the nonprofit sector? Do you feel like you’re a much better fit for one than the other?

Claire Walsh: In academia, there’s much more room for skepticism, and I think that I am potentially more suited to that. In nonprofits, I think it was really exciting to be working directly with people, providing people directly with services, and I miss that a lot, but I enjoy that within academia, there’s more room for questioning whether what we think is helping people is or isn’t.

Robert Wiblin: Is the challenge there that if internally, you’re not so optimistic about the value of the information that you’re delivering, it could be demoralizing for staff, or is it that it would discourage donors from supporting you and helping you to scale up the program?

Claire Walsh: I actually think that a lot of NGOs are very introspective and are always seeking to question their assumptions and improve their model. It’s more just me, personally, that I have a really hard time working on something if I don’t know it has an impact.

Robert Wiblin: Okay. Well, speaking of which, one irony that people sometimes point out about doing lots of randomized control trials of interventions or policy changes is that randomized control trials, as an approach, haven’t themselves been tested for a randomized control trial. You can imagine doing this, but it would be incredibly expensive where you took some policy areas, and you did lots of RCTs, and then, you came back to see whether the ones where you’ve done the RCTs were better than another randomly chosen bunch of policy errors where you didn’t, given that we haven’t yet done that. It might be just too impractical to do. How strong do you think is the empirical evidence behind the empiricism in development?

Claire Walsh: That’s a really great question. It’s true that we probably won’t ever get that lab coat approach where we evaluate some things with RCTs and other things not, but I think a couple of our affiliate professors are working on the question of whether evidence is a powerful tool in changing policy decisions, so randomly assigning certain public officials to get access to evidence about an intervention that works and others not, and seeing whether that changes what decisions they make. There are a couple of experiments ongoing, and I think those will be exciting to see the results of, to see whether evidence of actually something that can change policy-makers’ minds.

Robert Wiblin: Yes, that’s a very cool experiment. Do you have a sense, anecdotally, whether it’s the case that, in general, RCTs move funding most of the time or at least some of the time?

Claire Walsh: I can’t make a statement about the entire sector as a whole. I can only see what I’ve observed. What I’ve observed is that particularly for NGOs, when they’ve completed an RCT, and it’s shown that their program is effective, it can completely change their trajectory in terms of fund raising. I’ve also seen a lot of governments be persuaded, not just by data from RCTs, but also survey data that shows whether people like a policy or not, which is often collected during the course of an RCT, but has very useful information for them for making a decision. RCTs have a nice feature in that they don’t just report you one impact number. The surveys ask about a number of things and can actually provide you stats about what your constituents said about this new policy, whether they like it better than the old system. That can be just as persuasive and important as to whether it had a positive impact.

Robert Wiblin: J-PAL’s RCTs would be pretty reliable, but are you aware of sham RCTs that nonprofits or governments run, just to support, not fake trials but trials that aren’t run with very high standard, and that are mostly designed just to support whatever policy they wanted to endorse anyway?

Claire Walsh: The way I think about approaching that question is really case-by-case, and I don’t know how else to do it, but there are a set of criteria that you would look at to determine whether an individual RCT is of good quality or not such as whether they do balance test to determine that the treatment and control groups are statistically equivalent, on average, for the major factors that might be important to the intervention working or not. You might want to check if there were problems related to attrition or spill-overs that might be confounding their analysis and applying these principles and going to the ground truth and just looking at the papers themselves is the only way that I’ve been able to tell good RCTs from bad RCTs. I don’t think I can generalize that there’s no organization out there who’s doing bad ones. I will say that policy-based evidence-making is prolific.

Robert Wiblin: Yes, that’s what I was thinking about. I have worked in government, and when we were trying to evaluate impact evaluations of what we see, I think that you could sense that a lot of them that you would read, they weren’t designed with complete even-handedness in mind. They were designed with a slant towards supporting the policy that the government’s ordered, and look at the preexisting status quo.

Claire Walsh: That happens all the time, yes.

Robert Wiblin: Yes. Policy reform careers are one of the parts that we recommend that basically everyone thinks about pursuing, but there’s some people who are skeptical out there. They say, “How can you ever really tell that you’ve made a difference in a specific case?” Perhaps they have a preconception that it will be quite rare for someone like you or I to actually be able to influence government policy, because there’s so many other factors that are going into these decisions. As you were saying, evidence might be the eighth or ninth thing on the list. Is there anything that you could say to try to convince someone who’s a skeptic about working in think tanks or evidence collection or perhaps even going into elected office themselves?

Claire Walsh: What I didn’t understand before I started doing work with governments is how small the circle of people who control the purse strings is within a specific agency. There is a lot of room, if you can influence that person’s decision about what programs to move forward with and what programs to not move forward with. In that way, I think it’s possible to have a large degree of impact. I think for the people who are skeptical of working in government themselves, they can always consider working for organizations that are advocating for specific policy changes that are backed up by evidence.

Robert Wiblin: I think the kinds of people that I’m thinking of, they might believe that an individual politician can change spending outcomes, so if you can actually become a minister in a particular area, then, as you’re saying, you just have a lot of discretion. Maybe the longer the chain of causation, so the further back you’re working, if you’re just collecting evidence that, at some point, might be used by someone else to advocate to our government to try to convince someone to change the spending, the longer the chain, the more the concerned they become that we might be kidding ourselves about whether what we’re doing is ever actually going to make a difference. Are you mostly just confident that it works based on personal experience where you can see cases where funding decisions have changed? This is very hard to see how that would have happened without some research being done or the right conversations being had at a particular time?

Claire Walsh: A couple different responses. One, there’s worry that evidence is generated and not used is very real. We try to design GPI, the main thing I work on, Government Partnership Initiative, in order to account for that. It prioritizes government who have, upfront, committed to using evidence in a particular policy decision to try to prevent lack of use from coming into play. I think at J-PAL, we think about evidence used going much beyond the original implementing partner on a study. That’s why we’re committed to making all the papers publicly available on the website, putting the data up on the website, putting really short non-technical summaries up on the website, because we think these insights are not just useful for the people who were part of the original evaluation, but they might help us uncover some underlying things about human behavior that are useful for designing better programs anywhere like how people respond to incentives or how to get people to take up services, or how to more effectively deliver services.

Robert Wiblin: It’s interesting. I recently found a paper that looked at what fraction of published papers in economics in different fields where empirical versus theoretical or a combination of the two. Development economics was the most empirical field within economics. The movement that you’re a part of had really taken off. Interestingly, it had always been a majority, if we’re going back decades. I think people sometimes think the development economics isn’t empirical at all. Maybe the studies in the past were less likely to be randomized control trials, less likely to be the very high standard of evidence that they represent. It’s now over 90%, in fact, of papers in development economics are about collecting and analyzing data. They’re looking a little bit skeptical. I’ll stick up a link to that, and I’ll send it to you. You can say what you think.

The policy area you specialize in the most is energy and environment, right?

Claire Walsh: Correct, yes.

Robert Wiblin: I hadn’t really heard of RCTs or trials being conducted in that area so much. What are the issues that are at play there? How do you measure whether policies are working?

Claire Walsh: Yes, I think it’s a common misconception that impact evaluation is harder or somehow less possible of environment and energy policies. I think a bunch of our affiliate professors have proven that, that’s absolutely not the case. They are examining questions related to energy access. For instance, piloting solar micro-grids in communities that don’t have access to electricity and seeing if people are willing to pay for it and whether gaining access to electricity changes their lives in terms of test scores or how they spend their time. We also look at problems related to pollution reduction, both indoor air pollution, which kills between three and four million people per year and outdoor air pollution. We’re working with NGOs who are testing solutions to help protect people against pollution, but also major regulators and doing really large-scale experiments with industrial plants and how to get them to reduce their pollution.

Then, third, we look at questions related to climate change. I think how it relates to poverty is everyone knows that the poor in developing countries are going to be disproportionately harmed by climate change and a lot earlier than the rest of us. Looking at ways to both mitigate carbon emissions but also increase people’s resilience to the changes that are going to come with climate change. For instance, testing a drought-resistant and flood-resistant seed varieties that help small-holder farmers who rely only on rainfall cope with either too much of it or too little of it.

Robert Wiblin: Are there any RCTs that have been done in that area where you’ve got really positive results? You think this is a real winner of a policy that lots of people should be applying.

Claire Walsh: There are a lot of exciting interventions that have been evaluated once and found to be effective, but it might be useful to replicate them in other context or with different implementers or with different designs to see if they’re effective in multiple places. One really exciting paper that came out recently was an evaluation of payments for environmental services program in Uganda that paid land owners to refrain from cutting down trees on their land. It was a really cost-effective program that cut the deforestation rate from between 7 and 10% to between 2 and 5%. That’s a model that’s actually been tried by a lot of governments and a lot of NGOs all around the world. This is, to my knowledge, the first randomized control trial of it, but before going out and recommending it broadly, I’d love to see more evidence on this topic from others.

Robert Wiblin: Yes, I’ll see if we can find that study and stick up a link to it.

Claire Walsh: It’s in science [inaudible 00:30:42] and a couple other.

Robert Wiblin: Fantastic. I think you’re exactly right. We have to make sure that anything we’re going to scale up hugely has been tested more than once. I guess we saw what the worm was that you and probably most of our listeners would be familiar with that there was a high quality trial done in Kenya, taking the other two thousands that showed enormous effects from deworming children. Since then, there’s been other studies, and it probably still looks like it’s a pretty good thing to scale up, especially given how extremely cheap it is. Even if there’s only 50% chance that it’s a valuable, it could still be in a really good bang-for-buck in unexpected value terms. It’s a bit of a risk if you only have one study. Then, you scale something up enormously. Then, it turned a bit later on as, there’s a second study that doesn’t replicate the original result. You could potentially discredit the idea of evidence-based policy as a whole. It might be good, before we leap in, to double check the working.

Claire Walsh: That’s an important point. I think replication is obviously very important. We do a lot of studies of similar interventions in different contexts, but another way to think about that is an idea that a couple of our professors, including Paul Niehaus of GiveDirectly fame and Karthik Muralidharan have put out a paper recently. I think it’s called Experimenting At Scale where another way you could think about the results from one study being enough to scale up a policy is if we evaluate programs at scale in a sample that’s representative of the population that it could be scaled up to. It’s not always the case that you need to-

Robert Wiblin: Do it a second time. Sometimes, it could just be that the original RCT is done in such a way that it really is representative of what it would look like at scale?

Claire Walsh: Yes. I don’t know if it’s doing it the second time is no longer value. It’s more the latter that if you purposefully do represent the sample of the population that you’re thinking about scaling up to, the results might be much more likely to generalize.

Robert Wiblin: Yes, I guess one of the ongoing problems is that it’s a lot easier to offer a service in a really high quality way when you’re only delivering it to a thousand people. The trial of it is when you’re delivering it tens of millions across the entirety of India or something like that.

Claire Walsh: Yes. These nationally represented samples are often associated with more expensive study.

Robert Wiblin: How can you tell if this kind of work is a good fit for you?

Claire Walsh: You’re not afraid to go and live in a lower middle-income country and work for less pay than you would get in the private sector and just start doing research in the field, which involves long hours, writing complicated surveys and complex operations management, learning coding in lots of different languages. If that sounds exciting to you, I think you’re right for this type of work. Then, also, working with governments obviously requires a good deal of optimism, patience and communication skills, charisma, yes.

Robert Wiblin: Yes, yes, interesting. What’s the biggest downside of working in policy?

Claire Walsh: Personally, I really miss working in direct services. I would love to just be handing cash to really, really poor people.

Robert Wiblin: To be able to see the benefits that’s been provided directly?

Claire Walsh: Yes.

Robert Wiblin: Yes, okay, so it’s a bit remote from the actual impact that you’re having. You have to have some faith that someone’s being helped, but you don’t see them directly.

Claire Walsh: Yes.

Robert Wiblin: Yes, exactly. Where do you think, personally, that development economists can do the most good? Is there any particular pressing problem within the field where you think economic skills can have a really big impact?

Claire Walsh: I’ve been encouraged by a lot of the affiliate professors in the J-PAL network who are committed not only to answering questions that are relevant for economics as a discipline, but also important to governments making real decisions. If you want to read a good primer on that topic, Esther Duflo’s decent paper called the Economist as Plumber gets into these questions about how economists can help policy makers design programs and policies more effectively.

Robert Wiblin: Okay, well, we’ll stick up the link to that. I’ll be interested to read it.

Claire Walsh: Yes.

Robert Wiblin: What’s so good about plumbers?

Claire Walsh: The analogy of plumbers, they are always tinkering with piping and trying to figure out where the bottlenecks are and how the system as a whole could work better. I guess the idea is that economists, similarly, have really good insights about how to relieve bottlenecks in large systems and complex policies and programs that governments implement. This is an area where economists could add a lot of value and should engage in more, figuring out the nitty-gritty details of important big government programs because those details actually matter a lot more than many programs might assume.

Robert Wiblin: What is it that drew you to work on global poverty? Did you consider working on other problems in the world instead of poverty?

Claire Walsh: Not really. I studied abroad in Uganda when I was 19, and I saw how poverty completely limits people in terms of freedom and opportunity and the relatively small and cheap things that were missing in their lives that were present in my life that could help make their lives a lot better, including access to just primary preventative healthcare or primary acute care.

Robert Wiblin: There are definitely some people who have a romantic vision for poverty that people can be poor but still happy, and money is not everything. Do you think there’s some truth to that, or are they basically just … Have they just not been to countries where people are extremely poor?

Claire Walsh: I would never romanticize the suffering that comes with not having more than a dollar or two or three to live on a day. That being said, the type of stereotype about the poor that I try to fight against as much as possible is that they don’t have the same capabilities, or they’re bad at making decisions. I think actually randomized control trials have proven that they make very complex decisions related to savings or financial management. They might even be better-placed to make those choices than the rich because they actually are forced to make tough choices. That’s the stereotype about the poor that I think is most dangerous.

Robert Wiblin: They’re large in a day-to-day basis have quite a lot of unpleasantness in them, and that’s what motivates you to try to solve the problem?

Claire Walsh: Yes.

Robert Wiblin: Yes, so let’s move on now to try to think about a concrete advice that we might be able to learn from your career, or concrete opportunities that people listening might be able to take if they’re choosing what major to study or what to do at the post-graduate level or what jobs they might get, having done a masters degree. Where do you think young people should start if they want to work in global poverty in a similar way to you? What majors would you recommend that they do?

Claire Walsh: Economics, political science, international development.

Robert Wiblin: Snappy answer. Are there rather options for passing global poverty that you think are promising beyond the traditional ones like possibly doing entrepreneurships, starting a business in the developing world or just going and trying to work there in government just directly?

Claire Walsh: Definitely. You don’t have to go the research route. You don’t have to go the international aid route. I think, and I’m very pro entrepreneurship, I think sometimes, there’s a risk of, “I have the one silver bullet solution that’s going to solve anything.” It doesn’t exist. Therefore, I should create an organization around it. I would caution a little bit against that. There’s probably already someone who’s working on the problem. It might be more effective if you wanted to go work for their organization.

Robert Wiblin: You go and live in Indonesia for a couple of months every year, is that right?

Claire Walsh: That’s correct.

Robert Wiblin: What’s the reason for that?

Claire Walsh: We have J-PAL Southeast Asia is our regional office in the region, and it’s based at the University of Indonesia in Jakarta. I go there about two months every year, working on policy outreach, trying to get their research that’s been done by the office, taken up by government, also helping build and grow the office. We started in 2013 with two people, and now, it’s an office of 22 people. Also, starting work on a couple research projects there as well.

Robert Wiblin: Do you think it’s important to live in developing countries where you might be helping with policy change? This is something that people should consider doing in possibly their late teens or early 20’s when they have their opportunity to?

Claire Walsh: Definitely, and I regret not doing it more. I think if you want to work in international development, having at least two years of experience living in a developing country is a huge plus and can open a lot of doors.

Robert Wiblin: What is it specifically that you gain from doing that?

Claire Walsh: It’s so many different things you gain. It’s a deeper understanding of human difference, of cultural and ethnic diversity, of being able to live with less [preacher 00:39:10] comforts than you’re used to, being able to understand how to get things done in resource-poor environments, being able to put your personal comfort behind a greater cause and mission. Then, it also gains you a lot of credibility with a lot of major players and working in development to see that you’ve lived and worked and thrived in lower middle-income countries.

Robert Wiblin: Let’s say that you’re doing or you have completed a relevant undergraduate degree. How can you go about building a professional network in global poverty reduction? How can you experiment with different options that you might have in that whole field when you’re in your mid-20’s?

Claire Walsh: It’s not the prettiest answer. A lot of people, right out of undergraduate, first take an internship that’s often unpaid and often in a developing country. Then, they convert it later into a paid position. That’s what I did. That worries me because I think internship should be paid, but it is what a lot of people do. I think what really helped me build my professional network in international development because I wasn’t lucky enough to get into an important institution right out of undergrad. I was working for a very small niche in geos, was going back to get my masters. That completely exponentially grew my professional network and gave me connections to all the major organizations that I would hope to work for.

Robert Wiblin: We’ll talk about the masters in just a minute, but are there any conferences that you went to, to meet people before you’d done that? What are the big events that you go to now, where if people who wanted to meet you might be able to find you in the corridor?

Claire Walsh: I’m in economics, and so you’ll usually find me at the AEA conference every year, but also the other AEA, the American Evaluation Association where usually that as well.

Robert Wiblin: Those are the two that you go to mostly?

Claire Walsh: Yes, but those are not for development more broadly. The first is for economists, and the second is for evaluators, which includes economists but also includes a lot of qualitative researchers and anthropologists, sociologists, et cetera.

Robert Wiblin: Well, I guess if you had a conference that was just focused on Poverty Action as a whole, you could end up with a million people there. It’s quite a live sector, right?

Claire Walsh: Yes, and what happens is that there are just a million conferences a year.

Robert Wiblin: Yes, right, right. It sounds like perhaps the most important thing that we can learn from your career is the value of doing one of these excellent masters programs. Obviously, I’ve studied what you studied, but are there any other programs that people could try to get into that could really help to advance their career?

Claire Walsh: Yes, if you’re interested in public policy and international development particularly, there is a group of schools called the APSIA Schools. I don’t remember what the acronym stands for, but it’s a group of about 25 masters programs at universities around the US, and I think maybe a couple in Canada that have high quality masters degrees in this fields. They include Princeton’s Woodrow Wilson School, Harvard’s Kennedy School, Johns Hopkins’ School of Advanced International Studies, George Washington, Georgetown, American, Denver University, a lot of universities in California, and so it’s a lot of great public policy schools out there.

Robert Wiblin: That’s a very useful list. How do you think doing a masters compares to doing a PhD? Have you considered doing a PhD yourself?

Claire Walsh: Definitely considered doing a PhD myself. Doing a PhD in order to make policy change is not one of the top things that most people going to a PhD are trying to do. I’m really proud of the ones who are, and I would encourage more and more to think about it, but going into a PhD is really great for becoming an academic or a full-time researcher, but I knew I wanted to work in policy action rather than full-time research.

Robert Wiblin: That makes sense. I guess most of the people, the principal investigators all have PhDs at J-PAL, right? If you want to go into hardcore social science research, then, the PhD is necessary?

Claire Walsh: That’s correct, yes. If you want to be a principal investigator on an impact evaluation that you hope might someday be published in an academic journal, you need a PhD.

Robert Wiblin: Yes. It would be useful to list some of the organizations that people might actually be able to apply to work for, other than J-PAL and CEGA that we’ve talked about. First, people consider global poverty as a problem that quite a lot of people want to work on. It can be quite competitive, as you said early on in your career. What kinds of places can people apply to when they’re fairly at the earlier stages of their career where they might actually have a decent shot at getting a job or at least an internship that will let them get their foot in the door?

Claire Walsh: If you want to work in impact evaluation, there are lots of other organizations besides J-PAL and CEGA. There’s Innovations for Poverty Action, Evidence for Policy Design at Harvard. There are a couple private firms who do very similar work like Ideas42 and Idinsight that I think they’re areas where jobs have been growing over the past five years. I don’t see that growth slowing down yet, so I think there’s a reasonable chance of getting positions. There certainly are a lot of openings.

Robert Wiblin: What about later on in your career once you’ve built up some experience? What kinds of places can you go to where you might be able to do more good, having really, really advanced your skills?

Claire Walsh: We see a lot of people moving from positions from J-PAL into foundations where they can actually influence how nondominant resources are allocated, which his really exciting. Some people go on to be researchers who are focused on policy relevant research. I’m most excited about the potential to move into government research positions where you can both do research and then actually make a funding decision based on it.

Robert Wiblin: Are those governments in the developed world, or the developing world, or both?

Claire Walsh: They exist in both. I obviously will probably only be qualified to do them in the States, but for instance, the Office of Evaluation Services and the General Services Administration, the Consumer Financial Protection Bureau, the D.C. Lab, there all these city and state governments who have their own research wings that are just for the purpose of doing research to figure out how to better run programs for citizens. I think that’s a place where I hope to have policy impact in the future.

Robert Wiblin: Interesting. Here’s one a little bit out of my field. Most of our readers are people with at least an undergraduate degree in the US or the UK, but we do have a decent number of readers in Africa. Actually, maybe like 5% of our web traffic is from people living in India, people who are brought up in India. Do you have any ideas for what can you do if you were born in the developing world? How can you most effectively try to reform policy in your country, or do other things to help people in poverty?

Claire Walsh: So many things. For one, all of our regional offices are fully staffed by people from the countries where they’re based. Most of our South Asia office, which is based in Deli, and another office in Chennai is fully staffed by people from India. There’s tons of job opportunities there. If people are interested, they should check out our website. I mean part, the whole idea behind GPA is try to help government institutions in developing countries grow and become stronger. One of the best ways to do that is if really smart people from these countries go into government and try to make institutions better from within. I would encourage people to think about government as an option for them, especially in middle income countries that spend huge amounts of money on primary healthcare or cash transfers for the poor like India or Indonesia, et cetera.

Additionally, if government is not the right route for you, I think ultimately, the citizens of low and middle income countries will be in a position to hold their governments accountable. Any activism related to requiring the government to provide better services, I mean like one organization that does a really good job at that is called FAWEZA that’s based in East Africa. That’s all about helping citizens hold their governments more accountable to deliver better services. That’s an option if you don’t want to go into government yourself.

Robert Wiblin: I guess there’s a bunch of roles like that where actually, being a local person who was brought up in my country, puts you in a better position than a foreigner who’s coming in and perhaps doesn’t understand the culture or how to get things done, as well, and might not have as much traction if they’re trying to run a campaign because they’re seen as an outsider.

Claire Walsh: Completely. I mean the whole thing about GPI is we’re trying to work ourselves out of a job. Developing country governments can do all the same things that rich country governments can do. Soon, this will all be irrelevant, I hope.

Robert Wiblin: A question I regularly ask people I’m interviewing is whether you think you would do more good if instead of trying to do good through policy reform, instead you’re going out and try to make a lot of money, and then donated your earnings to help poverty, either doing cash transfers perhaps or providing medical services or perhaps funding a think tank or an organization like J-PAL. Is that something that you have ever thought about or have a view on?

Claire Walsh: I’ve definitely thought about it. I think in some ways, I think it’s quite likely that I could have more impact just giving all my money away to cash transfers for the poor. Sometimes, I wish I was just doing my daydreaming about that, but I have a broader goal in mind, which is helping improve institutions and having an optimism despite corruption, despite the hurdles of bureaucratic processes that governments in developing countries have the capacity and the capability to run high-quality programs that reduce poverty, and that in some small way, by helping them test their latest innovations before scaling them or scaling up things that RCTs have shown to be effective in the past, that I’m also contributing in some very, very small way to those institutions being more effective to deliver quality services to their citizen.

Robert Wiblin: Yes. It sounds like J-PAL has quite a lot of traction with developing our governments, and their budgets often very large. It would surprise me if you could have more impact by just donating the money. Maybe you have potential to earn more money than the one I appreciate, but I expect that the changes in budgets you’re getting through J-PAL are probably doing more good than that.

Claire Walsh: I would just have to get out my spreadsheet.

Robert Wiblin: Do you feel like your career track has been risky at all? I mean initially, you’re working at a nonprofit in the developing world. Did you ever feel like you weren’t sure what your next position was going to be, or whether you’d be able to transition into wider problems if you decided that you didn’t want to work on global poverty anymore?

Claire Walsh: I will say that early on in my career, I was very worried that by working for small NGOs, that I couldn’t transition into working for big NGOs, foundations or policy organizations or research organizations. I was quite pleased that, that wasn’t the case at all. Obviously, the masters program helped with that a lot, it was possible to go from a smaller NGO to a bigger organization like J-PAL. I do think that it is harder for people who go into public policy to then go into the private sector. That’s something lot of my peers think about and struggle with a lot. There is definitely organizations’ value skills from the private sector and from the business world that I feel like it’s usually easier for people to switch from a private sector to the policy space or the public sector, rather than the other way around.

Robert Wiblin: Yes, so perhaps early on in your career, if you want to be flexible, you might want to go into the private sector first?

Claire Walsh: I’ll just say that it’s easier to transition that way than the other way.

Robert Wiblin: I suppose that the skill-

Claire Walsh: You could just do what I did and not go into the private sector at all, and you can still be quite successful.

Robert Wiblin: Yes. Well, it seems like your career has jumped forward in leaps and bounds over the last five years. I really look forward to seeing where you end up in the next five years. Maybe you’ll be, I don’t know, running the Finance Ministry in India.

Claire Walsh: Hopefully not India.

Robert Wiblin: You’re not a fan of living in India?

Claire Walsh: No. It’s just that Indians can run their own Finance Industries.

Robert Wiblin: You’re right. That’s a pretty vague one. Maybe you’ll be running J-PAL instead.

Claire Walsh: No.

Robert Wiblin: My guest today has been Claire Walsh. Thanks so much for coming on the 80,000 Hours podcast, Claire.

Claire Walsh: Thanks so much, Rob. It’s been a pleasure.

Thanks for joining. After recording the show Claire realised she spent a lot of time talking about full time masters, which are are not an option for a lot of people. She wanted me to point out that there’s a new low-cost option for anyone who’s interested in studying development economics and impact evaluation: the J-PAL and the MIT Economics Department’s new Micromasters and Blended Master’s programs in “Data, Economics, and Public Policy”. It’s designed to give people anywhere in the world a high-quality, low-cost, flexible online Master’s program, even for people who can’t go to school full-time. In the program students take 5 online courses, and if they pass a proctored exam for each, they earn a Micromasters from MITx and are eligible to apply for an accelerated Masters on-campus at MIT. I’ll put up a link to that in the blog post.

Thanks for joining, talk to you next week.