The problem of undervaccinated kids
Varsha Venugopal: We have 19 million children who fail to receive their basic vaccinations by the time of their first birthdays each year. We know as a result, one child dies every minute because of vaccine-preventable disease.
Rob Wiblin: So at their first birthday they haven’t received any or most of the vaccinations, or is it just that 19 million haven’t done the full schedule that’s recommended?
Varsha Venugopal: So 19 million children are undervaccinated, and that’s a really good question because there is something called the zero-dose, which means they haven’t received any vaccinations. But with these 19 million children, what we are specifically referring to is that they are undervaccinated, which means they haven’t completed their full dose by the time they turn one.
Rob Wiblin: So you’re saying a child dies every minute, so that’s where you get kind of the 1.7 or 1.8 million people dying a year. Are there harms other than death?
Varsha Venugopal: Yeah, absolutely. So we know we are talking about vaccines saving two to three million child deaths every year, we know there are huge social and economic benefits at both individual and societal levels. So we know that there are costs related to medical treatments and also related to income and productivity losses, as well as, I mean, one can imagine the whole suffering of family and friends. There is also some evidence of association between vaccination and childhood development and educational outcomes. I think broadly there is a fair degree of consensus between experts that for every $1 invested in expanding immunization, there are about $60 returns in social, economic, and environmental benefits.
Rob Wiblin: What are the main countries where kids aren’t getting vaccinated?
Varsha Venugopal: So the big countries that are there are Afghanistan, Pakistan, DRC, Ethiopia, Indonesia, Nigeria, and of course India. India, which of course we’ll come to later, has about half of these undervaccinated children, so of the 19 million, we have about 10 million of them in India.
Evidence on SMS reminders
Rob Wiblin: I saw on your website that you have this kind of table summarizing the evidence on SMS reminders, and I took a quick look over and it seemed like there were five trials that had kind of positive effects and then five where it didn’t really seem like the SMS reminders had moved the needle very much. Did you go away and kind of, either yourself or get someone else to do this kind of literature review to learn other lessons about under what circumstances these things work, and what are things you can vary about the intervention to make them have a bigger impact?
Varsha Venugopal: Mmm, yeah, no, that’s a great question. So our colleagues at Charity Science Health did undertake this meta-evaluation in 2019 with the nine existing randomized trials, and they found a 7.4 percentage point increase in full immunization due to SMS reminders. So then to be conservative, they also ran the meta-analysis after subjectively making various substantial discounts in the reported effect sizes, and this reduced estimated effect size is around 3.2 percentage points. So our sense is the range of effect size is somewhere between 3.2 and 7.4, but in all these cases, the program is still cost effective.
Varsha Venugopal: I think one point we are aware of is we were quite clear on focusing on SMS versus, say, WhatsApp because based on our research, we could see that the smartphone penetration was still not as high as regular phones. So we were quite keen focusing on text messages, but we are also quite aware of the high correlation between mothers who haven’t got high literacy and the kids dropping off vaccinations. So the next iteration we are quite keen to introduce is voice recordings to see how they could further reinforce the messaging that’s going through the text messages.
Rob Wiblin: Yeah. Yeah, I was really happy to see the table both had studies with positive results and studies with null results where it couldn’t find an effect. Because an intervention, even one that’s effective, you should find some studies that by chance have found that it didn’t work in this particular case or perhaps were underpowered, there weren’t enough people in the sample to detect even a real effect. And if people aren’t aware of that, when they’re setting out, it suggests that they haven’t really been quite comprehensive enough in looking at all of the evidence out there to get a proper average kind of effect size across the full range of results, rather than just cherry-picking a few of them.
Varsha Venugopal: Yeah, no, we absolutely believe in making sure we look at the broad gamut of evidence that’s available. This is also the additional reason why we are quite keen on introducing the ambassador program along with the SMS reminders, because this seems like the effect sizes with the ambassadors is higher and we want to make sure we can capture some of that impact on the table.
Varsha Venugopal: In the original J-PAL study, the surveyors were sent to a random set of 17 households. And they asked several questions, but the main question is, “If there was a fair in town, who is most likely to tell you about it?” We then run an algorithm to identify the top gossips, or community influencers, and the surveyors then go back to these top influencers and recruit them as immunization ambassadors.
Varsha Venugopal: What we did as a result of COVID was pivot to a remote model, which already brings down the cost significantly, in that we call up a random set of households to identify these influencers and then call these influencers to recruit them as ambassadors. So compared to the SMS model, this model at the moment is still not as mature, and we’re still iterating to find the most cost-effective way of doing this.
Varsha Venugopal: Broadly [we tell influencers], “You have been identified as a community influencer, and we would like you to be an immunization ambassador. There is no monetary cost involved in this. It is a voluntary exercise. We would be sending you regular reminders on immunization camps. Are you willing to be an ambassador?”
Varsha Venugopal: So in more than 90% of the cases, when people were contacted randomly, they were able to give us not only the name of an influencer, but also their phone number. And then in more than 95% of the cases, the influencers agreed to be ambassadors. The challenge we had was reaching these influencers. Possibly because they are influencers, they’re not around most of the times. It’s unclear. That’s just a supposition. But that’s the bit we are now iterating, to try and call at different times of the day or different days of the week to get to them.
Varsha Venugopal: The other bit was, are these ambassadors then sharing the information? So one thing we did early on was give them a phone number that they could then share with parents and ask the parents to give a missed call to enroll into the SMS reminders program. And I think we had more than 30 people sign up in the first week, which again, gives us some confidence that the ambassadors are sharing this information and people are receiving it and somehow responding to it.
Varsha Venugopal: In the J-PAL study, they looked at both trusted people as well as community influencers and see the interaction effects, and it turns out it is the community influencer bit which seems to be causing the biggest impact. Other than the fact that we know about 18% of our influencers are women, which is kind of similar to the J-PAL study, we are very curious to better understand the motivations of the ambassadors and why they take it on. And that’s the kind of thing you don’t get in an RCT, so that’s something we want to go there and interrogate further.
Rob Wiblin: Yeah. Okay. So just to have a picture in the head of the magnitude of the benefit, you’re saying it increases vaccination rates by 10 percentage points and that’s about 27%, so huge magnitude. How large is the SMS reminder effect again, separately?
Varsha Venugopal: Somewhere between three and seven.
Why parents aren't already prioritizing vaccinations
Varsha Venugopal: So, to put this in context for India, we have about 26 million children born every year and we have 16 million children that are completing the vaccination schedule, so we are talking about the remaining 10 million.
Varsha Venugopal: Now of these 10 million, we have two million who do not touch the hospital system at all, the ones that are called the zero-dose kids. We have one million that get the first dose — that’s BCG and tuberculosis, Hep B and OPV, that’s polio — but then disappear. So just the birth dose. And then we have these seven million kids who are getting the birth dose and at least one other vaccination and are dropping off. And that’s our primary end user we have in mind, though we do think we can probably make some dent with the other three million kids as well.
Varsha Venugopal: So one of the big reasons we think that they drop off, and there has been some research, is to do with time inconsistency argument. I think it’s something referred to by Banerjee and Duflo in their book Poor Economics as well. This whole idea that I value my present very differently from the way I value my future, right? So even when I’m making decisions on exercising or gym, it’s all something I’d rather postpone for all these other myriad intrusions on my time in the present. And I think some of that is what’s at play for these parents as well.
Varsha Venugopal: So caregivers may miss their appointments for various reasons. They could just be forgetting about them. They may not have the right information to accurately understand the benefits. We do know sometimes they don’t know how many appointments they need to come for, or they just don’t want to take a day off work. They don’t want to deal with a crying child. They don’t want to take the bus to go somewhere for vaccinations. So all reasons which possibly by small nudges in the margins could be addressed.
Rob Wiblin: So their best guess is, it’s just the basic thing that we can all relate to of, you put something off and you put something off because it’s kind of a pain in the ass to do, and no particular day feels like the day that you want to go and do the vaccination when it requires crossing town, or dealing with your baby not wanting to get an injection.
Varsha Venugopal: Yes, and this is exactly where it’s really important to reinforce that it’s very different from COVID vaccine hesitancy or other kinds of vaccine hesitancy that may exist somewhere, which is possibly far more complicated and involves issues of trust. I mean, here we know, because they’ve come in for the birth dose and at least one other dose, that they broadly trust the vaccines and believe it’s a public good, but for a myriad of reasons are then dropping off because of just being overwhelmed with their daily lives.
Varsha Venugopal: We want to solve this problem of underimmunization and drop-off rate in India. So we have six priority states which have more than 70% of the underimmunized kids, and we have a few plans on how to get there. I think the default idea is that we would be running certain call centers and be able to reach at least across those six states and have a footprint.
Varsha Venugopal: But having said that, I think there may be some other endgame options we want to actively explore over the next few months. One is we advise the governments themselves and they take on some of this work, or we partner with other institutions working closely with the government, like WHO or UNICEF, who are already based in state government offices.
Varsha Venugopal: Another idea is potentially identifying other nonprofits within local contexts who understand the local area and can possibly quickly hire people or send out reminders, and we train them. I think our focus is very much on developing an operations manual over the next few months to then explore some quick exponential wins we can have in terms of scaling up through other ways. In addition to, of course, replicating and scaling up ourselves.
Rob Wiblin: Yeah. So what’s the biggest impediment or the biggest challenge that you foresee getting up to that much larger scale that you want to achieve?
Varsha Venugopal: So I think the first is the fundamental challenge with designing and implementing a program which has proven evidence at a much smaller scale. So any general challenge of scaling up and ensuring the effect sizes remain the same. How do we make sure there isn’t a reduction in the fidelity of the design? Can we have the same degree of oversight of these surveyors, is the big one.
Varsha Venugopal: I think the other big one for us is walking the line of innovating versus scaling. Something I mentioned earlier, there are certain funders who are quite keen on us growing in certain ways, which may or may not perfectly align with our strategic priorities. And especially at this stage for us, should we be reaching one additional district or a few more additional districts versus making some of those investments which may not see that direct impact on the ground in terms of additional lives saved, but may then build up to that exponential increase at a future date. So making some of those decisions is, I guess, high leverage but also high stress.
The importance of co-founders
Rob Wiblin: How essential do you think it is to have a co-founder if you are starting a project like this? To me intuitively, I imagine doing something like this on my own, and I’m like, “That would never work because I would get demoralized and then there’d be no one to pick me up when I was feeling down or facing challenges.” Do you have the same sense?
Varsha Venugopal: Yeah. I think for me, given where I was with my life trajectory and knowing I couldn’t possibly move to India, it made so much sense to have a co-founder who could actually be based in India. But even beyond that, the advice we got from CE and others is that having a co-founder is immensely useful in complementing some of the strengths and weaknesses and limitations you may have.
Varsha Venugopal: And from that perspective, it’s definitely been hugely useful to have someone who is just as stressed about carrying through with your day’s work. And yeah, just in terms of entrepreneurship being a lonely journey, having somebody whom you can constantly throw ideas back and forth and debate on how to take some of our decisions forward is immensely useful.
Rob Wiblin: It’s interesting that you’ve done this co-founder split where Fiona’s doing most of the management and I guess you’re on fundraising and strategy and planning and that kind of thing?
Varsha Venugopal: Yeah. It was broadly from this idea that she would be based in India and most of our team would be based there, so it would be easier for her to take on that role. And it’s worked out mostly fine, so I think we would be continuing with that model. It would be much harder for me sitting here to be managing a team on the ground. There is also this element of people being mostly in charge of their own selves. So the management is fairly light touch and more in terms of, “How can we support you to do your work better?”
Rob Wiblin: Yeah. That makes sense. I suppose the relationship between co-founders is so incredibly important for an organization’s success. Do you and Fiona spend time socially together to ensure that any potential frictions kind of get smoothed over?
Varsha Venugopal: Yes. Before our first trip to India, we spent quite a lot of time together, but even otherwise we are on call and chat pretty much all the time. Just again, I think just the nature of startup in early stage and lots of moving bits and pieces… I think we pretty much clicked fairly early and continue to be clicked. Yeah, I think we bring very different perspectives and skills and somehow are complementary and seem to agree on most things, and where we disagree we manage to work through in a fairly nonviolent way.
Rob Wiblin: Did you do a lot of screening to figure out whether you were a good match before you started the project? Or was it kind of, “Well, we’ll try getting the project going and that will test the relationship”?
Varsha Venugopal: I think CE has gotten pretty good at this. So they did some kind of… I think they even put us in some cat and dog category. They never told us which one of us was a cat or a dog, but I think they did a lot of behind-the-scenes calculations, I guess, based on several exercises we did with different people within the incubation. So I think there was some of just trusting their judgment. But also I think it is such a fundamentally important decision to get right. That it’s okay to, I guess, try out early on and see if it works or not, like probably with a lot of these decisions. And if not, being, I guess, brave to —
Rob Wiblin: Go find something else?
Varsha Venugopal: — to walk away. Yeah.