Interview with Caroline Fiennes about opportunities in effective philanthropy

Introduction

We recently interviewed Caroline Fiennes to find out about her ideas on opportunities to make a difference promoting effective philanthropy, and more about her organisation, Giving Evidence. The aim was to both inform our strategy as an organisation, and find opportunities for people who are interested in leading a career in this area.

The interview was conducted via phone call. Below we summarise the key messages of the conversation, followed by some key excerpts, which have been edited and reorganised for clarity.

In summary, Caroline told us:

• Billions of pounds are donated to charity in the UK each year, but there’s little evidence which can inform donors’ decisions about where to donate. Hence, this money probably doesn’t have as much impact as it could.
• One intervention would be to set up something like Charity Navigator for the UK, ideally rating charities both on organisation quality (as Charity Navigator does) and on the strength of the evidence behind the interventions they implement. There are many people interested in taking this project forward, but it’s difficult to raise money for it.
• Another intervention is creating a platform to publicly collect and share the monitoring and evaluation data that charities already produce. Over a billion pounds is spent on monitoring and evaluation each year, but it seems that only about two percent of the studies get shared. Giving Evidence recently raised funding to explore how to create a system for sharing evidence in the UK criminal justice sector.

The interview

Caroline Fiennes founded and directs Giving Evidence, which works to promote evidence-based charitable giving. She is on boards of the US Center for Effective Philanthropy, Charity Navigator, one of the world’s largest charity rating agencies, Evidence Aid (part of The Cochrane Collaboration), and the US-based Center for Global Development. She is the Corporation of London’s City Philanthropy Coach, and she has written an acclaimed book about evidence-based giving, “It Ain’t What You Give, It’s The Way That You Give It.”

What does Giving Evidence do?

You can see a two minute introduction to the organisation in this video. Find out more on their website.

Why is promoting effective philanthropy a promising cause?

Some charities are harmful in their work, some achieve nothing at all and some achieve a positive benefit. So wouldn’t it be a good idea if we figured out which ones are good, and directed resources towards them? Just as we’ve seen massive gains in medicine, from making decisions based on evidence, we need to do the same in charity.

Today, if you want to do your giving based on sound evidence, good luck. There’s so little evidence around and often there’s very little evidence about what the need is and where the need is. Money ends up going to the wrong place. In global health, for example, about 90% of the spend goes on 10% of the disease burden.

In most cases we don’t have data about where the money is. We also have frighteningly little data about the effectiveness of different interventions. On the radio this morning the guy who runs the NHS was calling for smaller hospitals rather than bigger… yet had hardly any evidence that that improves outcomes.

Many charities do not clearly articulate publicly what they actually do. Charity Navigator has a new algorithm for assessing charities which involves looking at an organisation and asking whether an intelligent outsider, in two minutes, can figure out what this organisation actually does. For loads of them, the answer is no. The next question is whether they publish anything about what their results are. Again, for loads of them, no. There’s a huge amount of evidence we need to get.

So if we had more evidence, we’d be able to make better decisions about where to give, and given the amount of money donated to charity each year, that could do a huge amount of good. This suggests (using our cause framework) it’s important to promote effective philanthropy, but is it tractable? i.e. why should we expect to be able to make progress promoting it?

A key question here is whether donors actually change their behaviour if they’re given them better evidence. Unfortunately, we don’t even know the answer to that! To my knowledge, until about a month ago, there were no studies at all attempting to answer the question.

Recently, Dean Karlan at Yale did one trial. The effect was nil on average, though larger donors gave more in response to better evidence. Still, it was one study at one time in one place etc. Before Dean’s study came out, the main evidence we had was the donations rise in GiveWell.

There are some other academics who are interested in the effect of evidence on donor behaviour. It seems to be quite an urgent task. There’s a lot of research about how get people to give more, but no one has really looked at how you get them to give better.

We also know from other fields it is possible to change people’s behaviour by providing evidence, but we also know it takes ages. I recently heard the former CEO of the NHS say that he time between a medical discovery and a corresponding change in doctor behavior is 15 years.

Could you get more traction promoting evidence-based practice to foundations or the charities themselves rather than individuals?

Well, shouldn’t we focus on where the money is? In the UK, about 40% of charities’ collective income is from individuals. Another 40% is from government and 20% from foundations. In the US, 80% comes from individuals. So, individual donors are important.

I don’t think charities are the audience, because people respond to incentives that they are under. Ninety-nine percent of my work is focussed on donors because “He who paid the piper calls the tune”. When asked why they measure what they measure, most charities say it is because the donors ask them to.

Charities compete with each other much more than one realises. The system doesn’t reward an inventor for collaborating in order to improve their programmes and doing a great programme does not generate the money with which to do a great programme. What generates money to do a great programme is having a nice fundraising dinner. Thus, charities run two completely separate businesses: one is talking to donors and the other is helping their actual beneficiaries.

Charity Navigator for the UK

What might be able to do to promote evidence-based philanthropy?

I’m wondering whether we should set up and fundraise for a Charity Navigator-like thing in the UK.

It costs about $1m per year in the US, where there are about 1m charities. It may be less in the UK because the sector is smaller here.

How would you make the algorithm for rating charities different from Charity Navigator?

Charity Navigator’s algorithm is very clever. If I were building a ‘Charity Navigator UK’ from scratch, I’d use a lot of the Charity Navigator algorithm with some minor amends. I like the fact that, for instance, it looks at whether you can figure out what the organisation does, what is it telling you about results, does it appear to be stable, and what evidence is there that it listens to their beneficiaries?.

After that, what you would really want is for the Charity Navigator to have enough resources so that it would look not only at the organisation itself, but it can also say “oh look, you seem to be doing XYZ intervention, what do we collectively as a human race know about XYZ intervention?” You want to be able to do a systematic review of the evidence on the interventions that that entity is doing.

You can imagine a table that has three columns, the first would have the charity’s name, the second would have how robust the organisation seems to be and the third would have the strength of evidence for the intervention that it’s doing. In most cases the strength of evidence will be very little.

How broad do you think the organisation should aim? Should it aim to rate most charities, or focus on finding the best?

I guess if you take a hardline effective altruism approach, then you wouldn’t like Charity Navigator because the hardline effective altruism says you’ve got to find where you will maximise your impact. I’ve thought about maximising too. Suppose a donor comes along and says “I want to put my money into cancer: What’s the best cancer charity?”. Then, as I understand it, a hardcore effective altruism response is “you don’t want to be giving to cancer: you want to be giving to bed nets”. The problem is, the person might then say “I don’t want to donate to bed nets, I want to do cancer”. You risk turning them off, and them not giving at all. My many years of dealing with many donors leads me to think that many donors would be turned off by that.

We don’t know the effect of trying to get people to change their cause. It’s an important assumption that trying to influence people’s causes is a net useful thing to do.

I have in my life dealt with a lot of donors who are, for various reasons, quite constrained in the range of causes they would give to. If you were a foundation and you were set up to support people with mental health, that’s what you’re legally obliged to do. Getting the Charity Commission to change your legal object is extremely hard.

So, I think there’s definitely room for more broadly focused analysis to accomplish a lot of good.

What would be required to set up an organisation like this?

The main problem is getting funding. The notion of doing Charity Navigators in different countries has been around a long time. I think about 20 groups have approached Charity Navigator with the idea of replicated it, and the conversation always falls on funding because it’s just really hard to get money for. This is partly why I’m interested in your community because there’s not many people who think about impact as the end in itself.

If you had the funding, how would you take it forward?

I would partner with Charity Navigator US. I would borrow some of their algorithm, not all of it, and I would borrow their web setup, because they have an enormous website. I would borrow whatever infrastructure they have behind that. They also have some really clever quality control around their ratings.

A platform to share charity evaluation data

What’s the problem?

We estimate that in the UK charities spend about £1.6bn a year on monitoring and evaluation (M&E) (calculation is at www.giving-evidence.com/book). We suspect that very little of that gets published. I used to be the CEO of a charity and I’ve long suspected very little gets published. My suspicion is supported by the fact Charity Navigator can hardly find any evidence online. I recently came across a study from Canada which implies that only about 2% of research done gets published. There’s more info here.

We are now looking now at creating a mechanism for all of that M&E data to get published because it’s potentially hugely useful when you’re doing systematic reviews. The fact is that donor money is being spent generating this evidence and if it’s not shared, that looks like money wasted. Surely it’s worth investigating and creating a system whereby it gets published and the norm is that it gets published.

How might we shift the norm? You can imagine, for example, a world in which, at the end of your grant, you’re not allowed to say “please find attached our impact report” you have to say “please find below a link to a public impact report”.We’re currently looking at what would it take to create that system and that norm.

Charities would need to clearly and publicly articulate four things: (A) the intervention which was carried out, in enough detail to replicate it, (B) the research question, (C) what was the research method, and (D) what was found.

There’s also a slightly separate approach, the FB Heron Foundation in the US is saying to its grantees and investees: don’t send us any information, put all of your information into a public data warehouse and we will cut it whichever way we want. We’re doing some exploratory work around this.

How are you taking this idea forward?

We recently raised some funding to explore how to create a system for sharing evidence within the UK criminal justice sector. We hope this will be a first step towards doing it across the entire nonprofit sector. You can read more about the general idea here. Our project proposal is here.

The analogy between charity and medicine

My overall impression is that in terms of the amount of and the sophistication of the use of evidence, the medics are miles outfront, international development is way behind, and most of philanthropy and domestic charities are miles behind that. So that’s why I feel very interested in the example of evidence-based methods in medicine.

We have just published a study looking at what the charity sector can learn from medicine.

We’re going to be looking more at the historical development, because medicine is 60 years ahead, and we want to know how we can get there, without it taking 60 years.

How can you get involved?

Giving Evidence is currently looking for volunteers.