End of year update on plan changes

NOTE: This piece is now out of date. More current information on our plans and impact can be found on our Evaluations page.


This is an update on the number of significant plan changes we’ve caused as of the end of Nov 2016.

We define a significant plan change as:

Someone tells us that 80,000 Hours caused them to change the career path they intend to pursue, in a way that they think increases their lifetime impact.

More on what counts as a significant plan change here.

Our total number of plan changes as of the end of Nov 2016 is 1,854, and after impact-adjusting these it’s 1,504.8.

Here’s a summary of our key figures:1

Year201120122013201420152016 (Jan-Nov)All-time total
Reach: unique visitors to site4,26646,92491,999149,164513,697834,3101,640,360
Year on year growth rateNA1000%96%62%244%62%NA
New newsletter subscribers added7061,6191,9432,28323,27156,17385,995
Year on year growth rateNA129%20%17%919%141%NA
New impact-adjusted significant plan changes recorded (at end of year)NANA125.0148.7320.2910.91,504.8
Year on year growth rateNANANA19%115%184%NA
Financial costs0£23,100£116,019£121,003£221,380£228,080£709,582
Labour costs (in person-years, inc. volunteers and freelancers)1.73.47.54.94.85.027.2
Financial costs per impact-adjusted plan changeNANA£928£814£691£250£472

Impact-adjustment of plan changes

In October 2015, we started rating plan changes with a value of 0.1, 1 or 10 based on our estimate of their counterfactual impact, in order to track the quality of plan changes over time. Read more.

Number of plan changes

Here’s the breakdown by year:
image-12

Note that we only started collecting plan changes in 2013, but started outreach in 2011. This means the 2013 figures reflect three years of work rather than one, so our growth from 2013 to 2014 was better than it looks from these charts. Also, the figures for 2016 don’t include December, so our year-on-year growth rates will be somewhat higher once all of 2016 is included.

Our month-on-month growth rates over 3 years are faster than our year-on-year growth rates, because we grew quickly during 2016. Here are our monthly impact-adjusted plan changes, with plan changes we learned about from our annual impact surveys amortised:
image-14

Here are the proportions of plan changes that we scored with the values 0.1, 1 and 10 each year. As you can see, most plan changes are scored as 0.1 or 1, and there are only a few 10s:
image-17

Here are our impact-adjusted plan changes per year, again broken down by their scores:
image-16

Finally, here our monthly impact-adjusted plan changes broken down by their scores (excluding those we learned about in our annual impact surveys):
image-26

Most of our growth has been driven by the 1s. This is because (as is shown below) growth has largely been driven by (i) workshops (ii) our online tools and the on-going impact survey (benefiting from higher web traffic) and (iii) more people taking the Giving What We Can pledge.

These sources tend to produce 1s rather than 10s, especially in the short-term. However, we think about 10% of these 1s will become 10s over the coming years. It’s hard to become a 10 right away because it requires a big shift.

How did we find out about the plan changes?

We learn about significant plan changes when our users fill out our online surveys, feedback forms, or through emailing users directly. The main sources are:

  • Impact survey – we have a survey on our website which about 5-10 people fill out per week. Once a year we also send the survey out to everyone on our newsletter — we call this our ‘annual impact survey’.
  • Online tools – our tools survey users on whether 80,000 Hours caused them to change their career plans.
  • Workshops and coaching – all workshop attendees and people we coach are asked to fill out a feedback form which asks them if they changed their plans.
  • Manual correspondence – we email a small fraction of our users to ask if they changed their careers plans due to us.
  • From Giving What We Can – we email Giving What We Can members who say they heard of GWWC through 80,000 Hours to ask whether they took the pledge due to us.

How we learned about impact-adjusted significant plan changes in 2016:

SourcePercentage
Tools30%
Workshops/coaching29%
Manual correspondence13%
Annual impact survey10%
From GWWC10%
Ongoing impact survey7%

Here are our monthly impact-adjusted plan changes broken down by how we learned about them:
image-25

Plan change statistics

What did the changes consist of?

You can see some examples of plan changes in our annual review (forthcoming). There are summary statistics below.

In October 2016 we added new questions to our plan change surveys, asking users for more information. One new multiple choice question we now ask is What did the change consist of?
screen-shot-2016-12-17-at-15-40-17

446 people who made significant plan changes have answered this question so far (out of a total 1,414 people who made a plan change in 2016).

Here are the proportions of impact-adjusted plan changes which included the different options (note that people could select multiple options):

Answer includesPercentage
Find a job that builds better career capital55%
Seek a different type of role (e.g. do research rather than direct work)52%
Be generally more focused on social impact47%
Become more involved in EA community41%
Seek to earn more income33%
Work on a different global problem28%
Donate to a different type of organisation26%
Take GWWC pledge24%
Work at a different organisation24%
Study a different university degree17%
Other0.4%

What did plan changes scored “10” switch into?

We categorised the plan changes which we scored as 10s in 2015 and 2016 by the paths they switched into:

CategoryNumberPercentage
EA org1650%
Earning to Give (expected donations above $100k/year)516%
High-net-worth donor to top causes39%
Running EA student group39%
AI safety research26%
AI safety capacity building13%
Machine learning grad study13%
For-profit start-up for global poor13%
Total32100%

What did plan changes scored “1” switch into?

We selected a random sample of 30 plan changes scored as 1s in 2016, and categorised them by what they now intend to do.

CategoryNumberPercentage
Took GWWC pledge723%
Policy (focused on top problem areas)517%
Corporate sector for skills (planning to work on top problem areas)310%
EA org27%
Learn programming (actively involved in EA community)27%
Startup27%
Data science (and planning to donate over 10%)27%
Econ/Machine learning PhD27%
Change to quantitative major (planning to work on top problem areas)27%
Donate 20% of income13%
Earning to Give (expected donations above $10k/year)13%
Non-profit (impact evaluation)13%

What did plan changes scored “0.1” switch into?

We did the same for a random sample of 30 plan changes scored as 0.1 in 2016:

CategoryNumberPercentage
Corporate sector for skills (less evidence planning to work on top problem areas)1033%
Earning to Give (expected donations less than $10k/year)517%
Policy (less evidence planning to work on top problem areas)310%
Biomedical research (smaller shift from previous intentions)310%
Software engineering (less evidence planning to work on top problem areas)27%
Change to quantitative major (less evidence planning to work on top problem areas)27%
Applied Maths PhD (less evidence planning to work on top problem areas)13%
Promote EA as teacher13%
Switch donations to effective charities13%
Data science (less evidence planning to work on top problem areas)13%
Non-profit (less evidence of focus on top problem areas)13%

How many people took the Giving What We Can pledge due to 80,000 Hours?

In 2016 we tracked 115 people who took the Giving What We Can pledge due to 80,000 Hours. This is 8% of our plan changes for the year, and 13% of our impact-adjusted plan changes.

We track this figure by emailing people who take the pledge and say that they first heard about Giving What We Can through 80,000 Hours, and ask them how likely it is that they would have taken the pledge if 80,000 Hours didn’t exist. We also track people who say in our impact surveys that they now intend to take the pledge due to 80,000 Hours, and who then become members.

Which causes are people planning to work on?

Another new question we added in October 2016 is: Which global problem or cause are you planning to work on with your career?

415 people who made significant plan changes have answered this question so far, with the following results (again, note that people could select multiple options):

Answer includesPercentage
Economic empowerment in poor countries37%
Health in poor countries36%
Promoting EA33%
Global priorities research32%
Climate change32%
Risks posed by artificial intelligence26%
Undecided21%
Factory farming19%
Biosecurity9%
Nuclear weapons5%
Other3%

15% of people who said that they now intend to work on a different global problem, chose at least one of the following, and didn’t choose any of the other causes:

  • Promoting effective altruism
  • Global priorities research
  • Risks from artificial intelligence
  • Biosecurity

So roughly 15% of the plan changes (based on the sample) are switching into our top priority areas.

How involved in the effective altruism community are people who make plan changes?

We also ask people who made a plan change: Do you consider yourself an active supporter of “effective altruism”? 466 people who made a significant plan change have answered this question so far, with the following responses:

AnswerPercentage
I like the ideas but I'm not yet actively involved68%
I'm actively involved in the community17%
I'm heavily involved in the community and promoting the ideas9%
I'm not sure what this is6%
No - I have reservations about it1%

How did they first hear about 80,000 Hours?

130 people who made plan changes answered the question How did you first find out about 80,000 Hours?, with the following results:

AnswerPercentage (not impact-adjusted)Percentage (impact-adjusted)
Recommendation from a friend18%30%
Through another effective altruist organisation22%21%
I don't remember13%14%
Search engine9%6%
Link on social media8%6%
Link on other website6%5%
Peter Singer5%5%
Through effectivealtruism.org / EA newsletter5%3%
Tim Ferriss podcast3%2%
Media coverage2%2%
Doing Good Better2%2%
Other5%2%

To sum up the main sources for the impact-adjusted plan changes: 30% came from word-of-mouth, 31% from the effective altruism community, and 21% from online outreach.

What caused the plan changes?

We ask each person who reports a plan change What was most significant in triggering these plan changes? Here are the breakdowns of what caused plan changes by year:
image-20

Compared with previous years, plan changes caused by online content and workshops have gone up, and the percentage of plan changes caused by our community and coaching have gone down. This is in line with our expectations because we spent 2016 focused on improving our online content and delivering workshops, and didn’t do much coaching.

Here is the same chart but with the absolute numbers of impact-adjusted plan changes each year:
image-22

As can be seen, most of our growth in 2016 was driven by plan changes caused by online content and by workshops.

Other ways we helped our users

We also asked our users if there are ways we’ve helped them with their careers besides changing their career plans. Here are the results:
screen-shot-2016-12-17-at-17-08-54

Notes and references

  1. In our previous report the number of plan changes for 2015 are reported as slightly higher (326.9 vs 320). This difference is due to people who made plan changes in 2015 making subsequent plan changes in 2016 with a higher impact adjusted value, meaning they are recorded in 2016 instead, and due to disconfirming 2015 plan changes in light of new information.

    Also note that our cost figures are preliminary estimates. Our UK accounts with CEA UK have been audited through the end of June 2015 and the US accounts with CEA USA through the end of Dec 2015. But costs after that time will be subject to revisions.

    We measure unique visitors to our website using the Google Analytics client side tracking library. We believe that a significant fraction (~20-50%) of our website visitors use browser extensions that block this tracking method, so our true visitor numbers are higher.