Our mistakes

These are some of the most important ways things could have gone better and mistakes we have made running 80,000 Hours, taken from our self-evaluations.

Annual review June 2014 – April 2015

Mistakes concerning our research and ideas

We let ourselves become too closely associated with earning to give.

This became especially obvious in August 2014 when we attended Effective Altruism Global in San Francisco, and found that many of the attendees – supposedly the people who know us best – saw us primarily as the people who advocate for earning to give. We’ve always believed, however, that earning to give is just one strategy among many, and think that a minority of people should pursue it. The cost is that we’ve put off people who would have been interested in us otherwise.

It was hard to avoid being tightly associated with earning to give, because it was our most memorable idea and the press loved to focus on it. However, we think there’s a lot we could have done to make it clearer that earning to give isn’t the only thing we care about. Read more.

We presented an overly simple view of replaceability, and didn’t correct common misconceptions about it

We think many of our previous applications of the replaceability argument were correct, but we don’t think it means that you shouldn’t take jobs with direct impact (e.g. working at a nonprofit) or that it’s okay to take harmful jobs for indirect benefits.

Unfortunately some of our early content suggested this might be the case, and we didn’t vigorously correct the misconception once it got out (although we never made replaceability a significant part of our career guide). We’re concerned that we may have encouraged some people to turn down jobs at high-impact organisations when it would have been better to accept them. Read more.

Not emphasising the importance of personal fit enough

We always thought personal fit – how likely you are to excel in a job – was important, but (i) over the last few years we’ve come to appreciate that it’s more important than we originally thought (most significantly due to conversations with Holden Karnofsky) and (ii) because we didn’t talk about it very often, we may have given the impression we thought it was less important than we in fact did. We’ve now made it a major part of our framework and career profiles.

Released an interview about multi-level marketing

We asked users to send us interviews about careers they knew about. One sent us a favourable interview about multi-level marketing, which we released. We were quickly told by a reader that multi-level marketing is highly ethically dubious, and took the post down within an hour. We should have better vetted user-submitted content before release.

Operational mistakes

Allowing a coaching backlog to build up in late 2014

We allowed a large backlog of over 100 coaching applicants to build up at the end of 2014, with the result that many had to wait several months for a response. This happened because our head of coaching was repeatedly on sick leave over the last half of 2014, and I didn’t step in quickly enough to close applications. To make it right, we apologised to everyone and gave email advice to about 50 of the applicants. When we set up our new coaching process in early 2015, we closely monitored the number of applicants and response times, closing applications when our capacity became stretched.

Not improving the online guide earlier

In September 2013 we took down our old career guide, going for a year without a summary of our key advice outside of the blog. I was aware of the problems this caused – most readers don’t visit useful old posts, and it was hard to find our most up-to-date views on a topic. I could have made a minimal replacement (e.g. a page listing our key articles) back in September 2013, which would have resulted in thousands of extra views to our best old content. Instead, we focused on coaching and new research, but in retrospect I think that was lower priority.
We should also have added a newsletter pop-up earlier. We were wary of annoying readers, but it dramatically increased our conversion rate from 0.2% to over 1%. In the end, we added a more complex appeal that just slides down under the header rather than popping up, and is only shown to engaged readers, with the aim of making it less annoying. However, we could easily have added a simpler pop-up a year ago, which would have resulted in 1000-2000 extra newsletter subscribers.

Simultaneously splitting our focus between the online guide and coaching

Perhaps an underlying cause of the previous two mistakes was that we attempted to push forward with both coaching and improving the online guide at the same time, despite only having the equivalent of two full-time staff working on them. We did this despite knowing the importance of being highly focused.
With more focus, we could have had clearer and shorter development cycles, better metrics, and generally better management, which would have helped the team to be more productive.

The reason we didn’t focus more was that we were reluctant to close the coaching, even temporarily, but in hindsight this wasn’t a strong consideration compared to the benefits of focus.

Not focusing more on hiring people we’ve already worked with

It’s widely seen as best practice in startups for the first couple of team members to be people who have worked together before and can work together really closely. We were aware of this advice but pushed forward with trying to hire new people. This mostly didn’t work out, costing significant time and straining relationships.

I think it would have been better either not to hire, or to focus on doing short, intense, in-person trials, since that’s the best way to test fit quickly. Instead, we did longer but less intense trials that were often remote and spread out over the year.

See the full review of progress

Annual review April 2013 – May 2014

Mistake: Team too large and not sufficiently focused on strategic progress. If the team had been smaller, more permanent, higher quality and more focused, we probably would have had less immediate impact, in the form of changed careers, research and outreach, but would have probably had more fundamental strategic progress, such as developing product plans or prototypes, testing the impact of our programs, recruiting staff and raising funding. Ultimately, it’s strategic progress that’s important for our chances of becoming much bigger in the future. In particular, a key cause was having too many interns. Interns allow us to have more immediate impact, but take up core staff time, reducing long-run progress, especially in the face of relatively complicated team plans. We analysed the issue of how many interns to hire in our last six month review, concluding that we should aim to have fewer in the future. In hindsight, we should have been even more aggressive in reducing the number.

What we did: First, we reduced the number of interns to one working on tech (Ozzie Gooen) and one on central CEA. After Ozzie left, we switched replaced them with a part-time paid web developer. Ozzie also significantly simplified the website, making it easier to maintain, and taught us more about how to edit it ourselves (more detail in the website review). We replaced the most useful functions of non-tech interns with long-term professional freelancers, including an oDesk editor, a volunteer editor, a virtual assistant and a contract researcher. We decided to only aim to have one or two interns over the next year, and to restrict these to people we are strongly considering hiring, or who can help with our strategic priorities. Besides reducing the number of interns, we raised the bar on hiring, and decided to focus on building a team of staff who are around to stay, and can take 80,000 Hours to scale. We decided to aim to make the team plans even more focused, by working on fewer activities at once and always having a clear top priority.

See our full list of mistakes.

Six Month Review Dec 2012 – March 2013

Mistake: We ran out of capacity in the operations team around March. This resulted in delays of over a month to the arrival of two interns, since we were unable to complete their visa applications in time, and a meant the Executive Director of 80,000 Hours had to spend a significant amount of time on operations (around 50% at peak). This happened because (i) we didn’€™t make a sufficiently detailed plan for operations at the beginning of the period, so we didn’€™t recruit enough interns to meet demand (ii) we were overoptimistic about how much time operations required (iii) The Director of Operations wasn’€™t given enough authority to make the decisions himself.

What we did: We decided to create a new role – the Executive Director of CEA – which was filled by Rob Wiblin around August 2013. The role was to (i) oversee the operations team (ii) take over fundraising from Will MacAskill and (iii) act as a single point of contact for issues that affect the whole of CEA. It was given equal status to the Executive Directors of 80,000 Hours and Giving What We Can, who decided to meet weekly as a three. The aim was to increase decision making capacity covering central issues, like the office, legal risks, the relationship with other organisations and recruitment, while also freeing up Will’s time from fundraising. We also decided to: (i) Ask the Director of Operations to draw up more detailed plans (ii) Hire additional interns for the central team (iii) consider hiring a second staff member for the central team (though we ended up deciding against). By our next annual review in May 2014, we were happy with the operation of the central team and Rob Wiblin’s performance in the new role.

See our full list of mistakes.

Six Month Review June – Nov 2012

Mistake: We could have been ahead of schedule if we had focused more on testing, product research and strategy from the start, rather than working as much as we did on outreach (although outreach had substantial benefits).

What we did: We addressed this by having two major strategy reassessments – one near the end of the summer and one in November – in which we assessed our competitive niche and analysed our success to date. Going forward, we’re making sure to include more time for strategy in our plans. We’ve designed an iterative product development process where we collect feedback early and use it to constantly adjust our content.

Mistake: CEA Central Operations (shared with Giving What We Can) had success in registering CEA as a charity and dealing with all the admin required to take on staff – sharing the department saved us months of staff time. However, it also had a number of failures, which wasted management attention.

What we did: We changed the role of the Director of Operations to officially answer to the Executive Director of 80,000 Hours and removed their responsibilities to work on Giving What We Can (which was dividing attention). The Executive Directors of Giving What We Can and 80,000 Hours started to meet weekly to discuss central issues, and we reviewed our allocation of interns to the central team.

See our full list of mistakes.