How to quantify research quality?

Introduction

You may have recently noticed a number appearing under our blog posts, in a little green square. That’s an attempt to better track the quality of our research, which is, as far we know, the first system of its kind.

This post explains why we added it, how it works, who does the ratings, and its benefits so far.

Image_0

Continue reading →

    What is the average net worth of Stanford entrepreneurs?

    In this post we estimate the mean net worth of Stanford alumni who made their wealth primarily through founding startups.

    Our estimate is that the mean net-worth of a Stanford alumnus who founded a corporation $10.8 million as of 2013.

    The reason we are interested in making this estimate is because it fits in with a larger research project to into entrepreneurship, and within that project, into the wealth that can be gained by becoming an entrepreneur.

    In this post, we estimate the total net worth of Stanford alumni who have founded corporations then we estimate the total number of Stanford alumni who have founded corporations. We then arrive at our estimate by dividing the total net worth of Stanford alumni founders by the total number of Stanford alumni founders. We close with some caveats and qualifications to our estimate.

    Continue reading →

      Increasing your earnings as a doctor

      Making a difference to patient’s lives is a gratifying part of medical work. However, an investigation by Dr Gregory Lewis suggests that doctors may be able to make a greater improvement to people’s lives through their donations than through their practice. In part, this is because the potentially large impact of charitable donations. For instance, research by GiveWell has shown that it’s likely to be possible to save a life for less than $10,000. This raises the question ‘how can doctors increase their earnings?’.

      In this post, we explore whether doctors can improve their earnings by:

      1. Moving to a different country.
      2. Choosing a highly-paid specialty.
      3. Pursuing locum (contract) shifts.

      Continue reading →

      Advisory board report June 2014

      As part of our annual review, we held an advisory board meeting. Our advisory board consists of: James Norris, Alex Flint and Jeff Kaufman. The members are chosen as supporters of our mission and effective altruism, who are not otherwise involved in running the organisation. We encourage them to act as impartial critics of our approach. One is a major donor to 80,000 Hours, two are highly active in building the effective altruism community and two have entrepreneurial experience.

      After the meeting, James Norris issued the following statement on behalf of the board:

      The advisory board met on May 9, 2014 to review the performance of 80,000 Hours over the past year. Alex Flint, Jeff Kaufman, James Norris, and Ben Todd were present. The members are in agreement that 80,000 Hours is executing well on its strategic plan and making strong progress overall.

      Fundraising and website metrics are both encouraging. William MacAskill’s commissioned book on effective altruism appears to be a big win for 80,000 Hours and the community at large. However, some members cautioned it might not ultimately yield as large of a benefit as anticipated.

      One ongoing concern is staffing. Salaries are slightly uncompetitive and senior leadership may be overly critical to 80,000 Hours’ success. Some members also voiced concern around maintaining a healthy public image in light of growing media attention.

      Overall, 80,000 Hours appears to be intelligently and thoughtfully run by a very capable team. The advisory board is confident 80,000 Hours will continue to execute well in the foreseeable future.

      See the previous report here.

      Continue reading →

      Case series – why and how to learn programming

      Software engineering is a lucrative career with an unusually low barrier to entry. Due to its appeal, some people in our community have switched into programming via many different routes. To help guide other individuals who are considering making this transition, we’ve gathered the five people in our community’s experiences learning to code and getting employed as a programmer.

      • Some programmers say that they enjoy their work because of the puzzles and problems involved in programming. They also say that they enjoy being drawn into a flow state.

      • One undesirable characteristic of software engineering is its white male monoculture.

      • Other common peeves are the need to understand large existing codebases and engaging in the boring aspects of fixing broken software.

      • People learn to program in a variety of ways including App Academy, computer science degrees, and teaching themselves while doing another job.

      • It’s easier to get hired if you’ve done an internship. Applying widely also helps. One App Academy graduate applied to 30-40 companies, out of which he got 5 phone screenings / code challenges, 2 in person interviews, and one offer.

      Continue reading →

      How much do Y Combinator founders earn?

      Paul Graham

      Introduction

      We’re interested in estimating how much tech entrepreneurs earn, since it’s one of our top recommended careers, and this is in part because it seems particularly high earning (enabling high donations, and potentially indicating the creation of a lot economic value). As part of this, we wanted to find out: if you can get into Y Combinator, how much will you earn? We’re particularly interested in Y Combinator because it’s the best known seed accelerator, and the data is available. In summary, here’s what we found:

      • The total value of Y Combinator companies is $26 billion, of which the founders own $8 billion.
      • Most of the returns have gone to a tiny minority of super-successes. The founders of AirBnB, Dropbox and Stripe are worth about US$7 billion, about 80% of all founders’ equity, although they account for 0.5% of the companies.
      • Outside of the most successful companies, it was still possible to earn significant returns. 12% of companies from the first five years of Y Combinator are now worth US$40 million or more, and a further 10% have sold for US$5-40 million. The remainder probably earned little more than their (low) salaries.
      • On average, founders from the first five years of Y Combinator are now worth US$18 million after 5-9 years, giving past average earnings of US$2.5 million per year
      • When it invests in its companies, Y Combinator values them at US$1.7 million, of which each founding team owns $1.6 million. This implies that founders must earn (in cash or equity) substantially more than $100,000 per year on average.
      • We expect the average earnings going forward to be less than $2.5 million per founder per year because of competitors to Y Combinator and regression to the mean.
      • Y Combinator accept 2.5% of applications.
      • Your personal expected earnings from applying to Y Combinator depend on your chance of being accepted and your chance of creating the next AirBnB or Dropbox.

      Continue reading →

      Summary of the annual review May 2014

      Tinted_images_and_options-02

      Introduction

      In this document, we present some concluding comments about our annual review and provide a guide to the rest of the documents. The summary of our previous review is here.

      Overview comments

      With the end of this annual review, 80,000 Hours has reached an important milestone. Over the last year we performed a major impact evaluation. We found that as of December 2013 we’ve caused 107 ‘significant plan changes’ – a metric we developed to track our impact. You can see some examples here. given this, we think we’ve achieved initial proof of concept that our prototype online content, coaching and community can change careers. we presented an overview of the evaluation here and an in-depth analysis of the plan changes here. we think this proof of concept is sufficiently compelling to move our focus on to improving the quality of our programs.

      On this basis, we’ve focused our strategy. To complete our ‘discovery phase’, for the rest of 2014 we’ll deepen our knowledge of social impact careers through research. We’ll also improve our online content. We’ll do this through successive rounds of coaching people and writing answers to their most pressing questions. We’ll summarise our findings on a series of six research pages, which we’ll submit for external evaluation.

      In 2015, we intend to move into our ‘execution phase’, in which we’ll aim to take the model we’ve discovered to its full potential. We envisage initially focusing on further improving the quality of our programs, in particular by writing a careers guide. We aim to finish this in time for the press campaign surrounding the launch of Will MacAskill’s book on effective altruism in August 2015. Securing this book deal with Gotham Books (Penguin US) and Guardian-Faber was another major success of 2013. As part of CEA’s effective altruism outreach project, it may become a major source of new users.

      After that, we may move our focus to outreach to ensure our programs connect with our entire target market. Or we may develop further programs, such as training a social impact career coach for every top university.

      Ultimately, we want to become the best source of advice in the world for the most talented, young graduates who want to make a positive impact, enabling them to best use their 80,000 hours to solve the world’s most pressing problems.

      While our key goals in the discovery phase are clarifying our strategy and testing our programs, we think our impact evaluation showed we’re justifying our costs through the value of the plan changes our users have adopted. Since we think the value of the plan changes we have caused is only a small portion of our total impact, this implies we’ve used resources highly effectively.

      Besides proof of concept and focusing our strategy, we had other important achievements, including good progress building the team, fundraising and continued growth in reach and impact.

      Overall, we’re very excited about the next year.

      Continue reading →

      Team plan May 2014

      In this document, which is part of our annual review, we outline our priorities over the rest of 2014. We also list the organisational metrics we intend to track and some of the challenges we anticipate facing.

      To see how this plan fits into our overall strategy, see the strategic review. To see what we did over the last year, see our review of progress.

      In summary, over the rest of 2014 we intend to focus on deepening our knowledge of social impact careers and improving the prototype of our online content. We’ll do this by expanding our research pages into a series of six separate pages, then completing several rounds of coaching and writing up research, with the aim of coaching at least 40 people, writing five reports on their most pressing questions, and writing five career profiles on the most asked about careers. At the end of the year, we’ll update the research pages based on what we’ve found, and perform a research evaluation to measure our progress.

      In addition, it’s highly important to build the capacity of our research team. Our main goal in this area is to find an outstanding candidate who can start working at 80,000 Hours as a staff member within the next 18 months, specialising in research.

      Other priorities include maintaining six months of reserves and building team capacity, through training the team and hiring new staff.

      Continue reading →

      Review of progress July 2013 to April 2014

      Introduction and summary

      The purpose of this document is to review what 80,000 Hours has achieved from July 2013 to February 2014 since our last review of progress. We also review how we performed relative to our targets, and our mistakes over the period. This document is part of our annual review.

      In summary:

      • We went through three stages during the period: website redesign, testing our content, and finally conducting our impact evaluation and strategic review. Other significant priorities included writing a book proposal on effective altruism, fundraising and staff recruitment.
      • Our main achievements were establishing proof of concept that our programs (research and online content, supported by coaching) can change career plans, and creating a clearer strategy.
      • While doing this, we continued to faciliate significant plan changes, which we think justify our costs.
      • Other achievements included: Will landed a major book deal to write about effective altruism, we continued to build the team and CEA, we increased our financial security by reaching our target of 12 months’ cash reserves, we implemented more professional branding, we had a meeting at the UK Prime Minister’s office on careers advice policy, we helped to foster the Global Priorities Project, we published over 40 research blog posts, the Cambridge student group had a strong first year, and we increased our organisational transparency.
      • We made progress on all of our key priorities and completed most of what we set out to achieve in our last review, but ended up several months behind schedule for a variety of reasons.
      • Our main mistake over the period was not keeping the team sufficiently focused on fundamental strategic progress. We think we’ve already corrected this mistake.

      You can find more detail on our key metrics in our review of program performance.

      Continue reading →

      80,000 Hours finance report

      Summary

      (N.B figures are rounded)

      Current financial situation

      As of April 2014, 80,000 Hours had £148,000 in reserves. This would represent 16 months of reserves at current rates of spending or 12.5 months of reserves if we follow our target budget, which would have us spend £130,000 over 2014. This £130,000 would be primarily spent on staff salaries (£68,000), intern support expenses and accommodation (£24,000) and office rent (£10,000).

      Historical spending

      In 2013, 80,000 Hours spent £124,000, or around £10,000 per month. From January through March 2014 80,000 Hours spent £28,000, or around £9,000 per month, slightly under the £31,000 budgeted. Since its founding in 2012 until April 2014 80,000 Hours has spent £176,000.

      Historical income

      In 2013 80,000 Hours received £175,000 in donations, of which £150,000 were specifically restricted to 80,000 Hours by donors and the remainder was given without restriction to our parent organisation, the Centre for Effective Altruism. From January through March 2014 80,000 Hours has received a further £92,000, of which £73,000 was restricted to 80,000 Hours by donors. Since its founding in 2012 until April 2014 80,000 Hours has received a total of £301,000.

      Fundraising targets

      We are currently looking to raise an additional £40,000. This would cover our remaining shortfall for 2014, and give us the option to hire an additional staff member to focus on research and coaching. Making this goal would mean we could end fundraising for the rest of 2014, and fully focus on developing our programs.

      Continue reading →

      Our strategic review May 2014

      Introduction – where have we come from?

      Less than two years ago, we were simply a group of student volunteers aiming to have the biggest possible impact. We thought we had some powerful ideas, which had caused some people to completely change their careers. But we didn’t know how to turn our ideas into a high impact organisation. We were doing a mixture of campaigning, community building, research and one-on-one coaching, and were unsure where to focus.

      Since then, we have focused our model, tested several prototype programs (online content and coaching) and gained an initial proof of concept by showing these can change careers. we also think we’ve had enough impact by changing career plans to justify our costs and have overall been a high impact use of resources.

      In light of these milestones, this document explains how our strategy has changed over time and where it stands now. It is part of our annual review.

      Summary

      We intend for the next year to be the final year of our ‘discovery phase’. The aim of the discovery phase is to identify the most high potential business model. (By ‘business model’ we mean the combination of programs, promotion, team structure and financing strategies we use to have a social impact).

      Our key focus will be on improving the quality of our prototype programs, in order to test some of the most important remaining uncertainties in our model.

      Within this, we have two aims. First we’ll focus on research to deepen our knowledge of how to choose the most high impact careers. We think further research is likely to be valuable, both because this body of knowledge is neglected but highly important, and because we think high quality research is crucial to the appeal and impact of our programs. High quality research is the most important way for us to become more credible (therefore able to persuade more people), it’s the key factor that sets us apart from our competition in attracting users, and it’ll ensure we highlight careers that are genuinely better than those our users would have taken otherwise (a key uncertainty in our impact evaluation). At the same time, we’re unsure how rapidly we’ll be able to make progress on research. Focusing on research for the rest of 2014, tracking our progress and submitting ourselves to an external research evaluation will reduce our uncertainty about the value of further research.

      Second, we’ll focus on improving our online content. We’ll expand our research page into a series of five, covering (i) the key criteria we suggest for comparing between careers, (ii) ranked lists of promising careers and causes (iii) supporting career profiles (iv) advice on how to find a career that fits and (v) a step-by-step planning process. We’ll also add pages to guide users to the best of our old content.

      We’re focusing on online content because we think our online content has more potential for impact than coaching. In our plan change analysis, we found over 30 of the 107 significant plan changes were attributable to online content alone. This would make the online content similarly important to the coaching in terms of changing careers. Given that online content is also far more scalable than coaching, it makes sense to prioritise it to test the hypothesis that it’s a better program. Moreover, developing the online content involves the least additional work over just doing research, and we think our current offering could be significantly improved relatively easily through better summaries of our existing content.

      We’ll continue with our one-on-one coaching as part of our research process. We’ll deepen our knowledge of social impact careers by doing rounds of coaching talented, altruistic, young people, then writing up answers to their most pressing questions for the blog. We’ll also write career profiles covering the careers our coachees most want to know more about.

      In 2015, we plan to start our ‘execution phase’, in which we aim to realise our model’s full potential for impact. We intend to initially focus on making our online content easier to use, most likely by developing it into an online careers guide, while continuing with research. The aim is to have the careers guide in place before the promotional campaign accompanying the launch of Will MacAskill’s book on effective altruism in August 2015. After that, we may increase our outreach work to connect with our entire target market.

      In the longer term, we’ll aim to develop further programs to deepen engagement with our users, such as expanding the coaching service. Our aim is to become the best source of advice in the world for the most talented, altruistic young graduates, enabling them to best use their 80,000 hours to solve the world’s most pressing problems.

      Continue reading →

      Plan change analysis and cost effectiveness

      Dsc04945

      Introduction

      This document is part of our annual review. In section one, it aims to resolve some key uncertainties within our review of program performance:

      1. How many significant plan changes has 80,000 Hours caused?
      2. What were these changes?

      In section two, we move on to consider:

      1. What costs has 80,000 Hours incurred in causing these changes?
      2. Does the value of the significant plan changes justify total historical costs?

      In the appendix, we also include 27 studies of career changes.

      Continue reading →

        How well are our programs performing?

        Introduction and summary

        In this document, which is part of our annual review, we overview the performance of our programs and their costs from founding to the end of 2013. Our programs consist of online research, coaching, community building and events. In this document, we examine how successful they have been in gaining audience, engaging and informing that audience, and ultimately in changing their career plans for the better.

        The key question we want to answer is: do we have proof of concept that our programs can repeatedly change the careers of our target audience for the better?

        Our key findings are:

        • We’ve been successful in reaching people and engaging them with little investment in outreach.
        • Out of several thousand engaged users, 107 have made significant career plan changes, and a significant proportion of these have followed through with their plan changes.
        • Overall, we think we have proof of concept that our programs can repeatedly change careers for the better.
        • In total, 80,000 Hours has received about £147,000 of donations and has taken 13 years of labour from the team.

        Our main uncertainties are:

        • How valuable is a plan change, taking into account what the person would have done otherwise? We have performed further analysis of this question in our analysis of plan changes. We plan to further investigate this question by performing a research evaluation and continuing to track our users over time. Some important sub-questions include:
        • How high value are ‘conventional careers’ compared to ‘effective altruist’ style careers?
        • What’s the chance of one or two people in the group having extreme impact, such as founding a highly influential organisation, donating tens of millions of pounds to charity, or being elected to office?
        • Are these plan changes due to 80,000 Hours or another group in effective altruism? We also investigate this issue in our analysis of plan changes. In the future, we may be able to carry out a randomised controlled trial of some of our programs to learn more.
        • How useful are the benefits of our programs compared to the information and support our users could have found elsewhere?
        • Will our users stay engaged for a couple of years, or many decades into the future?
        • How confident can we be that future work will lead to more plan changes?
        • How much personal value do our users gain from our programs? Would they pay for coaching?
        • How is impact allocated between our different programs? Would the plan changes attributed to coaching have been caused anyway due to our research and community?

        Continue reading →

        What impact has 80,000 Hours had?

        80000_hours

        Introduction

        What impact has 80,000 Hours had since we started in 2011? This document is not meant as an impact assessment; rather, its aim is to lay out the ways in which we’ve plausibly had an impact, and summarise what we currently know about them.

        Our impact can be compared to our historical costs. By the end of 2013, we had spent £147,000 and used about thirteen years of labour from the team, which we estimate had an opportunity cost of about £233,000 in forgone donations.

        Continue reading →

        What we can learn about career choice from the Terman study

        Terman2

        The Terman study is the longest running longitudinal studies ever to be carried out in psychology. The study included 1,528 of the most intelligent children born between 1900 and 1925. It started in 1921, and the participants have been followed up every four to five years ever since. Data was collected on their personality, habits, life-choices, health and much more. This allows researchers to track the results of different life choices over decades.

        Two of the leading researchers working on the Terman study recently released a book: The Longevity Project, which aims to uncover the factors that lead to the participants having long and healthy lives.

        The book has a fascinating chapter on career choice (though I’d recommend the whole thing).

        Here’s a summary of the key conclusions:

        The factors leading to career success

        • Intelligence predicts success, but it’s no guarantee. All of the participants in the Terman study were very bright, but a quarter ended up in less prestigious occupations, like clerical workers and craftsmen. Only one fifth ended up ‘highly successful’ – prominent doctors or lawyers, accomplished in the arts, or leading scientists. One fifth ended up ‘unsuccessful’ within their professions.
        • The more successful, the longer they lived. The most successful men lived on average five years longer than the least. In fact, Terman’s rating of success at age 30 predicted life-span decades later.
        • This effect was not explained by greater wealth, avoiding smoking and drinking, a happier marriage, more education, or conscientiousness (although conscientiousness did explain part of the effect).
        • A stable career with a clear progression of rising responsibilities also predicted longevity, compared to a ‘drifting’ career through many different professions.
        • Continuing to work into old age was a significant predictor of longevity.
        • Overall, the findings do not suggest that avoiding stress and responsibility is a good strategy for having a healthy life. Rather, they suggest that the becoming the type of person who perseveres to achieve ambitious goals leads to both success and health.
        • This links to a broader theme in positive psychology – in Flourish, Seligman proposes that achievement is one of the five key components of a flourishing life.

        Continue reading →

        Biographies of Top Entrepreneurs

        Lots of the people we coach want to know how to become a successful technological entrepreneur. One approach to this difficult question is to assess which unusual traits are common among the most successful tech entrepreneurs. In this post, we review the biographies of the ten richest tech entrepreneurs. Here is what we found:

        • All attended American Universities, though only half graduated (3 to start companies but 2 dropped out before they started their companies), and none have postgraduate qualifications.

        • 8 of the 10 entrepreneur’s Wikipedia page had stories or achievements demonstrating exceptional tech skills or interest in technology. (Azim Premji (Wipro) and Lawrence Ellison (Oracle)) are the only two whose Wikipedia pages do not demonstrate exceptional talent/interest in tech.

        • Fewer demonstrated early interest in business – Jeff Bezos and Michael Dell being the only exceptions

        • Only three took a job after finishing university and before starting a company.

          • Jeff Bezos, worked in multiple computer science-related jobs

          • Larry Ellison, worked for a data company while developing his product

          • Paul Allen, worked in programming before starting Microsoft

        Continue reading →

        Shared values predict startup success? An interview with Saberr

        Saberr

        Introduction

        As part of our ongoing research we have been looking at the best ways to go into entrepreneurship. When we talked to Matt Clifford, of Entrepreneur First about the question, he suggested talking to Saberr. Saberr are a small startup focussed on the question of predicting the success of teams in business settings, and they have already had some impressive successes.

        We spoke to Alistair Shepherd by phone, one of the two original founders of Saberr, about their perspective on forming a successful entrepreneurial team. The following is a selection of highlights from the call, edited and reorganised for clarity.

        Key points

        According to research by Noam Wasserman most startups fail because of their team, suggesting team composition is important for entrepreneurial success.
        While standard personality tests have not been shown to be very successful at predicting success in careers, Saberr have achieved some impressive, if small scale, predictive success using a model based on value alignment and behavioural diversity.

        Continue reading →

        App Academy interview with Buck Shlegeris

        Buck Shlegeris is a teaching assistant at App Academy in San Francisco. Buck plans to use his earnings in programming to give to charities that improve the future. We discussed whether 80,000 Hours members can start a career in programming by doing a coding bootcamp. Below are some edited notes from our conversation.

        Summary of main points:

        • People can enter training at App Academy from an unrelated background such as philosophy or other humanities with a few weeks of preparation.

        • The application includes some programming challenges and takes takes 10-20 hours to complete.

        • The course requires 60+ hours of work per week for 12 weeks.

        • 90% of App Academy students make it to graduation. By asking for help if you are failing to progress, you can probably further reduce the chance of dropping out.

        • Over 95% of App Academy graduates seeking employment as programmers attain it.

        • The average income of graduates is $100k in San Francisco’s Bay Area, with 90% securing an income from $80-120k. In New York City, the average income is $84k.

        Continue reading →

        The Undercover Economist speaks to 80,000 Hours

        Tim2

        Tim Harford recently spoke to us at Oxford. He’s a journalist for the Financial Times and the best-selling author of the Undercover Economist, which we’d recommend as a popular introduction to Economics. He also wrote Adapt, which argues that trial and error is the best strategy for solving important global problems. The arguments he makes fit with some of the arguments we have made for trial and error being a good way to plan your career.

        Tim gave a talk on innovation, similar to this. The talk introduced a distinction between two types of innovation, and asks, which one is more important?

        1. Marginal improvements – incremental improvements to existing systems.

        2. Revolutionary improvements – transformations of existing systems to create new ones.

        Continue reading →

        The Copenhagen Consensus speaks to 80,000 Hours about global prioritisation

        In October, Bjorn Lomborg from the Copenhagen Consensus Centre led a global priorities setting session at 80,000 Hours: Oxford in the Oxford Union. The video of the event has been uploaded by the Union.

        In the session, Lomborg guides the audience through the pros and cons of different uses of development aid, and asks them to put them in order of priority from the perspective of maximising the welfare of the global poor. Throughout the session, live votes are taken from the audience via wifi.

        More on the Copenhagen Consensus…

        Continue reading →