Guide to working in AI policy and strategy

Written by Miles Brundage a researcher who works on AI policy at the University of Oxford’s Future of Humanity Institute, with help from the 80,000 Hours team and others acknowledged below.

Why we wrote this

80,000 Hours’ research suggests that one of the highest-impact opportunities to improve the world may be by positively shaping the development of artificial intelligence. The issue is large in scale and neglected by most people. Recent experience suggests it’s possible to make steady progress in reducing the risks. See our profile of the problem for further explanation of why we believe this.

The last few years have seen dramatic growth in the number of people doing technical research to figure out how we can safely program an artificial general intelligence, and we have a guide for people who are considering doing this kind of work.

There is another topic that is just as important and has become relatively more neglected: improving AI policy and strategy. This includes questions like how can we avoid a dangerous arms race to develop powerful AI systems; how can the benefits of advanced AI systems be widely distributed; and how open should AI research be? If we handle these issues badly, it could lead to disaster, even if we can solve the technical challenges associated with controlling a machine intelligence.

We need answers to AI policy and strategy questions urgently because i) implementing solutions could take a long time,

Continue reading →

#1 – Miles Brundage on the world’s desperate need for AI strategists and policy experts

If a smarter-than-human AI system were developed, who would decide when it was safe to deploy? How can we discourage organisations from deploying such a technology prematurely to avoid being beaten to the post by a competitor? Should we expect the world’s top militaries to try to use AI systems for strategic advantage – and if so, do we need an international treaty to prevent an arms race?

Questions like this are the domain of AI policy experts.

We recently launched a detailed guide to pursuing careers in AI policy and strategy, put together by Miles Brundage at the University of Oxford’s Future of Humanity Institute.

It complements our article outlining the importance of positively shaping artificial intelligence and a podcast with Dr Dario Amodei of OpenAI on more technical artificial intelligence safety work which builds on this one. If you are considering a career in artificial intelligence safety, they’re all essential reading.

I interviewed Miles to ask remaining questions I had after he finished his career guide. We discuss the main career paths; what to study; where to apply; how to get started; what topics are most in need of research; and what progress has been made in the field so far.

Continue reading →

How much do hedge fund traders earn?

Hedge fund trading may be the highest paying job in the world, so to learn more, we spoke with a former manager at one of the world’s leading hedge funds. They gave us the following information, which allowed us to make a rough estimate of the typical earnings of hedge fund traders.

We also ran this document past several other people in the industry and asked them to point out mistakes.

We found that junior traders typically earn $300k – $3m per year, and it’s possible to reach these roles in 4 – 8 years. Senior portfolio managers can easily earn over $10m per year, though average earnings are probably lower. Read on for the details.

How do hedge funds make money and how is it shared among the employees?

Hedge funds trade in financial markets on behalf of clients in exchange for annual fees, and a cut of the profits. They’re similar to mutual funds but face fewer restrictions on what they can invest in, and can only be used by accredited investors.

The revenue of a hedge fund comes from the fees on the assets it manages. The typical fund charges a fee of 2% of assets under management per year, plus a performance fee. The performance fee is typically 20% of any returns it makes for the clients over and above the 2% base fee. So, if a fund makes 10% returns in a year,

Continue reading →

Most people report believing it’s incredibly cheap to save lives in the developing world

One way that people can have a social impact with their career is to donate money to effective charities. We mention this path in our career guide, suggesting that people donate to evidence-backed charities such as the Against Malaria Foundation, which is estimated by GiveWell to save the lives of children in the developing world for around $7,500 [Update: now $2,300 as of 2020].

Alyssa Vance told me that many people may see this as highly ineffective relatively to their optimistic expectations about how much it costs to improve the lives of people. I thought the reverse would be true – folks would be skeptical that charities in the developing world were effective at all. Fortunately Amazon Mechanical Turk makes it straightforward to survey public opinion at a low cost, so there was no need for us to sit around speculating. I suggested a survey on this question to someone in the effective altruism community with a lot of experience using Mechanical Turk – Spencer Greenberg of Clearer Thinking – and he went ahead and conducted one in just a few hours.

You can work through the survey people took yourself here and we’ve put the data and some details about the method in a footnote. The results clearly vindicated Alyssa:

It turns out that most Americans believe a child can be prevented from dying of preventable diseases for very little –

Continue reading →

Should you wait to make a difference?

If you’re committed to making a difference with your career, you may well find that there is a tension between doing good now and laying the groundwork for doing good later.

For example:

  • Next year, you have two choices. You could work for an effective charity, making an immediate difference to its beneficiaries. Or you could go to graduate school and build up your career capital, (hopefully) allowing you to have a larger impact later.
  • Alternatively, say you have a substantial sum of money. You could give it today, or you could invest it, allow it to grow, and then give the larger amount later.

How can you go about deciding between these options? Here we present a summary of our findings. The full research has been published by Oxford University’s Global Priorities Project.

Summary of our findings

Now vs Later flowchart

Summary: should you donate money now or later?

First, focus on saving enough to live for at least six months with no income, then start saving a reasonable amount for retirement (typical advice is 15% of income) and paying down any high interest rate debt. During this period, aim to give 1% so you stay engaged with giving and keep learning about which charities are most effective.

After that, whether to give now or later depends on a couple of factors.

Continue reading →

What should you look for in a job? Introducing our framework.

How can you compare two career options in terms of how satisfying and high-impact they’ll be?

It can seem near impossible to make comparisons like this, and it’s true that it’s hard. But we think it’s still possible to make a lot of progress.

In this article, we show how to break the comparison down into four factors. These form our career framework – the key ingredients of a fulfilling and high-impact career.

Why we chose these factors is more fully explained in our career guide. We’ve used this framework to analyse hundreds of real career decisions, and think it captures most of the important factors. However, it’s a work-in-progress, and we could see ourselves changing the emphasis of the framework over time.

How can we compare different careers?
Short-term impact

Let’s start by considering your impact only in the short-term. We can break it into two components:

  1. Role impact – the extent to which the role gives you opportunities to have a large social impact.
  2. Personal fit – the extent to which you will be able to take advantage of these opportunities, which largely boils down to how good you’ll be in the role.

We think these two factors roughly multiply, i.e. if you’re twice as good at the role, then your overall impact will be twice as high.

This applies whatever you think social impact consists of,

Continue reading →

4 biases to avoid in career decisions

Over the last couple of decades, a large and growing body of research has emerged which shows that our decisions are far from rational. We did a survey of this research to find out what it means for your career decisions.

It turns out that we likely don’t know as much as we think we do, are overconfident, tend to think too narrowly and continue with paths that are no longer best for us. We need to be more sceptical of our decisions than we might be inclined to be; find ways to broaden our options; and take a more systematic and evidence-based approach to career choice.

In what follows, we summarise the main sources of irrational bias and outline what you can do about them.

1. Thinking narrowly

We often think too narrowly when considering what options are available to us, and what’s important in comparing them.

What’s the evidence for this?

There’s evidence that in decision making, we “narrow frame” in two ways: first, we think too narrowly about what options are available to us. Second, we think too narrowly about what our objectives are in comparing those options. This is supported both by direct studies, and by the existence of more general biases: the availability heuristic, causing us to focus on options that are readily available; anchoring, a tendency to overweight the first piece of information given;

Continue reading →

How accurately does anyone know the global distribution of income?

World income distributionHow much should you believe the numbers in figures like this?

People in the effective altruism community often refer to the global income distribution to make various points:

  • The richest people in the world are many times richer than the poor.
  • People earning professional salaries in countries like the US are usually in the top 5% of global earnings and fairly often in the top 1%. This gives them a disproportionate ability to improve the world.
  • Many people in the world live in serious absolute poverty, surviving on as little as one hundredth the income of the upper-middle class in the US.

Measuring the global income distribution is very difficult and experts who attempt to do so end up with different results. However, these core points are supported by every attempt to measure the global income distribution that we’ve seen so far.

The rest of this post will discuss the global income distribution data we’ve referred to, the uncertainty inherent in that data, and why we believe our bottom lines hold up anyway.

Will MacAskill had a striking illustration of global individual income distribution in his book Doing Good Better, that has ended up in many other articles online, including our own career guide:
 
 

 
The data in this graph was put together back in 2012 using an approach suggested by Branko Milanovic,

Continue reading →

Allied health professional

Do you want to work in health, but don’t fancy nursing or medical school? Meet the ‘Allied Health Professions’. These are the people who helped your sister overcome her stutter (speech pathologists), prescribed your friend their glasses (optometrists), and took the x-ray that time you stubbed your toe so hard you thought you’d broken it (diagnostic radiographers).

The allied health professions give you the high job satisfaction of helping patients and will continue to be in demand as the population ages. The training pathways are shorter and less competitive than for medical doctors, and compared to most nurses you’ll have more say over how to treat your patients — plus the pay is often better, sometimes over $100 000 per year. The downsides are that the skills you build aren’t very helpful outside your profession, the best paying options still require up to 8 years at university, and as with any health career in the developed world it’s hard to have a large direct impact.

Overall, this is a reasonable option for earning to give, especially if you want an agreeable work schedule, could work in the US or Australia, and can get into the more highly paid professions like dentistry and optometry. Just keep in mind that the big investment in training required for this narrow career path means it’s especially worthwhile for you to explore your options before studying to become a health professional.

Continue reading →

What skills are effective altruist organisations short of? Results from our survey.

Note we’ve replaced this survey with one that’s more comprehensive and up-to-date.

In August 2016, we surveyed 16 organisations in the effective altruism community about their hiring needs, and to what extent they are constrained by talent compared to funding.

What follows is a summary of the results, grouped by question asked. You can see the list of organisations surveyed in the footnotes.

Note that since the survey was carried out over six months ago, some of the information may no longer be up to date. We intend to repeat the survey in August 2017, and will report back on how the situation has changed.

What types of talent does your organisation need?

Here are the options provided on the survey, along with the number of organisations which stated that they were looking to hire people for these roles:

(Note that this table is not weighted by budget or team size, although we do not expect that this would materially affect the results.)

In open feedback, several respondents also mentioned that the community is most in need of specialist researchers rather than generalist researchers. Some other skill sets which were not included as options, but which were mentioned more than twice in the open feedback include:

  1. Economists, in 3 cases.
  2. Math and AI researchers, in 2 cases.
  3. Policy experts, in 2 cases.

Continue reading →

Some further resources on high-impact careers

Career exploration illustrated by an old map

Below is a list of books, essays and resources which we have found helpful in drawing up our career guide, and which we hope will help if you’d like to go into more detail.

Enjoying your job and altruism

Stumbling Upon Happiness, by Prof. Dan Gilbert, outlines the science of measuring happiness, and explains why we’re so bad at judging what will make us happy.

Flourish, by Prof. Martin Seligman, is a survey of the findings of positive psychology from the last couple of decades, by the founder of the field. A bit rambly (compared to his excellent earlier work), but full of fascinating ideas and examples.

The How of Happiness, by Sonja Lyubomirsky, about the best ways to implement the findings of positive psychology to become a little happier each day.

Give and Take, by Prof. Adam Grant, outlines the evidence that having an altruistic mindset can make you more successful, so long as you avoid burnout. It then goes on to explain how you can avoid burnout.

Altruism, by Matthieu Ricard, makes a detailed argument that it’s best both for yourself and the world if you focus on helping others.

Giving Without Sacrifice, by Andreas Mogensen, which explores whether giving 10% of your income to charity will make you happier.

Continue reading →

5 reasons not to go into education

First published June 2015. Updated February 2017.

When we first speak to people interested in doing good with their careers, they often say they want to get involved in education in the US or the UK. This could mean donating to a school, doing education policy work, or becoming a teacher.

However, we haven’t prioritised careers in education at 80,000 Hours. We don’t dispute that education is a highly important problem – a more educated population could enable us to solve many other global challenges, as well as yield major economic benefits. The problem is that it doesn’t seem to be very easy to solve or neglected (important elements of our problem framework). So, it looks harder to have a large impact in education compared to many other areas. In the rest of this post, we’ll give five reasons why.

The following isn’t the result of in-depth research; it’s just meant to explain why we’ve deprioritised education so far. Our views could easily change. Note that in this post we’re not discussing education in the developing world.

1. It’s harder to help people in the US or UK

Everyone in the US or UK is rich by global standards: the poorest 5% of Americans are richer than the richest 5% of Indians (and that’s adjusted for the difference in purchasing power, see an explanation and the full data).

Continue reading →

The effective altruism guide to donating this giving season

People in the effective altruism community aim to use evidence and careful reasoning to work out how to best promote the wellbeing of all. To find the highest-impact charities this giving season, they’ve done tens of thousands of hours of research and published over 50,000 words of analysis this month. We read it all, and summed up the main recommendations by area.

But which of the 9 problem areas listed should you personally give to? We’ve got you covered here too. This tool asks you six questions and adjusts the ranking based on your beliefs:1

Quiz: Which problem should you give to? →

In the full post, you can find (i) how we came up with the list, (ii) more advice on how to narrow down the list, (iii) more information on each charity.

Continue reading →

80,000 Hours annual review Dec 2016

Summary

2016 was an excellent year for 80,000 Hours. Here are some highlights – full details follow.

Metrics

  • In our last review in May 2015, we set the goal of 50 significant plan changes per month by October 2016. That month, we actually recorded over 200.
  • To make it harder to grow by adding lots of small plan changes, in October 2015 we started “impact rating” the plan changes, and tracking the impact-weighted total. 31 Dec 2015, we set the target of tripling the monthly rate of impact-adjusted plan changes over the year, which we achieved in November 2016. We now track about 150 impact-adjusted significant plan changes (IASPC) per month.

Impact and cost-effectiveness

  • Our costs in 2016 were £250,000, up 13% on 2015. Considering that our staff could have earned to give instead, the total opportunity cost is perhaps £350,000 – £500,000.
  • Since our last review, the ratio of costs to IASPC fell almost 3-fold.
  • In 2016, we caused 115 people to take the Giving What We Can (GWWC) 10% pledge. GWWC estimates this is worth about £5 million in donations to their recommended charities (counterfactually-adjusted, time-discounted, dropout adjusted). So this alone plausibly justifies our costs, although our aim is to solve talent gaps rather than funding gaps.
  • In addition, the plan changes since our last review now include three people who each intend to donate over $100m over their lifetimes,

Continue reading →

Has 80,000 Hours justified its costs?

We set up 80,000 Hours because we thought it could become one of the most effective charities in the world. The idea was to achieve a multiplier – with a small amount of our time, we could enable thousands of others to spend their careers on whatever is most effective, and achieve thousands of times as much as we could individually.

In this post, we examine whether 80,000 Hours has generated enough impact to justify its costs over our history, and make some rough estimates of our multiplier.

Because it’s hard to estimate what would have happened if 80,000 Hours had never existed, all of these estimates are very uncertain, and can be debated. However, there are multiple ways we’ve plausibly justified our costs to date. In this document, we sketch out some of these pathways. We’re not aiming to be fully rigorous. Rather, consider the examples as a group. If only a few turn out to be genuine cases of impact, we’ll have justified our costs many times over.

Is this the wrong question?

80,000 Hours is a startup. Asking whether we’ve justified our costs to date is like asking whether Google was profitable in 2000. The aim of 80,000 Hours is to grow, and have a far larger impact years in the future.

Most of the value of donations to 80,000 Hours comes from the chance that these donations enable us to grow 10-times or 100-times.

Continue reading →

End of year update on plan changes

This is an update on the number of significant plan changes we’ve caused as of the end of Nov 2016.

We define a significant plan change as:

Someone tells us that 80,000 Hours caused them to change the career path they intend to pursue, in a way that they think increases their lifetime impact.

More on what counts as a significant plan change here.

Our total number of plan changes as of the end of Nov 2016 is 1,854, and after impact-adjusting these it’s 1,504.8.

Continue reading →

80,000 Hours has a funding gap

Over the past three years, we’ve grown almost 36-fold, more than tripling each year. This is measured in terms of our key metric – the number of impact-adjusted significant plan changes each month. At the same time, our budget has only increased 27% per year.

Given this success, we think it’s time to take 80,000 Hours to the next level of funding.

Over the next few weeks, we’ll be preparing our full annual review and fundraising documents, but here’s a preview.

chart

Overall, the 2017 target is to triple, measured in terms of impact-adjusted significant plan changes per month (which will mean over 3,000 over the year). We’ll do this by continuing to improve the advice, and starting to scale up marketing, with the aim of becoming the default source of career advice for talented, socially-motivated graduates.

Concretely, here’s some priorities we could pursue:

  • Dramatically improve the career reviews and problem profiles, so we have in-depth profiles of all the best options. This will help our existing users make better changes, and bring in more traffic.
  • Upgrading – develop mentors and specialist content for the most high-potential users, such as those who want to work on AI risk, policy, EA orgs and so on. We now have a large base of engaged users (1300+ through the workshop, 80,000+ on newsletter), so there’s a lot of follow-up we could do to get more valuable plan changes from them.

Continue reading →

Product manager in tech

Product management is one of the best non-programming roles in the tech industry, and tech is one of the most attractive industries to work in. It builds more widely-applicable skills than software engineering roles and has comparable pay. Programming experience isn’t necessary, but it’s also a great next step for software engineers.

Continue reading →

Update on 80,000 Hours May 2015 – June 2016

This is a quick update on our progress over the last year. Our next in-depth annual review, in which we’ll vet everything in more depth, will be in January 2017.

Our impact across the year

Here’s our key metrics for the top of our funnel. Ultimately we care about significant plan changes, which we report right below.

Unique visitors to site
New newsletter subscribers

Our newsletter now has a total of over 50,000 subscribers, which we think makes it the largest in the effective altruism community (most others have about 10,000). Our total traffic also just overtook GiveWell, which we think is the next largest by traffic (we had 880,000 users over the 12 months ending June, compared to 860,000 when calculated the same way.)

Continue reading →