In which career can you have the biggest positive impact on the world?

We’re a non-profit that for the last 7 years has done research on this question. Over this time, we’ve grown more confident in our thinking, and our answers have become more unusual.

Initially, we were most associated with “earning to give” — taking a higher paying career in order to donate more to charity, rather than working directly into a “social impact” job. Even at the time, we didn’t think it was typically the best option, and today we rarely think that’s where people should focus first.

Today, we think of the highest-impact careers as those where an additional person can most effectively contribute to the most pressing global problems. Some of our top recommendations include:

  • AI safety research in academia or industry.
  • Working in operations and other roles in “effective altruist” and existential risk research non-profits.
  • AI and biorisk policy and government careers.
  • Global priorities research in academia.
  • Being a “China specialist”.

How have we arrived at these ideas? In this article, we will:

  1. Outline a method anyone can use to generate a short-list of high-impact career options.

  2. Apply this method given our own view of global priorities and current knowledge, to produce a list of 5 key categories of high-impact careers.

  3. We’ll also list some especially promising (but also highly competitive) “priority paths”.

  4. Link to some tips for people who already have significant experience.

The purpose is to help you get more options, which we explain how to further narrow down elsewhere on the site.

This article is part of our advanced series, and assumes you’re already familiar with our key ideas. If you’re new to 80,000 Hours, you might like to start with our main article on high-impact careers. This article is also a work in progress and we expect to update it periodically.

Reading time: 15-40 minutes

Table of Contents

In a nutshell

To generate a short-list of high-impact career options given your personal situation, you can use the following process:

  1. Choose the 2-4 global problems you think are most pressing.
  2. Identify the ‘key bottlenecks’ to progress within each of these.
  3. Identify career paths that help relieve those bottlenecks.
  4. Focus on the options where you have the best personal fit.
  5. Make quantitative estimates of the impact of your top options.
  6. Start to narrow down, or if there are no suitable options, go back to the start.

When it comes to specific options, right now we often recommend the following five key categories, which should produce at least one good option for almost all graduates:

  1. Research in relevant areas
  2. Government and policy relevant to top problem areas
  3. Work at effective non-profits
  4. Apply an unusual strength to a needed niche
  5. Otherwise, earn to give

We’re especially excited about the following “priority paths”. They’re extremely competitive, but great options to test out if there’s a chance they’re a good fit. Use the table of contents to skip ahead and read a description of each path.

  1. AI policy and strategy
  2. AI safety technical research
  3. Grantmaker focused on top problem areas
  4. Work in effective altruist organisations
  5. Operations management in organisations focused on global catastrophic risks and effective altruism
  6. Global priorities researcher
  7. Biorisk strategy and research
  8. China specialists
  9. Earning to give in quantitative trading
  10. Decision-making psychology research and implementation

Six steps to generate high-impact career options

The best career path for you will depend on your values, strengths and situation, so the ideal approach is to generate your own list of promising options, rather than use a generic list.

In the first section, we’ll outline a process you can use to do this, focusing on the factors we think are most important. Later in the article we’ll apply the process (based on our list of problems and current knowledge) to come up with our five key categories and list of priority paths.

This process can be applied no matter your career stage – whether you’re an undergraduate or nearing retirement. If you already have significant experience in a skill, then also take a look at our advice by skill type.

The aim of the process is to give you ideas for options to aim for over the medium term. Elsewhere we cover how to further narrow down your shortlist in terms of your specific situation and personal fit. We wouldn’t usually encourage anyone to take a path where they have lower than average chances of success, or are likely to become unhappy.

1. Decide which global problems are most pressing

Should you focus on reducing climate change, improving education, lowering the chance of nuclear conflict, or another area entirely? The first stage is to come up with a short-list of 1-5 areas that you think are especially effective to work on.

This is perhaps the most important step, because we’ve argued some problem areas are 100 or even 1,000 times more pressing than those that people often focus on. By this we mean that we expect each unit of resources will lead to 100 times as much impact.1 So, if you start out focused on the wrong area, you might forgo over 99% of your impact.

We cover how to compare problem areas and why there are such big differences in effectiveness in the main career guide or in this more technical article.

Personally, we think the most important issues relate to improving the long-term future and reducing catastrophic risks, and include AI safety, biorisk, building the effective altruism community and global priorities research.

If you’d like to see more of our views on this question, see:

  1. Why to focus on future generations
  2. Why reducing extinction risk is the key priority.
  3. How our views about the world’s biggest problem have changed over the last 8 years.
  4. A ranked list of areas

2. Identify the key bottlenecks in each area

Each problem has different needs. For instance, we’d argue that developing a cure to cancer is not mainly held back by a lack of awareness, since everyone is already aware that cancer is a problem. Rather, it’s held back by a lack of progress on biomedical research, and that mainly requires talented biologists and funding.

We define the key bottleneck facing an area as the input it most needs to make progress.

More precisely, the input that’s the biggest bottleneck is the input which would yield the most progress towards solving the problem if one more person started working on it on average.

Again, we think there are large differences in the extent to which different inputs are bottlenecks, and so if you focus on the wrong one, you might give up most of your impact.

Some inputs we often consider include:

  • Funding – additional financial resources from donations or fundraising.
  • Insights – new ideas about how to solve the problem.
  • Awareness & support – how many people know and care about the issue, and how influential they are.
  • Political capital – the amount of political power that’s available for the issue.
  • Coordination – the extent to which existing resources effectively work together.
  • Community building – finding other people who want to work on the issue.
  • Logistics and operations – the extent to which programmes can be delivered at scale.
  • Leadership and management – the extent to which concrete plans can be formed and executed on using the resources already available.

One complication in what’s most relevant is the bottleneck in the future period while you’re working, so you need to try to predict how the area will unfold.

It’s useful to try to be as specific as possible about the bottleneck. For instance, within AI safety, we think that a key bottleneck right now is more talented researchers. We think this is a key bottleneck because:

  • In the long-term, to address major risks from AI, we’ll need a flourishing, credible field of AI safety research that can come up with solutions to the alignment problem.

  • Right now, the field isn’t held back by funding, since there are already several large funders and institutions who are willing to cover almost any good opportunities in the field (e.g. MILA, OpenAI, DeepMind, The Open Philanthropy Project, BERI / CHAI).

  • We think the field is also not mainly held back by a lack of general awareness, since recent press coverage about AI risks has reached many people and there’s a significant group of people who are concerned by the issues. (Though, further high-quality outreach could be useful, since much of the existing coverage misportrays the issue.) It’s also not held back by a lack of political capital, since we don’t know what policy changes would help.

  • Instead, we expect that what would most benefit the field is more talented researchers working on the issue, especially those who are able to publish well received papers on the topic.

  • More researchers would not only directly feed into progress on key research questions, but this could also start a positive feedback loop. If more great academic papers on the topic could be published, it would demonstrate that it’s a credible, tractable field, and attract even more researchers into the field.

This is borne out when we talk to experts in the area. These experts often estimate the value of an additional promising technical AI safety researcher is equivalent to $1-$10m of extra funding per year — much more than most people could donate.

Here are some general guidelines on where the bottlenecks are likely to be depending on the stage:

  1. Defining the field – Early on, what’s often needed is insight. First, insight is needed to work out that it’s an issue worth working on, and then it’s needed again to define the key issues in the field (a form of “disentanglement research”). Insight is needed a third time to work out what the best solutions
    to the problem are. This is the stage we’re at in AI strategy and policy.

  2. Community and field building – After it’s clearer what the solutions are, you can start to build a community focused on the issue. Building a community is often more effective than trying to solve the issue directly because it lets you mobilise others, and achieve a “multiplier” on your efforts. This is usually best achieved through targeted advocacy early on — it’s best to avoid broad advocacy until you’ve better worked out the message, because it’s easy to get wrong and hard to unwind. You may also need some work to make concrete progress on the problem to show that progress is possible, helping to mobilise others. This is closer to the stage that AI technical safety is in.

  3. Scaling up the best solutions – Once the low hanging fruit in community building has been taken, then it’s time to solve the problem through whichever means seem most effective. This might mean launching a research programme, advocacy campaign, or the scale up of a promising intervention. At this point the bottleneck will depend on what the best solution is. If the best solution is research, the bottleneck will eventually be insight; if it’s policy change then it will be political capital; if it’s rolling out an intervention it might be logistics, funding or entrepreneurship. Global health seems to have a logistics bottleneck: one major challenge is to scale up existing evidence-backed treatments such as malaria nets which mainly requires money, logistics and management. An area can also get held up at an earlier stage. For instance, a field might ultimately need insight, but if it’s unable to attract researchers due to a lack of funding, then the funding bottleneck has to be overcome first.

Besides looking at the stage, you can also try to determine the key bottleneck by directly estimating which of the listed inputs are most needed at the margin. For this, it’s often useful to investigate which are most relatively neglected.

Before moving to the next step, aim to identify the 1-3 key bottlenecks in each of the top global problems you want to focus on. You can find some of our own assessments within our problem profiles.

3. Identify the career paths that best address these bottlenecks

In this stage, the aim is to generate specific career paths that help to resolve these bottlenecks.

Below, we list some general categories to consider. Try to think of at least one interesting concrete career option within each.

  1. Direct work — find the best organisations addressing the problem, and work there. These are usually non-profits, but could also be for-profit organisations with a social mission. This could mean working in management, operations, outreach and using many other skills.
  2. Entrepreneurship — help found a new organisation addressing a key bottleneck in the area.
  3. Research — try to make progress on whichever research questions are most important in the area. This usually means seeking a PhD and aiming to work in academia, but there are also research options in non-profits, think tanks and companies.
  4. Mass advocacy — make people more aware of the issue and how it can be solved, or encourage them to take action. This often means working in the media or a campaigning non-profit.
  5. Targeted advocacy — as above, but focused on a niche audience. This could be done alongside another job, especially one that gives you access to influential people.
  6. Government and policy — take a job in a relevant area of government / think tanks / party politics, then try to improve policy, or otherwise enable government to better manage the issue.
  7. Earning to give — take a job where you can earn more, then donate to the best organisations in the area.

Don’t forget you can address bottlenecks either directly or indirectly. For instance, if insights are the key bottleneck, you could try to contribute directly by becoming a researcher, but you could also earn to give and fund researchers or do targeted advocacy to recruit more researchers.

Rather than contribute right away, another option is to invest in yourself (or a community) and explore in order to have a greater impact in the future. We will discuss whether or not to do this in a future article (for now, see our old article).

We’d roughly estimate that by focusing on the career paths that effectively address the key bottlenecks in an area, you can increase your impact about 2-5 times (compared to choosing randomly).2 This makes this stage significantly less important than your choice of area, but still important.

4. Focus on the career options with the best personal fit

As we argue in our main career guide, the most productive contributors in a complex job have 10 or even 100 times as much impact as the median. Not all of this difference is predictable, but even if it only partially is, then “personal fit” is still a key consideration. Among options that you might seriously consider, we’d roughly estimate that the best in terms of personal fit are 2-10 times better than the median.2

This means that although your choice of problem area is likely overall a more important factor than personal fit, once you have a list of options that are plausibly high-impact, then personal fit becomes the key consideration.

More technically, we define personal fit as:

Personal fit: how productive you expect to be in the job in the long-term compared to the average of others who typically take that job.

With this definition, then the total expected immediate impact of a role is:

Expected impact = (average impact of role) x (personal fit)

The “impact of role” depends on the other two factors — how pressing the problem and whether the role makes a large contribution to it on average. If we think of these as “effectiveness factors”, then we can write:

Expected impact = (avg pressingness of problem) x (avg effectiveness of method) x (personal fit)

This formula is the first element in our career framework. However, note that to make an all-considered comparison of options, you also need to consider career capital, job satisfaction, coordination with a community, and other personal factors. We’ll cover these factors in later articles.

We can now see that because the factors roughly multiply, balance is key. If an option is terrible on any dimension then you should probably eliminate it.

We can also see that if we can achieve an increase on each factor, the increases will multiply producing a very large total increase. For instance, if you can find a problem that’s 100-times more pressing, a method that’s 5-times more effective and an option that’s a 10-times better personal fit, then the total increase in impact would be 5000-times. However, in practice the factors will conflict. For instance, your best option for personal fit might not be in the best problem area. Still, we often think it’s possible to achieve a 10 to 100 times increase in impact depending on where you start.

5. Consider making quantitative estimates of the value of different career options

The process we’ve covered so far is mainly based on a broad qualitative analysis, but it’s also useful to compare your outputs to some quantitative estimates, and this section will cover one way of doing that.

The first point is that we can think of different career paths as contributing different resources to global problems. We can then try to compare the value of these different contributions in a standard unit.

One unit we’ve used is dollars of donations to the problem.

You can measure the value of different contributions in dollars of donations by considering tradeoffs like the following:

Which makes a bigger contribution?

  1. An additional researcher willing to work on the area (at a specific level of personal fit).

  2. $X of donations per year.

The value of “X” at which you’re indifferent between the two is an estimate of the dollar value of the contributions of this research.

If you do this for each career option, then you’ll have a quantitative estimate of the value of your contribution to each area.

If you ask the question for a specific person (e.g. consider Jane working as a researcher), then the estimate should consider all of the factors we’ve listed above (key bottleneck, appropriate career path, personal fit).

Note as well that if you phrase the question appropriately, then it should also consider the “replaceability” of different staff, since we’re asking about the value of an additional researcher considering who else the organisation could hire instead.3

That said, the difficulty of taking account of all of these factors means that people’s estimates will be highly uncertain, so should be used with caution.

We’ve tried to perform this kind of analysis for the problem areas we’ve reviewed in-depth. We usually do this by asking experts in the problem areas to make these kinds of tradeoffs.4

For instance, we did this for the effective altruism community as a whole in our 2017 talent survey, which found results like the following:

 Average weighted by org sizeAverageMedian
Senior hire$12.8m ($4.1m excluding an outlier)$7.6m ($3.6m excluding an outlier)$1.0m
Junior hire$1.8m$1.2m$0.25m

Unfortunately, these estimates will involve a huge amount of uncertainty and disagreement. But the presence of uncertainty doesn’t mean we shouldn’t at least try to make estimates. Rather, we should make the best estimates we can, while also bearing in mind that they could easily change, and combine quantitative analysis with (hopefully) more robust qualitative arguments. The process of making a quantitative estimate also helps to clarify and improve our reasoning.

Once you have the estimates for the value of your contribution to different areas, you need to combine them with your estimates of the relative effectiveness of working on different problem areas (or at different organisations). Then, the multiple of the two gives you the expected value of the career paths i.e.:

Expected impact = (effectiveness of area) * (dollar value of contribution to area)

For instance, suppose you’re comparing two options:

  1. Earning to give supporting factory farming charities.
  2. Working in a non-profit focused on global health.

You determine the following (these are entirely hypothetical figures):

  • Each dollar contributed to global health produces 1 “unit” of “good done” or “value”.
  • Each dollar contributed to opposing factory farming produces 3 units of value.
  • If you earn to give, you can donate $10,000 per year.
  • If you work at a global health non-profit, they’d value having you working there as equivalent to their next best hire at an additional $20,000 of donations each year.

Then, the value of the earning to give for anti-factory farming work is:

3 * 10,000 = 30,000 units

The expected value of the global health non-profit option is:

1 * 20,000 = 20,000 units

So, with these figures, earning to give to support factory farming charities comes out ahead.

However, given the huge uncertainties in these kinds of estimates, the factory farming option is only narrowly ahead so this is close to a draw. Usually, we’d look for one option to be several times better before putting much weight on the estimate. When the differences are more narrow, then we’d mainly focus on more qualitative analysis (e.g. where you’ll have the best personal fit), or other factors that we will cover in later articles (e.g. career capital, value of information).

6. Start to narrow down, or go back to the start

Summing up what we’ve covered, the highest-impact career option for you is the one that does best based on a combination of whether it:

  1. Problem: Contributes to a pressing global problem (perhaps 100-fold increase in impact).
  2. Method: Makes a large contribution to a key bottleneck in one of these problems (perhaps 2-5 fold increase).
  3. Fit: Is an excellent personal fit (2-10 fold increase among reasonable options, maybe 100-fold increase among a wide sample of options).

You can analyse these factors both qualitatively and quantitatively.

Having applied the process, if you don’t have any plausible options at this point, then go back to the start and widen your search. In general, you can either focus on the same problem area, but consider less pressing bottlenecks (e.g. earn to give); or you can consider a wider range of problem areas. We don’t recommend aiming for something with below average personal fit, but you could aim for “good” rather than “excellent”.

If you have a reasonable short-list, then you could start to narrow down and make your plan.

In brief, this will involve the following steps, which we cover in more depth in our career guide:

  1. Decide how highly to weigh gaining career capital and flexibility compared to immediate impact (read more).
  2. Assess your long-term options in terms of impact, career capital, personal fit and comparative advantage, to work out which is best (read more about our framework and assessing your options).
  3. Decide whether to commit to entering the option that seems best or doing more to learn about which option is best. It may even be worth spending several years trying out different paths before revisiting your list (read more about how to try out your options)
  4. If you decide to commit, work out the most effective next step to enter your top option.
  5. Whether you explore or commit, make sure you have some nearby back-up options and a plan Z (read more about ABZ plans).

Now we’ll cover what options we think are best if we apply the steps we’ve just covered.

Which career paths do we think are often highest-impact? Five key categories

If you take an effective altruism style approach to social impact, and roughly share our long-term focused view of global priorities, then we think the following five broad categories currently stand out as long-term options to aim at.

Try to think of at least one specific option that might be a good personal fit within each category. We’ve chosen these categories to be broad enough that there’s usually something for everyone.

We’ll start by listing the categories, and then we give more explanation of why we chose them.

Later, we list some narrower “priority paths” that cover the very highest-impact options we know.

Many of the top problem areas we focus on are mainly constrained by a need for additional research, and we’ve argued research seems like a high-impact path in general.

Following this path usually means pursuing graduate study in a relevant area where you have good personal fit, then aiming to do research relevant to a top problem area, or by supporting other researchers.

Research is the most difficult to enter of the five categories, but going to graduate school gives you useful career capital for the other four categories, so it’s often a good path to start with (though we still recommend exploring other options for 1-2 years before starting a PhD).

After your PhD, it’s hard to re-enter academia if you leave, so at this stage if you’re still in doubt it’s often best to continue within academia. Eventually, however, it may well be best to do research in non-profits, corporations, governments and think tanks instead of academia, since it can let you focus more on the most practically relevant issues and might suit you better.

You can also support the work of other researchers in a complementary role, such as a manager, executive assistant, fundraiser or operations role. We’ve argued these roles are often neglected, therefore especially high-impact. It’s often useful to have graduate training in the relevant area before taking these roles.

Some especially relevant areas to study include (not in order): machine learning, neuroscience, statistics, economics / international relations / security studies / political science / public policy, synthetic biology / bioengineering / genetic engineering, China studies, philosophy, decision psychology. (See more on the question of what to study.)

Read more about research careers.

Government is often the most important force in addressing pressing global problems, and there are many positions that seem to offer a good network and a great deal of influence relative to how competitive they are.

In this category, aim to develop expertise in a relevant area, then take any government or policy job where you can help to improve policy relevant to a pressing global problem.

If you’re a U.S. citizen, working on U.S. federal policy can be particularly valuable because the U.S. federal government is so large and has so much influence over many of our priority problems. People whose career goal is to influence the U.S. federal government often switch between many different types of roles as they advance. In the U.S., we know about four main types of roles that can lead to a big impact on our priority problems. (We focus on the U.S. here because of its influence. We think working in policy can also be quite valuable in other countries although the potential career paths look slightly different.)

  1. Working in the executive branch such as the Defense Department, the State Department, intelligence agencies, or the White House. See our profile on the UK civil service, which starts with a general case for working in government. Note, though, that in the U.S. top executive branch officials are often hired from outside the traditional career civil service. So even if your goal is to eventually be a top executive branch official, the best path might include spending much of your career in the other types of roles we describe next.

  2. Working as a Congressional staffer. Congressional staffers can have a lot of influence over legislation, especially if they work on a committee relevant to one of our priority problems. It’s possible to achieve seniority and influence as a Congressional staffer surprisingly quickly. Our impression, though, is that the very top staffers often have graduate degrees sometimes including degrees from top law schools. From this path it’s also common to move into the executive branch, or to seek elected office.

  3. Working for a political campaign. We doubt that political campaign work is the highest impact option in the long run but if the candidate you work for wins this can be a great way to get a high impact staff position. For example, many people who work on a winning presidential campaign eventually get high impact positions in the White House or elsewhere in the executive branch.

  4. Influencer positions outside of government, covering policy research and advocacy. For example, you might work at a think tank or a company interested in a relevant policy area. In this kind of job, you can develop original proposals for policy improvements and/or help to set the agenda around a specific area of policy. You can also often build expertise and connections to let you switch into the executive branch, working on a campaign, or other policy positions. Journalists can also be very influential but our impression is that there is not as clear of a path from working as a journalist to getting other policy jobs.

In the UK, the options are similar except there is more separation between political careers and careers in the civil service (which is the equivalent of the executive branch). Read more in our profile on UK civil service careers and UK party political careers.

People also often start policy careers by doing graduate studies in an area that’s relevant to the type of policy you want to work on. In the US, it’s also common to enter from law school, a master of public policy, or a career in business.

Some especially relevant areas of policy expertise to gain and work within include: technology policy; security studies, international relations, especially China-West relations; and public health with a focus on pandemics and bioterrorism.

There are many government positions that require a wide range of skill types, so there should be some options in this category for everyone. For instance, think tank roles involve more analytical skills (though more applied than the pure research pathway), while more political positions require relatively good social skills. Some positions are very intense and competitive, while many government positions offer reasonable work-life balance and don’t have tough entry conditions.

Although we suspect many non-profits don’t have much impact, there are still many great non-profits addressing pressing global issues, and they’re sometimes constrained by a lack of talent, which can make them a high-impact option.

One major advantage of non-profits is that they can tackle the issues that get most neglected by other actors, such as addressing market failures, carrying out research that doesn’t earn academic prestige, or doing political advocacy on behalf of disempowered groups such as animals or future generations.

To focus on this category, start by making a list of non-profits that address the top problem areas, have a large scale solution to that problem, and are well run. Then, consider any job where you might have great personal fit.

The top non-profits in an area are often very difficult to enter, but you can always expand your search to consider a wider range of organisations. These roles also cover a wide variety of skills, including outreach, management, operations, research, and others.

We list some organisations to consider on our job board, which includes some top picks as well as an expanded list at the bottom. Read more about working at effective non-profits in our full career review.

If you already have a strong existing skill-set, is there a way to apply that to one of the key problems?

If there’s any option in which you might excel, it’s usually worth considering, both for the potential impact and especially for the career capital; excellence in one field can often give you opportunities in others.

This is even more likely if you’re part of a community that’s coordinating. Communities tend to need a small number of experts covering each of their main bases.

For instance, anthropology isn’t the field we’d most often recommend someone learn, but it turned out that during the Ebola crisis, anthropologists played a vital role, since they understood how burial practices might affect transmission and how to change them. So, the biorisk community needs at least a few people with anthropology expertise.

This means that if you have an existing skill-set that covers a base for a community within a top area, it can be a promising option, even if it’s obscure.

However, there are limits to what can be made relevant. We struggle to think of a way to apply a PhD in medieval portraiture directly to the top problem areas, so sometimes it will be better to retrain rather than apply an existing skill.

By donating to the very most effective organisations in an area, any graduate in a high income country can have a significant impact.

This means that a reasonable career option is to take a higher-earning job than you would have otherwise, and donate the extra income to effective charities.

When earning to give, it’s also important to pick a job with good personal fit, that doesn’t cause significant harm, and that builds career capital, so you have the option to switch out of earning to give later.

We list some of the highest-earning jobs available in a separate article, however, you don’t need to enter one of these in order to earn to give. You’re earning to give whenever you earn more than you would have otherwise, so everyone should have some earning to give options open to them.

The most obvious high-earning areas are sectors like technology, law, finance, consulting and medicine. But some higher-earning options outside of these sectors include the following: real estate, allied health, corporate management, sales & marketing and engineering.

Considering both income and career capital leads us to favour jobs in high-performing organisations where you can develop a concrete skill that’s useful in one of the other paths, such as management or operations. Tech startups with 20-100 employees are often a good place to consider.

Read more about earning to give.

Why did we choose these categories?

Why do we especially highlight research, policy and non-profit jobs; deprioritize earning to give; and omit advocacy and entrepreneurship?

In brief, we think our list of top problems (AI safety, biorisk, EA, GPR, nuclear security, institutional decision-making) are mainly constrained by research insights, either those that directly solve the issue or insights about policy solutions.

Our top problem areas are also complex, and it’s easy to make the situation worse rather than better through poorly informed actions or poor coordination.

This means that what’s most needed is people able to develop deep expertise in the area, have relevant insights, and effectively coordinate. We call people able to do this “dedicated talent”. In contrast, it’s hard to contribute to these areas part-time or indirectly.

People have the best chances of contributing to the insights bottleneck by taking roles in academia, corporate research labs, government & policy positions, and relevant non-profits. They can also take complementary positions in the same sectors, such as in operations and management.

On the other hand, there is currently more funding available than is being spent in these areas, so earning to give doesn’t seem like the key bottleneck. It also often seems like you can raise more money by working as a fundraiser in a relevant organisation than you can by earning to give.

Advocacy can be useful if well targeted, but doesn’t typically seem like the top priority bottleneck because many of the key people are already aware of the issues (they’re instead constrained by a lack of solutions), we don’t have simple messages to promote, and advocacy often misfires.

Likewise, non-profit entrepreneurship can also easily have unintended consequences, so we prefer people to build up expertise first, and it’s difficult to see for-profit ways to contribute to many of these issues.5

These positions are both our own assessment and backed up by results of our surveys of community leaders about talent constraints, skill needs and key bottlenecks.

You can see more detail about our reasoning in the appendix.

If you’re more focused on global health and factory farming, how do these recommendations change?

If you’re focused on global health or factory farming, then earning to give, for-profit work and advocacy become much more attractive; though, research, non-profit and government positions remain great choices.

Earning to give is more attractive because there are bigger funding gaps in these fields. GiveWell estimates their top charities will have a funding gap of about $200m in 2018, even after taking account of the donations they expect to cause, and this amount would be much higher if GiveDirectly were included.

Likewise, factory farming also seems to face greater funding constraints, though Lewis Bollard from the Open Philanthropy Project reports that funding constraints have decreased significantly in the last few years. This would again make it more important to try to work directly in the key non-profit organisations.

This said, we think that people focused on global health often underestimate options beyond earning to give, perhaps because their impact is less tangible.

For instance, we’re excited to see efforts by Evidence Action and Charity Entrepreneurship to found new non-profits focusing on evidence-backed interventions that might get recommended by GiveWell in the future, and we think working at these organisations or GiveWell non-profits or GiveWell itself could be good options. Read more about founding effective international development non-profits and listen to our podcasts on working at GiveWell and setting up a new charity.

Likewise, we think working in relevant think tanks and government agencies can be higher-impact due to the potentially huge scale of the impact of changing policy. One stand out organisation is the Center for Global Development, which has been funded by GiveWell. You can also see a more general argument for government careers in our career review and listen to our podcast about promoting evidence-based development policy.

Besides earning to give, mass advocacy is also relatively more attractive within these areas because you can promote a simple message like “donate here to end malaria”, “vote for the candidate that will increase foreign aid for health”, or “don’t eat factory farmed meat”. These messages are more robust, and allow many people to take useful action. This means that if you’re focused on these areas, then options like working in the media are more attractive.

In contrast, it’s much less clear how to run a successful mass outreach campaign about AI safety or even the idea of effective altruism — past efforts to spread the idea of effective altruism have often led to it being simplified down to “individuals should donate to evidence-backed health charities”, which put off many of the people we’d most like to appeal to, because it seems oversimplified and like there’s little role for researchers and policy experts (some of the people who are most needed).

Finally, building new for-profit companies is more attractive in these areas. For instance, developing the meat substitute industry seems like one of the best ways to end factory farming, because if meat substitutes can compete with factory farmed meat on both taste and cost, then they could drive factory farms out of business. This area is taking off right now, is being driven by for-profit companies, and is in large part constrained by people with business skill-sets able to lead and scale up companies (though also by research). To learn more, see our profile on factory farming and podcasts with Bruce Friedrich and Marie Gibbons.

Likewise, in international development, for-profit companies can play a role by creating products for the world’s poorest people, such as with SendWave or Segovia Technology.

In contrast, it’s not possible to set up companies that directly help people who don’t yet exist, and therefore make progress on long-term issues.

Our list of priority paths

In this section, we list some narrower career options that we think are among the highest-impact options we know. They’re mostly sub-options from the five key categories above that seem especially promising — focusing both on a top problem and a key bottleneck — and also offer reasonable career capital.

These options are also extremely competitive, and few jobs exist within each. So, they’re good options to strive for, but it’s important to have a back-up plan.

They’re also high-stakes, by which we mean it’s easy to accidentally cause harm within them. This is a further reason to check that you have good personal fit and the right expertise before going ahead.

The list is tentative. We’ve likely missed lots of good options, and the order could easily change, but it’s a starting point.

Note that we just sketch out the main arguments in favour of the option; counter-arguments are usually listed in the further reading.

If you’re aligned with our prioritisation, and want to pursue these options, then here’s a simplified version of a process to take.

  1. Start by working through the following list (roughly in order) to try to determine which might be a reasonable personal fit.
  2. Do more in-depth investigations over several months to work out which is best for you.
  3. Then, try to pursue whichever option seems to have the best personal fit and comparative advantage. Try it for several years, and if you’re on track to succeed, continue, and otherwise switch to another path.
  4. While doing this, make sure you have 1-3 solid back-up options to fall back on if your initial choice doesn’t work out.

Remember that we think personal fit is really important and we don’t encourage people to take options that would make them unhappy. But if one of these paths might be a good fit, it will usually be the top recommendation we’d make to you.

As we’ve argued, the next few decades might see the development of powerful machine learning algorithms with the potential to transform society.6 This could potentially have both huge upsides and downsides, including the possibility of catastrophic risks.

To manage these risks, one need is technical research into the design of safe AI systems (the “alignment problem”), which we cover later. But even if the technical problems are solved, there remain many other important questions to address. These can be roughly categorised into the three key challenges of transformative AI strategy:

  • Ensuring broad sharing of the benefits from developing powerful AI systems, as opposed to letting AI’s development harm humanity or unduly concentrate power.
  • Avoiding exacerbating military competition or conflict caused by increasingly powerful AI systems.
  • Ensuring that the groups that develop AI are working together to develop and implement safety features.

To overcome these challenges, we need a community of experts who understand the intersection of modern AI systems and policy, and work together to mitigate long-term risks. These experts would broadly carry out two overlapping activities: (i) research – to develop strategy and policy proposals, and (ii) implementation – working together to put policy into practice.

Ultimately, we see these issues as equally important as the technical ones, but currently they are more neglected. Many of the top academic centres and AI companies have started to hire researchers working on technical AI safety, and there’s perhaps a community of 20-50 full-time researchers focused on the issue. However, there are only a handful of researchers focused on strategic issues or working in AI policy.

Note that there is already a significant amount of work being done on nearer-term issues in AI policy, such as the regulation of self-driving cars. What’s neglected is work on issues that are likely to arise as AI systems become substantially more powerful than those in existence today — so called “transformative AI” — such as the three non-technical challenges outlined above.

Some examples of top jobs to work towards long-term in this path include the following, which fit a variety of skill types:

  • Work at top AI labs, such as DeepMind or OpenAI, especially in relevant policy team positions or other influential roles.
  • In think tanks, aim to become a researcher at one that works on relevant issues.
  • In the US government, develop expertise in a relevant issue, then take a position somewhere like the Office of Science and Technology Policy, National Security Council staff, Office of the Secretary of Defence, Joint AI Research Center, or the US Government’s defence and intelligence research funding agencies (DARPA and IARPA). Other governments are relevant as well, but the US is probably the most important right now.
  • In academia, become a researcher at one of the institutes focused on long-term AI policy, especially the Future of Humanity Institute at Oxford, which already has several researchers working on these issues in the Governance of AI Program.
  • In party politics, aim to get an influential position, especially as an advisor with a focus on emerging technology policy (e.g. start as a staffer in Congress).

How to enter

In the first few years of this path, you’d focus on learning more about the issues and how government works, as well as meeting key people in the field, rather than pushing for a specific proposal. AI policy and strategy is a deeply complicated area, and it’s easy to make things worse by accident (e.g. see the Unilateralists Curse).

Some common early career steps include:

  1. Relevant graduate study. Some especially useful fields include international relations, strategic studies, machine learning, economics, law, public policy, and political science. Our top recommendations right now are international relations if you want to focus on research and strategic studies if you want to focus on implementation.
  2. Working at a top AI company, especially DeepMind and OpenAI.
  3. Any general entry-level government and policy positions (as listed earlier), which let you gain expertise and connections, such as think tank internships, being a researcher or staffer for a politician, joining a campaign, and government leadership schemes.

This research field is at a very early stage, which creates multiple challenges. For one, the key questions have not been formalised, which creates a need for “distentanglement research” to enable other researchers to get traction. For another, there is a lack of mentors and positions, which can make it hard for people to break into the area.

These problems mean that it’s hard to enter this path as a researcher unless you’re able to become one of the top approximately 30 people in the field relatively quickly. On the other hand, if you are able to do this, then you’ll be able to make an especially big contribution, because you can be one of the founders of the field.

If you’re not able to land a research position now, then you can either (i) continue to build up expertise and contribute to research when the field is more developed, or (ii) focus more on the policy positions, which could absorb hundreds of people.

Most of the first steps on this path also offer widely useful career capital. For instance, you could often switch into other areas of policy, the application of AI to other social problems, or earning to give. So, the risks of starting down this path if you may want to switch later are not too high.

Since this is one of our top priority paths, we recently hired a specialist advisor, Niel Bowerman, to focus on finding and helping people who want to enter it. He is especially focused on roles aimed an improving US AI public policy. If you would like advice, get in touch here.

Could this be a good fit for you?

One key question is whether you have a reasonable chance of getting some of the top jobs listed earlier.

The government and political positions require people with a well-rounded skill-set, able to meet lots of people and maintain relationships, and the patience to work with a slow moving bureaucracy. It’s also ideal if you’re a US citizen that might be able to get security clearance, and don’t have an unconventional past that could get used against you.

The more research focused positions would typically require the ability to get into a top 10 grad school in a relevant area and deep interest in the issues. For instance, when you read about the issues, do you get ideas for new approaches to them? Read more about predicting fit in research.

Turning to other factors, you should only enter this path if you’re convinced of the importance of long-term AI safety. This path also requires making controversial decisions under huge uncertainty, so it’s also important to have excellent judgement, caution and a willingness to work with others, or it would be easy to have an unintended negative impact. This is hard to judge, but you can get some information early on by seeing how well you’re able to work with others in the field.

However, if you can succeed in this area, then you have the opportunity to make a significant contribution to what might well be the most important issue of the next century.

Key further reading

Other reading

As we’ve argued, the next few decades might see the development of powerful machine learning algorithms with the potential to transform society. This could potentially have both huge upsides and downsides, including the possibility of catastrophic risks.

Besides strategy and policy work discussed above, another key way to limit these risks is research into the technical challenges raised by powerful AI systems, such as the alignment problem. In short, how do we design powerful AI systems so they’ll do what we want, and not have unintended consequences?

This field of research has started to take off, and there are now major academic centres and AI labs where you can work on these issues, such as MILA in Vancouver, FHI at Oxford, CHAI at Berkeley, DeepMind in London and OpenAI in San Francisco. We’ve advised over 100 people on this path, with several already working at the above institutions.

There is plenty of funding available for talented researchers, including academic grants, and philanthropic donations from major grantmakers like the Open Philanthropy Project. It’s also possible to get funding for your PhD programme. The main need of the field is more people capable of using this funding to carry out the research.

In this path, the aim is to get a position at one of the top AI safety research centres, either in industry, non-profits or academia, and then try to work on the most pressing questions, with the eventual aim of becoming a research lead overseeing safety research.

Broadly, AI safety technical positions can be divided into (i) research and (ii) engineering. Researchers direct the research programme. Engineers create the systems and do the analysis needed to carry out the research. Although engineers have less influence over the high-level research goals, it can still be important that engineers are concerned about safety. This concern means they’ll better understand the ultimate goals of the research (and so prioritise better), be more motivated, shift the culture towards safety, and use the career capital they gain to benefit other safety projects in the future. This means that engineering can be a good alternative for those who don’t want to pursue research for whatever reason.

It can also be useful to have people who understand and are concerned by AI safety in AI research teams that aren’t directly focused on AI safety to help promote concern for safety in general, so this is another back-up option. This is especially true if you can end up in a management position with some influence over the organisation’s priorities.

How to enter

The first step on this path is usually to pursue a PhD in machine learning at a good school. It’s possible to enter without a PhD, but it’s close to a requirement in research roles at the academic centres and DeepMind, which represent a large fraction of the best positions. A PhD in machine learning also opens up options in AI policy, applied AI and earning to give, so this path has good backup options.

However, if you want to pursue engineering over research, then the PhD is not necessary. Instead, you can do a Masters programme or train up in industry.

It’s also possible to enter this path from neuroscience, especially computational neuroscience, so if you already have a background in that area you may not have to return to study. In the future, there might be social science opportunities to contribute to the alignment problem as well (to be covered in future work).

Could this be a good fit for you?

  • Might you have a shot of getting into a top 5 graduate school in machine learning? This is a reasonable proxy for whether you can get a job at a top AI research centre, though it’s not a requirement. Needless to say these places are very academically demanding.
  • Are you convinced of the importance of long-term AI safety?
  • Might you have a shot at making a contribution to one of the relevant research questions? For instance, are you highly interested in the topic, sometimes have ideas for questions to look into, and can’t resist pursuing them? Read more about predicting success in research.

Further reading

The Open Philanthropy Project takes an effective altruism approach to advising philanthropists on where to give. It likely has over $10bn of committed funds from Dustin Moskovitz and Cari Tuna, and is aiming to advise other philanthropists. There are other “angel” donors in the community who could give in the $1-$10m range per year, but aren’t at their maximum level of giving. And we know a number of other billionaires who are interested in effective altruism and might want to start new foundations.

One reason why these donors don’t give more is a lack of concrete “shovel-ready” opportunities. This is partly due to a lack of qualified leaders able to run projects in the top problem areas (especially to found non-profits working on research, policy and community building). But another reason is a lack of grantmakers able to vet these opportunities or start new projects themselves. A randomly chosen new project in this area likely has little expected impact — since there’s some chance it helps and some chance it makes the situation worse — so it’s vital to have grantmakers able to distinguish good projects from the bad.

The skill of grantmaking involves being able to survey the opportunities available in an area, and come to reasonable judgements about their likelihood of success, and probable impact if they do succeed. Grantmakers also need to build a good network, both so they can identify opportunities early, and identify groups with good judgement and the right intentions.

In addition, grantmakers need to get into a position where they’re trusted by the major funders, and this requires having some kind of relevant track record.

All of this makes it difficult to become a grantmaker. However, this also means that if you are able to get into one of these positions, then you can have a huge impact. A small scale grantmaker might advise on where several million dollars of donations are given each year. Meanwhile, a grantmaker at a large foundation — typically called a “programme officer” — might oversee $5-$40m of grants per year.

Given the current situation, it’s likely that a significant fraction of the money a grantmaker oversees wouldn’t have been donated otherwise for at least several years, so they increase the total amount of giving and get good projects started sooner.

What’s more, by having more talented grantmakers, the money can be donated more effectively. If you can improve the effectiveness of $10m per year to a top problem area by 10%, that’s equivalent to donating about $1m yourself. This often seems achievable because the grantmakers have significant influence over where the funds go and there’s a lot of potential to do more detailed research than what currently exists.

Overall, we think top grantmakers working in effective altruism can create value equal to millions or even tens of millions of dollars per year in donations to top problem areas, making it one of the highest-impact positions right now.

Finally, these positions offer good career capital because you’ll make lots of important connections within the top problem areas, so it seems easy to exit into direct work. Another exit option is government and policy. Or you could switch into operations or management, and have an impact by enabling other grantmakers to be more effective.

Related paths

One related path is to work as a grantmaker in a foundation that doesn’t explicitly identify with effective altruism, in order to help bring in an effective altruism perspective. The advantage of this path is that it might be easier to add value. However, the downside is that most foundations are not willing to change their focus areas, and we think choice of focus area is the most important decision. Existing foundations also often require significant experience in the area, and sometimes it’s not possible to work from junior positions up to programme officer.

Another related path is philanthropic advising. One advantage of this path is that you can pursue it part-time to build a track record. This also means you could combine it with earning to give and donating your own money, or with advocacy positions that might let you meet potential philanthropists. We’ve seen several people give informal advice to philanthropists, or be given regranting funds to donate on their behalf.

A third related path is to work at a government agency that funds relevant research, such as IARPA, DARPA and NIH. Grantmakers in these agencies often oversee even larger pools of funding, but you’ll face more restrictions on where it can go. They also often require a PhD.

How to enter

One entry route is to take a junior position at one of these foundations (e.g. research analyst), then work your way up to being a grantmaker. We think the best place to work if you’re taking this path is the Open Philanthropy Project (disclaimer: we’ve received grants from them). Founder’s Pledge also has a philanthropic advising team, though it has less of a track record and is less focused on long-term focused problem areas. You could also consider research positions at other effective altruism organisations — wherever will let you build a track record of this kind of research (e.g. the Future of Humanity Institute).

Another key step is to build up a track record of grant making. You could start by writing up your thoughts about where to give your own money on the effective altruism forum. From there, it might be possible to start doing part-time philanthropic advice, and then work up to joining a foundation or having a regranting pool (funds given to you by another donor to allocate).

A third option is to pursue work in the problem area where you want to make grants, perhaps in non-profits, policy or research, in order to build up expertise and connections in the area. This is the usual route into grantmaking roles. For instance, the Open Philanthropy Project hired Lewis Bollard to work on factory farming grants after he was Policy Advisor & International Liaison to the CEO at The Humane Society of the United States, one of the leading organisations in the area.

Could this be a good fit for you?

  • This position requires a well-rounded skill-set. You need to be analytical, but also able to meet and build relationships with lots of people in your problem area of focus.
  • Like AI policy, it requires excellent judgement.
  • Some indications of potential: Do you sometimes have ideas for grants others haven’t thought of, or only came to support later? Do you think you could persuade a major funder of a new donation opportunity? Can you clearly explain the reasons you hold particular views, and their biggest weaknesses? Could you develop expertise and strong relationships with the most important actors in a top problem area? Could you go to graduate school in a relevant area at a top 20 school? (This isn’t needed, but is an indication of analytical ability.)
  • Note that working as support or research staff for an effective grantmaker is also high-impact, so that’s a good backup option.

Further reading

We think building the effective altruism community is a promising way to build capacity to address pressing global problems in the future. This is because it seems possible to grow a great deal and contains people who are willing to switch area to work on whichever issues turn out to be most urgent in the future, so is robust to changes in priorities.

We realise this seems self-promotional, since we ourselves run an effective altruism organisation. However, if we didn’t recommend what we ourselves do, then we’d be contradicting ourselves. We also wouldn’t want everyone to work on this area, since then we’d only build a community and never do anything. But we think recommending it as one path among about ten makes sense.

A key way to contribute to building the effective altruism community is to take a job at one of the organisations in the community — see a list of organisations. Many of these organisations have a solid track record, are growing and have significant funding, so their biggest bottleneck is finding staff who are a good fit. An additional staff member can often grow the community by several additional people each year, achieving a multiplier on their effort. These organisations don’t only grow the community, they also do other useful work, such as research and fundraising.

Not many people have the characteristics needed to fill these positions, and many of the organisations operate close to “threshold hiring”, in which they’ll hire anyone who meets their criteria. This means these jobs are hard to get, but if you can get one, you won’t be significantly replaceable and it’ll be high-impact.

In our survey, leaders in the community estimated that recent junior staff members generate the equivalent of $80,000 – $600,000 in donations of value per year (interquartile range); and that recent senior hires generate the equivalent of $330,000 – $1.4m per year. For the most impactful recent hires, people gave figures over $3m, and the most successful in this field would be even higher again.

These roles let you develop expertise in effective altruism, top global problem areas and running startup non-profits. They put you at the heart of the effective altruism movement and long-term future communities, letting you build a great network there. Many of the organisations also put a lot of emphasis on personal development.

Role types

There are a variety of roles available, broadly categorised into the following:

  • Management, operations and administration – e.g. hiring staff, setting strategy, creating internal processes, setting budgets.
  • Research and advice – e.g. developing the ideas of effective altruism, writing and talking about them, helping people in the community.
  • Outreach, marketing and community – e.g. running social media accounts, content marketing, running promotional accounts, visual design, moderating forums, market research, responding to the media.
  • Systems and engineering – e.g. web engineering, data capture and analysis, web design, creating internal tools.

How to enter

To enter these roles, you can apply directly to the organisations. However, most of the organisations want to hire people who are committed to the community, so they often hire people who are already involved. This means that if you want to aim towards these positions, the most important step is to start meeting people in the community, and doing small projects to build your reputation (e.g. writing on the forum, volunteering at EAG, starting a local group, or doing freelance consulting for an organisation). We list more advice in our full profile.

If you want to get a job that puts you in a better position to enter these roles in the future, then do something that lets you develop a concrete skill that’s relevant to one of the role types listed above. Well-run tech startups with 10-100 people are often a good place to learn these skills in a similar context.

Could this be a good fit for you?

Whether you might be a good fit in part depends on the type of role you’re going for, however, there are some common characteristics the organisations typically look for:

  • A track record that demonstrates intelligence and an ability to work hard.
  • Evidence of deep interest in effective altruism — you need to be happy to talk about it much of the day. This breaks down into a focus on social impact and a scientific mindset, as well as knowledge of the community.
  • Flexibility and independence – the organisations are relatively small, so staff need to be happy to work on lots of different projects with less structure.
  • Several years of experience in a relevant skill is a bonus, but not required.

Further reading

Within working at effective altruism organisations, we’d like to especially highlight roles in operations.

Operations staff enable everyone else in the organisation to focus on their core tasks and maximise their productivity. They’re especially crucial for enabling an organisation to increase in scale. The best operations staff do this by setting up scalable systems, rather than addressing individual tasks. This could involve creating a financial system to make budgets and track expenses; creating an HR system to hire staff, onboard them and give feedback; or being an executive assistant to a project founder. Some operations staff manage a significant part of their organisation, and several have worked their way up to CEO or COO.

As we’ve argued, these positions seem especially in-demand right now, both within the core effective altruism organisations and the broader global catastrophic risk community. There are many open positions and organisation leaders often say this skill-set is their biggest bottleneck. A recent influx in funding has also created a systems bottleneck — it’s hard to rapidly hire and retain staff without an excellent operations team.

Many expect that it must be easy to hire operations staff from outside of the community, but we don’t think that’s true, especially right now while the relevant organisations are small. Since operations staff are so central to an organisation, they need to deeply understand the mission and be deeply embedded in the team.

Instead, we think these roles likely get unfairly neglected, because they are less glamorous and their impact is more indirect.

Within academic settings, it’s also because university regulations often prevent the institutes from paying as much as these roles merit.

This can make operations roles an especially good way to contribute to these top problem areas.

Operations roles also let you gain a skill-set that’s needed in basically every organisation — especially by managers — and so is highly transferable. In addition, in this path you can gain the other career capital benefits from working at effective altruism organisations we mentioned earlier.

How to enter

To get these roles, it’s often possible to enter directly and work your way up.

If you’re not able to land a full-time job right away, one common way people have entered these roles before is by volunteering to help run an EAG conference.

Otherwise, you could seek operations experience in top organisations in the private sector, such as a growing a well-run tech startup, while also meeting people in the relevant communities.

Could this be a good fit for you?

In addition to the traits we list for working at effective altruism organisations, operations staff are especially good at optimising systems, anticipating problems, making plans, having attention to detail, staying calm in the face of urgent tasks, prioritising among a large number of tasks, and communicating with the rest of the team.

We often find that people who are good at operations don’t realise that they are. Some clues you might be a good fit include:

  • Have you gone above and beyond in a previous job? For instance, did you notice a problem and take steps to fix it without anyone telling you what to do?
  • Have you been able to quickly learn new skills, and apply that knowledge? For instance, did you succeed in your studies or quickly pick up hobbies in your spare time?
  • Can you work independently? For example, have you pushed ahead with side projects that you weren’t required to do for work or school?
  • Have you run a significant event, such as a conference, exhibition or concert? This involves some similar skills, and people have often had the opportunity during university.

Further reading

We’ve argued that one of the most important priorities is working out what the priorities should be. There’s a huge amount that’s not known about how to do the most good, and although this is one of the most important questions someone would ask, it has received little systematic study. This is why effective altruism is so important.

The study of which actions do the most good is especially neglected if you take a long-term perspective, in which what most matters is the effects of our actions on future generations. This position has only been recently elucidated, and we know little about its practical implications. Given this, we could easily see our current perspective on global priorities shifting given more research, so these questions have practical significance.

The study of how to help others is also especially neglected from a high-level perspective. People have done significant work on questions like “how can we reduce climate change”, but much less on the question “how pressing is climate change compared to health?” and “what methods should we use to make that comparison?”. It’s these high-level questions we especially want to see addressed.

We call the study of high-level questions about how best to help others “global priorities research”. It’s primarily a combination of moral philosophy and economics, but it also draws on decision theory, decision-making psychology, moral psychology, and a wide variety of other disciplines, especially those concerning technology and public policy. You can see a research agenda produced by the Global Priorities Institute at Oxford.

We’d like to see global priorities research turned into a flourishing field, both within and outside of academia.

To make this happen, perhaps the biggest need right now is to find more researchers able to make progress on the key questions of the field. There is already enough funding available to hire more people if they could demonstrate potential in the area (though there’s a greater need for funding than with AI safety). Demonstrating potential is hard, especially because the field is even more nascent than AI safety, resulting in a lack of mentorship. However, if you are able to enter, then it’s extremely high-impact — you might help define a whole new discipline.

Another bottleneck to progress on global priorities research might be operations staff, as discussed earlier, so that’s another option to consider if you want to work on this issue.

Role types

You can broadly pursue this path either in academia or non-profits.

We think building this field within academia is a vital goal, because if it becomes accepted there, then it will attract the attention of hundreds of other researchers.

The only major academic centre currently focused on this research is the
Global Priorities Institute at Oxford (GPI), so if you want to pursue this path as an academic, that’s the ideal place to work.

One problem is that GPI only has a couple of open positions, and you’d usually need to have a top academic background to get one of them (e.g. if you did well in a PhD from a top 10 school in your subject that’s a good sign). Positions are especially competitive in philosophy.

That said, we expect that other centres will be established over the coming years. In the meantime, you could try to build expertise or pursue relevant research in other academic positions. For instance, doing an Economics PhD (and postdoc) opens up lots of other options, so is a reasonable path to pursue even if you’re not sure that global priorities research is a good fit for you.

One downside of academia, however, is that you need to work on topics that are publishable, and these are often not those that are most relevant to real decisions. This means it’s also important to have researchers working elsewhere on more practical questions.

We think the leading applied centre working on this research is the Open Philanthropy Project. One other advantage of working there is that your findings will directly feed into how billions of dollars are spent (disclaimer: we have received grants from them). However, you can also pursue this research at other effective altruism organisations. 80,000 Hours, for instance, does a form of applied global priorities research focused on career strategy.

There have also been people who have pursued global priorities research independently, such as Carl Shulman and Paul Christiano. These researchers often start with blogging, and then take freelance work from donors and organisations once they’ve proven their abilities.

How to enter?

The best entry route to the academic end of the field is to study a PhD in economics or philosophy. This is both because PhDs provide useful training and because they are required for most academic positions. Currently, economics is in shorter supply than philosophy, and also gives you better back-up options, so is preferable if you have the choice.

It’s also possible to enter from other disciplines. A number of people in the field have backgrounds in maths, computer science and physics. Psychology is perhaps the next most relevant subject, especially the areas around decision-making psychology and moral psychology. The field also crosses into AI and emerging technology strategy, so the options we listed in the earlier sections are also relevant, as well as knowledge of relevant areas of science. Finally, as the field develops there will be more demand for people with a policy focus, who might have studied political science, international relations, or security studies. In general, this is a position where wide general knowledge is more useful than most.

With the non-academic positions, a PhD isn’t necessary, but you do ideally need to find a way to demonstrate potential in this kind of research. It’s useful to develop skills in clear writing and basic quantitative analysis. Sometimes people enter the non-academic roles directly from undergrad if they’re sufficiently talented.

Could this be a good fit for you?

  • Might you be able to get into a PhD in economics or philosophy at a top 10 school? (This isn’t to say this qualification is required, it’s just that if you would be able to succeed in such a path, it’s an indicator of ability.)
  • Do you have excellent judgement? By this we mean, can you take on messy, ill-defined questions, and come up with reasonable assessments about them? This is not required in all roles, but it is especially useful right now given the nascent nature of the field and nature of the questions that are ultimately being addressed.
  • Do you have general knowledge or an interest in a wide range of academic disciplines?
  • Might you have a shot at making a contribution to one of the relevant research questions? For instance, are you highly interested in the topic, and sometimes have ideas for questions to look into? Are you able to work independently for many days at a time? Are you able to stick with or lead a research project over many years? Read more about predicting success in research.

Key further reading

Other reading

We’ve argued that pandemics pose a global catastrophic risk, and this risk could increase as advances in bioengineering make it possible to create engineered pandemics that are more deadly than naturally occurring ones.

There is already a significant community working on pandemic prevention, and there are many ways to contribute to this field. However, (while this is starting to change) most of the existing work is focused on conventional, naturally-caused pandemics, while we think what’s most important are catastrophic risks, especially those that might end civilization. These are much more likely to be deliberately caused, so involve issues more naturally covered by the defence & bioterrorism community than public health, and involve a different set of interventions. What’s more, we’ve estimated that most of the past funding for work on bioterrorism has focused on risks like anthrax, which can’t spread from person-to-person, and so don’t pose a catastrophic risk.

This means that despite significant existing work on pandemic prevention, “global catastrophic biological risks” remain highly neglected.

We rate biorisk as a less pressing issue than AI safety, mainly because we think biorisks are less likely to be existential, and AI seems more likely to play a key role in shaping the long-term future in other ways. This is why we rank this path lower than the AI paths above. However, it can easily be your top option if you have a comparative advantage in this path (e.g. a background in medicine).

To mitigate these risks, what’s most needed at a high-level is people able to develop strategy and policy proposals, and work with governments to aid their implementation. We call this path “biorisk strategy and policy research”. We can further divide this into an academic and a government path, as follows.

The main line of defence against these risks is government, but there are currently few people concerned with existential risks working there. So, we think it’s valuable to build up a community of experts in relevant areas of national government and intergovernmental organisations, such as the US Centers for Disease Control, European Centre for Disease Prevention and Control, FBI’s Weapons of Mass Destruction Directorate and the World Health Organization. You could also work in relevant think tanks, such as the Center for Health Security and Nuclear Threat Initiative. These experts can help to implement better policies when they’re known, and they can also help to improve policy proposals in the meantime. We list key organisations in our profile.

Another option is to work in academia. This involves developing a relevant area of expertise, such as synthetic biology, genetics, public health, epidemiology, international relations, security studies and political science. Note that it’s possible, and perhaps even beneficial, to start by studying a quantitative subject (sometimes even to graduate level), and then switch into biology later. Quantitative skills are in-demand in biology and give you better back-up options.

Once you’ve completed training, you could: do research on directly useful technical questions, such as how to create broad-spectrum diagnostics or rapidly deploy vaccines; do research on strategic questions, such as how dangerous technologies should be controlled; or you could advise policy-makers and other groups on the issues. A top research centre to aim to work at is the Center for International Security and Cooperation at Stanford.

As with AI strategy, global catastrophic biological risk is still a nascent field. This again can make it hard to contribute, since we don’t yet know which research questions are most important, and there is often a shortage of mentorship.

This means that there’s an especially pressing need for more “field building” or “disentanglement” research, with the aim of defining the field. If you might be able to do this kind of work, then your contribution is especially valuable since you can unlock the efforts of other researchers. The main home for most of this kind of research with a long-term focus right now is the Future of Humanity Institute in Oxford. There’s also a significant need for mentors who can help the next generation enter.

If you’re not able to contribute to the strategic research right now, then you can (i) try to identify more straightforward research questions that are relevant, (ii) work in more conventional biorisk organisations to build up expertise, (iii) focus on policy positions with the aim of building a community and expertise, (iv) become an expert on a relevant area of biology.

One advantage of biorisk is that many of the top positions seem somewhat less competitive than in AI, because they don’t require world-class quantitative skills.

Besides pandemic risks, we’re also interested in how to safely manage the introduction of other potentially transformative discoveries in biology, such as genetic engineering, which could be used to fundamentally alter human characteristics and values, or anti-ageing research. We see these issues as somewhat less pressing and neglected than engineered pandemics, but they provide another reason to develop expertise in these kinds of areas.

Often the way to enter this path is to pursue relevant graduate studies (such as in the subjects listed above) because this takes you along the academic path, and is also helpful in the policy path, where many positions require graduate study. Alternatively, you can try to directly enter relevant jobs in government, international organisations and non-profits, and build expertise on the job.

The main backup option from this path depends on what expertise you have, but one direction is other options in policy — it’s usually possible to switch your focus within a policy career. You could also work on adjacent research questions, such as those relevant to global health.

Unfortunately, overall the backup options often seem a little worse than AI safety, because qualifications in biology don’t open up as many options (many more people get biology PhDs than there are academic positions, leading to a high rate of drop out). This is another reason why we list this path lower. That said, you could still exit into the biotechnology or health industries to earn to give, and ultimately a wide range of other paths, such as non-profit careers.

Could this be a good fit for you?

  • Are you deeply convinced of the importance of reducing extinction risks?
  • Compared to similar roles in AI safety, this field doesn’t require equally strong quantitative skills.
  • In research, might you be capable of getting a PhD from a top 30 school in one of these areas? This isn’t required but is a good indicator. Read more about predicting success in research.
  • If focused on field building research, can you take on messy, ill-defined questions, and come up with reasonable assessments about them?
  • Are you able to be discreet about sensitive information concerning biodefence?
  • If focused on policy, might you be capable of getting and be satisfied in a relevant position in government? In policy, it’s useful to have relatively stronger social skills, such as being happy to speak to people all day, and being able to maintain a network. Policy careers also require patience to work with large bureaucracies and sometimes public scrutiny.
  • Do you already have experience in a relevant area of biology?

Key reading

  • Our profile on biorisk, which includes a list of recommended organisations and research centres (update in progress), and relevant podcasts, which are linked to from the profile.

Other reading

China will play a role in solving many of the biggest challenges of the next century, including emerging technologies and global catastrophic risks, and China is the most influential country in AI development after the US.

However, it often seems like there’s a lack of understanding and coordination between China and the West. For instance, even today, 3 times as many Americans study French as study Chinese. For this reason, we’d like to see more people (both Chinese and non-Chinese) develop a deep understanding of the intersection of effective altruism and China, and help to coordinate between the two countries.

In particular, we want people to learn about the aspects of China most relevant to our top problem areas, which means topics like artificial intelligence, international relations, pandemic response, bioengineering, political science, and so on. China is also crucial in ending factory farming, though we currently rate this as a lower priority at the margin.

More concretely, this could mean options like:

  • Graduate study in a relevant area, such as machine learning or synthetic biology; economics, international relations, or security studies with a focus on China or emerging technology; or Chinese language, history and politics.
  • If possible, a prestigious China-based fellowship, such as the Schwarzman Scholarship programme or Yenching Scholars, is a great option.
  • Research at a think tank or academic institute focused on these topics.
  • If from the West, learn Chinese in China, or find another option that lets you live there.
  • If Chinese, work at a Western effective altruism organisation.
  • Work at a Chinese technology company or philanthropic organisation.

Once you have this expertise, you could aim to contribute to AI and biorisk strategy & policy questions that involve China. You could also advise and assist Western organisations that want to coordinate with Chinese organisations. You might also directly work for Chinese organisations that are concerned with these problem areas.

Note that we’re not in favour of promoting effective altruism in China, working in the government, or attempting to fundraise from Chinese philanthropists. This could easily backfire if the message was poorly framed or if its intent was misperceived in China.

Rather, the aim of this path is to learn more about China, and then aim to improve cooperation between Western and Chinese groups. If you are considering doing outreach in China, get in touch and we can introduce you to people who can help you navigate the downsides.

To help with this priority path, we work with a part-time advisor who’s a specialist in China. You can get help from them here.

Could this be a good fit for you?

  • Do you already have knowledge of China? If not, could you see yourself becoming interested in Chinese politics, economy, culture, and so on, and also being involved in the effective altruism community?
  • Compared to other options on the list, this path requires more humanities skills (e.g. understanding other politics and cultures) rather than scientific skill-set.
  • Otherwise the skill-set required is fairly similar to the AI strategy and policy path earlier.

Further reading

If you can find a job with good fit, it usually seems possible to make a larger contribution to the problem areas we highlight by working directly in them rather than earning to give — so in this sense they’re more “talent constrained” than “funding constrained”.

However, additional funding is still useful, so earning to give is still a high-impact option. Earning to give can also be your top option if you have fit with an unusually well paid career. One path we’ve seen work especially well is quant trading in hedge funds and proprietary firms.

Quant trading means using algorithms to trade the financial markets for profit. We think, for the most part, that the direct impact of the work is likely neutral. But it might be the highest-paid career path out there.

Compensation starts around $100k-$300k per year, and can reach $1m per year within a couple of years at the best firms, and eventually over $10m per year if you make partner. We estimate that if you can enter the path at a good firm, the expected earnings are around $1m per year over a career. This is similar to being a tech startup founder, except that for startup founders who make it into a top accelerator, the deal is more like a 10% chance of getting $30m in 3-10 years, so the startup option involves much more risk and a major delay.

This means that the earnings in quant trading can enable huge donations to effective charities relatively quickly. We know several people in this path, such as Sam, who donated six figures within their first couple of years of work. This is enough to set you up as an “angel donor” in the community — you can try to identify promising new projects that larger donors can later scale up.

Many people also find the work surprisingly engaging. Creating a winning strategy is an intellectual challenge, but unlike academia, you get to work closely with a team and receive rapid feedback on how you’re performing. These firms often have “geeky” cultures, making them quite unlike the finance stereotype. The hours are often a reasonable 45-60h per week, rather than the 60-80h reported in entry-level investment banking.

These jobs are prestigious due to their earnings, and you can learn useful technical skills, as well as transferable skills like teamwork. The main downside on career capital is that these positions do not especially help you make connections, since you’ll mainly only work with others in your firm.

The other main downside of these jobs is that they’re highly competitive. There are more comments on fit below.

You may also find it surprisingly hard to set aside the time to figure out where is best to give, especially if you’re hoping to provide funding to early projects, working in unusual areas.

Role types and top firms

There are two broad pathways:

  • Traders – who develop and oversee the strategies.
  • Engineers – who create the systems to collect data, implement trades, and track performance.

It varies from firm to firm, but typically the engineers get paid less, but with more stability in their earnings.

It’s also important to know that the salaries can vary significantly by firm. There are perhaps only a couple of firms where it’s possible to progress to seven figures relatively quickly without a graduate degree, including Jane St, Hudson River Trading, DE Shaw, and maybe several others. The pay is often significantly less in other firms, though still often several hundred thousand dollars per year. Other firms also offer earnings on this level, but require a PhD.

In addition, note that quant trading positions are very different from “quant” jobs at other financial companies, such as investment banks or non-quantitative investment firms. Usually “quants” are “middle office” staff who provide analysis to the “front office” staff who oversee the key decisions. This makes them more stable but significantly lower paid, and sometimes less prestigious. Such firms also typically have a less geek-friendly culture.

Could this be a good fit for you?

  • One indication of potential is that you’d be capable of finishing in the top half of the class at a top 30 school in mathematics, theoretical physics or computer science at undergraduate level.
  • Another option is to enter based on your programming skills as an engineer. Might you be able to get a top software engineering job such as at Google?
  • Besides intelligence, the firms also look for good judgement and rapid decision-making skills. One indication of these is that you like playing strategy games or poker.
  • Would you be capable of reliably giving a large fraction of your income to charity?
  • Compared to academia, you need to have relatively better communication and team work skills, since you’ll work closely with your colleagues hour-by-hour in potentially stressful situations.

Further reading

Governments and other important institutions frequently have to make complex, high-stakes decisions based on the judgement calls of just a handful of people. There’s reason to believe that human judgements can be flawed in a number of ways, but can be substantially improved using more systematic processes and techniques. Improving the quality of decision-making in important institutions could improve our ability to solve almost all other problems.

We’d like to help form a new community of researchers and practitioners who develop and implement these techniques. We’re especially keen to help people who want to work on the areas of policy most relevant to global catastrophic risks, such as nuclear security, AI, and biorisk. Note that we’re not talking about the popular “nudge” work in behavioural sciences, which is focused on making small improvements to personal behaviours. Rather, we’re interested in neglected work relevant to high-stakes decisions like whether to go to war, such as Tetlock’s research into forecasting.

This path divides into two main options: (i) developing better decision-making techniques (ii) getting them applied in important organisations, especially those relevant to catastrophic risks.

To enter, the first step is to gain relevant expertise. This is most naturally done by getting a PhD in behavioural or decision science. However, you could also take a more practical route by starting your career in government and policy, and learning about the science on the side.

Once you have the expertise, you can either try to make progress on key research questions in the field, or work with an important organisation to improve their processes. We can introduce you to people working on this.

As with global priorities research, this is a nascent field that could become much bigger, and now is an exciting time to enter.

Could this be a good fit for you?

  • Might you be able to get a job in a relevant area of government? Do you know how to influence choices within a bureaucracy?
  • On the research path, might you be able to get into a psychology PhD at a top 30 school?
  • Might you have a shot at making a contribution to one of the relevant research questions? For instance, are you highly interested in the topic, and sometimes have ideas for questions to look into? Are you able to work independently for many days at a time? Are you able to stick with or lead a research project over many years? Read more about predicting success in research.

Further reading

Key resources:

Other resources:

If none of these fit, what to do?

Go back to the general 6-step process and five key categories we outlined earlier. In particular, go back to your list of problem areas, and consider whether there might be any relevant jobs in non-profits, government & policy, research, or using a particular skill you have. There should be options within at least one of these categories for any graduate.

In general, one option is to consider a wider range of problem areas. Another option is to stick with the same problem areas but contribute to a less pressing bottleneck. We don’t recommend compromising too much on personal fit.

One option that’s always there is to take any job you find satisfying and donate, or you can focus on building career capital that might let you take one of these jobs later.

If you already have skills, where to focus?

We’ve provided some advice on how to quickly narrow down the options we’ve considered based on your existing skills and experience. See this separate article.

Want to enter one of our priority paths?

If you’d like to try to have a big impact in one of these paths, our one-on-one advice might be able to help you choose an option and enter. As well as advice, we often know mentors, job opportunities and sources of funding in each path, and all our help is free. So, if you think you have a shot at entering one of them, let us know.

Apply now

Appendix – Outline of the reasoning behind the five key categories

In summary:

  • They do more to resolve the key bottlenecks facing our top recommended problem areas.
  • When leaders in the community are forced to trade funding against additional people in these roles, they prefer the people to the amount they could likely donate in earning to give.
  • These options let you learn the skills that are most needed in the top problems.
  • The additional flexibility of earning to give isn’t enough to offset the other downsides.

The main reason we highlight these paths is that the key bottlenecks to the problems we prioritise most often relate to research or policy change. In particular, the bottlenecks either involve technical fixes or more research to work out what the best solutions are, especially from a policy perspective. Otherwise, the key bottlenecks relate to community building. Either way, this means three broad types of positions are needed: those working in research or supporting it; those in relevant areas of government, and those in non-profits that work on research, policy or community building.

When we survey organisations working in our top problem areas, they tend to report that their fields are most constrained by people who are talented and aligned – those able to develop relevant expertise and work directly on solving the problem. This is especially true because the solutions to these problems are uncertain and require innovation.

In contrast, experts don’t usually say that additional funding — although it’s still useful — is the key bottleneck. This is because these areas are small, and there are already major donors willing to donate more than they currently do (read more). What most often holds back more funding is a lack of aligned and trusted people who are able to deploy it, and this means we need more people in non-profit or academic positions.

What’s more, even if funding was a key bottleneck, it often seems like you can raise more money by working as a fundraiser in a relevant organisation, doing advocacy for the issue rather than earning to give, or by simply making progress in the area, which has historically attracted more donors.

This prioritisation is reflected when we ask area experts to make quantitative tradeoffs. In our 2017 talent survey of the effective altruism community, we asked leaders at 17 key organisations to compare between donations and typical recent hires. (We also found similar results in our not yet published 2018 survey.)

In particular, we asked how organisations would have to be compensated in donations for their last ‘junior hire’ or ‘senior hire’ to disappear and not do valuable work for a 3 year period, yielding the following results:

 Average weighted by org sizeAverageMedian
Senior hire$12.8m ($4.1m excluding an outlier)$7.6m ($3.6m excluding an outlier)$1.0m
Junior hire$1.8m$1.2m$0.25m

It’s unclear exactly how to interpret these estimates, and we discuss some of their weaknesses in the full article about the survey. However, it certainly suggests that a talented person working directly on the issues can often have an impact equivalent to hundreds of thousands of dollars per year. This is more than most people could donate, which suggests that if you have good fit with one of these positions, they’re often higher-impact than earning to give.

The survey was mainly about jobs at the relevant non-profits rather than academic or government positions, but we expected the figures would be similar for these positions if someone was a good personal fit, and perhaps higher. (And this was borne out by our not-yet-published 2018 survey.)

We think the trade-off would hold for less competitive positions. For instance, if faced with the choice of someone being a software engineer and donating 10-20% of their salary per year (maybe $6-50k) or working in a relevant area of government, we think most in our survey would prefer the person in government.

Another reason why we highlight these three paths is that they seem to resolve the key skill gaps reported in the survey. We asked leaders in the community about which skills are most needed in the community as a whole, they fell into the following broad categories:

  1. Government and policy experts.
  2. Management & operations & administration.
  3. Generalist researchers.
  4. Experts in machine learning, the life sciences, economics.
  5. Movement builders.

The most direct way to gain these skills and fill the gaps is probably to take the three paths we highlight: do relevant graduate study and research; work in government & policy; or work in a non-profit where you can learn about these issues and learn these skills.

The other highly rated skill in the survey was “good judgement” (phrased as “good calibration, wide knowledge, and ability to work out what’s important”) and again, if you want to develop good judgement about issues concerning the world’s most pressing problems, then it seems like usually the best option is probably just to start working on those problems. You can improve your judgement as you go by tracking your predictions and following the other steps listed here. (Though there might be some exceptions, such as jobs in financial trading, which require making lots of judgement calls and give you feedback on them.)

The main exception on skill building is that in the right corporate sector job you could also develop relevant skills in management, operations, administration, some types of research, machine learning, marketing, and so on. So, if you were to earn to give, we’d encourage you to do in a job that simultaneously lets you develop one of these skills, to give you the option of filling a skill gap later. This might mean something like working at a tech startup on operations, or doing ML research at Google.

The main counterpoint in favour of earning to give is likely its flexibility over time: it’s easier to redirect your donations to a different problem area than to change where you work. This remains a good reason to earn to give over the other paths, especially for people who are especially uncertain about which problem area to focus on.

However, we think we might have overstated the differences in flexibility in the past. One key point is that some of the other options we highlight also often provide significant flexibility about problem areas. For instance, in government and policy careers, it’s relatively easy to switch which area of policy-making you work on. Likewise, if you gain transferable skills like management and outreach, then you can apply them to different problem areas over time. It’s also possible to design organisations that can flexibly support different areas, such as 80,000 Hours or Founders Pledge.

You gain even more flexibility about jobs to pursue within an area, because you’ll gain expertise and connections relative to the problem, which will mean you’ll have plenty of opportunities in the future. People who earn to give, however, often end up with skills that aren’t especially relevant to the top problem areas.

We also value flexibility a little less than we did in the past. In part, this is because we think our top problems are urgent, so it’s important to prioritise immediate impact compared to putting yourself in a good position decades in the future. We also feel more confident in our choice of problem areas, since our ranking has stabilised in the last three years. Finally, when you’re working with a community, it seems like there are greater reasons to specialise. We’ll cover these factors in upcoming articles.

Raising awareness to a mass audience doesn’t seem like a key bottleneck for many of our top problems for a couple of reasons:

  1. Often the most relevant people are already aware of the issues we highlight. For instance, AI safety has had plenty of press coverage recently, so almost all AI scientists and many relevant policy makers are aware of it. The bigger bottleneck is having the right people take the ideas seriously, and this requires in-depth engagement, rather than mass advocacy.
  2. Advocacy can easily backfire, since if you present an idea in a bad way, that can ruin first impressions, create misunderstandings that are hard to reverse, and make progress harder in the future. Read more.
  3. The problem areas we’re focused on are complex, uncertain and mainly in need of innovation, so we often don’t have simple actions to promote. This makes it difficult for people who are superficially involved to contribute, which makes mass advocacy less useful. Instead, we need to develop better proposals for action, and this means working directly in research, non-profits or policy.

We are keen to see more targeted advocacy around top problem areas, and “movement building” was listed as a key skill bottleneck, but this is very difficult to do well, so we don’t highlight it as one of the top five categories. If you’re interested in doing targeted advocacy, then we’d suggest first working in one of the other paths to build up expertise, especially in relevant non-profits or academia.

Entrepreneurship is similar to targeted advocacy — we’d like to see more, but it’s hard to do well, so we don’t highlight it as a top five category. Again, we think most people should first focus on building up expertise in an area so that they might spot a good idea down the line, and then be more likely to succeed in their venture, while being less likely to accidentally cause harm by setting up a second tier project. You could help to scale up an existing project, which are often bottlenecked by management capacity.

We’re open to for-profit work as a strategy for social impact, especially due to its scalability.

However, given our current choice of problem areas, we find it hard to identify for-profit ways to directly contribute. As mentioned, most of the key bottlenecks relate to theoretical research and policy change, and it’s hard for for-profits to contribute to this.

This is not surprising. We think the most pressing global issues relate to improving the long-term future, and issues that affect the long-term future are severe market failures (because the people who are affected don’t exist yet) and this makes it very hard to use for-profit methods to make progress.

There can sometimes be exceptions. For instance, it’s possible to create a for-profit mechanism for funding research. This is what Elon Musk has tried to do with SpaceX, which sells satellite launches in order to fund research relevant to space colonisation in general. Similarly, DeepMind has so much money that it’s able to fund AI strategy research, even though this doesn’t have any immediate relevance to its bottom line (though the incentives are not always purely focused on impact). This can mean sometimes it’s high-impact to work in a for-profit company, even if the for-profit isn’t directly solving the problem. You could also try to set up a new for-profit company like this, which is an earning to give style strategy (though potentially better since you build a network and credibility that’s more relevant to the key problem).

The area where for-profits seem most likely to be able to play a directly relevant role is in biorisk reduction. For instance, for-profit companies could develop cheap diagnostics, to enable to early detection of new diseases. That said, these kinds of for-profit strategies usually come at a significant cost of being less focused on the most crucial issues within an area.

Notes and references

  1. It's helpful to focus your career on a broad 'area' rather than a narrow intervention as a way to deal with uncertainty. Over time, which intervention seems best is likely to change, so if you aim at an overly narrow intervention that turns out to not be promising, then you could waste a lot of effort. Instead, it's useful to focus your efforts on a "cluster" of promising but similar interventions. This means that if one intervention doesn't work out, there are plenty of others you can switch into. It also means that the expertise you accumulate will be relevant to lots of relevant interventions, giving you better options in the future.

    We've become convinced this approach is useful through our work with people over the years, where narrowing down in terms of "areas" often seems like one of the most productive steps of our advice. We've also seen it productively applied elsewhere in the effective altruism community, especially at the Open Philanthropy Project.

    This means that the definition of a "problem area" is a pragmatic one. We define problem areas as whatever clusters of promising interventions are useful for career planning. Similarly, Open Phil defines them as the clusters that are useful to treat as a group for the purpose of grantmaking.

    Read more here
  2. Though if we consider your options more broadly, in the worst you’ll have near zero impact, so the ratio could be arbitrarily large. Likewise, in the worst case you'll have negative impact which would make the ratio negative.
  3. Though, this tradeoff doesn't usually include the opportunity cost of the donations i.e. the value of what else could have been done with the the money.

    So, if the organisation says a new hire is worth $100,000, it doesn't follow that a donor should be willing to pay this much for a new hire, since they might have an even better use of the money.
  4. Ideally experts who are sympathetic with the idea of comparing paths in terms of impact, and have some sense of the distribution of opportunities and resources in the area, and are well calibrated.

    We discuss this in our podcast with Prof Tetlock
  5. Large for-profits can fund relevant research, such as Google, and often these can be good places to work, but this is more like an example of earning to give than a direct contribution by a for-profit.
  6. For arguments that transformative AI may have an outsized impact on humanity’s long-term trajectory see this post by the Open Philanthropy Project, one of our funders. A longer discussion can be found in Superintelligence by Prof. Nick Bostrom (we are affiliated with Prof. Bostrom’s Future of Humanity Institute at the University of Oxford). For a popular introduction to this issue, see this article, and some caveats/corrections here

    For discussion of timelines of transformative AI see ‘what do we know about AI timelines?’ again by the Open Philanthropy Project, one of our funders. A survey of AI expert opinion also appears to support this claim. However it is unclear how we should weight the expertise of AI research when predicting technological trends in AI, as discussed in our podcast with survey lead author Katja Grace.

    Archived links retrieved 18-Sep-2018.