How to pursue a career in research to lower the risks from superintelligent machines: a new career review.

google-deepmind-artificial-intelligence

This is a summary of our full career review on artificial intelligence risk research.

Have you read the profile and think you want to contribute to artificial intelligence risk research? Fill out this form and we’ll see if we can help.

Many people we coach are interested in doing research into artificial intelligence (AI), in particular how to lower the risk that superintelligent machines do harmful things not intended by their creators – a field usually referred to as ‘AI risk research’. The reasons people believe this is a particularly pressing area of research are outlined in sources such as:

Our goal with this career review was not to assess the cause area of AI risk research – on that we defer to the authors above. Rather we wanted to present some concrete guidance for the growing number of people who want to work on the problem.

We spoke to the leaders in the field, including top academics, the head of MIRI and managers in AI companies, and the key findings are:

  • Some organisations working on this problem,

Continue reading →

Which cause is most effective?

In previous posts, we explained what causes are and presented a method for assessing them in terms of expected effectiveness.

In this post, we apply this method to identify a list of causes that we think represent some particularly promising opportunities for having a social impact in your career (though there are many others we don’t cover!).

We’d like to emphasise that these are just informed guesses over which there’s disagreement. We don’t expect the results to be highly robust. However, you have to choose something to work on, so we think it’ll be useful to share our guesses to give you ideas and so we can get feedback on our reasoning – we’ve certainly had lots of requests to do so. In the future, we’d like more people to independently apply the methodology to a wider range of causes and do more research into the biggest uncertainties.

The following is intended to be a list of some of the most effective causes in general to work on, based on broad human values. Which cause is most effective for an individual to work on also depends on what resources they have (money, skills, experience), their comparative advantages and how motivated they are. This list is just intended as a starting point, which needs to be combined with individual considerations. An individual’s list may differ due also to differences in values. After we present the list, we go over some of the key assumptions we made and how these assumptions affect the rankings.

We intend to update the list significantly over time as more research is done into these issues. Fortunately, more and more cause prioritisation research is being done, so we’re optimistic our answers will become more solid over the next couple of years. This also means we think it’s highly important to stay flexible, build career capital, and keep your options open.

In the rest of this post we:
1. Provide a summary list of high priority causes
2. Explain what each cause is and overview our reasons for including it
3. Explain how key judgement calls alter the ranking
4. Overview how we came up with the list and how we’ll take it forward
5. Answer other common questions

Continue reading →

Influencing the Far Future

future_generations

Introduction

In an earlier post we reviewed the arguments in favor of the idea that we should primarily assess causes in terms of whether they help build a society that’s likely to survive and flourish in the very long-term. We think this is a plausible position, but it raises the question: what activities in fact do help improve the world over the very long term, and of those, which are best? We’ve been asked this question several times in recent case studies.

First, we propose a very broad categorisation of how our actions today might affect the long-run future.

Second, as a first step to prioritising different methods, we compiled a list of approaches to improve the long-run future that are currently popular among the community of people who explicitly believe the long-run future is important.

The list was compiled from our knowledge of the community. Please let us know if you think there are other important types of approach that have been neglected. Further, note that this post is not meant as an endorsement of any particular approach; just an acknowledgement that it has significant support.

Third, we comment on how existing mainstream philanthropy may or may not influence the far future.

Continue reading →

High impact interview 1: Existential risk research at SIAI

The plan: to conduct a series of interviews with successful workers in various key candidates for high impact careers.

The first person to agree to an interview is Luke Muehlhauser (aka lukeprog of Less Wrong), the executive director of the Singularity Institute for Artificial Intelligence, whose mission is to influence the development of greater-than-human intelligence to try and ensure that it’s a force for human flourishing rather than extinction.

Continue reading →