New releases

#47 – PhD or programming? Fast paths into aligning AI as a machine learning engineer, according to ML engineers Catherine Olsson & Daniel Ziegler

Podcast by and · Published November 2nd, 2018

After dropping out of his ML PhD at Stanford, Daniel Ziegler needed to decide what to do next. He’d always enjoyed building stuff and wanted to help shape the development of AI, so he thought a research engineering position at an org dedicated to aligning AI with human interests could be his best option.

He decided to apply to OpenAI, spent 6 weeks preparing for the interview, and actually landed the job. His PhD, by contrast, might have taken 6 years. Daniel thinks this highly accelerated career path may be possible for many others.

On today’s episode Daniel is joined by Catherine Olsson, who has also worked at OpenAI, and left her computational neuroscience PhD to become a research engineer at Google Brain. They share this piece of advice for those interested in this career path: just dive in. If you’re trying to get good at something, just start doing that thing, and figure out that way what’s necessary to be able to do it well.

To go with this episode, Catherine has even written a simple step-by-step guide to help others copy her and Daniel’s success.

Daniel thinks the key for him was nailing the job interview.

OpenAI needed him to be able to demonstrate the ability to do the kind of stuff he’d be working on day-to-day. So his approach was to take a list of 50 key deep reinforcement learning papers, read one or two a day, and pick a handful to actually reproduce. He spent a bunch of time coding in Python and TensorFlow, sometimes 12 hours a day, trying to debug and tune things until they were actually working.

Continue reading →

ML engineering for AI safety & robustness: a Google Brain engineer’s guide to entering the field

Article by and · Published November 2018

Technical AI safety is a multifaceted area of research, with many sub-questions in areas such as reward learning, robustness, and interpretability. These will all need to be answered in order to make sure AI development will go well for humanity as systems become more and more powerful.

Not all of these questions are best tackled with abstract mathematics research; some can be approached with concrete coding experiments and machine learning (ML) prototypes. As a result, some AI safety research teams are looking to hire a growing number of Software Engineers and ML Research Engineers.

Additionally, some research teams that may not think of themselves as focussed on ‘AI Safety’ per se, nonetheless work on related problems like verification of neural nets or learning from human feedback, and are often hiring engineers.

Note that this guide was written in November 2018 to complement an in-depth conversation on the 80,000 Hours Podcast with Catherine Olsson and Daniel Ziegler on how to transition from computer science and software engineering in general into ML engineering, with a focus on alignment and safety. If you like this guide, we’d strongly encourage you to check out the podcast episode where we discuss some of the instructions here, and other relevant advice.

What are the necessary qualifications for these positions?

Software Engineering: Some engineering roles on AI safety teams do not require ML experience. You might already be prepared to apply to these positions if you have the following qualifications:

  • BSc/BEng degree in computer science or another technical field (or comparable experience)
  • Strong knowledge of software engineering (as a benchmark: could pass a Google software engineering interview)
  • Interest in working on AI safety
  • (usually) Willingness to move to London or the San Francisco Bay Area

If you’re a software engineer with an interest in these roles,

Continue reading →

Second October job board update

Blog post by · Published October 24th, 2018

Our job board now lists 142 vacancies, with 38 additional opportunities since the last update 3 weeks ago.

If you’re actively looking for a new role, we recommend checking out the job board regularly – when a great opening comes up, you’ll want to maximise your preparation time.

The job board remains a curated list of the most promising positions to apply for that we’re currently aware of. They’re all high-impact opportunities at organisations that are working on some of the world’s most pressing problems:

Check out the job board →

They’re demanding positions, but if you’re a good fit for one of them, it could be your best opportunity to have an impact.

If you apply for one of these jobs, or intend to, please do let us know.

A few highlights from the last month

Continue reading →

New article: Have a particular strength? Already an expert in a field? Here are the socially impactful careers 80,000 Hours suggests you consider first.

Blog post by · Published October 24th, 2018

We’ve published a new article that summarises our advice based on your strengths and link you to the most relevant articles for you to read:

This list is preliminary. We wanted to publish our existing thoughts on what to do with each skill, but can easily see ourselves changing our minds over the coming years.

You can read about our general process and what career paths we recommend in our full article.

Sometimes, however, it’s possible to give more specific advice about what options to consider to people who already have pre-existing experience or qualifications, or are unusually good at a certain type of work.

In this article, we provide a list of skills, and for each one give a list of socially-impactful options that people who are unusually good in that area should most often consider.

We start with three “strengths” (quantitative, verbal & social, and visual). Then we go on to give advice for people with existing experience in fifteen specific fields.

Bear in mind it’s often possible to completely change field: we’ve seen people switch from philosophy to software engineering, and architecture into economics. Nonetheless, these are good starting points.

The skill types also overlap, and you probably also have several of them. The aim is just to give you some tips on narrowing down your options more quickly.

Continue reading →

#46 – Philosopher Hilary Greaves on moral cluelessness, population ethics, probability within a multiverse, & harnessing the brainpower of academia to tackle the most important research questions

Podcast by and · Published October 23rd, 2018

The barista gives you your coffee and change, and you walk away from the busy line. But you suddenly realise she gave you $1 less than she should have. Do you brush your way past the people now waiting, or just accept this as a dollar you’re never getting back? According to philosophy professor Hilary Greaves – Director of Oxford University’s Global Priorities Institute, which is hiring now – this simple decision will completely change the long-term future by altering the identities of almost all future generations.

How? Because by rushing back to the counter, you slightly change the timing of everything else people in line do during that day — including changing the timing of the interactions they have with everyone else. Eventually these causal links will reach someone who was going to conceive a child.

By causing a child to be conceived a few fractions of a second earlier or later, you change the sperm that fertilizes their egg, resulting in a totally different person. So asking for that $1 has now made the difference between all the things that this actual child will do in their life, and all the things that the merely possible child – who didn’t exist because of what you did – would have done if you decided not to worry about it.

As that child’s actions ripple out to everyone else who conceives down the generations, ultimately the entire human population will become different, all for the sake of your dollar. Will your choice cause a future Hitler to be born, or not to be born? Probably both!

Continue reading →

New article: Ways people trying to do good accidentally make things worse, and how to avoid them

Blog post by · Published October 22nd, 2018

We’ve published a new article about how to avoid accidentally causing harm through your career:

“We encourage people to work on problems that are neglected by others and large in scale. Unfortunately those are precisely the problems where people can do the most damage if their approach isn’t carefully thought through.

If a problem is very important, then setting back the cause is very bad. If a problem is so neglected that you’re among the first focused on it, then you’ll have a disproportionate influence on the field’s reputation, how likely others are to enter it, and many early decisions that could have path-dependent effects on the field’s long-term success.

We don’t particularly enjoy writing about this admittedly demotivating topic. Ironically, we expect that cautious people – the folks who least need this advice – will be the ones most likely to take it to heart.

Nonetheless we think cataloguing these risks is important if we’re going to be serious about having an impact in important but ‘fragile’ fields like reducing extinction risk.

In this article, we’ll list six ways people can unintentionally set back their cause. You may already be aware of most of these risks, but we often see people neglect one or two of them when new to a high stakes area – including us when we were starting 80,000 Hours.”

Read the full article…

Continue reading →

#45 – Economist Tyler Cowen says our overwhelming priorities should be maximising economic growth and making civilisation more stable. Is he right?

Podcast by and · Published October 17th, 2018

I’ve probably spent more time reading Tyler Cowen – Professor of Economics at George Mason University – than any other author. Indeed it’s his incredibly popular blog Marginal Revolution that prompted me to study economics in the first place. Having spent thousands of hours absorbing Tyler’s work, it was a pleasure to be able to question him about his latest book and personal manifesto: Stubborn Attachments: A Vision for a Society of Free, Prosperous, and Responsible Individuals.

Tyler makes the case that, despite what you may have heard, we can make rational judgments about what is best for society as a whole. He argues:

  1. Our top moral priority should be preserving and improving humanity’s long-term future
  2. The way to do that is to maximise the rate of sustainable economic growth
  3. We should respect human rights and follow general principles while doing so.

We discuss why Tyler believes all these things, and I push back where I disagree. In particular: is higher economic growth actually an effective way to safeguard humanity’s future, or should our focus really be elsewhere?

In the process we touch on many of moral philosophy’s most pressing questions: Should we discount the future? How should we aggregate welfare across people? Should we follow rules or evaluate every situation individually? How should we deal with the massive uncertainty about the effects of our actions? And should we trust common sense morality or follow structured theories?

After covering the book, the conversation ranges far and wide. Will we leave the galaxy, and is it a tragedy if we don’t? Is a multi-polar world less stable? Will humanity ever help wild animals? Why do we both agree that Kant and Rawls are overrated?

Continue reading →

What skills or experience are most needed within professional effective altruism in 2018? And which problems are most effective to work on? New survey of organisational leaders.

Blog post by and · Published October 8th, 2018

Read this to see the 2019 data.

Update April 2019: We think that our use of the term ‘talent gaps’ in this post (and elsewhere) has caused some confusion. We’ve written a post clarifying what we meant by the term and addressing some misconceptions that our use of it may have caused. Most importantly, we now think it’s much more useful to talk about specific skills and abilities that are important constraints on particular problems rather than talking about ‘talent constraints’ in general terms. This page may be misleading if it’s not read in conjunction with our clarifications.

What are the most pressing needs in the effective altruism community right now? What problems are most effective to work on? Who should earn to give and who should do direct work? We surveyed managers at organisations in the community to find out their views. These results help to inform our recommendations about the highest impact career paths available.

Our key finding is that for the questions that we asked 12 months ago, the results have not changed very much. This gives us more confidence in our survey results from 2017.

We also asked some new questions, including about the monetary value placed on our priority paths, discount rates on talent and how current leaders first discovered and got involved in effective altruism.

Continue reading →

New career review on becoming an academic researcher: Highlights on your chances of success, which fields have highest GRE scores, & having impact outside research

Blog post by · Published October 4th, 2018

We recently published a new career review on becoming an academic researcher by Jess Whittlestone. It covers issues such as:

  • Entry requirements and what it takes to excel.
  • What are your chances of success?
  • How to maximise your impact within academia.
  • How to assess your personal fit at each stage of your career.
  • Which field are best to enter?
  • How to establish your career early on, and trade-off impact against career advancement.
  • Review of the pros and cons of the path.

Check out our career review of academic research →

Here are some extracts from the full profile.

—

Research isn’t the only way academics can have a large impact

When we think of academic careers, research is what first comes to mind, but academics have many other pathways to impact which are less often considered. Academics can also influence public opinion, advise policy-makers, or manage teams of other researchers to help them be more productive.

If any of these routes might turn out to be a good fit for you, then it makes the path even more attractive. We’ll sketch out some of these other paths:

1. Public outreach

Peter Singer’s career began in an ordinary enough way for a promising young academic, studying philosophy at Oxford University. But he soon started moving in a different direction from his peers,

Continue reading →

October job board update

Blog post by · Published October 3rd, 2018

Our job board now lists 128 vacancies, with 45 additional opportunities since last month.

If you’re actively looking for a new role, we recommend checking out the job board regularly – when a great opening comes up, you’ll want to maximise your preparation time.

The job board remains a curated list of the most promising positions to apply for that we’re currently aware of. They’re all high-impact opportunities at organisations that are working on some of the world’s most pressing problems:

Check out the job board →

They’re demanding positions, but if you’re a good fit for one of them, it could be your best opportunity to have an impact.

If you apply for one of these jobs, or intend to, please do let us know.

A few highlights from the last month

Continue reading →

#44 – Paul Christiano on how OpenAI is developing real solutions to the ‘AI alignment problem’, and his vision of how humanity will progressively hand over decision-making to AI systems

Podcast by and · Published October 2nd, 2018

Paul Christiano is one of the smartest people I know and this episode has one of the best explanations for why AI alignment matters and how we might solve it. After our first session produced such great material, we decided to do a second recording, resulting in our longest interview so far. While challenging at times I can strongly recommend listening – Paul works on AI himself and has a very unusually thought through view of how it will change the world. Even though I’m familiar with Paul’s writing I felt I was learning a great deal and am now in a better position to make a difference to the world.

A few of the topics we cover are:

  • Why Paul expects AI to transform the world gradually rather than explosively and what that would look like
  • Several concrete methods OpenAI is trying to develop to ensure AI systems do what we want even if they become more competent than us
  • Why AI systems will probably be granted legal and property rights
  • How an advanced AI that doesn’t share human goals could still have moral value
  • Why machine learning might take over science research from humans before it can do most other tasks
  • Which decade we should expect human labour to become obsolete, and how this should affect your savings plan.

—

Here’s a situation we all regularly confront: you want to answer a difficult question, but aren’t quite smart or informed enough to figure it out for yourself. The good news is you have access to experts who are smart enough to figure it out. The bad news is that they disagree.

Continue reading →

List of 80,000 Hours content from the last 4 months, summary of what was most popular, and plans for future releases.

Blog post by · Published October 1st, 2018

Cross posted from the Effective Altruism Forum.

Here’s your regular reminder of everything 80,000 Hours has released over the last four months, since our last roundup. If you’d like to get these updates more regularly, you can join our newsletter.

We’ve done a major redesign of our job board, increasing the number of vacancies listed there from ~20 to over 100. It has doubled its traffic since the first half of the year, and is now one of the five most popular pages on the whole site.

  1. High impact job board

We’ve released two in-depth articles that should be of special interest to the community:

  1. These are the world’s highest impact career paths according to our research. This is an update of our top recommended careers.
  2. Should you play to your comparative advantage when choosing your career? New theoretical content on the relevance of comparative advantage, and thoughts on how to practically evaluate it.

We released 9 podcast episodes totalling 19.5 hours, covering lots of key topics in EA in significant depth (in chronological order):

  1. How the audacity to fix things without asking permission can change the world, demonstrated by Tara Mac Aulay
  2. Tanya Singh on ending the operations management bottleneck in effective altruism
  3. Finding the best charity requires estimating the unknowable.

Continue reading →

Recent research we’ve published: Our top 10 careers for social impact; Congressional staffing; Comparative advantage; And can you guess which psychology experiments will replicate?

Blog post by · Published September 30th, 2018

We recently published a number of new articles that you might have missed if you don’t follow us on social media (Facebook and Twitter) or our research newsletter.

Probably our most important release for this year is this article summarising many of our key findings since we started in 2011:

It outlines our new suggested process anyone can use to generate a short-list of high-impact career options given their personal situation.

It then describes the top five key categories of career we most often recommend, which should produce at least one good option for almost all graduates, and why we’re enthusiastic about them.

It goes on to list and explains the top 10 “priority paths” we want to draw attention to, because we think they can enable to right person to do a particularly large amount of good for the world.

Second, if you’re trying to figure out which job is the best fit for you, or how to coordinate with other people – for example the effective altruism community – you will want to read:

Third, if you’d like to influence government or work in politics, you should check out our comprehensive review of the pros and cons of being a Congressional Staffer and how to become one:

Continue reading →

#43 – Daniel Ellsberg on the creation of nuclear doomsday machines, the institutional insanity that maintains them, and a practical plan for dismantling them

Podcast by and · Published September 24th, 2018

In Stanley Kubrick’s iconic film Dr. Strangelove, the American president is informed that the Soviet Union has created a secret deterrence system which will automatically wipe out humanity upon detection of a single nuclear explosion in Russia. With US bombs heading towards the USSR and unable to be recalled, Dr Strangelove points out that “the whole point of this Doomsday Machine is lost if you keep it a secret – why didn’t you tell the world, eh?” The Soviet ambassador replies that it was to be announced at the Party Congress the following Monday: “The Premier loves surprises”.

Daniel Ellsberg – leaker of the Pentagon Papers which helped end the Vietnam War and Nixon presidency – claims in his new book The Doomsday Machine: Confessions of a Nuclear War Planner that Dr. Strangelove might as well be a documentary. After attending the film in Washington DC in 1964, he and a military colleague wondered how so many details of the nuclear systems they were constructing had managed to leak to the filmmakers.

The USSR did in fact develop a doomsday machine, Dead Hand, which probably remains active today.

If the system can’t contact military leaders, it checks for signs of a nuclear strike. Should its computers determine that an attack occurred, it would automatically launch all remaining Soviet weapons at targets across the northern hemisphere.

As in the film, the Soviet Union long kept Dead Hand completely secret, eliminating any strategic benefit, and rendering it a pointless menace to humanity.

Continue reading →

American with a science PhD? Get a fast-track into AI and STEM policy by applying for the acclaimed AAAS Science & Technology Fellowship by Nov 1.

Blog post by · Published September 18th, 2018

Within just four years of finishing her PhD in biophysics, Jessica Tuchman Mathews was Director of Global Issues for President Carter’s National Security Council. In her first year in the role she helped put together a nuclear non-proliferation pact among 15 countries including the US and the Soviet Union.

Later in her career, Jessica served as Deputy to the Undersecretary of State for Global Affairs, wrote a weekly column for the Washington Post, and most recently served as President of the Carnegie Endowment for International Peace, an influential Washington-based foreign policy think tank.

What launched such an successful career? In our conversation with Jessica, she argued it was the AAAS Science & Technology (S&T) Policy Fellowship. Jessica was selected as one of their inaugural fellows in 1973.

In this article we argue that for eligible people interested in our top recommended problem areas and S&T policy careers the AAAS S&T Policy Fellowship is a valuable springboard that could rapidly advance your career as it did for Jessica.

The opportunity

At 80,000 Hours, we think the AAAS Policy Fellowship is one of the best routes into the US Government for people with a STEM or social science PhD, or an engineering masters and three years of industry experience.

Policy fellows work within the US Government for one year in policy-related roles relevant to science and technology. Nearly 300 fellows are accepted each year,

Continue reading →

#42 – Amanda Askell on tackling the ethics of infinity, being clueless about the effects of our actions, and having moral empathy for intellectual adversaries

Podcast by and · Published September 11th, 2018

Consider two familiar moments at a family reunion.

Our host, Uncle Bill, is taking pride in his barbequing skills. But his niece Becky says that she now refuses to eat meat. A groan goes round the table; the family mostly think of this as an annoying picky preference. But were it viewed as a moral position rather than personal preference – as they might if instead Becky were avoiding meat on religious grounds – it would usually receive a very different reaction.

An hour later Bill expresses a strong objection to abortion. Again, a groan goes round the table: the family mostly think that he has no business in trying to foist his regressive preferences on other people’s personal lives. But if considered not as a matter of personal taste, but rather as a moral position – that Bill genuinely believes he’s opposing mass-murder – his comment might start a serious conversation.

Amanda Askell, who recently completed a PhD in philosophy at NYU focused on the ethics of infinity, thinks that we often betray a complete lack of moral empathy. Across the political spectrum, we’re unable to get inside the mindset of people who expresses views that we disagree with, and see the issue from their point of view.

A common cause of conflict, as above, is confusion between personal preferences and moral positions. Assuming good faith on the part of the person you disagree with, and actually engaging with the beliefs they claim to hold, is perhaps the best remedy for our inability to make progress on controversial issues.

Continue reading →

Have a particular strength? Already an expert in a field? Here are the socially impactful careers 80,000 Hours suggests you consider first.

Article by · Published October 2018

This list is preliminary. We wanted to publish our existing thoughts on what to do with each skill, but can easily see ourselves changing our minds over the coming years.

You can read about our general process and what career paths we recommend in our full article.

Sometimes, however, it’s possible to give more specific advice about what options to consider to people who already have pre-existing experience or qualifications, or are unusually good at a certain type of work.

In this article, we provide a list of skills, and for each one give a list of socially-impactful options that people who are unusually good in that area should most often consider.

We start with three “strengths” (quantitative, verbal & social, and visual). Then we go on to give advice for people with existing experience in fifteen specific fields.

Bear in mind it’s often possible to completely change field: we’ve seen people switch from philosophy to software engineering, and architecture into economics. Nonetheless, these are good starting points.

The skill types also overlap, and you probably also have several of them. The aim is just to give you some tips on narrowing down your options more quickly.

This article assumes you’re already familiar with our problem profiles and top careers. It just summarises our bottom lines on how to narrow down, without giving all our reasoning.

Continue reading →

#41 – If the US put fewer people in prison, would crime go up? Not at all, according to Open Philanthropy’s renowned researcher David Roodman.

Podcast by and · Published August 28th, 2018

With 698 inmates per 100,000 citizens, the U.S. is the world’s leader in incarcerating people. But what effect does this actually have on crime?

According to David Roodman, Senior Advisor to Open Philanthropy, the marginal effect is zero.

This stunning rebuke to the American criminal justice system comes from the man Holden Karnofsky called “the gold standard for in-depth quantitative research”. His other investigations include the risk of geomagnetic storms, whether deworming improves health and test scores, and the development impacts of microfinance – all of which we also cover in this episode.

In his comprehensive review of the evidence, David says the effects of crime can be split into three categories; before, during, and after.

Does having tougher sentences deter people from committing crime? After reviewing studies on gun laws and ‘three strikes’ in California, David concluded that the effect of deterrence is zero.

Does imprisoning more people reduce crime by incapacitating potential offenders? Here he says yes, noting that crimes like motor vehicle theft have gone up in a way that seems pretty clearly connected with recent Californian criminal justice reforms (though the effect on violent crime is far lower).

Finally, do the after-effects of prison make you more or less likely to commit future crimes?

This one is more complicated.

His literature review suggested that more time in prison made people substantially more likely to commit future crimes when released. But concerned that he was biased towards a comfortable position against incarceration, David did a cost-benefit analysis using both his favoured reading of the evidence and the devil’s advocate view; that there is deterrence and that the after-effects are beneficial.

Continue reading →

Randomised experiment: If you’re genuinely unsure whether to quit your job or break up, then you probably should

Blog post by · Published August 25th, 2018

One of my favourite studies ever is ‘Heads or Tails: The Impact of a Coin Toss on Major Life Decisions and Subsequent Happiness’ by economist Steven Levitt of ‘Freakonomics’.

Levitt collected tens of thousands of people who were deeply unsure whether to make a big change in their life. After offering some advice on how to make hard choices, those who remained truly undecided were given the chance to use a flip of a coin to settle the issue. 22,500 did so. Levitt then followed up two and six months later to ask people whether they had actually made the change, and how happy they were out of 10.

People who faced an important decision and got heads – which indicated they should quit, break up, propose, or otherwise mix things up – were 11 percentage points more likely to do so.

It’s very rare to get a convincing experiment that can help us answer as general and practical a question as ‘if you’re undecided, should you change your life?’ But this experiment can!

I wish there were much more social science like this, for example, to figure out whether or not people should explore a wider variety of different jobs during their career (for more on that one see our articles on how to find the right career for you and what job characteristics really make people happy).

The widely reported headline result was that people who made a change in their life as a result of the coin flip were 0.48 points happier out of 10,

Continue reading →

#40 – How well can we actually predict the future? Katja Grace on why expert opinion isn’t a great guide to AI’s impact and how to do better

Podcast by and · Published August 21st, 2018

Experts believe that artificial intelligence will be better than humans at driving trucks by 2027, working in retail by 2031, writing bestselling books by 2049, and working as surgeons by 2053. But how seriously should we take these predictions?

Katja Grace, lead author of ‘When Will AI Exceed Human Performance?’, thinks we should treat such guesses as only weak evidence. But she also says there might be much better ways to forecast transformative technology, and that anticipating such advances could be one of our most important projects.

Note: Katja’s organisation AI Impacts is currently hiring part- and full-time researchers.

There’s often pessimism around making accurate predictions in general, and some areas of artificial intelligence might be particularly difficult to forecast.

But there are also many things we’re now able to predict confidently — like the climate of Oxford in five years — that we no longer give ourselves much credit for.

Some aspects of transformative technologies could fall into this category. And these easier predictions could give us some structure on which to base the more complicated ones.

One controversial debate surrounds the idea of an intelligence explosion; how likely is it that there will be a sudden jump in AI capability?

And one way to tackle this is to investigate a more concrete question: what’s the base rate of any technology having a big discontinuity?

A significant historical example was the development of nuclear weapons. Over thousands of years, the energy density of explosives didn’t increase by much. Then within a few years, it got thousands of times better. Discovering what leads to such anomalies may allow us to better predict the possibility of a similar jump in AI capabilities.

Continue reading →