OpenAI can teach algorithms to write articles, win video games & manipulate objects. How can policy keep up with AI advances?

I would recommend everyone who has calibrated intuitions about AI timelines spend some time doing stuff with real robots and it will probably … how should I put this? … further calibrate your intuitions in quite a humbling way.

Jack Clark

Dactyl is an AI system that can manipulate objects with a human-like robot hand. OpenAI Five is an AI system that can defeat humans at the video game Dota 2. The strange thing is they were both developed using the same general-purpose reinforcement learning algorithm.

How is this possible and what does it show?

In today’s interview Jack Clark, Policy Director at OpenAI, explains that from a computational perspective using a hand and playing Dota 2 are remarkably similar problems.

A robot hand needs to hold an object, move its fingers, and rotate it to the desired position. In Dota 2 you control a team of several different people, moving them around a map to attack an enemy.

Your hand has 20 or 30 different joints to move. The number of main actions in Dota 2 is 10 to 20, as you move your characters around a map.

When you’re rotating an objecting in your hand, you sense its friction, but you don’t directly perceive the entire shape of the object. In Dota 2, you’re unable to see the entire map and perceive what’s there by moving around — metaphorically ‘touching’ the space.

Read our new in-depth article on becoming an AI policy specialist: The case for building expertise to work on US AI policy, and how to do it

This is true of many apparently distinct problems in life. Compressing different sensory inputs down to a fundamental computational problem which we know how to solve only requires the right general purpose software.

OpenAI used an algorithm called Proximal Policy Optimization (PPO), which is fairly robust — in the sense that you can throw it at many different problems, not worry too much about tuning it, and it will do okay.

Jack emphasises that this algorithm wasn’t easy to create, and they were incredibly excited about it working on both tasks. But he also says that the creation of such increasingly ‘broad-spectrum’ algorithms has been the story of the last few years, and that the invention of software like PPO will have unpredictable consequences, heightening the huge challenges that already exist in AI policy.

Today’s interview is a mega-AI-policy-quad episode; Jack is joined by his colleagues Amanda Askell and Miles Brundage, on the day they released their fascinating and controversial large general language model GPT-2.

We discuss:

  • What are the most significant changes in the AI policy world over the last year or two?
  • How much is the field of AI policy still in the phase of just doing research and figuring out what should be done, versus actually trying to change things in the real world?
  • What capabilities are likely to develop over the next five, 10, 15, 20 years?
  • How much should we focus on the next couple of years, versus the next couple of decades?
  • How should we approach possible malicious uses of AI?
  • What are some of the potential ways OpenAI could make things worse, and how can they be avoided?
  • Publication norms for AI research
  • Where do we stand in terms of arms races between countries or different AI labs?
  • The case for creating a newsletter
  • Should the AI community have a closer relationship to the military?
  • Working at OpenAI vs. working in the US government
  • How valuable is Twitter in the AI policy world?

Rob is then joined by two of his colleagues — Niel Bowerman and Michelle Hutchinson — to quickly discuss:

  • The reaction to OpenAI’s release of GPT-2
  • Jack’s critique of our US AI policy article
  • How valuable are roles in government?
  • Where do you start if you want to write content for a specific audience?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

Find your highest impact role: 104 new vacancies in our February 2019 job board updates

Our job board continues to get big updates each 2 week, and now lists 235 vacancies, with 104 additional opportunities in the last month.

If you’re actively looking for a new role, we recommend checking out the job board regularly – when a great opening comes up, you’ll want to maximise your time to prepare.

The job board is a curated list of the most promising positions to apply for that we’re currently aware of. They’re all high-impact opportunities at organisations that are working on some of the world’s most pressing problems:

Check out the job board →

They’re demanding positions, but if you’re a good fit for one of them, it could be your best opportunity to have an impact.

If you apply for one of these jobs, or intend to, please do let us know.

A few highlights from the last month

Continue reading →

Can journalists still write about important things?

I think it’s certainly fair to say that there is more good journalism than people realize. And the reason for that is that a lot of the stuff that gets people very angry is not the best stuff out there.

Kelsey Piper

“Politics. Business. Opinion. Science. Sports. Animal welfare. Existential risks.” Is this a plausible future lineup for major news outlets?

Funded by the Rockefeller Foundation and given very little editorial direction, Vox’s Future Perfect aspires to be more or less that.

Competition in the news business creates pressure to write quick pieces on topical political issues that can drive lots of clicks with just a few hours’ work.

But according to Kelsey Piper, staff writer for this new section on Vox’s website focused on effective altruist themes, Future Perfect’s goal is to run in the opposite direction and make room for more substantive coverage that’s not tied to the news cycle.

They hope that in the long-term, talented writers from other outlets across the political spectrum, can also be attracted to tackle these topics.

Some skeptics of the project have questioned whether this general coverage of global catastrophic risks actually helps reduce them.

Kelsey responds: if you decide to dedicate your life to AI safety research, what’s the likely reaction from your family and friends? Do they think of you as someone about to join “that weird Silicon Valley apocalypse thing”? Or do they, having read about the issues widely, simply think “Oh, yeah. That seems important. I’m glad you’re working on it.”

Kelsey believes that really matters, and is determined by broader coverage of these kinds of topics.

If that’s right, is journalism a plausible pathway for doing the most good with your career, or did Kelsey just get particularly lucky? After all, journalism is a shrinking industry without an obvious revenue model to fund many writers looking into the world’s most pressing problems.

Kelsey points out that one needn’t take the risk of committing to journalism at an early age. Instead listeners can specialise in an important topic, while leaving open the option of switching into specialist journalism later on, should a great opportunity happen to present itself.

In today’s episode we discuss that path, as well as:

  • What’s the day to day life of a Vox journalist like?
  • How can good journalism get funded?
  • Are there meaningful tradeoffs between doing what’s in the interest of Vox, and doing what’s good?
  • How concerned should we be about the risk of effective altruism being perceived as partisan?
  • How well can short articles effectively communicate complicated ideas?
  • Are there alternative business models that could fund high quality journalism on a larger scale?
  • How do you approach the case for taking AI seriously to a broader audience?
  • How valuable might it be for media outlets to do Tetlock-style forecasting?
  • Is it really a good idea to heavily tax billionaires?
  • How do you avoid the pressure to get clicks?
  • How possible is it to predict which articles are going to be popular?
  • How did Kelsey build the skills necessary to work at Vox?
  • General lessons for people dealing with very difficult life circumstances

Rob is then joined by two of his colleagues – Keiran Harris and Michelle Hutchinson – to quickly discuss:

  • The risk political polarisation poses to long-termist causes
  • How should specialists keep journalism available as a career option?
  • Should we create a news aggregator that aims to make someone as well informed as possible in big-picture terms?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

Radical institutional reforms that make capitalism & democracy work better, and how to get them

…a lot of libertarians are into the idea of decentralised knowledge when it comes to Hayek and the market. But when we talk about politics, suddenly they think that there’s no knowledge out there. And I think that… that’s nuts. … politics is just like all these other things – there’s lots of local information out there.

Prof Glen Weyl

Imagine you were put in charge of planning out a country’s economy – determining who should work where and what they should make – without prices. You would surely struggle to collect all the information you need about what people want and who can most efficiently make it from an office building in the capital city.

Pro-market economists love to wax rhapsodic about the capacity of markets to pull together the valuable local information spread across all of society and solve this so-called ‘knowledge problem’.

But when it comes to politics and voting – which also aim to aggregate the preferences and knowledge found in millions of individuals – the enthusiasm for finding clever institutional designs turns to skepticism.

Today’s guest, freewheeling economist Glen Weyl, won’t have it, and is on a warpath to reform liberal democratic institutions in order to save them. Just last year he wrote Radical Markets: Uprooting Capitalism and Democracy for a Just Society with Eric Posner, but he has already moved on, saying “in the 6 months since the book came out I’ve made more intellectual progress than in the whole 10 years before that.”

He believes we desperately need more efficient, equitable and decentralised ways to organise society that take advantage of what each person knows, and his research agenda has already made some breakthroughs.

Despite a background in the best economics departments in the world – Harvard, Princeton, Yale and the University of Chicago – he is too worried for the future to sit in his office writing papers. Instead he has left the academy to try to inspire a social movement, RadicalxChange, with a vision of social reform as expansive as his own. (You can sign up for their conference in March here.)

Economist Alex Tabarrok called his latest proposal, known as ‘liberal radicalism’, “a quantum leap in public-goods mechanism-design.” The goal is to accurately measure how much the public actually values a good they all have to share, like a scientific research finding. Alex observes that under liberal radicalism “almost magically… citizens will voluntarily contribute exactly the amount that correctly signals how much society as a whole values the public good. Amazing!” But the proposal, however good in theory, might struggle in the real world because it requires large subsidies, and compensates for people’s selfishness so effectively that it might even be an overcorrection.

An earlier proposal – ‘quadratic voting’ (QV) – would allow people to express the relative strength of their preferences in the democratic process. No longer would 51 people who support a proposal, but barely care about the issue, outvote 49 incredibly passionate opponents, predictably making society worse in the process.

Instead everyone would be given ‘voice credits’ which they could spread across elections as they chose. QV follows a square root rule: 1 voice credit gets you 1 vote, 4 voice credits gets you 2 votes, 9 voice credits gives you 3 votes, and so on. It’s not immediately apparent, but this method is on average the ideal way of allowing people to more and more impose their desires on the rest of society, but at an ever escalating cost. To economists it’s an idea that’s obvious, though only in retrospect, and is already being taken up by business.

Weyl points to studies showing that people are more likely to vote strongly not only about issues they care more about, but issues they know more about. He expects that allowing people to specialise and indicate when they know what they’re talking about will create a democracy that does more to aggregate careful judgement, rather than just passionate ignorance.

But these and indeed all of Weyl’s proposals have faced criticism. Some say the risk of unintended consequences are too great, or that they solve the wrong problem. Others see these proposals as unproven, impractical, or just another example of overambitious social planning on the part of intellectuals. I raise these concerns to see how he responds.

Weyl hopes a creative spirit in figuring out how to make collective decision-making work for the modern world can restore faith in liberal democracy and prevent a resurgence of reactionary ideas during a future recession. But as big a topic as all that is, this extended conversation covers more:

  • How should we think about blockchain as a technology, and the community dedicated to it?
  • How could auctions inspire an alternative to private property?
  • Why is Glen wary of mathematical styles of approaching issues?
  • Is high modernism underrated?
  • Should we think of the world as going well or badly?
  • What are the biggest intellectual errors of the effective altruism community? And the rationality community?
  • Should migrants be sponsored by communities?
  • Could we provide people with a sustainable living by treating their data as labour?
  • The potential importance of artists in promoting ideas
  • How does liberal radicalism actually work

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

The case for building expertise to work on US AI policy, and how to do it

At 80,000 Hours we think a significant number of people should build expertise to work on United States (US) policy relevant to the long-term effects of the development and use of artificial intelligence (AI).

In this article we go into more detail on this claim, as well as discussing arguments in favor and against. We also briefly outline which specific career paths to aim for and discuss which sorts of people we think might suit these roles best.

This article is based on multiple conversations with three senior US Government officials, three federal employees working on science and technology issues, three congressional staffers, and several other people who have served as advisors to government from within academia and non-profits. We also spoke with several research scientists at top AI labs and in academia, as well as relevant experts from foundations and nonprofits.

We have hired Niel Bowerman as our in-house specialist on AI policy careers. If you are a US citizen interested in pursuing a career in AI public policy, please let us know and Niel may be able to work with you to help you enter this career path.

Still in her 20s, Terah Lyons has risen to the top of the artificial intelligence (AI) policy world.

Less than two years after finishing her Harvard undergraduate, she was working in the Obama White House, writing a report laying out the administration’s policies on AI.

Continue reading →

The CIA analyst who foresaw Trump in 2013 and his theory of why politics is changing

…the elites that ran our institutions had the authority to provide information, frame it & explain the world. That’s completely gone, and with it there’s been a bleeding away of expert authority, and a public has been created that’s essentially very angry…

Martin Gurri

Politics in rich countries seems to be going nuts. What’s the explanation? Rising inequality? The decline of manufacturing jobs? Excessive immigration?

Martin Gurri spent decades as a CIA analyst and in his 2014 book The Revolt of The Public and the Crisis of Authority in the New Millennium, predicted political turbulence for an entirely different reason: new communication technologies were flipping the balance of power between the public and traditional authorities.

In 1959 the President could control the narrative by leaning on his friends at four TV stations, who felt it was proper to present the nation’s leader in a positive light, no matter their flaws. Today, it’s impossible to prevent someone from broadcasting any grievance online, whether it’s a contrarian insight or an insane conspiracy theory.

According to Gurri, trust in society’s institutions – police, journalists, scientists and more – has been undermined by constant criticism from outsiders, and exposed to a cacophony of conflicting opinions on every issue the public takes fewer truths for granted. We are now free to see our leaders as the flawed human beings they always have been, and are not amused.

Suspicious they are being betrayed by elites, the public can also use technology to coordinate spontaneously and express its anger. Keen to ‘throw the bastards out’ – protesters take to the streets, united by what they don’t like, but without a shared agenda for how to move forward or the institutional infrastructure to figure out how to fix things. Some popular movements have come to view any attempt to exercise power over others as suspect.

If Gurri is to be believed, protest movements in Egypt, Spain, Greece and Israel in 2011 followed this script, while Brexit, Trump and the French yellow vests movement subsequently vindicated his theory.

In this model, politics won’t return to its old equilibrium any time soon. The leaders of tomorrow will need a new message and style if they hope to maintain any legitimacy in this less hierarchical world. Otherwise, we’re in for decades of grinding conflict between traditional centres of authority and the general public, who doubt both their loyalty and competence.

But how much should we believe this theory? Why do Canada and Australia remain pools of calm in the storm? Aren’t some malcontents quite concrete in their demands? And are protest movements actually more common (or more nihilistic) than they were decades ago?

In today’s episode we ask these questions and add an hour-long discussion with two of Rob’s colleagues – Keiran Harris and Michelle Hutchinson – to further explore the ideas in the book.

The conversation covers:

  • What’s changed about the public’s relationship to information and authority?
  • Are protesters today usually united for or against something?
  • What sorts of people are participating in these new movements?
  • Are we elites or the public?
  • Is the number of street protests and the level of dissatisfaction with governments actually higher than before?
  • How do we know that the internet is driving this rather than some other phenomenon?
  • How do technological changes enable social and political change?
  • The historical role of television
  • Are people also more disillusioned now with sports heroes and actors?
  • What are the best arguments against this thesis?
  • How should we think about countries like Canada, Australia, Spain, and China using this model?
  • Has public opinion shifted as much as it seems?
  • How can we get to a point where people view the system and politicians as legitimate and respectable, given the competitive pressures against being honest about the limits of your power and knowledge?
  • Which countries are finding good ways to make politics work in this new era?
  • What are the implications for the threat of totalitarianism?
  • What is this is going to do to international relations? Will it make it harder for countries to cooperate and avoid conflict?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

We could feed all 8 billion people through a nuclear winter. Dr David Denkenberger is working to make it practical.

I was reading this paper called Fungi & Sustainability – the premise was that after an asteroid impact, humans would go extinct and the world would be ruled by mushrooms, which would grow just fine in the dark. I thought… why don’t we just eat the mushrooms and not go extinct?

Dr David Denkenberger

If a nuclear winter or asteroid impact blocked the sun for years, our inability to grow food would result in billions dying of starvation, right? According to Dr David Denkenberger, co-author of Feeding Everyone No Matter What: no. If he’s to be believed, nobody need starve at all.

Even without the sun, David sees the Earth as a bountiful food source. Mushrooms farmed on decaying wood. Bacteria fed with natural gas. Fish and mussels supported by sudden upwelling of ocean nutrients – and many more.

Dr Denkenberger is an Assistant Professor at the University of Alaska Fairbanks, and he’s out to spread the word that while a nuclear winter might be horrible, experts have been mistaken to assume that mass starvation is an inevitability. In fact, he says, the only thing that would prevent us from feeding the world is insufficient preparation.

Not content to just write a book pointing this out, David has gone on to found a growing non-profit – the Alliance to Feed the Earth in Disasters – to brace the world to feed everyone come what may. He expects that today 10% of people would find enough food to survive a massive disaster. In principle, if we did everything right, nobody need go hungry. But being more realistic about how much we’re likely to invest, David hopes a plan to inform people ahead of time would save 30%, and a decent research and development scheme 80%.

According to David’s published cost-benefit analyses, work on this problem may be able to save lives, in expectation, for under $100 each, making it an incredible investment.

These preparations could also help make humanity more resilient to global catastrophic risks, by forestalling an ‘everyone for themselves’ mentality, which then causes trade and civilization to unravel.

But some worry that David’s cost-effectiveness estimates are exaggerations, so I challenge him on the practicality of his approach, and how much his non-profit’s work would actually matter in a post-apocalyptic world. In our extensive conversation, we cover:

  • How could the sun end up getting blocked, or agriculture otherwise be decimated?
  • What are all the ways we could we eat nonetheless? What kind of life would this be?
  • Can these methods be scaled up fast?
  • What is his organisation, ALLFED, actually working on?
  • How does he estimate the cost-effectiveness of this work, and what are the biggest weaknesses of the approach?
  • How would more food affect the post-apocalyptic world? Won’t people figure it out at that point anyway?
  • Why not just leave guidebooks with this information in every city?
  • Would these preparations make nuclear war more likely?
  • What kind of people is ALLFED trying to hire?
  • What would ALLFED do with more money? What have been their biggest mistakes?
  • How he ended up doing this work. And his other engineering proposals for improving the world, including how to prevent a supervolcano explosion.

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

What’s the best charity to donate to?

First published Nov 2016. Last updated Dec 2018.

If you want to make a difference, and are not already wedded to a cause, what’s the best charity to donate to? This is a brief summary of the most useful information we’ve been able to find.

First, we’ll sketch a process to use to compare options, then we’ll give our recommendations.

If you don’t have much time for research, our top recommendation is to give to the Effective Altruism Funds.

How to choose an effective charity
First, plan your research

  1. Do you trust someone else? If you know someone who shares your values and has already put a lot of thought into where to give, then consider simply going with their recommendations. You can skip ahead to see some recommendations from experts in charity evaluation. If you still want to do your own research, go to the next step.
  2. If you have under $10,000 to give, consider entering a donor lottery. It’s now possible to put $5,000 into a fund with other small donors, in exchange for a 5% chance of being able to choose where $100,000 from that fund gets donated. Why might you want to do this? In the case where you win, you can do a great deal of research into where’s best to give, to allocate that $100,000 as well as possible. Otherwise, you don’t have to do any research,

Continue reading →

Find your highest impact role: 77 new vacancies in our December job board updates

Thanks to the sterling work of Maria Gutierrez, our job board continues to get big updates each 2 week, and now lists 169 vacancies, with 77 additional opportunities in the last month.

If you’re actively looking for a new role, we recommend checking out the job board regularly – when a great opening comes up, you’ll want to maximise your time to prepare.

The job board is a curated list of the most promising positions to apply for that we’re currently aware of. They’re all high-impact opportunities at organisations that are working on some of the world’s most pressing problems:

Check out the job board →

They’re demanding positions, but if you’re a good fit for one of them, it could be your best opportunity to have an impact.

If you apply for one of these jobs, or intend to, please do let us know.

A few highlights from the last month

Continue reading →

A year’s worth of education for under a dollar and other ‘best buys’ in development, from the UK aid agency’s Chief Economist

…more teachers, more books, more inputs, like smaller class sizes – at least in the developing world – seem to have no impact, and that’s where most government money gets spent….

Dr Rachel Glennerster

If I told you it’s possible to deliver an extra year of ideal primary-level education for 30 cents, would you believe me? Hopefully not – the claim is absurd on its face.

But it may be true nonetheless. The very best education interventions are phenomenally cost-effective, but they’re not the kinds of things you’d expect, says this week’s guest, Dr Rachel Glennerster.

She’s Chief Economist at the UK’s foreign aid agency DFID, and used to run J-PAL, the world-famous anti-poverty research centre based at MIT’s Economics Department, where she studied the impact of a wide range of approaches to improving education, health, and political institutions. According to Glennerster:

“…when we looked at the cost effectiveness of education programs, there were a ton of zeros, and there were a ton of zeros on the things that we spend most of our money on. So more teachers, more books, more inputs, like smaller class sizes – at least in the developing world – seem to have no impact, and that’s where most government money gets spent.”

“But measurements for the top ones – the most cost effective programs – say they deliver 460 LAYS per £100 spent ($US130). LAYS are Learning-Adjusted Years of Schooling. Each one is the equivalent of the best possible year of education you can have – Singapore-level.”

“…the two programs that come out as spectacularly effective… well, the first is just rearranging kids in a class.”

“You have to test the kids, so that you can put the kids who are performing at grade two level in the grade two class, and the kids who are performing at grade four level in the grade four class, even if they’re different ages – and they learn so much better. So that’s why it’s so phenomenally cost effective because, it really doesn’t cost anything.”1

“The other one is providing information. So sending information over the phone [for example about how much more people earn if they do well in school and graduate]. So these really small nudges. Now none of those nudges will individually transform any kid’s life, but they are so cheap that you get these fantastic returns on investment – and we do very little of that kind of thing.”

(See the links section below to learn more about these kinds of results.)

In this episode, Dr Glennerster shares her decades of accumulated wisdom on which anti-poverty programs are overrated, which are neglected opportunities, and how we can know the difference, across a range of fields including health, empowering women and macroeconomic policy.

Regular listeners will be wondering – have we forgotten all about the lessons from episode 30 of the show with Dr Eva Vivalt? She threw several buckets of cold water on the hope that we could accurately measure the effectiveness of social programs at all.

According to Eva, her dataset of hundreds of randomised controlled trials indicates that social science findings don’t generalize well at all. The results of a trial at a school in Namibia tell us remarkably little about how a similar program will perform if delivered at another school in Namibia – let alone if it’s attempted in India instead.

Rachel offers a different and more optimistic interpretation of Eva’s findings.

Firstly, Rachel thinks it will often be possible to anticipate where studies will generalise and where they won’t. Studies are being lumped together that vary a great deal in i) how serious the problem is to start, ii) how well the program is delivered, iii) the details of the intervention itself. It’s no surprise that they have very variable results.

Rachel also points out that even if randomised trials can never accurately measure the effectiveness of every individual program, they can help us discover regularities of human behaviour that can inform everything we do. For instance, dozens of studies have shown that charging for preventative health measure like vaccinations will greatly reduce the number of people who take them up.

To learn more and figure out who you sympathise with, you’ll just have to listen to the the episode.

Regardless, Vivalt and Glennerster agree that we should continue to run these kinds of studies, and today’s episode delves into the latest ideas in global health and development. We discuss:

  • The development of aid work over the past 3 decades?
  • What’s the right balance of RCT work?
  • Do RCTs distract from broad economic growth and progress in these societies?
  • Overrated/underrated: charter cities, getting along with colleagues, cash transfers, cracking down on tax havens, micronutrient supplementation, pre-registration
  • The importance of using your judgement, experience, and priors
  • Things that reoccur in every culture
  • Do we produce too many programs where the quality of implementation matters?
  • Has the “empirical revolution” gone too far?
  • The increasing usage of Bayesian statistics
  • High impact gender equality interventions
  • Should we mostly focus on reforming macroeconomic policy in developing countries?
  • How important are markets for carbon?
  • What should we think about the impact the US and UK had in eastern Europe after the Cold War?

Footnote 1: You may understandably want to read the research on this! Unfortunately it is from an as-yet unpublished analysis by Noam Angrist at the World Bank. He worked on constructing a measure of Learning-Adjusted School Years for impact evaluations, which builds on Rachel’s 2013 paper in Science which attempts to determine the cost-effectiveness of a range of different health interventions. He says it should be available “sometime in the early new year” – we’ll link to it when it comes out.

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

A simple checklist for overcoming life and career setbacks

At 80,000 Hours we focus a lot on developing ambitious plans to dramatically improve the world.

Something we haven’t written so much about is how to overcome the challenges – heartbreak, rejection, failure, illness, grief, conflict and more – that are sure to arise as we attempt to follow through on those plans, and which risk throwing us off course.

We don’t have particular expertise on this topic, but I wanted to share an approach that me and some friends have found useful, and which might help you as well.

When bad things happen in life, the thoughts we then have about them have a big impact on how much they harm us. Even where we can’t avoid the direct suffering inflicted by a problem, we can at least avoid hurting ourselves further, by ruminating about it and getting trapped in a cycle of negative thoughts.

In the case of the minor annoyances we face every day, maintaining our equanimity can almost entirely eliminate the harm they cause us. And even when we face serious adversity, ensuring we think about it the right way can limit the damage, and save us from falling into depression or another negative spiral. (Though this list isn’t really suitable for seriously traumatic events.)

To help myself with this, I’ve made a checklist of questions I try to work through when something unpleasant happens, in order to reframe the situation and get over it as quickly as possible.

Continue reading →

Computer science algorithms tackle fundamental and universal problems. Can they help us live better, or is that a false hope?

We tend to think of deciding whether to commit to a partner, or where to go out for dinner, as uniquely and innately human problems. The message of the book is simply: they are not. In fact they correspond – really precisely in some cases – to some of the fundamental problems of computer science.

Brian Christian

Ever felt that you were so busy you spent all your time paralysed trying to figure out where to start, and couldn’t get much done? Computer scientists have a term for this – thrashing – and it’s a common reason our computers freeze up. The solution, for people as well as laptops, is to ‘work dumber’: pick something at random and finish it, without wasting time thinking about the bigger picture.

Ever wonder why people reply more if you ask them for a meeting at 2pm on Tuesday, than if you offer to talk at whatever happens to be the most convenient time in the next month? The first requires a two-second check of the calendar; the latter implicitly asks them to solve a vexing optimisation problem.

What about estimating the probability of something you can’t model, and which has never happened before? Math has got your back: the likelihood is no higher than 1 in the number of times it hasn’t happened, plus one. So if 5 people have tried a new drug and survived, the chance of the next one dying is at most 1 in 6.

Bestselling author Brian Christian studied computer science, and in the book Algorithms to Live By he’s out to find the lessons it can offer for a better life. In addition to the above he looks into when to quit your job, when to marry, the best way to sell your house, how long to spend on a difficult decision, and how much randomness to inject into your life.

In each case computer science gives us a theoretically optimal solution. In this episode we think hard about whether its models match our reality.

One genre of problems Brian explores in his book are ‘optimal stopping problems’, the canonical example of which is ‘the secretary problem’. Imagine you’re hiring a secretary, you receive n applicants, they show up in a random order, and you interview them one after another. You either have to hire that person on the spot and dismiss everybody else, or send them away and lose the option to hire them in future.

It turns out most of life can be viewed this way – a series of unique opportunities you pass by that will never be available in exactly the same way again.

So how do you attempt to hire the very best candidate in the pool? There’s a risk that you stop before you see the best, and a risk that you set your standards too high and let the best candidate pass you by.

Mathematicians of the mid-twentieth century produced the elegant solution: spend exactly one over e, or approximately 37% of your search, just establishing a baseline without hiring anyone, no matter how promising they seem. Then immediately hire the next person who’s better than anyone you’ve seen so far.

It turns out that your odds of success in this scenario are also 37%. And the optimal strategy and the odds of success are identical regardless of the size of the pool. So as n goes to infinity you still want to follow this 37% rule, and you still have a 37% chance of success. Even if you interview a million people.

But if you have the option to go back, say by apologising to the first applicant and begging them to come work with you, and you have a 50% chance of your apology being accepted, then the optimal explore percentage rises all the way to 61%.

Today’s episode focuses on Brian’s book-length exploration of how insights from computer algorithms can and can’t be applied to our everyday lives. We cover:

  • Is it really important that people know these different models and try to apply them?
  • What’s it like being a human confederate in the Turing test competition? What can you do to seem incredibly human?
  • Is trying to detect fake social media accounts a losing battle?
  • The canonical explore/exploit problem in computer science: the multi-armed bandit
  • How can we characterize a computational model of what people are actually doing, and is there a rigorous way to analyse just how good their instincts actually are?
  • What’s the value of cardinal information above and beyond ordinal information?
  • What’s the optimal way to buy or sell a house?
  • Why is information economics so important?
  • The martyrdom of being a music critic
  • ‘Simulated annealing’, and the best practices in optimisation
  • What kind of decisions should people randomize more in life?
  • Is the world more static than it used to be?
  • How much time should we spend on prioritisation? When does the best solution require less precision?
  • How do you predict the duration of something when you you don’t even know the scale of how long it’s going to last?
  • How many heists should you go on if you have a certain fixed probability of getting arrested and having all of your assets seized?
  • Are pro and con lists valuable?
  • Computational kindness, and the best way to schedule meetings
  • How should we approach a world of immense political polarisation?
  • How would this conversation have changed if there wasn’t an audience?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

Think twice before talking about ‘talent gaps’ – clarifying nine misconceptions

After pushing the idea of ‘talent gaps’ in 2015, we’ve noticed increasing confusion about the term.

This is partly our fault. So, here’s a quick list of common misconceptions about talent gaps and how they can be fixed. This is all pretty rough and we’re still refining our own views, but we hope this might start to clarify this issue, while we work on better explaining the idea in our key content.

1. Problem areas are constrained by specific skills, not ‘talent’

Problem areas are rarely generically ‘talent constrained’. They’re instead constrained by specific skills and abilities. It’s nearly always clearer to talk about the specific needs of the field, ideally down to the level of specific profiles of people, rather than talent and funding in general.

For instance, work to positively shape the development of AI is highly constrained by the following:

  • ML researchers, especially those able to do field-defining work, who are interested in and understand AI safety, the alignment problem, and other issues relevant to the long-term development of AI.
  • People skilled in operations, especially those able to run non-profits with under 50 people or academic institutes, and who are interested in and understand issues related to the long-term development of AI.
  • Strategy and policy researchers able to do disentanglement research in pre-paradigmatic fields.
  • People with the policy expertise and career capital to work in influential government positions who are also knowledgeable about and dedicated to the issue.

Continue reading →

PhD or programming? Fast paths into aligning AI as a machine learning engineer, according to ML engineers Catherine Olsson & Daniel Ziegler

If you are a talented software engineer, the state of the questions right now is that some of them are just ready to throw engineers on. And so if you haven’t just tried applying to the position that you want, just try. Just see. You might actually be ready for it.

Catherine Olsson

After dropping out of his ML PhD at Stanford, Daniel Ziegler needed to decide what to do next. He’d always enjoyed building stuff and wanted to help shape the development of AI, so he thought a research engineering position at an org dedicated to aligning AI with human interests could be his best option.

He decided to apply to OpenAI, spent 6 weeks preparing for the interview, and actually landed the job. His PhD, by contrast, might have taken 6 years. Daniel thinks this highly accelerated career path may be possible for many others.

On today’s episode Daniel is joined by Catherine Olsson, who has also worked at OpenAI, and left her computational neuroscience PhD to become a research engineer at Google Brain. They share this piece of advice for those interested in this career path: just dive in. If you’re trying to get good at something, just start doing that thing, and figure out that way what’s necessary to be able to do it well.

To go with this episode, Catherine has even written a simple step-by-step guide to help others copy her and Daniel’s success.

Please let us know how we’ve helped you – take 5 minutes to fill out our 2018 annual impact survey. This is one of the best quick things you can do to support our work.

Daniel thinks the key for him was nailing the job interview.

OpenAI needed him to be able to demonstrate the ability to do the kind of stuff he’d be working on day-to-day. So his approach was to take a list of 50 key deep reinforcement learning papers, read one or two a day, and pick a handful to actually reproduce. He spent a bunch of time coding in Python and TensorFlow, sometimes 12 hours a day, trying to debug and tune things until they were actually working.

Daniel emphasizes that the most important thing was to practice exactly those things that he knew he needed to be able to do. He also received an offer from the Machine Intelligence Research Institute, and so he had the opportunity to decide between two organisations focused on the global problem that most concerns him.

Daniel’s path might seem unusual, but both he and Catherine expect it can be replicated by others. If they’re right, it could greatly increase our ability to quickly get new people into ML roles in which they can make a difference.

Catherine says that her move from OpenAI to an ML research team at Google now allows her to bring a different set of skills to the table. Technical AI safety is a multifaceted area of research, and the many sub-questions in areas such as reward learning, robustness, and interpretability all need to be answered to maximize the probability that AI development goes well for humanity.

Today’s episode combines the expertise of two pioneers and is a key resource for anyone wanting to follow in their footsteps. We cover:

  • What is the field of AI safety? How could your projects contribute?
  • What are OpenAI and Google Brain doing?
  • Why would one decide to work on AI?
  • The pros and cons of ML PhDs
  • Do you learn more on the job, or while doing a PhD?
  • Why did Daniel think OpenAI had the best approach? What did that mean?
  • Controversial issues within ML
  • What are some of the problems that are ready for software engineers?
  • What’s required to be a good ML engineer? Is replicating papers a good way of determining suitability?
  • What fraction of software developers could make similar transitions?
  • How in-demand are research engineers?
  • The development of Dota 2 bots
  • What’s the organisational structure of ML groups? Are there similarities to an academic lab?
  • The fluidity of roles in ML
  • Do research scientists have more influence on the vision of an org?
  • What’s the value of working in orgs not specifically focused on safety?
  • Has learning more made you more or less worried about the future?
  • The value of AI policy work
  • Advice for people considering 23andMe

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

ML engineering for AI safety & robustness: a Google Brain engineer’s guide to entering the field

Note that this guide was written in November 2018 to complement an in-depth conversation on the 80,000 Hours Podcast with Catherine Olsson and Daniel Ziegler on how to transition from computer science and software engineering in general into ML engineering, with a focus on alignment and safety. If you like this guide, we’d strongly encourage you to check out the podcast episode where we discuss some of the instructions here, and other relevant advice.

Technical AI safety is a multifaceted area of research, with many sub-questions in areas such as reward learning, robustness, and interpretability. These will all need to be answered in order to make sure AI development will go well for humanity as systems become more and more powerful.

Not all of these questions are best tackled with abstract mathematics research; some can be approached with concrete coding experiments and machine learning (ML) prototypes. As a result, some AI safety research teams are looking to hire a growing number of Software Engineers and ML Research Engineers.

Additionally, some research teams that may not think of themselves as focussed on ‘AI Safety’ per se, nonetheless work on related problems like verification of neural nets or learning from human feedback, and are often hiring engineers.

What are the necessary qualifications for these positions?

Software Engineering: Some engineering roles on AI safety teams do not require ML experience. You might already be prepared to apply to these positions if you have the following qualifications:

  • BSc/BEng degree in computer science or another technical field (or comparable experience)
  • Strong knowledge of software engineering (as a benchmark: could pass a Google software engineering interview)
  • Interest in working on AI safety
  • (usually) Willingness to move to London or the San Francisco Bay Area

If you’re a software engineer with an interest in these roles,

Continue reading →

Second October job board update

Our job board now lists 142 vacancies, with 38 additional opportunities since the last update 3 weeks ago.

If you’re actively looking for a new role, we recommend checking out the job board regularly – when a great opening comes up, you’ll want to maximise your preparation time.

The job board remains a curated list of the most promising positions to apply for that we’re currently aware of. They’re all high-impact opportunities at organisations that are working on some of the world’s most pressing problems:

Check out the job board →

They’re demanding positions, but if you’re a good fit for one of them, it could be your best opportunity to have an impact.

If you apply for one of these jobs, or intend to, please do let us know.

A few highlights from the last month

Continue reading →

New article: Have a particular strength? Already an expert in a field? Here are the socially impactful careers 80,000 Hours suggests you consider first.

We’ve published a new article that summarises our advice based on your strengths and link you to the most relevant articles for you to read:

This list is preliminary. We wanted to publish our existing thoughts on what to do with each skill, but can easily see ourselves changing our minds over the coming years.

You can read about our general process and what career paths we recommend in our full article.

Sometimes, however, it’s possible to give more specific advice about what options to consider to people who already have pre-existing experience or qualifications, or are unusually good at a certain type of work.

In this article, we provide a list of skills, and for each one give a list of socially-impactful options that people who are unusually good in that area should most often consider.

We start with three “strengths” (quantitative, verbal & social, and visual). Then we go on to give advice for people with existing experience in fifteen specific fields.

Bear in mind it’s often possible to completely change field: we’ve seen people switch from philosophy to software engineering, and architecture into economics. Nonetheless, these are good starting points.

The skill types also overlap, and you probably also have several of them. The aim is just to give you some tips on narrowing down your options more quickly.

Continue reading →

    Philosophy Prof Hilary Greaves on moral cluelessness, population ethics, probability within a multiverse, & harnessing the brainpower of academia to tackle the most important research questions

    You might think, OK, I know that the immediate effects of funding anti-malarial bed nets are positive – I know that I’m going to save lives. But I also know that there are going to be further downstream effects and side-effects of my intervention. For example, effects on the size of future populations. It’s notoriously unclear how to think about the value of future population size, whether it’ll be a good thing to increase population in the short term, or whether that would in the end be a bad thing. There are lots of uncertainties here.

    Hilary Greaves

    The barista gives you your coffee and change, and you walk away from the busy line. But you suddenly realise she gave you $1 less than she should have. Do you brush your way past the people now waiting, or just accept this as a dollar you’re never getting back? According to philosophy professor Hilary Greaves – Director of Oxford University’s Global Priorities Institute, which is hiring now – this simple decision will completely change the long-term future by altering the identities of almost all future generations.

    How? Because by rushing back to the counter, you slightly change the timing of everything else people in line do during that day — including changing the timing of the interactions they have with everyone else. Eventually these causal links will reach someone who was going to conceive a child.

    By causing a child to be conceived a few fractions of a second earlier or later, you change the sperm that fertilizes their egg, resulting in a totally different person. So asking for that $1 has now made the difference between all the things that this actual child will do in their life, and all the things that the merely possible child – who didn’t exist because of what you did – would have done if you decided not to worry about it.

    As that child’s actions ripple out to everyone else who conceives down the generations, ultimately the entire human population will become different, all for the sake of your dollar. Will your choice cause a future Hitler to be born, or not to be born? Probably both!

    Some find this concerning. The actual long term effects of your decisions are so unpredictable, it looks like you’re totally clueless about what’s going to lead to the best outcomes. It might lead to decision paralysis — you won’t be able to take any action at all.

    Prof Greaves doesn’t share this concern for most real life decisions. If there’s no reasonable way to assign probabilities to far-future outcomes, then the possibility that you might make things better in completely unpredictable ways is more or less canceled out by the equally plausible possibility that you might make things worse in equally unpredictable ways.

    But, if instead we’re talking about a decision that involves highly-structured, systematic reasons for thinking there might be a general tendency of your action to make things better or worse — for example if we increase economic growth — Prof Greaves says that we don’t get to just ignore the unforeseeable effects.

    When there are complex arguments on both sides, it’s unclear what probabilities you should assign to this or that claim. Yet, given its importance, whether you should take the action in question actually does depend on figuring out these numbers.

    So, what do we do?

    Today’s episode blends philosophy with an exploration of the mission and research agenda of the Global Priorities Institute: to develop the effective altruism movement within academia. We cover:

    • What’s the long term vision of the Global Priorities Institute?
    • How controversial is the multiverse interpretation of quantum physics?
    • What’s the best argument against academics just doing whatever they’re interested in?
    • How strong is the case for long-termism? What are the best opposing arguments?
    • Are economists getting convinced by philosophers on discount rates?
    • Given moral uncertainty, how should population ethics affect our real life decisions?
    • How should we think about archetypal decision theory problems?
    • The value of exploratory vs. basic research
    • Person affecting views of population ethics, fragile identities of future generations, and the non-identity problem
    • Is Derek Parfit’s repugnant conclusion really repugnant? What’s the best vision of a life barely worth living?
    • What are the consequences of cluelessness for those who based their donation advice on GiveWell style recommendations?
    • How could reducing global catastrophic risk be a good cause for risk-averse people?
    • What’s the core difficulty in forming proper credences?
    • The value of subjecting EA ideas to academic scrutiny
    • The influence of academia in society
    • The merits of interdisciplinary work
    • The case for why operations is so important in academia
    • The trade off between working on important problems and advancing your career

    Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

    The 80,000 Hours Podcast is produced by Keiran Harris.

    Continue reading →

    New article: Ways people trying to do good accidentally make things worse, and how to avoid them

    We’ve published a new article about how to avoid accidentally causing harm through your career:

    “We encourage people to work on problems that are neglected by others and large in scale. Unfortunately those are precisely the problems where people can do the most damage if their approach isn’t carefully thought through.

    If a problem is very important, then setting back the cause is very bad. If a problem is so neglected that you’re among the first focused on it, then you’ll have a disproportionate influence on the field’s reputation, how likely others are to enter it, and many early decisions that could have path-dependent effects on the field’s long-term success.

    We don’t particularly enjoy writing about this admittedly demotivating topic. Ironically, we expect that cautious people – the folks who least need this advice – will be the ones most likely to take it to heart.

    Nonetheless we think cataloguing these risks is important if we’re going to be serious about having an impact in important but ‘fragile’ fields like reducing extinction risk.

    In this article, we’ll list six ways people can unintentionally set back their cause. You may already be aware of most of these risks, but we often see people neglect one or two of them when new to a high stakes area – including us when we were starting 80,000 Hours.”

    Read the full article…

    Continue reading →

      Economics Prof Tyler Cowen says our overwhelming priorities should be maximising economic growth and making civilization more stable. Is he right?

      We can already see that three key questions should be elevated in their political and philosophical importance. Namely: number one, what can we do to boost the rate of economic growth? Number two, what can we do to make civilization more stable? And number three, how should we deal with environmental problems?

      Tyler Cowen

      I’ve probably spent more time reading Tyler Cowen – Professor of Economics at George Mason University – than any other author. Indeed it’s his incredibly popular blog Marginal Revolution that prompted me to study economics in the first place. Having spent thousands of hours absorbing Tyler’s work, it was a pleasure to be able to question him about his latest book and personal manifesto: Stubborn Attachments: A Vision for a Society of Free, Prosperous, and Responsible Individuals.

      Tyler makes the case that, despite what you may have heard, we can make rational judgments about what is best for society as a whole. He argues:

      1. Our top moral priority should be preserving and improving humanity’s long-term future
      2. The way to do that is to maximise the rate of sustainable economic growth
      3. We should respect human rights and follow general principles while doing so.

      We discuss why Tyler believes all these things, and I push back where I disagree. In particular: is higher economic growth actually an effective way to safeguard humanity’s future, or should our focus really be elsewhere?

      In the process we touch on many of moral philosophy’s most pressing questions: Should we discount the future? How should we aggregate welfare across people? Should we follow rules or evaluate every situation individually? How should we deal with the massive uncertainty about the effects of our actions? And should we trust common sense morality or follow structured theories?

      After covering the book, the conversation ranges far and wide. Will we leave the galaxy, and is it a tragedy if we don’t? Is a multi-polar world less stable? Will humanity ever help wild animals? Why do we both agree that Kant and Rawls are overrated?

      Today’s interview is released on both the 80,000 Hours Podcast and Tyler’s own show: Conversation with Tyler.

      Tyler may have had more influence on me than any other writer but this conversation is richer for our remaining disagreements. If the above isn’t enough to tempt you to listen, we also look at:

      • Why couldn’t future technology make human life a hundred or a thousand times better than it is for people today?
      • Why focus on increasing the rate of economic growth rather than making sure that it doesn’t go to zero?
      • Why shouldn’t we dedicate substantial time to the successful introduction of genetic engineering?
      • Why should we completely abstain from alcohol and make it a social norm?
      • Why is Tyler so pessimistic about space? Is it likely that humans will go extinct before we manage to escape the galaxy?
      • Is improving coordination and international cooperation a major priority?
      • Why does Tyler think institutions are keeping up with technology?
      • Given that our actions seem to have very large and morally significant effects in the long run, are our moral obligations very onerous?
      • Can art be intrinsically valuable?
      • What does Tyler think Derek Parfit was most wrong about, and what was he was most right about that’s unappreciated today?
      • How should we think about animal suffering?
      • Do self-aware entities have to be biological in some sense?
      • What’s the most likely way that the worldview presented in Stubborn Attachments could be fundamentally wrong?
      • During ‘underrated vs overrated’, should guests say ‘appropriately rated’ more often?

      Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

      The 80,000 Hours podcast is produced by Keiran Harris.

      Continue reading →