Podcast

#58 – Pushmeet Kohli of DeepMind on designing robust & reliable AI systems and how to succeed in AI

By and · Published June 3rd, 2019

When you’re building a bridge, responsibility for making sure it won’t fall over isn’t handed over to a few ‘bridge not falling down engineers’. Making sure a bridge is safe to use and remains standing in a storm is completely central to the design, and indeed the entire project.

When it comes to artificial intelligence, commentators often distinguish between enhancing the capabilities of machine learning systems and enhancing their safety. But to Pushmeet Kohli, principal scientist and research team leader at DeepMind, research to make AI robust and reliable is no more a side-project in AI design than keeping a bridge standing is a side-project in bridge design.

Far from being an overhead on the ‘real’ work, it’s an essential part of making AI systems work in any sense. We don’t want AI systems to be out of alignment with our intentions, and that consideration must arise throughout their development.

Professor Stuart Russell — co-author of the most popular AI textbook — has gone as far as to suggest that if this view is right, it may be time to retire the term ‘AI safety research’ altogether.

With the goal of designing systems that reliably do what we want, DeepMind have recently published work on important technical challenges for the ML community.

For instance, Pushmeet is looking for efficient ways to test whether a system conforms to the desired specifications, even in peculiar situations, by creating an ‘adversary’ that proactively seeks out the worst failures possible. If the adversary can efficiently identify the worst-case input for a given model, DeepMind can catch rare failure cases before deploying a model in the real world. In the future single mistakes by autonomous systems may have very large consequences, which will make even small failure probabilities unacceptable.

He’s also looking into ‘training specification-consistent models’ and formal verification’, while other researchers at DeepMind working on their AI safety agenda are figuring out how to understand agent incentives, avoid side-effects, and model AI rewards.

In today’s interview, we focus on the convergence between broader AI research and robustness, as well as:

  • DeepMind’s work on the protein folding problem
  • Parallels between ML problems and past challenges in software development and computer security
  • How can you analyse the thinking of a neural network?
  • Unique challenges faced by DeepMind’s technical AGI safety team
  • How do you communicate with a non-human intelligence?
  • How should we conceptualize ML progress?
  • What are the biggest misunderstandings about AI safety and reliability?
  • Are there actually a lot of disagreements within the field?
  • The difficulty of forecasting AI development

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.


As an addendum to the episode, we caught up with some members of the DeepMind team to learn more about roles at the organization beyond research and engineering, and how these contribute to the broader mission of developing AI for positive social impact.

A broad sketch of the kinds of roles listed on the DeepMind website may be helpful for listeners:

  • Program Managers keep the research team moving forward in a coordinated way, enabling and accelerating research.
  • The Ethics & Society team explores the real-world impacts of AI, from both an ethics research and policy perspective.
  • The Public Engagement & Communications team thinks about how to communicate about AI and its implications, engaging with audiences ranging from the AI community to the media to the broader public.
  • The Recruitment team focuses on building out the team in all of these areas, as well as research and engineering, bringing together the diverse and multidisciplinary group of people required to fulfill DeepMind’s ambitious mission.

There are many more listed opportunities across other teams, from Legal to People & Culture to the Office of the CEO, where our listeners may like to get involved.

They invite applicants from a wide range of backgrounds and skill sets so interested listeners should take a look at their open positions.

Continue reading →

#57 – Tom Kalil on how to do the most good in government

By and · Published April 23rd, 2019

You’re 29 years old, and you’ve just been given a job in the White House. How do you quickly figure out how the US Executive Branch behemoth actually works, so that you can have as much impact as possible – before you quit or get kicked out?

That was the challenge put in front of Tom Kalil in 1993.

He had enough success to last a full 16 years inside the Clinton and Obama administrations, working to foster the development of the internet, then nanotechnology, and then cutting-edge brain modelling, among other things.

But not everyone figures out how to move the needle. In today’s interview, Tom shares his experience with how to increase your chances of getting an influential role in government, and how to make the most of the opportunity if you get in.

He believes that Congressional gridlock leads people to greatly underestimate how much the Executive Branch can and does do on its own every day. Decisions by individuals change how billions of dollars are spent; regulations are enforced, and then suddenly they aren’t; and a single sentence in the State of the Union can get civil servants to pay attention to a topic that would otherwise go ignored.

Over years at the White House Office of Science and Technology Policy, ‘Team Kalil’ built up a white board of principles. For example, ‘the schedule is your friend’: setting a meeting date with the President can force people to finish something, where they otherwise might procrastinate.

Or ‘talk to who owns the paper’. People would wonder how Tom could get so many lines into the President’s speeches. The answer was “figure out who’s writing the speech, find them with the document, and tell them to add the line.” Obvious, but not something most were doing.

Not everything is a precise operation though. Tom also tells us the story of NetDay, a project that was put together at the last minute because the President incorrectly believed it was already organised – and decided he was going to announce it in person.

American interested in working on AI policy?

We’ve helped dozens of people transition into policy careers. We can offer introductions to people and funding opportunities, and we can help answer specific questions you might have.

If you are a US citizen interested in building expertise to work on US AI policy, apply for our free coaching service.

Apply for coaching

In today’s episode we get down to nuts & bolts, and discuss:

  • How did Tom spin work on a primary campaign into a job in the next White House?
  • Why does Tom think hiring is the most important work he did, and how did he decide who to bring onto the team?
  • How do you get people to do things when you don’t have formal power over them?
  • What roles in the US government are most likely to help with the long-term future, or reducing existential risks?
  • Is it possible, or even desirable, to get the general public interested in abstract, long-term policy ideas?
  • What are ‘policy entrepreneurs’ and why do they matter?
  • What is the role for prizes in promoting science and technology? What are other promising policy ideas?
  • Why you can get more done by not taking credit.
  • What can the White House do if an agency isn’t doing what it wants?
  • How can the effective altruism community improve the maturity of our policy recommendations?
  • How much can talented individuals accomplish during a short-term stay in government?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

#56 – Persis Eskander on wild animal welfare and what, if anything, to do about it

By and · Published April 15th, 2019

Elephants in chains at travelling circuses; pregnant pigs trapped in coffin-sized crates at factory farms; deers living in the wild. We should welcome the last as a pleasant break from the horror, right?

Maybe, but maybe not. While we tend to have a romanticised view of nature, life in the wild includes a range of extremely negative experiences.

Most animals are hunted by predators, and constantly have to remain vigilant lest they be killed, and perhaps experience the terror of being eaten alive. Resource competition often leads to chronic hunger or starvation. Their diseases and injuries are never treated. In winter wild animals freeze to death and in droughts they die of heat or thirst.

There are fewer than 20 people in the world dedicating their lives to researching these problems.

But according to Persis Eskander, researcher at Open Philanthropy, if we sum up the negative experiences of all wild animals, their sheer number – trillions to quintillions, depending on which you count – could make the scale of the problem larger than most other near-term concerns.

Persis urges us to recognise that nature isn’t inherently good or bad, but rather the result of an amoral evolutionary process. For those that can’t survive the brutal indifference of their environment, life is often a series of bad experiences, followed by an even worse death.

But should we actually intervene? How do we know what animals are sentient? How often do animals really feel hunger, cold, fear, happiness, satisfaction, boredom, and intense agony? Are there long-term technologies that could some day allow us to massively improve wild animal welfare?

For most of these big questions, the answer is: we don’t know. And Persis thinks we’re far from knowing enough to start interfering with ecosystems. But that’s all the more reason to start considering these questions.

There are a few concrete steps we could take today, like improving the way wild caught fish are slaughtered. Fish might lack the charisma of a lion or the intelligence of a pig, but if they have the capacity to suffer — and evidence suggests that they do — we should be thinking of ways to kill them painlessly rather than allowing them to suffocate to death over hours.

In today’s interview we explore wild animal welfare as a new field of research, and discuss:

  • Do we have a moral duty towards wild animals?
  • How should we measure the number of wild animals?
  • What are some key activities that generate a lot of suffering or pleasure for wild animals that people might not fully appreciate?
  • Is there a danger in imagining how we as humans would feel if we were put into their situation?
  • Should we eliminate parasites and predators?
  • How important are insects?
  • Interventions worth rolling out today
  • How strongly should we focus on just avoiding humans going in and making things worse?
  • How does this compare to work on farmed animal suffering?
  • The most compelling arguments for not dedicating resources to wild animal welfare
  • Is there much of a case for the idea that this work could improve the very long-term future of humanity?
  • Would increasing concern for wild animals improve our values?
  • How do you get academics to take an interest in this?
  • How could autonomous drones improve wild animal welfare?

Rob is then joined by two of his colleagues — Niel Bowerman and Michelle Hutchinson — to quickly cover:

  • The importance of figuring out your values
  • Chemistry, psychology, and other different paths towards working on wild animal welfare
  • How to break into new fields

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

#55 – Mark Lutter & Tamara Winter on founding charter cities with outstanding governance to end poverty

By and · Published March 31st, 2019

Governance matters. Policy change quickly took China from famine to fortune; Singapore from swamps to skyscrapers; and Hong Kong from fishing village to financial centre. Unfortunately, many governments are hard to reform and — to put it mildly — it’s not easy to found a new country.

This has prompted poverty-fighters and political dreamers to look for creative ways to get new and better ‘pseudo-countries’ off the ground. The poor could then voluntarily migrate to in search of security and prosperity. And innovators would be free to experiment with new political and legal systems without having to impose their ideas on existing jurisdictions.

The ‘seasteading movement’ imagined founding new self-governing cities on the sea, but obvious challenges have kept that one on the drawing board. Nobel Prize winner and World Bank President Paul Romer suggested ‘charter cities’, where a host country would volunteer for another country with better legal institutions to effectively govern some of its territory. But that idea too ran aground for political, practical and personal reasons.

Now Dr Mark Lutter and Tamara Winter, of The Center for Innovative Governance Research (CIGR), are reviving the idea of ‘charter cities’, with some modifications. Gone is the idea of transferring sovereignty. Instead these cities would look more like the ‘special economic zones’ that worked miracles for Taiwan and China among others. But rather than keep the rest of the country’s rules with a few pieces removed, they hope to start from scratch, opting in to the laws they want to keep, in order to leap forward to “best practices in commercial law.”

Also listen to: Rob on The Good Life: Andrew Leigh in Conversation — on ‘making the most of your 80,000 hours’.

The project has quickly gotten attention, with Mark and Tamara receiving funding from Tyler Cowen’s Emergent Ventures (discussed in episode 45) and winning a Pioneer tournament.

Starting afresh with a new city makes it possible to clear away thousands of harmful rules without having to fight each of the thousands of interest groups that will viciously defend their privileges. Initially the city can fund infrastructure and public services by gradually selling off its land, which appreciates as the city flourishes. And with 40 million people relocating to cities every year, there are plenty of prospective migrants.

CIGR is fleshing out how these arrangements would work, advocating for them, and developing supporting services that make it easier for any jurisdiction to implement. They’re currently in the process of influencing a new prospective satellite city in Zambia.

Of course, one can raise many criticisms of this idea: Is it likely to be taken up? Is CIGR really doing the right things to make it happen? Will it really reduce poverty if it is?

We discuss those questions, as well as:

  • How did Mark get a new organisation off the ground, with fundraising and other staff?
  • What made China’s ‘special economic zones’ so successful?
  • What are the biggest challenges in getting new cities off the ground?
  • What are the top criticisms of charter cities, and why aren’t they worried?
  • How did Mark find and hire Tamara? How did he know this was a good idea?
  • Who do they need to talk to to make charter cities happen?
  • How does their idea fit into the broader story of governance innovation?
  • Should people care about this idea if they aren’t focussed on tackling poverty?
  • Why aren’t people already doing this?
  • Why does Tamara support more people starting families?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

#54 – Askell, Brundage & Clark from OpenAI on publication norms, malicious uses of AI, and general-purpose learning algorithms

By and · Published March 19th, 2019

Dactyl is an AI system that can manipulate objects with a human-like robot hand. OpenAI Five is an AI system that can defeat humans at the video game Dota 2. The strange thing is they were both developed using the same general-purpose reinforcement learning algorithm.

How is this possible and what does it show?

In today’s interview Jack Clark, Policy Director at OpenAI, explains that from a computational perspective using a hand and playing Dota 2 are remarkably similar problems.

A robot hand needs to hold an object, move its fingers, and rotate it to the desired position. In Dota 2 you control a team of several different people, moving them around a map to attack an enemy.

Your hand has 20 or 30 different joints to move. The number of main actions in Dota 2 is 10 to 20, as you move your characters around a map.

When you’re rotating an objecting in your hand, you sense its friction, but you don’t directly perceive the entire shape of the object. In Dota 2, you’re unable to see the entire map and perceive what’s there by moving around — metaphorically ‘touching’ the space.

Read our new in-depth article on becoming an AI policy specialist: The case for building expertise to work on US AI policy, and how to do it

This is true of many apparently distinct problems in life. Compressing different sensory inputs down to a fundamental computational problem which we know how to solve only requires the right general purpose software.

OpenAI used an algorithm called Proximal Policy Optimization (PPO), which is fairly robust — in the sense that you can throw it at many different problems, not worry too much about tuning it, and it will do okay.

Jack emphasises that this algorithm wasn’t easy to create, and they were incredibly excited about it working on both tasks. But he also says that the creation of such increasingly ‘broad-spectrum’ algorithms has been the story of the last few years, and that the invention of software like PPO will have unpredictable consequences, heightening the huge challenges that already exist in AI policy.

Today’s interview is a mega-AI-policy-quad episode; Jack is joined by his colleagues Amanda Askell and Miles Brundage, on the day they released their fascinating and controversial large general language model GPT-2.

We discuss:

  • What are the most significant changes in the AI policy world over the last year or two?
  • How much is the field of AI policy still in the phase of just doing research and figuring out what should be done, versus actually trying to change things in the real world?
  • What capabilities are likely to develop over the next five, 10, 15, 20 years?
  • How much should we focus on the next couple of years, versus the next couple of decades?
  • How should we approach possible malicious uses of AI?
  • What are some of the potential ways OpenAI could make things worse, and how can they be avoided?
  • Publication norms for AI research
  • Where do we stand in terms of arms races between countries or different AI labs?
  • The case for creating a newsletter
  • Should the AI community have a closer relationship to the military?
  • Working at OpenAI vs. working in the US government
  • How valuable is Twitter in the AI policy world?

Rob is then joined by two of his colleagues — Niel Bowerman and Michelle Hutchinson — to quickly discuss:

  • The reaction to OpenAI’s release of GPT-2
  • Jack’s critique of our US AI policy article
  • How valuable are roles in government?
  • Where do you start if you want to write content for a specific audience?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

#53 – Kelsey Piper on the room for important advocacy within journalism

By and · Published February 27th, 2019

“Politics. Business. Opinion. Science. Sports. Animal welfare. Existential risks.” Is this a plausible future lineup for major news outlets?

Funded by the Rockefeller Foundation and given very little editorial direction, Vox’s Future Perfect aspires to be more or less that.

Competition in the news business creates pressure to write quick pieces on topical political issues that can drive lots of clicks with just a few hours’ work.

But according to Kelsey Piper, staff writer for this new section on Vox’s website focused on effective altruist themes, Future Perfect’s goal is to run in the opposite direction and make room for more substantive coverage that’s not tied to the news cycle.

They hope that in the long-term, talented writers from other outlets across the political spectrum, can also be attracted to tackle these topics.

Some skeptics of the project have questioned whether this general coverage of global catastrophic risks actually helps reduce them.

Kelsey responds: if you decide to dedicate your life to AI safety research, what’s the likely reaction from your family and friends? Do they think of you as someone about to join “that weird Silicon Valley apocalypse thing”? Or do they, having read about the issues widely, simply think “Oh, yeah. That seems important. I’m glad you’re working on it.”

Kelsey believes that really matters, and is determined by broader coverage of these kinds of topics.

If that’s right, is journalism a plausible pathway for doing the most good with your career, or did Kelsey just get particularly lucky? After all, journalism is a shrinking industry without an obvious revenue model to fund many writers looking into the world’s most pressing problems.

Kelsey points out that one needn’t take the risk of committing to journalism at an early age. Instead listeners can specialise in an important topic, while leaving open the option of switching into specialist journalism later on, should a great opportunity happen to present itself.

In today’s episode we discuss that path, as well as:

  • What’s the day to day life of a Vox journalist like?
  • How can good journalism get funded?
  • Are there meaningful tradeoffs between doing what’s in the interest of Vox, and doing what’s good?
  • How concerned should we be about the risk of effective altruism being perceived as partisan?
  • How well can short articles effectively communicate complicated ideas?
  • Are there alternative business models that could fund high quality journalism on a larger scale?
  • How do you approach the case for taking AI seriously to a broader audience?
  • How valuable might it be for media outlets to do Tetlock-style forecasting?
  • Is it really a good idea to heavily tax billionaires?
  • How do you avoid the pressure to get clicks?
  • How possible is it to predict which articles are going to be popular?
  • How did Kelsey build the skills necessary to work at Vox?
  • General lessons for people dealing with very difficult life circumstances

Rob is then joined by two of his colleagues – Keiran Harris and Michelle Hutchinson – to quickly discuss:

  • The risk political polarisation poses to long-termist causes
  • How should specialists keep journalism available as a career option?
  • Should we create a news aggregator that aims to make someone as well informed as possible in big-picture terms?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

#52 – Glen Weyl on uprooting capitalism and democracy for a just society

By and · Published February 8th, 2019

Imagine you were put in charge of planning out a country’s economy – determining who should work where and what they should make – without prices. You would surely struggle to collect all the information you need about what people want and who can most efficiently make it from an office building in the capital city.

Pro-market economists love to wax rhapsodic about the capacity of markets to pull together the valuable local information spread across all of society and solve this so-called ‘knowledge problem’.

But when it comes to politics and voting – which also aim to aggregate the preferences and knowledge found in millions of individuals – the enthusiasm for finding clever institutional designs turns to skepticism.

Today’s guest, freewheeling economist Glen Weyl, won’t have it, and is on a warpath to reform liberal democratic institutions in order to save them. Just last year he wrote Radical Markets: Uprooting Capitalism and Democracy for a Just Society with Eric Posner, but he has already moved on, saying “in the 6 months since the book came out I’ve made more intellectual progress than in the whole 10 years before that.”

He believes we desperately need more efficient, equitable and decentralised ways to organise society that take advantage of what each person knows, and his research agenda has already made some breakthroughs.

Despite a background in the best economics departments in the world – Harvard, Princeton, Yale and the University of Chicago – he is too worried for the future to sit in his office writing papers. Instead he has left the academy to try to inspire a social movement, RadicalxChange, with a vision of social reform as expansive as his own. (You can sign up for their conference in March here.)

Economist Alex Tabarrok called his latest proposal, known as ‘liberal radicalism’, “a quantum leap in public-goods mechanism-design.” The goal is to accurately measure how much the public actually values a good they all have to share, like a scientific research finding. Alex observes that under liberal radicalism “almost magically… citizens will voluntarily contribute exactly the amount that correctly signals how much society as a whole values the public good. Amazing!” But the proposal, however good in theory, might struggle in the real world because it requires large subsidies, and compensates for people’s selfishness so effectively that it might even be an overcorrection.

An earlier proposal – ‘quadratic voting’ (QV) – would allow people to express the relative strength of their preferences in the democratic process. No longer would 51 people who support a proposal, but barely care about the issue, outvote 49 incredibly passionate opponents, predictably making society worse in the process.

Instead everyone would be given ‘voice credits’ which they could spread across elections as they chose. QV follows a square root rule: 1 voice credit gets you 1 vote, 4 voice credits gets you 2 votes, 9 voice credits gives you 3 votes, and so on. It’s not immediately apparent, but this method is on average the ideal way of allowing people to more and more impose their desires on the rest of society, but at an ever escalating cost. To economists it’s an idea that’s obvious, though only in retrospect, and is already being taken up by business.

Weyl points to studies showing that people are more likely to vote strongly not only about issues they care more about, but issues they know more about. He expects that allowing people to specialise and indicate when they know what they’re talking about will create a democracy that does more to aggregate careful judgement, rather than just passionate ignorance.

But these and indeed all of Weyl’s proposals have faced criticism. Some say the risk of unintended consequences are too great, or that they solve the wrong problem. Others see these proposals as unproven, impractical, or just another example of overambitious social planning on the part of intellectuals. I raise these concerns to see how he responds.

Weyl hopes a creative spirit in figuring out how to make collective decision-making work for the modern world can restore faith in liberal democracy and prevent a resurgence of reactionary ideas during a future recession. But as big a topic as all that is, this extended conversation covers more:

  • How should we think about blockchain as a technology, and the community dedicated to it?
  • How could auctions inspire an alternative to private property?
  • Why is Glen wary of mathematical styles of approaching issues?
  • Is high modernism underrated?
  • Should we think of the world as going well or badly?
  • What are the biggest intellectual errors of the effective altruism community? And the rationality community?
  • Should migrants be sponsored by communities?
  • Could we provide people with a sustainable living by treating their data as labour?
  • The potential importance of artists in promoting ideas
  • How does liberal radicalism actually work

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

#51 – Martin Gurri on the revolt of the public & crisis of authority in the information age

By and · Published January 29th, 2019

Politics in rich countries seems to be going nuts. What’s the explanation? Rising inequality? The decline of manufacturing jobs? Excessive immigration?

Martin Gurri spent decades as a CIA analyst and in his 2014 book The Revolt of The Public and the Crisis of Authority in the New Millennium, predicted political turbulence for an entirely different reason: new communication technologies were flipping the balance of power between the public and traditional authorities.

In 1959 the President could control the narrative by leaning on his friends at four TV stations, who felt it was proper to present the nation’s leader in a positive light, no matter their flaws. Today, it’s impossible to prevent someone from broadcasting any grievance online, whether it’s a contrarian insight or an insane conspiracy theory.

According to Gurri, trust in society’s institutions – police, journalists, scientists and more – has been undermined by constant criticism from outsiders, and exposed to a cacophony of conflicting opinions on every issue the public takes fewer truths for granted. We are now free to see our leaders as the flawed human beings they always have been, and are not amused.

Suspicious they are being betrayed by elites, the public can also use technology to coordinate spontaneously and express its anger. Keen to ‘throw the bastards out’ – protesters take to the streets, united by what they don’t like, but without a shared agenda for how to move forward or the institutional infrastructure to figure out how to fix things. Some popular movements have come to view any attempt to exercise power over others as suspect.

If Gurri is to be believed, protest movements in Egypt, Spain, Greece and Israel in 2011 followed this script, while Brexit, Trump and the French yellow vests movement subsequently vindicated his theory.

In this model, politics won’t return to its old equilibrium any time soon. The leaders of tomorrow will need a new message and style if they hope to maintain any legitimacy in this less hierarchical world. Otherwise, we’re in for decades of grinding conflict between traditional centres of authority and the general public, who doubt both their loyalty and competence.

But how much should we believe this theory? Why do Canada and Australia remain pools of calm in the storm? Aren’t some malcontents quite concrete in their demands? And are protest movements actually more common (or more nihilistic) than they were decades ago?

In today’s episode we ask these questions and add an hour-long discussion with two of Rob’s colleagues – Keiran Harris and Michelle Hutchinson – to further explore the ideas in the book.

The conversation covers:

  • What’s changed about the public’s relationship to information and authority?
  • Are protesters today usually united for or against something?
  • What sorts of people are participating in these new movements?
  • Are we elites or the public?
  • Is the number of street protests and the level of dissatisfaction with governments actually higher than before?
  • How do we know that the internet is driving this rather than some other phenomenon?
  • How do technological changes enable social and political change?
  • The historical role of television
  • Are people also more disillusioned now with sports heroes and actors?
  • What are the best arguments against this thesis?
  • How should we think about countries like Canada, Australia, Spain, and China using this model?
  • Has public opinion shifted as much as it seems?
  • How can we get to a point where people view the system and politicians as legitimate and respectable, given the competitive pressures against being honest about the limits of your power and knowledge?
  • Which countries are finding good ways to make politics work in this new era?
  • What are the implications for the threat of totalitarianism?
  • What is this is going to do to international relations? Will it make it harder for countries to cooperate and avoid conflict?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

#50 – David Denkenberger on how to feed all 8 billion people through an asteroid/nuclear winter

By and · Published December 27th, 2018

If a nuclear winter or asteroid impact blocked the sun for years, our inability to grow food would result in billions dying of starvation, right? According to Dr David Denkenberger, co-author of Feeding Everyone No Matter What: no. If he’s to be believed, nobody need starve at all.

Even without the sun, David sees the Earth as a bountiful food source. Mushrooms farmed on decaying wood. Bacteria fed with natural gas. Fish and mussels supported by sudden upwelling of ocean nutrients – and many more.

Dr Denkenberger is an Assistant Professor at the University of Alaska Fairbanks, and he’s out to spread the word that while a nuclear winter might be horrible, experts have been mistaken to assume that mass starvation is an inevitability. In fact, he says, the only thing that would prevent us from feeding the world is insufficient preparation.

Not content to just write a book pointing this out, David has gone on to found a growing nonprofit – the Alliance to Feed the Earth in Disasters – to brace the world to feed everyone come what may. He expects that today 10% of people would find enough food to survive a massive disaster. In principle, if we did everything right, nobody need go hungry. But being more realistic about how much we’re likely to invest, David hopes a plan to inform people ahead of time would save 30%, and a decent research and development scheme 80%.

According to David’s published cost-benefit analyses, work on this problem may be able to save lives, in expectation, for under $100 each, making it an incredible investment.

These preparations could also help make humanity more resilient to global catastrophic risks, by forestalling an ‘everyone for themselves’ mentality, which then causes trade and civilization to unravel.

But some worry that David’s cost-effectiveness estimates are exaggerations, so I challenge him on the practicality of his approach, and how much his nonprofit’s work would actually matter in a post-apocalyptic world. In our extensive conversation, we cover:

  • How could the sun end up getting blocked, or agriculture otherwise be decimated?
  • What are all the ways we could we eat nonetheless? What kind of life would this be?
  • Can these methods be scaled up fast?
  • What is his organisation, ALLFED, actually working on?
  • How does he estimate the cost-effectiveness of this work, and what are the biggest weaknesses of the approach?
  • How would more food affect the post-apocalyptic world? Won’t people figure it out at that point anyway?
  • Why not just leave guidebooks with this information in every city?
  • Would these preparations make nuclear war more likely?
  • What kind of people is ALLFED trying to hire?
  • What would ALLFED do with more money? What have been their biggest mistakes?
  • How he ended up doing this work. And his other engineering proposals for improving the world, including how to prevent a supervolcano explosion.

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

#49 – Rachel Glennerster on a year's worth of education for under $1 and other development best buys

By and · Published December 20th, 2018

If I told you it’s possible to deliver an extra year of ideal primary-level education for 30 cents, would you believe me? Hopefully not – the claim is absurd on its face.

But it may be true nonetheless. The very best education interventions are phenomenally cost-effective, but they’re not the kinds of things you’d expect, says this week’s guest, Dr Rachel Glennerster.

She’s Chief Economist at the UK’s foreign aid agency DFID, and used to run J-PAL, the world-famous anti-poverty research centre based at MIT’s Economics Department, where she studied the impact of a wide range of approaches to improving education, health, and political institutions. According to Glennerster:

“…when we looked at the cost effectiveness of education programs, there were a ton of zeros, and there were a ton of zeros on the things that we spend most of our money on. So more teachers, more books, more inputs, like smaller class sizes – at least in the developing world – seem to have no impact, and that’s where most government money gets spent.”

“But measurements for the top ones – the most cost effective programs – say they deliver 460 LAYS per £100 spent ($US130). LAYS are Learning-Adjusted Years of Schooling. Each one is the equivalent of the best possible year of education you can have – Singapore-level.”

“…the two programs that come out as spectacularly effective… well, the first is just rearranging kids in a class.”

“You have to test the kids, so that you can put the kids who are performing at grade two level in the grade two class, and the kids who are performing at grade four level in the grade four class, even if they’re different ages – and they learn so much better. So that’s why it’s so phenomenally cost effective because, it really doesn’t cost anything.”

“The other one is providing information. So sending information over the phone [for example about how much more people earn if they do well in school and graduate]. So these really small nudges. Now none of those nudges will individually transform any kid’s life, but they are so cheap that you get these fantastic returns on investment – and we do very little of that kind of thing.”

(See the links section below to learn more about these kinds of results.)

In this episode, Dr Glennerster shares her decades of accumulated wisdom on which anti-poverty programs are overrated, which are neglected opportunities, and how we can know the difference, across a range of fields including health, empowering women and macroeconomic policy.

Regular listeners will be wondering – have we forgotten all about the lessons from episode 30 of the show with Dr Eva Vivalt? She threw several buckets of cold water on the hope that we could accurately measure the effectiveness of social programs at all.

According to Eva, her dataset of hundreds of randomised controlled trials indicates that social science findings don’t generalize well at all. The results of a trial at a school in Namibia tell us remarkably little about how a similar program will perform if delivered at another school in Namibia – let alone if it’s attempted in India instead.

Rachel offers a different and more optimistic interpretation of Eva’s findings.

Firstly, Rachel thinks it will often be possible to anticipate where studies will generalise and where they won’t. Studies are being lumped together that vary a great deal in i) how serious the problem is to start, ii) how well the program is delivered, iii) the details of the intervention itself. It’s no surprise that they have very variable results.

Rachel also points out that even if randomised trials can never accurately measure the effectiveness of every individual program, they can help us discover regularities of human behaviour that can inform everything we do. For instance, dozens of studies have shown that charging for preventative health measure like vaccinations will greatly reduce the number of people who take them up.

To learn more and figure out who you sympathise with, you’ll just have to listen to the the episode.

Regardless, Vivalt and Glennerster agree that we should continue to run these kinds of studies, and today’s episode delves into the latest ideas in global health and development. We discuss:

  • The development of aid work over the past 3 decades?
  • What’s the right balance of RCT work?
  • Do RCTs distract from broad economic growth and progress in these societies?
  • Overrated/underrated: charter cities, getting along with colleagues, cash transfers, cracking down on tax havens, micronutrient supplementation, pre-registration
  • The importance of using your judgement, experience, and priors
  • Things that reoccur in every culture
  • Do we produce too many programs where the quality of implementation matters?
  • Has the “empirical revolution” gone too far?
  • The increasing usage of Bayesian statistics
  • High impact gender equality interventions
  • Should we mostly focus on reforming macroeconomic policy in developing countries?
  • How important are markets for carbon?
  • What should we think about the impact the US and UK had in eastern Europe after the Cold War?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

#48 – Brian Christian on better living through the wisdom of computer science

By and · Published November 22nd, 2018

Ever felt that you were so busy you spent all your time paralysed trying to figure out where to start, and couldn’t get much done? Computer scientists have a term for this – thrashing – and it’s a common reason our computers freeze up. The solution, for people as well as laptops, is to ‘work dumber’: pick something at random and finish it, without wasting time thinking about the bigger picture.

Ever wonder why people reply more if you ask them for a meeting at 2pm on Tuesday, than if you offer to talk at whatever happens to be the most convenient time in the next month? The first requires a two-second check of the calendar; the latter implicitly asks them to solve a vexing optimisation problem.

What about estimating the probability of something you can’t model, and which has never happened before? Math has got your back: the likelihood is no higher than 1 in the number of times it hasn’t happened, plus one. So if 5 people have tried a new drug and survived, the chance of the next one dying is at most 1 in 6.

Bestselling author Brian Christian studied computer science, and in the book Algorithms to Live By he’s out to find the lessons it can offer for a better life. In addition to the above he looks into when to quit your job, when to marry, the best way to sell your house, how long to spend on a difficult decision, and how much randomness to inject into your life.

In each case computer science gives us a theoretically optimal solution. In this episode we think hard about whether its models match our reality.

One genre of problems Brian explores in his book are ‘optimal stopping problems’, the canonical example of which is ‘the secretary problem’. Imagine you’re hiring a secretary, you receive n applicants, they show up in a random order, and you interview them one after another. You either have to hire that person on the spot and dismiss everybody else, or send them away and lose the option to hire them in future.

It turns out most of life can be viewed this way – a series of unique opportunities you pass by that will never be available in exactly the same way again.

So how do you attempt to hire the very best candidate in the pool? There’s a risk that you stop before you see the best, and a risk that you set your standards too high and let the best candidate pass you by.

Mathematicians of the mid-twentieth century produced the elegant solution: spend exactly one over e, or approximately 37% of your search, just establishing a baseline without hiring anyone, no matter how promising they seem. Then immediately hire the next person who’s better than anyone you’ve seen so far.

It turns out that your odds of success in this scenario are also 37%. And the optimal strategy and the odds of success are identical regardless of the size of the pool. So as n goes to infinity you still want to follow this 37% rule, and you still have a 37% chance of success. Even if you interview a million people.

But if you have the option to go back, say by apologising to the first applicant and begging them to come work with you, and you have a 50% chance of your apology being accepted, then the optimal explore percentage rises all the way to 61%.

Today’s episode focuses on Brian’s book-length exploration of how insights from computer algorithms can and can’t be applied to our everyday lives. We cover:

  • Is it really important that people know these different models and try to apply them?
  • What’s it like being a human confederate in the Turing test competition? What can you do to seem incredibly human?
  • Is trying to detect fake social media accounts a losing battle?
  • The canonical explore/exploit problem in computer science: the multi-armed bandit
  • How can we characterize a computational model of what people are actually doing, and is there a rigorous way to analyse just how good their instincts actually are?
  • What’s the value of cardinal information above and beyond ordinal information?
  • What’s the optimal way to buy or sell a house?
  • Why is information economics so important?
  • The martyrdom of being a music critic
  • ‘Simulated annealing’, and the best practices in optimisation
  • What kind of decisions should people randomize more in life?
  • Is the world more static than it used to be?
  • How much time should we spend on prioritisation? When does the best solution require less precision?
  • How do you predict the duration of something when you you don’t even know the scale of how long it’s going to last?
  • How many heists should you go on if you have a certain fixed probability of getting arrested and having all of your assets seized?
  • Are pro and con lists valuable?
  • Computational kindness, and the best way to schedule meetings
  • How should we approach a world of immense political polarisation?
  • How would this conversation have changed if there wasn’t an audience?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

#47 – Catherine Olsson & Daniel Ziegler on the fast path into high-impact ML engineering roles

By and · Published November 2nd, 2018

After dropping out of his ML PhD at Stanford, Daniel Ziegler needed to decide what to do next. He’d always enjoyed building stuff and wanted to help shape the development of AI, so he thought a research engineering position at an org dedicated to aligning AI with human interests could be his best option.

He decided to apply to OpenAI, spent 6 weeks preparing for the interview, and actually landed the job. His PhD, by contrast, might have taken 6 years. Daniel thinks this highly accelerated career path may be possible for many others.

On today’s episode Daniel is joined by Catherine Olsson, who has also worked at OpenAI, and left her computational neuroscience PhD to become a research engineer at Google Brain. They share this piece of advice for those interested in this career path: just dive in. If you’re trying to get good at something, just start doing that thing, and figure out that way what’s necessary to be able to do it well.

To go with this episode, Catherine has even written a simple step-by-step guide to help others copy her and Daniel’s success.

Daniel thinks the key for him was nailing the job interview.

OpenAI needed him to be able to demonstrate the ability to do the kind of stuff he’d be working on day-to-day. So his approach was to take a list of 50 key deep reinforcement learning papers, read one or two a day, and pick a handful to actually reproduce. He spent a bunch of time coding in Python and TensorFlow, sometimes 12 hours a day, trying to debug and tune things until they were actually working.

Daniel emphasizes that the most important thing was to practice exactly those things that he knew he needed to be able to do. He also received an offer from the Machine Intelligence Research Institute, and so he had the opportunity to decide between two organisations focused on the global problem that most concerns him.

Daniel’s path might seem unusual, but both he and Catherine expect it can be replicated by others. If they’re right, it could greatly increase our ability to quickly get new people into ML roles in which they can make a difference.

Catherine says that her move from OpenAI to an ML research team at Google now allows her to bring a different set of skills to the table. Technical AI safety is a multifaceted area of research, and the many sub-questions in areas such as reward learning, robustness, and interpretability all need to be answered to maximize the probability that AI development goes well for humanity.

Today’s episode combines the expertise of two pioneers and is a key resource for anyone wanting to follow in their footsteps. We cover:

  • What is the field of AI safety? How could your projects contribute?
  • What are OpenAI and Google Brain doing?
  • Why would one decide to work on AI?
  • The pros and cons of ML PhDs
  • Do you learn more on the job, or while doing a PhD?
  • Why did Daniel think OpenAI had the best approach? What did that mean?
  • Controversial issues within ML
  • What are some of the problems that are ready for software engineers?
  • What’s required to be a good ML engineer? Is replicating papers a good way of determining suitability?
  • What fraction of software developers could make similar transitions?
  • How in-demand are research engineers?
  • The development of Dota 2 bots
  • What’s the organisational structure of ML groups? Are there similarities to an academic lab?
  • The fluidity of roles in ML
  • Do research scientists have more influence on the vision of an org?
  • What’s the value of working in orgs not specifically focused on safety?
  • Has learning more made you more or less worried about the future?
  • The value of AI policy work
  • Advice for people considering 23andMe

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

#46 – Hilary Greaves on moral cluelessness, population ethics, & harnessing the brainpower of academia to tackle the most important research questions

By and · Published October 23rd, 2018

The barista gives you your coffee and change, and you walk away from the busy line. But you suddenly realise she gave you $1 less than she should have. Do you brush your way past the people now waiting, or just accept this as a dollar you’re never getting back? According to philosophy professor Hilary Greaves – Director of Oxford University’s Global Priorities Institute, which is hiring now – this simple decision will completely change the long-term future by altering the identities of almost all future generations.

How? Because by rushing back to the counter, you slightly change the timing of everything else people in line do during that day — including changing the timing of the interactions they have with everyone else. Eventually these causal links will reach someone who was going to conceive a child.

By causing a child to be conceived a few fractions of a second earlier or later, you change the sperm that fertilizes their egg, resulting in a totally different person. So asking for that $1 has now made the difference between all the things that this actual child will do in their life, and all the things that the merely possible child – who didn’t exist because of what you did – would have done if you decided not to worry about it.

As that child’s actions ripple out to everyone else who conceives down the generations, ultimately the entire human population will become different, all for the sake of your dollar. Will your choice cause a future Hitler to be born, or not to be born? Probably both!

Some find this concerning. The actual long term effects of your decisions are so unpredictable, it looks like you’re totally clueless about what’s going to lead to the best outcomes. It might lead to decision paralysis — you won’t be able to take any action at all.

Prof Greaves doesn’t share this concern for most real life decisions. If there’s no reasonable way to assign probabilities to far-future outcomes, then the possibility that you might make things better in completely unpredictable ways is more or less canceled out by the equally plausible possibility that you might make things worse in equally unpredictable ways.

But, if instead we’re talking about a decision that involves highly-structured, systematic reasons for thinking there might be a general tendency of your action to make things better or worse — for example if we increase economic growth — Prof Greaves says that we don’t get to just ignore the unforeseeable effects.

When there are complex arguments on both sides, it’s unclear what probabilities you should assign to this or that claim. Yet, given its importance, whether you should take the action in question actually does depend on figuring out these numbers.

So, what do we do?

Today’s episode blends philosophy with an exploration of the mission and research agenda of the Global Priorities Institute: to develop the effective altruism movement within academia. We cover:

  • What’s the long term vision of the Global Priorities Institute?
  • How controversial is the multiverse interpretation of quantum physics?
  • What’s the best argument against academics just doing whatever they’re interested in?
  • How strong is the case for long-termism? What are the best opposing arguments?
  • Are economists getting convinced by philosophers on discount rates?
  • Given moral uncertainty, how should population ethics affect our real life decisions?
  • How should we think about archetypal decision theory problems?
  • The value of exploratory vs. basic research
  • Person affecting views of population ethics, fragile identities of future generations, and the non-identity problem
  • Is Derek Parfit’s repugnant conclusion really repugnant? What’s the best vision of a life barely worth living?
  • What are the consequences of cluelessness for those who based their donation advice on GiveWell style recommendations?
  • How could reducing global catastrophic risk be a good cause for risk-averse people?
  • What’s the core difficulty in forming proper credences?
  • The value of subjecting EA ideas to academic scrutiny
  • The influence of academia in society
  • The merits of interdisciplinary work
  • The case for why operations is so important in academia
  • The trade off between working on important problems and advancing your career

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.

Continue reading →

#45 – Tyler Cowen's stubborn attachments to maximising economic growth, making civilization more stable & respecting human rights

By and · Published October 17th, 2018

I’ve probably spent more time reading Tyler Cowen – Professor of Economics at George Mason University – than any other author. Indeed it’s his incredibly popular blog Marginal Revolution that prompted me to study economics in the first place. Having spent thousands of hours absorbing Tyler’s work, it was a pleasure to be able to question him about his latest book and personal manifesto: Stubborn Attachments: A Vision for a Society of Free, Prosperous, and Responsible Individuals.

Tyler makes the case that, despite what you may have heard, we can make rational judgments about what is best for society as a whole. He argues:

  1. Our top moral priority should be preserving and improving humanity’s long-term future
  2. The way to do that is to maximise the rate of sustainable economic growth
  3. We should respect human rights and follow general principles while doing so.

We discuss why Tyler believes all these things, and I push back where I disagree. In particular: is higher economic growth actually an effective way to safeguard humanity’s future, or should our focus really be elsewhere?

In the process we touch on many of moral philosophy’s most pressing questions: Should we discount the future? How should we aggregate welfare across people? Should we follow rules or evaluate every situation individually? How should we deal with the massive uncertainty about the effects of our actions? And should we trust common sense morality or follow structured theories?

After covering the book, the conversation ranges far and wide. Will we leave the galaxy, and is it a tragedy if we don’t? Is a multi-polar world less stable? Will humanity ever help wild animals? Why do we both agree that Kant and Rawls are overrated?

Today’s interview is released on both the 80,000 Hours Podcast and Tyler’s own show: Conversation with Tyler.

Tyler may have had more influence on me than any other writer but this conversation is richer for our remaining disagreements. If the above isn’t enough to tempt you to listen, we also look at:

  • Why couldn’t future technology make human life a hundred or a thousand times better than it is for people today?
  • Why focus on increasing the rate of economic growth rather than making sure that it doesn’t go to zero?
  • Why shouldn’t we dedicate substantial time to the successful introduction of genetic engineering?
  • Why should we completely abstain from alcohol and make it a social norm?
  • Why is Tyler so pessimistic about space? Is it likely that humans will go extinct before we manage to escape the galaxy?
  • Is improving coordination and international cooperation a major priority?
  • Why does Tyler think institutions are keeping up with technology?
  • Given that our actions seem to have very large and morally significant effects in the long run, are our moral obligations very onerous?
  • Can art be intrinsically valuable?
  • What does Tyler think Derek Parfit was most wrong about, and what was he was most right about that’s unappreciated today?
  • How should we think about animal suffering?
  • Do self-aware entities have to be biological in some sense?
  • What’s the most likely way that the worldview presented in Stubborn Attachments could be fundamentally wrong?
  • During ‘underrated vs overrated’, should guests say ‘appropriately rated’ more often?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours podcast is produced by Keiran Harris.

Continue reading →

#44 – Paul Christiano on how OpenAI is developing real solutions to the 'AI alignment problem', and his vision of how humanity will progressively hand over decision-making to AI systems

By and · Published October 2nd, 2018

Paul Christiano is one of the smartest people I know and this episode has one of the best explanations for why AI alignment matters and how we might solve it. After our first session produced such great material, we decided to do a second recording, resulting in our longest interview so far. While challenging at times I can strongly recommend listening – Paul works on AI himself and has a very unusually thought through view of how it will change the world. Even though I’m familiar with Paul’s writing I felt I was learning a great deal and am now in a better position to make a difference to the world.

A few of the topics we cover are:

  • Why Paul expects AI to transform the world gradually rather than explosively and what that would look like
  • Several concrete methods OpenAI is trying to develop to ensure AI systems do what we want even if they become more competent than us
  • Why AI systems will probably be granted legal and property rights
  • How an advanced AI that doesn’t share human goals could still have moral value
  • Why machine learning might take over science research from humans before it can do most other tasks
  • Which decade we should expect human labour to become obsolete, and how this should affect your savings plan.

—

Here’s a situation we all regularly confront: you want to answer a difficult question, but aren’t quite smart or informed enough to figure it out for yourself. The good news is you have access to experts who are smart enough to figure it out. The bad news is that they disagree.

If given plenty of time – and enough arguments, counterarguments and counter-counter-arguments between all the experts – should you eventually be able to figure out which is correct? What if one expert were deliberately trying to mislead you? And should the expert with the correct view just tell the whole truth, or will competition force them to throw in persuasive lies in order to have a chance of winning you over?

In other words: does ‘debate’, in principle, lead to truth?

According to Paul Christiano – researcher at the machine learning research lab OpenAI and legendary thinker in the effective altruism and rationality communities – this question is of more than mere philosophical interest. That’s because ‘debate’ is a promising method of keeping artificial intelligence aligned with human goals, even if it becomes much more intelligent and sophisticated than we are.

It’s a method OpenAI is actively trying to develop, because in the long-term it wants to train AI systems to make decisions that are too complex for any human to grasp, but without the risks that arise from a complete loss of human oversight.

If AI-1 is free to choose any line of argument in order to attack the ideas of AI-2, and AI-2 always seems to successfully defend them, it suggests that every possible line of argument would have been unsuccessful.

But does that mean that the ideas of AI-2 were actually right? It would be nice if the optimal strategy in debate were to be completely honest, provide good arguments, and respond to counterarguments in a valid way. But we don’t know that’s the case.

According to Paul, it’s clear that if the judge is weak enough, there’s no reason that an honest debater would be at an advantage. But the hope is that there is some threshold of competence above which debates tend to converge on more accurate claims the longer they continue.

Most real world debates are set up under highly suboptimal conditions; judges usually don’t have a lot of time to think about how to best get to the truth, and often have bad incentives themselves. But for AI safety via debate, researchers are free to set things up in the way that gives them the best shot. And if we could understand how to construct systems that converge to truth, we would have a plausible way of training powerful AI systems to stay aligned with our goals.

This is our longest interview so far for good reason — we cover a fascinating range of topics:

  • What could people do to shield themselves financially from potentially losing their jobs to AI?
  • How important is it that the best AI safety team ends up in the company with the best ML team?
  • What might the world look like if several states or actors developed AI at the same time (aligned or otherwise)?
  • Would artificial general intelligence grow in capability quickly or slowly?
  • How likely is it that transformative AI is an issue worth worrying about?
  • What are the best arguments against being concerned?
  • What would cause people to take AI alignment more seriously?
  • Concrete ideas for making machine learning safer, such as iterated amplification.
  • What does it mean to say that a crow-like intelligence could be much better at science than humans?
  • What is ‘prosaic AI’?
  • How do Paul’s views differ from those of the Machine Intelligence Research Institute?
  • The importance of honesty for people and organisations
  • What are the most important ways that people in the effective altruism community are approaching AI issues incorrectly?
  • When would an ‘unaligned’ AI nonetheless be morally valuable?
  • What’s wrong with current sci-fi?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours podcast is produced by Keiran Harris.

Continue reading →

#43 – Daniel Ellsberg on the creation of nuclear doomsday machines, the institutional insanity that maintains them, & how they could be dismantled

By and · Published September 24th, 2018

In Stanley Kubrick’s iconic film Dr. Strangelove, the American president is informed that the Soviet Union has created a secret deterrence system which will automatically wipe out humanity upon detection of a single nuclear explosion in Russia. With US bombs heading towards the USSR and unable to be recalled, Dr Strangelove points out that “the whole point of this Doomsday Machine is lost if you keep it a secret – why didn’t you tell the world, eh?” The Soviet ambassador replies that it was to be announced at the Party Congress the following Monday: “The Premier loves surprises”.

Daniel Ellsberg – leaker of the Pentagon Papers which helped end the Vietnam War and Nixon presidency – claims in his new book The Doomsday Machine: Confessions of a Nuclear War Planner that Dr. Strangelove might as well be a documentary. After attending the film in Washington DC in 1964, he and a military colleague wondered how so many details of the nuclear systems they were constructing had managed to leak to the filmmakers.

The USSR did in fact develop a doomsday machine, Dead Hand, which probably remains active today.

If the system can’t contact military leaders, it checks for signs of a nuclear strike. Should its computers determine that an attack occurred, it would automatically launch all remaining Soviet weapons at targets across the northern hemisphere.

As in the film, the Soviet Union long kept Dead Hand completely secret, eliminating any strategic benefit, and rendering it a pointless menace to humanity.

You might think the United States would have a more sensible nuclear launch policy. You’d be wrong.

As Ellsberg explains based on first-hand experience as a nuclear war planner in the early stages of the Cold War, the notion that only the president is able to authorize the use of US nuclear weapons is a carefully cultivated myth.

The authority to launch nuclear weapons is delegated alarmingly far down the chain of command – significantly raising the chance that a lone wolf or communication breakdown could trigger a nuclear catastrophe.

The whole justification for this is to defend against a ‘decapitating attack’, where a first strike on Washington disables the ability of the US hierarchy to retaliate. In a moment of crisis, the Russians might view this as their best hope of survival.

Ostensibly, this delegation removes Russia’s temptation to attempt a decapitating attack – the US can retaliate even if its leadership is destroyed. This strategy only works, though, if you tell the enemy you’ve done it.

Instead, since the 50s this delegation has been one of the United States most closely guarded secrets, eliminating its strategic benefit, and rendering it another pointless menace to humanity.

Even setting aside the above, the size of the Russian and American nuclear arsenals today makes them doomsday machines of necessity. According to Ellsberg, if these arsenals are ever launched, whether accidentally or deliberately, they would wipe out almost all human life, and all large animals.

Strategically, the setup is stupid. Ethically, it is monstrous.

If the US or Russia sent its nuclear arsenal to destroy the other, would it even make sense to retaliate? Ellsberg argues that it doesn’t matter one way or another. The nuclear winter generated by the original attack would be enough to starve to death most people in the aggressor country within a year anyway. Retaliation would just slightly speed up their demise.

So – how was such a system built? Why does it remain to this day? And how might we shrink our nuclear arsenals to the point they don’t risk the destruction of civilization?

Daniel explores these questions eloquently and urgently in his book (that everyone should read), and this conversation is a gripping introduction. We cover:

  • Why full disarmament today would be a mistake
  • What are our greatest current risks from nuclear weapons?
  • What has changed most since Daniel was working in and around the government in the 50s and 60s?
  • How well are secrets kept in the government?
  • How much deception is involved within the military?
  • The capacity of groups to commit evil
  • How Hitler was more cautious than America about nuclear weapons
  • What was the risk of the first atomic bomb test?
  • The effect of Trump on nuclear security
  • What practical changes should we make? What would Daniel do if he were elected president?
  • Do we have a reliable estimate of the magnitude of a ‘nuclear winter’?
  • What would be the optimal number of nuclear weapons for the US and its allies to hold?
  • What should we make of China’s nuclear stance? What are the chances of war with China?
  • Would it ever be right to respond to a nuclear first strike?
  • Should we help Russia get better attack detection methods to make them less anxious?
  • How much power do lobbyists really have?
  • Has game theory had any influence over nuclear strategy?
  • Why Gorbachev allowed Russia’s covert biological warfare program to continue
  • Is it easier to help solve the problem from within the government or at outside orgs?
  • What gives Daniel hope for the future?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours podcast is produced by Keiran Harris.

Continue reading →

#42 – Amanda Askell on moral empathy, the value of information & the ethics of infinity

By and · Published September 11th, 2018

Consider two familiar moments at a family reunion.

Our host, Uncle Bill, is taking pride in his barbequing skills. But his niece Becky says that she now refuses to eat meat. A groan goes round the table; the family mostly think of this as an annoying picky preference. But were it viewed as a moral position rather than personal preference – as they might if instead Becky were avoiding meat on religious grounds – it would usually receive a very different reaction.

An hour later Bill expresses a strong objection to abortion. Again, a groan goes round the table: the family mostly think that he has no business in trying to foist his regressive preferences on other people’s personal lives. But if considered not as a matter of personal taste, but rather as a moral position – that Bill genuinely believes he’s opposing mass-murder – his comment might start a serious conversation.

Amanda Askell, who recently completed a PhD in philosophy at NYU focused on the ethics of infinity, thinks that we often betray a complete lack of moral empathy. Across the political spectrum, we’re unable to get inside the mindset of people who expresses views that we disagree with, and see the issue from their point of view.

A common cause of conflict, as above, is confusion between personal preferences and moral positions. Assuming good faith on the part of the person you disagree with, and actually engaging with the beliefs they claim to hold, is perhaps the best remedy for our inability to make progress on controversial issues.

One seeming path to progress involves contraception. A lot of people who are anti-abortion are also anti-contraception. But they’ll usually think that abortion is much worse than contraception – so why can’t we compromise and agree to have much more contraception available?

According to Amanda, a charitable explanation is that people who are anti-abortion and anti-contraception engage in moral reasoning and advocacy based on what, in their minds, is the best of all possible worlds: one where people neither use contraception nor get abortions.

So instead of arguing about abortion and contraception, we could discuss the underlying principle that one should advocate for the best possible world, rather than the best probable world. Successfully break down such ethical beliefs, absent political toxicity, and it might be possible to actually figure out why we disagree and perhaps even converge on agreement.

Today’s episode blends such practical topics with cutting-edge philosophy. We cover:

  • The problem of ‘moral cluelessness’ – our inability to predict the consequences of our actions – and how we might work around it
  • Amanda’s biggest criticisms of social justice activists, and of critics of social justice activists
  • Is there an ethical difference between prison and corporal punishment? Are both or neither justified?
  • How to resolve ‘infinitarian paralysis’ – the inability to make decisions when infinities get involved.
  • What’s effective altruism doing wrong?
  • How should we think about jargon? Are a lot of people who don’t communicate clearly just trying to scam us?
  • How can people be more successful while they remain within the cocoon of school and university?
  • How did Amanda find her philosophy PhD, and how will she decide what to do now?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours podcast is produced by Keiran Harris.

Continue reading →

#41 – David Roodman on incarceration, geomagnetic storms, & becoming a world-class researcher

By and · Published August 28th, 2018

With 698 inmates per 100,000 citizens, the U.S. is the world’s leader in incarcerating people. But what effect does this actually have on crime?

According to David Roodman, Senior Advisor to Open Philanthropy, the marginal effect is zero.

This stunning rebuke to the American criminal justice system comes from the man Holden Karnofsky called “the gold standard for in-depth quantitative research”. His other investigations include the risk of geomagnetic storms, whether deworming improves health and test scores, and the development impacts of microfinance – all of which we also cover in this episode.

In his comprehensive review of the evidence, David says the effects of crime can be split into three categories; before, during, and after.

Does having tougher sentences deter people from committing crime? After reviewing studies on gun laws and ‘three strikes’ in California, David concluded that the effect of deterrence is zero.

Does imprisoning more people reduce crime by incapacitating potential offenders? Here he says yes, noting that crimes like motor vehicle theft have gone up in a way that seems pretty clearly connected with recent Californian criminal justice reforms (though the effect on violent crime is far lower).

Finally, do the after-effects of prison make you more or less likely to commit future crimes?

This one is more complicated.

His literature review suggested that more time in prison made people substantially more likely to commit future crimes when released. But concerned that he was biased towards a comfortable position against incarceration, David did a cost-benefit analysis using both his favoured reading of the evidence and the devil’s advocate view; that there is deterrence and that the after-effects are beneficial.

For the devil’s advocate position David used the highest assessment of the harm caused by crime, which suggests a year of prison prevents about $92,000 in crime. But weighed against a lost year of liberty, valued at $50,000, and the cost of operating prisons, the numbers came out exactly the same.

So even using the least-favourable cost-benefit valuation of the least favourable reading of the evidence — it just breaks even.

The argument for incarceration melts further when you consider the significant crime that occurs within prisons, de-emphasised because of a lack of data and a perceived lack of compassion for inmates.

In today’s episode we discuss how to conduct such impactful research, and how to proceed having reached strong conclusions.

We also cover:

  • How do you become a world class researcher? What kinds of character traits are important?
  • Are academics aware of following perverse incentives?
  • What’s involved in data replication? How often do papers replicate?
  • The politics of large orgs vs. small orgs
  • How do you decide what questions to research?
  • How concerned should a researcher be with their own biases?
  • Geomagnetic storms as a potential cause area
  • How much does David rely on interviews with experts?
  • The effects of deworming on child health and test scores
  • Is research getting more reliable? Should we have ‘data vigilantes’?
  • What are David’s critiques of effective altruism?
  • What are the pros and cons of starting your career in the think tank world? Do people generally have a high impact?
  • How do we improve coordination across groups, given our evolutionary history?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours podcast is produced by Keiran Harris.

Continue reading →

#40 – Katja Grace on forecasting future technology & how much we should trust expert predictions.

By and · Published August 21st, 2018

Experts believe that artificial intelligence will be better than humans at driving trucks by 2027, working in retail by 2031, writing bestselling books by 2049, and working as surgeons by 2053. But how seriously should we take these predictions?

Katja Grace, lead author of ‘When Will AI Exceed Human Performance?’, thinks we should treat such guesses as only weak evidence. But she also says there might be much better ways to forecast transformative technology, and that anticipating such advances could be one of our most important projects.

Note: Katja’s organisation AI Impacts is currently hiring part- and full-time researchers.

There’s often pessimism around making accurate predictions in general, and some areas of artificial intelligence might be particularly difficult to forecast.

But there are also many things we’re now able to predict confidently — like the climate of Oxford in five years — that we no longer give ourselves much credit for.

Some aspects of transformative technologies could fall into this category. And these easier predictions could give us some structure on which to base the more complicated ones.

One controversial debate surrounds the idea of an intelligence explosion; how likely is it that there will be a sudden jump in AI capability?

And one way to tackle this is to investigate a more concrete question: what’s the base rate of any technology having a big discontinuity?

A significant historical example was the development of nuclear weapons. Over thousands of years, the energy density of explosives didn’t increase by much. Then within a few years, it got thousands of times better. Discovering what leads to such anomalies may allow us to better predict the possibility of a similar jump in AI capabilities.

Katja likes to compare our efforts to predict AI with those to predict climate change. While both are major problems (though Katja and 80,000 Hours have argued that we should prioritise AI safety), only climate change has prompted hundreds of millions of dollars of prediction research.

That neglect creates a high impact opportunity, and Katja believes that talented researchers should strongly consider following her path.

Some promising research questions include:

  • What’s the relationship between brain size and intelligence?
  • How frequently, and when, do technological trends undergo discontinuous progress?
  • What’s the explanation for humans’ radical success over other apes?
  • What are the best arguments for a local, fast takeoff?

In today’s interview we also discuss:

  • Why is AI impacts one of the most important projects in the world?
  • How do you structure important surveys? Why do you get such different answers when asking what seem to be very similar questions?
  • How does writing an academic paper differ from posting a summary online?
  • When will unguided machines be able to produce better and cheaper work than humans for every possible task?
  • What’s one of the most likely jobs to be automated soon?
  • Are people always just predicting the same timelines for new technologies?
  • How do AGI researchers different from other AI researchers in their predictions?
  • What are attitudes to safety research like within ML? Are there regional differences?
  • Are there any other types of experts we ought to talk to on this topic?
  • How much should we believe experts generally?
  • How does the human brain compare to our best supercomputers? How many human brains are worth all the hardware in the world?
  • How quickly has the processing capacity for machine learning problems been increasing?
  • What can we learn from the development of previous technologies in figuring out how fast transformative AI will arrive?
  • What are the best arguments for and against discontinuous development of AI?
  • Comparing our predictions of climate change and AI development
  • How should we measure human capacity to predict generally?
  • How have things changed in the AI landscape over the last 5 years?
  • How likely is an AI explosion?
  • What should we expect from a post AI dominated economy?
  • Should people focus specifically on the early timeline scenarios even if they consider them unlikely?
  • How much influence can people ever have on things that will happen in 20 years? Are there any examples of people really trying to do this?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours podcast is produced by Keiran Harris.

Continue reading →

#39 – Spencer Greenberg on the scientific approach to solving difficult everyday questions.

By and · Published August 7th, 2018

Will Trump be re-elected? Will North Korea give up their nuclear weapons? Will your friend turn up to dinner?

Spencer Greenberg, founder of ClearerThinking.org, has a process for working out such real life problems.

Let’s work through one here: how likely is it that you’ll enjoy listening to this episode?

The first step is to figure out your ‘prior probability’: your estimate of how likely you are to enjoy the interview before getting any further evidence.

Other than applying common sense, one way to figure this out is ‘reference class forecasting’. That is, looking at similar cases and seeing how often something is true, on average.

Spencer is our first ever return guest (Dr Anders Sandberg appeared on episodes 29 and 33 – but only because his one interview was so fascinating that we split it into two).

So one reference class might be, how many Spencer Greenberg episodes of the 80,000 Hours Podcast have you enjoyed so far? Being this specific limits bias in your answer, but with a sample size of just one – you’ll want to add more data points to reduce the variance of the answer (100% or 0% are both too extreme answers).

Zooming out, how many episodes of the 80,000 Hours Podcast have you enjoyed? Let’s say you’ve listened to 10, and enjoyed 8 of them. If so 8 out of 10 might be a reasonable prior.

If we want a bigger sample we can zoom out further: what fraction of long-form interview podcasts have you ever enjoyed?

Having done that you’d need to update whenever new information became available. Do the topics seem more interesting than average? Did Spencer make a great point in the first 5 minutes? Was this description unbearably self-referential?

In the episode we’ll explain the mathematically correct way to update your beliefs over time as new information comes in: Bayes Rule. You take your initial odds, multiply them by a ‘Bayes Factor’ and boom – updated probabilities. Once you know the trick it’s even easy to do it in your head. We’ll run through several diverse case studies of updating on evidence.

Speaking of the Question of Evidence: in a world where Spencer was not worth listening to, how likely is it that we’d invite him back for a second episode?

Also in this episode:

  • How could we generate 20-30 new happy thoughts a day? What would that do to our welfare?
  • What do people actually value? How do EAs differ from non EAs?
  • Why should we care about the distinction between intrinsic and instrumental values?
  • Should hedonistic utilitarians really want to hook themselves up to happiness machines?
  • What types of activities are people generally under-confident about? Why?
  • When should you give a lot of weight to your existing beliefs?
  • When should we trust common sense?
  • Does power posing have any effect?
  • Are resumes worthless?
  • Did Trump explicitly collude with Russia? What are the odds of him getting re-elected?
  • What’s the probability that China and the US go to War in the 21st century?
  • How should we treat claims of expertise on nutrition?
  • Why were Spencer’s friends suspicious of Theranos for years?
  • How should we think about the placebo effect?
  • Does a shift towards rationality typically cause alienation from family and friends? How do you deal with that?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours podcast is produced by Keiran Harris.

Continue reading →