New releases

#154 – Rohin Shah on DeepMind and trying to fairly hear out both AI doomers and doubters

Podcast by and · Published June 9th, 2023

Can there be a more exciting and strange place to work today than a leading AI lab? Your CEO has said they’re worried your research could cause human extinction. The government is setting up meetings to discuss how this outcome can be avoided. Some of your colleagues think this is all overblown; others are more anxious still.

Today’s guest — machine learning researcher Rohin Shah — goes into the Google DeepMind offices each day with that peculiar backdrop to his work.

He’s on the team dedicated to maintaining ‘technical AI safety’ as these models approach and exceed human capabilities: basically that the models help humanity accomplish its goals without flipping out in some dangerous way. This work has never seemed more important.

In the short-term it could be the key bottleneck to deploying ML models in high-stakes real-life situations. In the long-term, it could be the difference between humanity thriving and disappearing entirely.

For years Rohin has been on a mission to fairly hear out people across the full spectrum of opinion about risks from artificial intelligence — from doomers to doubters — and properly understand their point of view. That makes him unusually well placed to give an overview of what we do and don’t understand. He has landed somewhere in the middle — troubled by ways things could go wrong, but not convinced there are very strong reasons to expect a terrible outcome.

Today’s conversation is wide-ranging and Rohin lays out many of his personal opinions to host Rob Wiblin, including:

  • What he sees as the strongest case both for and against slowing down the rate of progress in AI research.
  • Why he disagrees with most other ML researchers that training a model on a sensible ‘reward function’ is enough to get a good outcome.
  • Why he disagrees with many on LessWrong that the bar for whether a safety technique is helpful is “could this contain a superintelligence.”
  • That he thinks nobody has very compelling arguments that AI created via machine learning will be dangerous by default, or that it will be safe by default. He believes we just don’t know.
  • That he understands that analogies and visualisations are necessary for public communication, but is sceptical that they really help us understand what’s going on with ML models, because they’re different in important ways from every other case we might compare them to.
  • Why he’s optimistic about DeepMind’s work on scalable oversight, mechanistic interpretability, and dangerous capabilities evaluations, and what each of those projects involves.
  • Why he isn’t inherently worried about a future where we’re surrounded by beings far more capable than us, so long as they share our goals to a reasonable degree.
  • Why it’s not enough for humanity to know how to align AI models — it’s essential that management at AI labs correctly pick which methods they’re going to use and have the practical know-how to apply them properly.
  • Three observations that make him a little more optimistic: humans are a bit muddle-headed and not super goal-orientated; planes don’t crash; and universities have specific majors in particular subjects.
  • Plenty more besides.

Continue reading →

Practical steps to take now that AI risk is mainstream

Blog post by · Published June 6th, 2023

AI risk has gone mainstream. So what’s next?

Last Tuesday’s statement on AI risk has hit headlines across the world. Hundreds of leading AI scientists and other prominent figures — including the CEOs of OpenAI, Anthropic and Google DeepMind — signed the one-sentence statement:

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

This mainstreaming of concerns about the risk of extinction from AI represents a substantial shift to the strategic landscape — and should, as a result, have implications on how best to reduce the risk.

How has the landscape shifted?

Pictures from the White House Press Briefing. Meme from @kristjanmoore. The relevant video is here.

So far, I think the most significant effect of the changes in the way these risks are viewed can be seen in changes in political activity.

World leaders — including Joe Biden, Rishi Sunak, Emmanuel Macron — have all met leaders in AI in the last few months. AI regulation was a key topic of discussion at the G7. And now it’s been announced that Biden and Sunak will discuss extinction risks from AI as part of talks in DC next week.

At the moment,

Continue reading →

#153 – Elie Hassenfeld on two big-picture critiques of GiveWell’s approach, and six lessons from their recent work

Podcast by and · Published June 2nd, 2023

GiveWell is one of the world’s best-known charity evaluators, with the goal of “searching for the charities that save or improve lives the most per dollar.” It mostly recommends projects that help the world’s poorest people avoid easily prevented diseases, like intestinal worms or vitamin A deficiency.

But should GiveWell, as some critics argue, take a totally different approach to its search, focusing instead on directly increasing subjective wellbeing, or alternatively, raising economic growth?

Today’s guest — cofounder and CEO of GiveWell, Elie Hassenfeld — is proud of how much GiveWell has grown in the last five years. Its ‘money moved’ has quadrupled to around $600 million a year.

Its research team has also more than doubled, enabling them to investigate a far broader range of interventions that could plausibly help people an enormous amount for each dollar spent. That work has led GiveWell to support dozens of new organisations, such as Kangaroo Mother Care, MiracleFeet, and Dispensers for Safe Water.

But some other researchers focused on figuring out the best ways to help the world’s poorest people say GiveWell shouldn’t just do more of the same thing, but rather ought to look at the problem differently.

Currently, GiveWell uses a range of metrics to track the impact of the organisations it considers recommending — such as ‘lives saved,’ ‘household incomes doubled,’ and for health improvements, the ‘quality-adjusted life year.’ To compare across opportunities, it then needs some way of weighing these different types of benefits up against one another. This requires estimating so-called “moral weights,” which Elie agrees is far from the most mature part of the project.

Continue reading →

The public is more concerned about AI causing extinction than we thought

Blog post by · Published May 26th, 2023

What does the public think about risks of human extinction?

We care a lot about reducing extinction risks and think doing so is one of the best ways you can have a positive impact with your career. But even before considering career impact, it can be natural to worry about these risks — and as it turns out, many people do!

In April 2023, the US firm YouGov polled 1,000 American adults on how worried they were about nine different potential extinction threats. It found the following percentages of respondents were either “concerned” or “very concerned” about extinction from each threat:

We’re particularly interested in this poll now because we have recently updated our page on the world’s most pressing problems, which includes several of these extinction risks at the top.

Knowing how the public feels about these kinds of threats can impact how we communicate about them.

For example, if we take the results at face value, 46% of the poll’s respondents are concerned about human extinction caused by artificial intelligence. Maybe this surprisingly high figure means we don’t need to worry as much as we have over the last 10 years about sounding like ‘sci fi’ when we talk about existential risks from AI, since it’s quickly becoming a common concern!

How does our view of the world’s most pressing problems compare?

Continue reading →

Give feedback on the new 80,000 Hours career guide

Blog post by · Published May 25th, 2023

We’ve spent the last few months updating 80,000 Hours’ career guide (which we previously released in 2017 and which you’ve been able to get as a physical book). This week, we’ve put our new career guide live on our website. Before we formally launch and promote the guide — and republish the book — we’d like to gather feedback from our readers!

How can you help?

First, take a look at the new career guide.

Note that our target audience for this career guide is approximately the ~100k young adults most likely to have high-impact careers, in the English-speaking world. Many of them may not yet be familiar with many of the ideas that are widely discussed in the effective altruism community. Also, this guide is primarily aimed at people aged 18–24.

When you’re ready, there’s a simple form to fill in:

Give feedback

Thank you so much!

Extra context: why are we making this change?

In 2018, we deprioritised 80,000 Hours’ career guide in favour of our key ideas series.

Our key ideas series had a more serious tone, and was more focused on impact. It represented our best and most up-to-date advice. We expected that this switch would reduce engagement time on our site, but that the key ideas series would better appeal to people more likely to change their careers to do good.

Continue reading →

#152 – Joe Carlsmith on navigating serious philosophical confusion

Podcast by , and · Published May 19th, 2023

What is the nature of the universe? How do we make decisions correctly? What differentiates right actions from wrong ones?

Such fundamental questions have been the subject of philosophical and theological debates for millennia. But, as we all know, and surveys of expert opinion make clear, we are very far from agreement. So… with these most basic questions unresolved, what’s a species to do?

In today’s episode, philosopher Joe Carlsmith — Senior Research Analyst at Open Philanthropy — makes the case that many current debates in philosophy ought to leave us confused and humbled. These are themes he discusses in his PhD thesis, A stranger priority? Topics at the outer reaches of effective altruism.

To help transmit the disorientation he thinks is appropriate, Joe presents three disconcerting theories — originating from him and his peers — that challenge humanity’s self-assured understanding of the world.

The first idea is that we might be living in a computer simulation, because, in the classic formulation, if most civilisations go on to run many computer simulations of their past history, then most beings who perceive themselves as living in such a history must themselves be in computer simulations. Joe prefers a somewhat different way of making the point, but, having looked into it, he hasn’t identified any particular rebuttal to this ‘simulation argument.’

If true, it could revolutionise our comprehension of the universe and the way we ought to live.

The second is the idea that “you can ‘control’ events you have no causal interaction with, including events in the past.” The thought experiment that most persuades him of this is the following:

Perfect deterministic twin prisoner’s dilemma: You’re a deterministic AI system, who only wants money for yourself (you don’t care about copies of yourself). The authorities make a perfect copy of you, separate you and your copy by a large distance, and then expose you both, in simulation, to exactly identical inputs (let’s say, a room, a whiteboard, some markers, etc.). You both face the following choice: either (a) send a million dollars to the other (“cooperate”), or (b) take a thousand dollars for yourself (“defect”).

Continue reading →

#151 – Ajeya Cotra on accidentally teaching AI models to deceive us

Podcast by , and · Published May 12th, 2023

Ajeya Cotra on accidentally teaching AI models to deceive us

Imagine you are an orphaned eight-year-old whose parents left you a $1 trillion company, and no trusted adult to serve as your guide to the world. You have to hire a smart adult to run that company, guide your life the way that a parent would, and administer your vast wealth. You have to hire that adult based on a work trial or interview you come up with. You don’t get to see any resumes or do reference checks. And because you’re so rich, tonnes of people apply for the job — for all sorts of reasons.

Today’s guest Ajeya Cotra — senior research analyst at Open Philanthropy — argues that this peculiar setup resembles the situation humanity finds itself in when training very general and very capable AI models using current deep learning methods.

As she explains, such an eight-year-old faces a challenging problem. In the candidate pool there are likely some truly nice people, who sincerely want to help and make decisions that are in your interest. But there are probably other characters too — like people who will pretend to care about you while you’re monitoring them, but intend to use the job to enrich themselves as soon as they think they can get away with it.

Like a child trying to judge adults, at some point humans will be required to judge the trustworthiness and reliability of machine learning models that are as goal-oriented as people, and greatly outclass them in knowledge, experience, breadth, and speed. Tricky!

Continue reading →

How 80,000 Hours has changed some of our advice after the collapse of FTX

Blog post by · Published May 12th, 2023

Following the bankruptcy of FTX and the federal indictment of Sam Bankman-Fried, many members of the team at 80,000 Hours were deeply shaken. As we have said, we had previously featured Sam on our site as a positive example of earning to give, a mistake we now regret. We felt appalled by his conduct and at the harm done to the people who had relied on FTX.

These events were emotionally difficult for many of us on the team, and we were troubled by the implications it might have for our attempts to do good in the world. We had linked our reputation with his, and his conduct left us with serious questions about effective altruism and our approach to impactful careers.

We reflected a lot, had many difficult conversations, and worked through a lot of complicated questions. There’s still a lot we don’t know about what happened, there’s a diversity of views within the 80,000 Hours team, and we expect the learning process to be ongoing.

Ultimately, we still believe strongly in the principles that drive our work, and we stand by the vast majority of our advice. But we did make some significant updates in our thinking, and we’ve changed many parts of the site to reflect them. We wrote this post to summarise the site updates we’ve made and to explain the motivations behind them, for transparency purposes and to further highlight the themes that unify the changes.

Continue reading →

#150 – Tom Davidson on how quickly AI could transform the world

Podcast by , and · Published May 5th, 2023

It’s easy to dismiss alarming AI-related predictions when you don’t know where the numbers came from.

For example: what if we told you that within 15 years, it’s likely that we’ll see a 1,000x improvement in AI capabilities in a single year? And what if we then told you that those improvements would lead to explosive economic growth unlike anything humanity has seen before?

You might think, “Congratulations, you said a big number — but this kind of stuff seems crazy, so I’m going to keep scrolling through Twitter.”

But this 1,000x yearly improvement is a prediction based on real economic models created by today’s guest Tom Davidson, Senior Research Analyst at Open Philanthropy. By the end of the episode, you’ll either be able to point out specific flaws in his step-by-step reasoning, or have to at least consider the idea that the world is about to get — at a minimum — incredibly weird.

As a teaser, consider the following:

Developing artificial general intelligence (AGI) — AI that can do 100% of cognitive tasks at least as well as the best humans can — could very easily lead us to an unrecognisable world.

You might think having to train AI systems individually to do every conceivable cognitive task — one for diagnosing diseases, one for doing your taxes, one for teaching your kids, etc. — sounds implausible, or at least like it’ll take decades.

But Tom thinks we might not need to train AI to do every single job — we might just need to train it to do one: AI research.

Continue reading →

Some thoughts on moderation in doing good

Blog post by · Published May 5th, 2023

Here’s one of the deepest tensions in doing good:

How much should you do what seems right to you, even if it seems extreme or controversial, vs how much should you moderate your views and actions based on other perspectives?

If you moderate too much, you won’t be doing anything novel or ambitious, which really reduces how much impact you might have. The people who have had the biggest impact historically often spoke out about entrenched views and were met with hostility — think of the civil rights movement or Galileo.

Moreover, simply following ethical ‘common sense’ has a horrible track record. It used to be common sense to think that homosexuality was evil, slavery was the natural order, and that the environment was there for us to exploit.

And there is still so much wrong with the world. Millions of people die of easily preventable diseases, society is deeply unfair, billions of animals are tortured in factory farms, and we’re gambling our entire future by failing to mitigate threats like climate change. These huge problems deserve radical action — while conventional wisdom appears to accept doing little about them.

On a very basic level, doing more good is better than doing less. But this is a potentially endless and demanding principle, and most people don’t give it much attention or pursue it very systematically. So it wouldn’t be surprising if a concern for doing good led you to positions that seem radical or unusual to the rest of society.

Continue reading →

Why we’re adding information security to our list of priority career paths

Blog post by · Published April 28th, 2023

Information security could be a top option for people looking to have a high-impact career.

This might be a surprising claim — information security is a relatively niche field, and it doesn’t typically appear on canonical lists of do-gooder careers.

But we think there’s an unusually strong case that information security skills (which allow you to protect against unauthorised use, hacking, leaks, and tampering) will be key to addressing problems that are extremely important, neglected, and tractable. We now rank this career among the highest-impact paths we’ve researched.

In the introduction to our recently updated career review of information security, we discuss how poor information security decisions may have played a decisive role in the 2016 US presidential campaign. If an organisation is big and influential, it needs good information security to ensure that it functions as intended. This is true whether it’s a political campaign, a major corporation, a biolab, or an AI company.

These last two cases could be quite important. We rank the risks from pandemic viruses and the chances of an AI-related catastrophe among the most pressing problems in the world — and information security is likely a key part of reducing these dangers.

That’s because hackers and cyberattacks — from a range of actors with varying motives — could try to steal crucial information, such as instructions for making a super-virus or the details of an extremely powerful AI model.

Continue reading →

Luisa and Keiran on free will, and the consequences of never feeling enduring guilt or shame

Podcast (80k After Hours) by and · Published April 2023

In this episode of 80k After Hours, Luisa Rodriguez and Keiran Harris chat about the consequences of letting go of enduring guilt, shame, anger, and pride.

They cover:

  • Keiran’s views on free will, and how he came to hold them
  • What it’s like not experiencing sustained guilt, shame, and anger
  • Whether Luisa would become a worse person if she felt less guilt and shame, specifically whether she’d work fewer hours, or donate less money, or become a worse friend
  • Whether giving up guilt and shame also means giving up pride
  • The implications for love
  • The neurological condition ‘Jerk Syndrome’
  • And some practical advice on feeling less guilt, shame, and anger

Who this episode is for:

  • People sympathetic to the idea that free will is an illusion
  • People who experience tons of guilt, shame, or anger
  • People worried about what would happen if they stopped feeling tons of guilt, shame, or anger

Who this episode isn’t for:

  • People strongly in favour of retributive justice
  • Philosophers who can’t stand random non-philosophers talking about philosophy
  • Non-philosophers who can’t stand random non-philosophers talking about philosophy

Continue reading →

Are we doing enough to stop the worst pandemics?

Blog post by · Published April 21st, 2023

COVID-19 has been devastating for the world. While people debate how the response could’ve been better, it should be easy to agree that we’d all be better off if we can stop any future pandemic before it occurs. But we’re still not taking pandemic prevention very seriously.

A recent report in The Washington Post highlighted one major danger: some research on potential pandemic pathogens may actually increase the risk, rather than reduce it.

Back in 2017, we talked about what we thought were several warning signs that something like COVID might be coming down the line. It’d be a big mistake to ignore these kinds of warning signs again.

It seems unfortunate that so much of the discussion of the risks in this space is backward-looking. The news has been filled with commentary and debates about the chances that COVID accidentally emerged from a biolab or that it crossed over directly from animals to humans.

We’d appreciate a definitive answer to this question as much as anyone, but there’s another question that matters much more but gets asked much less:

What are we doing to reduce the risk that the next dangerous virus — which could come from an animal, a biolab, or even a bioterrorist attack — causes a pandemic even worse than COVID-19?

80,000 Hours ranks preventing catastrophic pandemics as among the most pressing problems in the world.

Continue reading →

How much should you research your career?

Blog post by · Published April 18th, 2023

In career decisions, we advise that you don’t aim for confidence — aim for a stable best guess.

Career decisions have a big impact on your life, so it’s natural to want to feel confident in them.

Unfortunately, you don’t always get this luxury.

For years, I’ve faced the decision of whether to focus more on writing, organisation building, or something else. And despite giving it a lot of thought, I’ve rarely felt more than 60% confident in one of the options.

How should you handle these kinds of situations?

The right response isn’t just to guess, flip a coin, or “follow your heart.”

It’s still worth identifying your key uncertainties, and doing your research: speak to people, do side projects, learn about each path, etc.

Sometimes you’ll quickly realise one answer is best. If we plot your confidence against how much research you’ve done, it’ll look like this:
Deliberation graph

But sometimes that doesn’t happen. What then?

Stop your research when your best guess stops changing.

That might look more like this:
Deliberation graph

This can be painful. You might only be 51% confident in your best guess, and it really sucks to have to make a decision when you feel so uncertain.

But certainty is not always achievable. You might face questions that both (i) are important but (ii) can’t realistically be resolved — which I think is the situation I faced.

Continue reading →

#149 – Tim LeBon on how altruistic perfectionism is self-defeating

Podcast by and · Published April 12th, 2023

Being a good and successful person is core to your identity. You place great importance on meeting the high moral, professional, or academic standards you set yourself.

But inevitably, something goes wrong and you fail to meet that high bar. Now you feel terrible about yourself, and worry others are judging you for your failure. Feeling low and reflecting constantly on whether you’re doing as much as you think you should makes it hard to focus and get things done. So now you’re performing below a normal level, making you feel even more ashamed of yourself. Rinse and repeat.

This is the disastrous cycle today’s guest, Tim LeBon — registered psychotherapist, accredited CBT therapist, life coach, and author of 365 Ways to Be More Stoic — has observed in many clients with a perfectionist mindset.

Tim has provided therapy to a number of 80,000 Hours readers — people who have found that the very high expectations they had set for themselves were holding them back. Because of our focus on “doing the most good you can,” Tim thinks 80,000 Hours both attracts people with this style of thinking and then exacerbates it.

But Tim, having studied and written on moral philosophy, is sympathetic to the idea of helping others as much as possible, and is excited to help clients pursue that — sustainably — if it’s their goal.

Tim has treated hundreds of clients with all sorts of mental health challenges. But in today’s conversation, he shares the lessons he has learned working with people who take helping others so seriously that it has become burdensome and self-defeating — in particular, how clients can approach this challenge using the treatment he’s most enthusiastic about: cognitive behavioural therapy.

Continue reading →

#148 – Johannes Ackva on unfashionable climate interventions that work, and fashionable ones that don’t

Podcast by and · Published April 3rd, 2023

If you want to work to tackle climate change, you should try to reduce expected carbon emissions by as much as possible, right? Strangely, no.

Today’s guest, Johannes Ackva — the climate research lead at Founders Pledge, where he advises major philanthropists on their giving — thinks the best strategy is actually pretty different, and one few are adopting.

In reality you don’t want to reduce emissions for its own sake, but because emissions will translate into temperature increases, which will cause harm to people and the environment.

Crucially, the relationship between emissions and harm goes up faster than linearly. As Johannes explains, humanity can handle small deviations from the temperatures we’re familiar with, but adjustment gets harder the larger and faster the increase, making the damage done by each additional degree of warming much greater than the damage done by the previous one.

In short: we’re uncertain what the future holds and really need to avoid the worst-case scenarios. This means that avoiding an additional tonne of carbon being emitted in a hypothetical future in which emissions have been high is much more important than avoiding a tonne of carbon in a low-carbon world.

That may be, but concretely, how should that affect our behaviour? Well, the future scenarios in which emissions are highest are all ones in which clean energy tech that can make a big difference — wind, solar, and electric cars — don’t succeed nearly as much as we are currently hoping and expecting. For some reason or another, they must have hit a roadblock and we continued to burn a lot of fossil fuels.

Continue reading →

What our research has found about AI — and why it matters

Blog post by · Published March 31st, 2023

Everyone’s suddenly talking a lot about artificial intelligence — and we have many helpful resources for getting up to speed.

With the release of GPT-4, Bing, DALL-E, Claude, and many other AI systems, it can be hard to keep track of all the latest developments in artificial intelligence. It can also be hard to keep sight of the big picture: what does this emerging technology actually mean for the world?

This is a huge topic — and a lot is still unknown. But at 80,000 Hours, we’ve been interested in and concerned about AI for many years, and we’ve researched the issue extensively. Now, even major media outlets are taking seriously the kinds of things we’ve been worried about. Given all the excitement in this area, we wanted to share a round-up of some of our top content and findings about AI from recent years.

Some of our top articles on AI:

Continue reading →

Longtermism: a call to protect future generations

Article by · Published March 2023

When the 19th-century amateur scientist Eunice Newton Foote filled glass cylinders with different gases and exposed them to sunlight, she uncovered a curious fact. Carbon dioxide became hotter than regular air and took longer to cool down.

Remarkably, Foote saw what this momentous discovery meant.

“An atmosphere of that gas would give our earth a high temperature,” she wrote in 1857.

Though Foote could hardly have been aware at the time, the potential for global warming due to carbon dioxide would have massive implications for the generations that came after her.

If we ran history over again from that moment, we might hope that this key discovery about carbon’s role in the atmosphere would inform governments’ and industries’ choices in the coming century. They probably shouldn’t have avoided carbon emissions altogether, but they could have prioritised the development of alternatives to fossil fuels much sooner in the 20th century, and we might have prevented much of the destructive climate change that present people are already beginning to live through — which will affect future generations as well.

We believe it would’ve been much better if previous generations had acted on Foote’s discovery, especially by the 1970s, when climate models were beginning to reliably show the future course of warming global trends.

If this seems right, it’s because of a commonsense idea: to the extent that we are able to, we have strong reasons to consider the interests and promote the welfare of future generations.

Continue reading →

Ways people trying to do good accidentally make things worse, and how to avoid them

Article by and · Last updated March 2023 · First published October 2018

We advise people to work on problems that are important but neglected, and to try to increase the contribution they’re able to have.

These steps make it easier to have a big impact, but they also increase your potential to make things worse: the more important the problem, the worse it is to set it back; the more neglected an area, the more effect you have on its trajectory; and the more influence you have, the more it matters if you’re wrong.

This holds even if you’re not doing anything directly harmful and trying to be cautious — it’s easy to make things worse by accident, and indeed to make them much worse.

In some areas of life, your downsides are relatively capped. If you try to write a great novel, and no one wants to publish it, the worst thing you’ve done is waste some time.

But we’ll show that when it comes to doing good — especially in ‘fragile fields’ — there are many ways to set back the broader field, and so the downsides aren’t limited in the same way. The potential for negative impact can be as big or greater than the potential for positive.

So if you’re going to try to have an impact, and especially if you’re going to be ambitious about it, it’s very important to carefully consider how you might accidentally make things worse.

This doesn’t mean sticking to ‘sure things’

Continue reading →