New releases

#140 – Bear Braumoeller on the case that war isn't in decline

Podcast by and · Published November 8th, 2022

Is war in long-term decline? Steven Pinker’s The Better Angels of Our Nature brought this previously obscure academic question to the centre of public debate, and pointed to rates of death in war to argue energetically that war is on the way out.

But that idea divides war scholars and statisticians, and so Better Angels has prompted a spirited debate, with datasets and statistical analyses exchanged back and forth year after year. The lack of consensus has left a somewhat bewildered public (including host Rob Wiblin) unsure quite what to believe.

Today’s guest, professor in political science Bear Braumoeller, is one of the scholars who believes we lack convincing evidence that warlikeness is in long-term decline. He collected the analysis that led him to that conclusion in his 2019 book, Only the Dead: The Persistence of War in the Modern Age.

The question is of great practical importance. The US and PRC are entering a period of renewed great power competition, with Taiwan as a potential trigger for war, and Russia is once more invading and attempting to annex the territory of its neighbours.

If war has been going out of fashion since the start of the Enlightenment, we might console ourselves that however nerve-wracking these present circumstances may feel, modern culture will throw up powerful barriers to another world war. But if we’re as war-prone as we ever have been, one need only inspect the record of the 20th century to recoil in horror at what might await us in the 21st.

Bear argues that the second reaction is the appropriate one. The world has gone up in flames many times through history, with roughly 0.5% of the population dying in the Napoleonic Wars, 1% in World War I, 3% in World War II, and perhaps 10% during the Mongol conquests. And with no reason to think similar catastrophes are any less likely today, complacency could lead us to sleepwalk into disaster.

He gets to this conclusion primarily by analysing the datasets of the decades-old Correlates of War project, which aspires to track all interstate conflicts and battlefield deaths since 1815. In Only the Dead, he chops up and inspects this data dozens of different ways, to test if there are any shifts over time which seem larger than what could be explained by chance variation alone.

Among other metrics, Bear looks at:

  • Battlefield deaths alone, as a percentage of combatants’ populations, and as a percentage of world population.
  • The total number of wars starting in a given year.
  • Rates of war initiation as a fraction of all country pairs capable of fighting wars.
  • How likely it was during different periods that a given war would double in size.
Image source.

In a nutshell, and taking in the full picture painted by these different measures, Bear simply finds no general trend in either direction from 1815 through today. It seems like, as philosopher George Santayana lamented in 1922, “only the dead have seen the end of war”.

That’s not to say things are the same in all periods. Depending on which indication of warlikeness you give the greatest weight, you can point to some periods that seem violent or pacific beyond what might be explained by random variation.

For instance, Bear points out that war initiation really did go down a lot at the end of the Cold War, with peace probably fostered by a period of unipolar US dominance, and the end of great power funding for proxy wars.

But that drop came after a period of somewhat above-average warlikeness during the Cold War. And surprisingly, the most peaceful period in Europe turns out not to be 1990–2015, but rather 1815–1855, during which the monarchical ‘Concert of Europe,’ scarred by the Napoleonic Wars, worked together to prevent revolution and interstate aggression.

Why haven’t modern ideas about the immorality of violence led to the decline of war, when it’s such a natural thing to expect? Bear is no Enlightenment scholar, but his book notes (among other reasons) that while modernity threw up new reasons to embrace pacifism, it also gave us new reasons to embrace violence: as a means to overthrow monarchy, distribute the means of production more equally, or protect people a continent away from ethnic cleansing — all motives that would have been foreign in the 15th century.

In today’s conversation, Bear and Rob discuss all of the above in more detail than even a usual 80,000 Hours podcast episode, as well as:

  • What would Bear’s critics say in response to all this?
  • What do the optimists get right?
  • What are the biggest problems with the Correlates of War dataset?
  • How does one do proper statistical tests for events that are clumped together, like war deaths?
  • Why are deaths in war so concentrated in a handful of the most extreme events?
  • Did the ideas of the Enlightenment promote nonviolence, on balance?
  • Were early states more or less violent than groups of hunter-gatherers?
  • If Bear is right, what can be done?
  • How did the ‘Concert of Europe’ or ‘Bismarckian system’ maintain peace in the 19th century?
  • Which wars are remarkable but largely unknown?
  • What’s the connection between individual attitudes and group behaviour?
  • Is it a problem that this dataset looks at just the ‘state system’ and ‘battlefield deaths’?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type ‘80,000 Hours’ into your podcasting app. Or read the transcript below.

Producer: Keiran Harris
Audio mastering: Ryan Kessler
Transcriptions: Katy Moore

Continue reading →

The importance of considering speculative ideas

Blog post by · Published October 30th, 2022

Let’s admit it: some of the things we think about at 80,000 Hours are considered weird by a lot of other people.

Our list of the most pressing problems has some pretty widely accepted concerns, to be sure: we care about mitigating climate change, preventing nuclear war, and ensuring good governance.

But one of our highest priorities is preventing an AI-related catastrophe, which sounds like science fiction to a lot of people. And, though we know less about them, we’re also interested in speculative issues — such as atomically precise manufacturing, artificial sentience, and wild animal suffering. These aren’t typically the kind of issues activists distribute flyers about.

Should it make us nervous that some of our ideas are out of the mainstream? It’s probably a good idea in these cases to take a step back, reexamine our premises, and consult others we trust about our conclusions. But we shouldn’t be too shocked if some of our beliefs end up at odds with common sense — indeed, I think everyone has good reason to be open to believing weird ideas.

One of the best reasons for this view relates to another of 80,000 Hours’ top priorities: preventing catastrophic pandemics. I’d guess few people think it’s strange to be concerned about pandemics now, as COVID-19 has killed more than 6 million people worldwide and thrown the global economy into chaos.

Continue reading →

#139 – Alan Hájek on puzzles and paradoxes in probability and expected value

Podcast by and · Published October 28th, 2022

A casino offers you a game. A coin will be tossed. If it comes up heads on the first flip you win $2. If it comes up on the second flip you win $4. If it comes up on the third you win $8, the fourth you win $16, and so on. How much should you be willing to pay to play?

The standard way of analysing gambling problems, ‘expected value‘ — in which you multiply probabilities by the value of each outcome and then sum them up — says your expected earnings are infinite. You have a 50% chance of winning $2, for ‘0.5 * $2 = $1’ in expected earnings. A 25% chance of winning $4, for ‘0.25 * $4 = $1’ in expected earnings, and on and on. A never-ending series of $1s added together comes to infinity. And that’s despite the fact that you know with certainty you can only ever win a finite amount!

Today’s guest — philosopher Alan Hájek of the Australian National University — thinks of much of philosophy as “the demolition of common sense followed by damage control” and is an expert on paradoxes related to probability and decision-making rules like “maximise expected value.”

The problem described above, known as the St. Petersburg paradox, has been a staple of the field since the 18th century, with many proposed solutions. In the interview, Alan explains how very natural attempts to resolve the paradox — such as factoring in the low likelihood that the casino can pay out very large sums, or the fact that money becomes less and less valuable the more of it you already have — fail to work as hoped.

We might reject the setup as a hypothetical that could never exist in the real world, and therefore of mere intellectual curiosity. But Alan doesn’t find that objection persuasive. If expected value fails in extreme cases, that should make us worry that something could be rotten at the heart of the standard procedure we use to make decisions in government, business, and nonprofits.

These issues regularly show up in 80,000 Hours’ efforts to try to find the best ways to improve the world, as the best approach will arguably involve long-shot attempts to do very large amounts of good.

Consider which is better: saving one life for sure, or three lives with 50% probability? Expected value says the second, which will probably strike you as reasonable enough. But what if we repeat this process and evaluate the chance to save nine lives with 25% probability, or 27 lives with 12.5% probability, or after 17 more iterations, 3,486,784,401 lives with a 0.00000009% chance. Expected value says this final offer is better than the others — 1,000 times better, in fact.

Insisting that people give up a sure thing in favour of a vanishingly low chance of a very large impact strikes some people as peculiar or even fanatical. But one of Alan’s PhD students, Hayden Wilkinson, discovered that rejecting expected value on this basis requires you to swallow even more bitter pills, like giving up on the idea that if A is better than B, and B is better than C, then A is also better than C.

Ultimately Alan leans towards the view that our best choice is to “bite the bullet” and stick with expected value, even with its sometimes counterintuitive implications. Where we want to do damage control, we’re better off looking for ways our probability estimates might be wrong.

In today’s conversation, Alan and Rob explore these issues and many others:

  • Simple rules of thumb for having philosophical insights
  • A key flaw that hid in Pascal’s wager from the very beginning
  • Whether we have to simply ignore infinities because they mess everything up
  • What fundamentally is ‘probability’?
  • Some of the many reasons ‘frequentism’ doesn’t work as an account of probability
  • Why the standard account of counterfactuals in philosophy is deeply flawed
  • And why counterfactuals present a fatal problem for one sort of consequentialism

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type ‘80,000 Hours’ into your podcasting app. Or read the transcript below.

Producer: Keiran Harris
Audio mastering: Ben Cordell and Ryan Kessler
Transcriptions: Katy Moore

Continue reading →

Open position: Recruiter

Blog post by · Published October 19th, 2022

The role

You’ll be managed by Sashika Coxhead, our Head of Recruiting, and will have the opportunity to work closely with hiring managers from other teams.

Initial responsibilities will include:

  • Project management of active recruiting rounds. For example, overseeing the candidate pipeline and logistics of hiring rounds, making decisions on initial applications, and managing candidate communications.
  • Sourcing potential candidates. This might include generating leads for specific roles, publicising new positions, reaching out to potential candidates, and answering any questions they have about working at 80,000 Hours.
  • Taking on special projects to improve our recruiting systems. For example, you might help to build an excellent applicant tracking system, test ways to improve our ability to generate leads, or introduce strategies to make our hiring rounds more efficient.

Depending on your skills and interests, you might also:

  • Take ownership of a particular area of our recruiting process, e.g. proactive outreach to potential candidates, our applicant tracking system, or metrics for the recruiting team’s success.
  • Conduct screening interviews where needed, to assess applicants’ fit for particular roles at 80,000 Hours.

After some time in the role, we’d hope for you to sit on internal hiring committees. This involves forming an inside view on candidates’ performance; discussing uncertainties with the hiring manager and committee; and, with the other committee members, giving final approval on who to make offers to.

Continue reading →

    Anonymous advice: If you want to reduce AI risk, should you take roles that advance AI capabilities?

    Article by · Published October 2022

    We’ve argued that preventing an AI-related catastrophe may be the world’s most pressing problem, and that while progress in AI over the next few decades could have enormous benefits, it could also pose severe, possibly existential risks. As a result, we think that working on some technical AI research — research related to AI safety — may be a particularly high-impact career path.

    But there are many ways of approaching this path that involve researching or otherwise advancing AI capabilities — meaning making AI systems better at some specific skills — rather than only doing things that are purely in the domain of safety. In short, this is because:

    • Capabilities work and some forms of safety work are intertwined.
    • Many available ways of learning enough about AI to contribute to safety are via capabilities-enhancing roles.

    So if you want to help prevent an AI-related catastrophe, should you be open to roles that also advance AI capabilities, or steer clear of them?

    We think this is a hard question! Capabilities-enhancing roles could be beneficial or harmful. For any role, there are a range of considerations — and reasonable people disagree on whether, and in what cases, the risks outweigh the benefits.

    So we asked the 22 people we thought would be most informed about this issue — and who we knew had a range of views —

    Continue reading →

    #138 – Sharon Hewitt Rawlette on why pleasure and pain are the only things that intrinsically matter

    Podcast by and · Published September 30th, 2022

    What in the world is intrinsically good — good in itself even if it has no other effects? Over the millennia, people have offered many answers: joy, justice, equality, accomplishment, loving god, wisdom, and plenty more.

    The question is a classic that makes for great dorm-room philosophy discussion. But it’s hardly just of academic interest. The issue of what (if anything) is intrinsically valuable bears on every action we take, whether we’re looking to improve our own lives, or to help others. The wrong answer might lead us to the wrong project and render our efforts to improve the world entirely ineffective.

    Today’s guest, Sharon Hewitt Rawlette — philosopher and author of The Feeling of Value: Moral Realism Grounded in Phenomenal Consciousness — wants to resuscitate an answer to this question that is as old as philosophy itself.

    That idea, in a nutshell, is that there is only one thing of true intrinsic value: positive feelings and sensations. And similarly, there is only one thing that is intrinsically of negative value: suffering, pain, and other unpleasant sensations.

    Lots of other things are valuable too: friendship, fairness, loyalty, integrity, wealth, patience, houses, and so on. But they are only instrumentally valuable — that is to say, they’re valuable as means to the end of ensuring that all conscious beings experience more pleasure and other positive sensations, and less suffering.

    As Sharon notes, from Athens in 400 BC to Britain in 1850, the idea that only subjective experiences can be good or bad in themselves — a position known as ‘philosophical hedonism’ — has been one of the most enduringly popular ideas in ethics.

    And few will be taken aback by the notion that, all else equal, more pleasure is good and less suffering is bad. But can they really be the only intrinsically valuable things?

    Over the 20th century, philosophical hedonism became increasingly controversial in the face of some seemingly very counterintuitive implications. For this reason the famous philosopher of mind Thomas Nagel called The Feeling of Value “a radical and important philosophical contribution.”

    So what convinces Sharon that philosophical hedonism deserves another go?

    Stepping back for a moment, any answer to the question “What has intrinsic value?” faces a serious challenge: “How do we know?” It’s far from clear how something having intrinsic value can cause us to believe that it has intrinsic value. And if there’s no causal or rational connection between something being valuable and our believing that it has value, we could only get the right answer by some extraordinary coincidence. You may feel it’s intrinsically valuable to treat people fairly, but maybe there’s just no reason to trust that intuition.

    Since the 1700s, many philosophers working on so-called ‘metaethics’ — that is, the study of what ethical claims are and how we could know if they’re true — have despaired of us ever making sense of or identifying the location of ‘objective’ or ‘intrinsic’ value. They conclude that when we say things are ‘good,’ we aren’t really saying anything about their nature, but rather just expressing our own attitudes, or intentions, or something else.

    Sharon disagrees. She says the answer to all this has been right under our nose all along.

    We have a concept of value because of our experiences of positive sensations — sensations that immediately indicate to us that they are valuable and that if someone could create more of them, they ought to do so. Similarly, we have a concept of badness because of our experience of suffering — sensations that scream to us that if suffering were all there were, it would be a bad thing.

    How do we know that pleasure is valuable, and that suffering is the opposite of valuable? Directly!

    While I might be mistaken that a painting I’m looking at is in real life as it appears to me, I can’t be mistaken about the nature of my perception of it. If it looks red to me, it may or may not be red, but it’s definitely the case that I am perceiving redness. Similarly, while I might be mistaken that a painting is intrinsically valuable, I can’t be mistaken about the pleasurable sensations I’m feeling when I look at it, and the fact that among other qualities those sensations have the property of goodness.

    While intuitive on some level, this arguably implies some very strange things. Most famously, the philosopher Robert Nozick challenged it with the idea of an ‘experience machine’: if you could enter into a simulated world and enjoy a life far more pleasurable than the one you experience now, should you do so, even if it would mean none of your accomplishments or relationships would be ‘real’? Nozick and many of his colleagues thought not.

    The idea has also been challenged for failing to value human freedom and autonomy for its own sake. Would it really be OK to kill one person to use their organs to save the lives of five others, if doing so would generate more pleasure and less suffering? Few believe so.

    In today’s interview, Sharon explains the case for a theory of value grounded in subjective experiences, and why she believes these counterarguments are misguided. A philosophical hedonist shouldn’t get in an experience machine, nor override an individual’s autonomy, except in situations so different from the classic thought experiments that it no longer seems strange they would do so.

    Host Rob Wiblin and Sharon cover all that, as well as:

    • The essential need to disentangle intrinsic, instrumental, and other sorts of value
    • Why Sharon’s arguments lead to hedonistic utilitarianism rather than hedonistic egoism (in which we only care about our own feelings)
    • How do people react to the ‘experience machine’ thought experiment when surveyed?
    • Why hedonism recommends often thinking and acting as though it were false
    • Whether it’s crazy to think that relationships are only useful because of their effects on our subjective experiences
    • Whether it will ever be possible to eliminate pain, and whether doing so would be desirable
    • If we didn’t have positive or negative experiences, whether that would cause us to simply never talk about goodness and badness
    • Whether the plausibility of hedonism is affected by our theory of mind
    • And plenty more

    Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type ‘80,000 Hours’ into your podcasting app. Or read the transcript below.

    Producer: Keiran Harris
    Audio mastering: Ryan Kessler
    Transcriptions: Katy Moore

    Continue reading →

    Kuhan Jeyapragasan on effective altruism university groups

    Podcast (80k After Hours) by and · Published September 2022

    In this episode of 80k After Hours, Rob Wiblin interviews Kuhan Jeyapragasan about effective altruism university groups.

    From 2015 to 2020, Kuhan did an undergrad and then a master’s in maths and computer science at Stanford — and did a lot to organise and improve the EA group on campus.

    Rob and Kuhan cover:

    • The challenges of making a group appealing and accepting of everyone
    • The concrete things Kuhan did to grow the successful Stanford EA group
    • Whether local groups are turning off some people who should be interested in effective altruism, and what they could do differently
    • Lessons Kuhan learned from Stanford EA
    • The Stanford Existential Risks Initiative (SERI)

    Who this episode is for:

    • People already involved in EA university groups
    • People interested in getting involved in EA university groups

    Who this episode isn’t for:

    • People who’ve never heard of ‘effective altruism groups’
    • People who’ve never heard of ‘effective altruism’
    • People who’ve never heard of ‘university’

    Get this episode by subscribing to our more experimental podcast on the world’s most pressing problems and how to solve them: type ’80k After Hours’ into your podcasting app. Or read the transcript below.

    Producer: Keiran Harris
    Audio mastering: Ryan Kessler
    Transcriptions: Katy Moore

    “Gershwin – Rhapsody in Blue, original 1924 version” by Jason Weinberger is licensed under creative commons

    Continue reading →

    ‘S-risks’

    Problem profile by · Published September 2022

    People working on suffering risks or s-risks attempt to reduce the risk of something causing vastly more suffering than has existed on Earth so far. We think research to work out how to mitigate these risks might be particularly important. You may also be able to do important work by building this field, which is currently highly neglected — with fewer than 50 people working on this worldwide.

    Continue reading →

    #137 – Andreas Mogensen on whether effective altruism is just for consequentialists

    Podcast by and · Published September 8th, 2022

    Effective altruism, in a slogan, aims to ‘do the most good.’ Utilitarianism, in a slogan, says we should act to ‘produce the greatest good for the greatest number.’ It’s clear enough why utilitarians should be interested in the project of effective altruism. But what about the many people who reject utilitarianism?

    Today’s guest, Andreas Mogensen — senior research fellow at Oxford University’s Global Priorities Institute — does reject utilitarianism, but as he explains, this does little to dampen his enthusiasm for effective altruism.

    Andreas leans towards ‘deontological’ or rule-based theories of ethics, rather than ‘consequentialist’ theories like utilitarianism which look exclusively at the effects of a person’s actions.

    Like most people involved in effective altruism, he parts ways with utilitarianism in rejecting its maximal level of demandingness, the idea that the ends justify the means, and the notion that the only moral reason for action is to benefit everyone in the world considered impartially.

    However, Andreas believes any plausible theory of morality must give some weight to the harms and benefits we provide to other people. If we can improve a stranger’s wellbeing enormously at negligible cost to ourselves and without violating any other moral prohibition, that must be at minimum a praiseworthy thing to do.

    In a world as full of preventable suffering as our own, this simple ‘principle of beneficence’ is probably the only premise one needs to grant for the effective altruist project of identifying the most impactful ways to help others to be of great moral interest and importance.

    As an illustrative example Andreas refers to the Giving What We Can pledge to donate 10% of one’s income to the most impactful charities available, a pledge he took in 2009. Many effective altruism enthusiasts have taken such a pledge, while others spend their careers trying to figure out the most cost-effective places pledgers can give, where they’ll get the biggest ‘bang for buck’.

    For someone living in a world as unequal as our own, this pledge at a very minimum gives an upper-middle class person in a rich country the chance to transfer money to someone living on about 1% as much as they do. The benefit an extremely poor recipient receives from the money is likely far more than the donor could get spending it on themselves.

    What arguments could a non-utilitarian moral theory mount against such giving?

    Perhaps it could interfere with the achievement of other important moral goals? In response to this Andreas notes that alleviating the suffering of people in severe poverty is an important goal that should compete with alternatives. And furthermore, giving 10% is not so much that it likely disrupts one’s ability to, for instance, care for oneself or one’s family, or participate in domestic politics.

    Perhaps it involves the violation of important moral prohibitions, such as those on stealing or lying? In response Andreas points out that the activities advocated by effective altruism researchers almost never violate such prohibitions — and if a few do, one can simply rule out those options and choose among the rest.

    Many approaches to morality will say it’s permissible not to give away 10% of your income to help others as effectively as is possible. But if they will almost all regard it as praiseworthy to benefit others without giving up something else of equivalent moral value, then Andreas argues they should be enthusiastic about effective altruism as an intellectual and practical project nonetheless.

    In this conversation, Andreas and Rob discuss how robust the above line of argument is, and also cover:

    • Should we treat philosophical thought experiments that feature very large numbers with great suspicion?
    • If we had to allow someone to die to avoid preventing the football World Cup final from being broadcast to the world, is that permissible or not? If not, what might that imply?
    • What might a virtue ethicist regard as ‘doing the most good’?
    • If a deontological theory of morality parted ways with common effective altruist practices, how would that likely be?
    • If we can explain how we came to hold a view on a moral issue by referring to evolutionary selective pressures, should we disbelieve that view?

    Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type ‘80,000 Hours’ into your podcasting app. Or read the transcript below.

    Producer: Keiran Harris
    Audio mastering: Ben Cordell and Beppe Rådvik
    Transcriptions: Katy Moore

    Continue reading →

    Should you go to law school in the US to have a high-impact career?

    Career review · Published September 2022

    This article outlines considerations for readers deciding whether to get a US law degree, explains how it might help them have a positive impact, and describes the law school experience. It is based on public resources about law school and careers in law and policy, as well as the experiences of people who have recently attended or considered attending law school. Consider this post a series of “best guesses” about how to approach the decision about whether to attend law school, rather than a definitive guide or empirical study.

    I. Why law school could be an important step in a promising career path

    The most common jobs for lawyers are probably not the most promising from the perspective of doing as much good as you can. A large share of US lawyers, especially graduates of top-ranked law schools, work in large private law firms, organising corporate transactions or defending large companies from lawsuits.

    But law school can be a promising route into high-impact careers in policy and government, especially for people interested in eventually holding senior roles for which formal credentials are highly valued.

    Since we believe working in policy to address some of the most pressing problems in the world is among the most promising career paths for people who want to have a high positive impact, law school may be a particularly appealing place to start, especially for people who are early in their career. Policy work in the US is a particularly high-priority path,

    Continue reading →

    Andrés Jiménez Zorrilla on the Shrimp Welfare Project

    Podcast (80k After Hours) by and · Published September 2022

    In this episode of 80k After Hours, Rob Wiblin interviews Andrés Jiménez Zorrilla about the Shrimp Welfare Project, which he cofounded in 2021. It’s the first project in the world focused on shrimp welfare specifically and now has six full-time staff.

    They cover:

    • The evidence for shrimp sentience
    • How farmers and the public feel about shrimp
    • The scale of the problem
    • What shrimp farming looks like
    • The killing process, and other welfare issues
    • Shrimp Welfare Project’s strategy
    • History of shrimp welfare work
    • What it’s like working in India and Vietnam
    • How to help

    Who this episode is for:

    • People who care about animal welfare
    • People interested in new and unusual problems
    • People open to shrimp sentience

    Who this episode isn’t for:

    • People who think shrimp couldn’t possibly be sentient
    • People who got called ‘shrimp’ a lot in high school and get anxious when they hear the word over and over again

    Get this episode by subscribing to our more experimental podcast on the world’s most pressing problems and how to solve them: type ’80k After Hours’ into your podcasting app. Or read the transcript below.

    Producer: Keiran Harris
    Audio mastering: Ben Cordell and Ryan Kessler
    Transcriptions: Katy Moore

    “Gershwin – Rhapsody in Blue, original 1924 version” by Jason Weinberger is licensed under creative commons

    Continue reading →

    What could an AI-caused existential catastrophe actually look like?

    Article by · Published August 2022

    At 5:29 AM on July 16, 1945, deep in the Jornada del Muerto desert in New Mexico, the Manhattan Project carried out the world’s first successful test of a nuclear weapon.

    From that moment, we’ve had the technological capacity to wipe out humanity.

    But if you asked someone in 1945 to predict exactly how this risk would play out, they would almost certainly have got it wrong. They may have thought there would have been more widespread use of nuclear weapons in World War II. They certainly would not have predicted the fall of the USSR 45 years later. Current experts are concerned about India–Pakistan nuclear conflict and North Korean state action, but 1945 was before even the partition of India or the Korean War.

    That is to say, you’d have real difficulty predicting anything about how nuclear weapons would be used. It would have been even harder to make these predictions in 1933, when Leo Szilard first realised that a nuclear chain reaction of immense power could be possible, without any concrete idea of what these weapons would look like.

    Despite this difficulty, you wouldn’t be wrong to be concerned.

    In our problem profile on AI, we describe a very general way in which advancing AI could go wrong. But there are lots of specifics we can’t know much about at this point.

    Continue reading →