Technical AI safety is a multifaceted area of research, with many sub-questions in areas such as reward learning, robustness, and interpretability. These will all need to be answered in order to make sure AI development will go well for humanity as systems become more and more powerful.
Not all of these questions are best tackled with abstract mathematics research; some can be approached with concrete coding experiments and machine learning (ML) prototypes. As a result, some AI safety research teams are looking to hire a growing number of Software Engineers and ML Research Engineers.
Additionally, some research teams that may not think of themselves as focussed on ‘AI Safety’ per se, nonetheless work on related problems like verification of neural nets or learning from human feedback, and are often hiring engineers.
Note that this guide was written in November 2018 to complement an in-depth conversation on the 80,000 Hours Podcast with Catherine Olsson and Daniel Ziegler on how to transition from computer science and software engineering in general into ML engineering, with a focus on alignment and safety. If you like this guide, we’d strongly encourage you to check out the podcast episode where we discuss some of the instructions here, and other relevant advice.
What are the necessary qualifications for these positions?
Software Engineering: Some engineering roles on AI safety teams do not require ML experience. You might already be prepared to apply to these positions if you have the following qualifications:
BSc/BEng degree in computer science or another technical field (or comparable experience)
Strong knowledge of software engineering (as a benchmark: could pass a Google software engineering interview)
Interest in working on AI safety
(usually) Willingness to move to London or the San Francisco Bay Area
If you’re a software engineer with an interest in these roles,
Blog post by Robert Wiblin · Published October 24th, 2018
Our job board now lists 142 vacancies, with 38 additional opportunities since the last update 3 weeks ago.
If you’re actively looking for a new role, we recommend checking out the job board regularly – when a great opening comes up, you’ll want to maximise your preparation time.
The job board remains a curated list of the most promising positions to apply for that we’re currently aware of. They’re all high-impact opportunities at organisations that are working on some of the world’s most pressing problems:
Blog post by Robert Wiblin · Published October 24th, 2018
We’ve published a new article that summarises our advice based on your strengths and link you to the most relevant articles for you to read:
This list is preliminary. We wanted to publish our existing thoughts on what to do with each skill, but can easily see ourselves changing our minds over the coming years.
You can read about our general process and what career paths we recommend in our full article.
Sometimes, however, it’s possible to give more specific advice about what options to consider to people who already have pre-existing experience or qualifications, or are unusually good at a certain type of work.
In this article, we provide a list of skills, and for each one give a list of socially-impactful options that people who are unusually good in that area should most often consider.
We start with three “strengths” (quantitative, verbal & social, and visual). Then we go on to give advice for people with existing experience in fifteen specific fields.
Bear in mind it’s often possible to completely change field: we’ve seen people switch from philosophy to software engineering, and architecture into economics. Nonetheless, these are good starting points.
The skill types also overlap, and you probably also have several of them. The aim is just to give you some tips on narrowing down your options more quickly.
The barista gives you your coffee and change, and you walk away from the busy line. But you suddenly realise she gave you $1 less than she should have. Do you brush your way past the people now waiting, or just accept this as a dollar you’re never getting back? According to philosophy professor Hilary Greaves – Director of Oxford University’s Global Priorities Institute, which is hiring now – this simple decision will completely change the long-term future by altering the identities of almost all future generations.
How? Because by rushing back to the counter, you slightly change the timing of everything else people in line do during that day — including changing the timing of the interactions they have with everyone else. Eventually these causal links will reach someone who was going to conceive a child.
By causing a child to be conceived a few fractions of a second earlier or later, you change the sperm that fertilizes their egg, resulting in a totally different person. So asking for that $1 has now made the difference between all the things that this actual child will do in their life, and all the things that the merely possible child – who didn’t exist because of what you did – would have done if you decided not to worry about it.
As that child’s actions ripple out to everyone else who conceives down the generations, ultimately the entire human population will become different, all for the sake of your dollar. Will your choice cause a future Hitler to be born, or not to be born? Probably both!
Some find this concerning. The actual long term effects of your decisions are so unpredictable, it looks like you’re totally clueless about what’s going to lead to the best outcomes. It might lead to decision paralysis — you won’t be able to take any action at all.
Prof Greaves doesn’t share this concern for most real life decisions. If there’s no reasonable way to assign probabilities to far-future outcomes, then the possibility that you might make things better in completely unpredictable ways is more or less canceled out by the equally plausible possibility that you might make things worse in equally unpredictable ways.
But, if instead we’re talking about a decision that involves highly-structured, systematic reasons for thinking there might be a general tendency of your action to make things better or worse — for example if we increase economic growth — Prof Greaves says that we don’t get to just ignore the unforeseeable effects.
When there are complex arguments on both sides, it’s unclear what probabilities you should assign to this or that claim. Yet, given its importance, whether you should take the action in question actually does depend on figuring out these numbers.
So, what do we do?
Today’s episode blends philosophy with an exploration of the mission and research agenda of the Global Priorities Institute: to develop the effective altruism movement within academia. We cover:
What’s the long term vision of the Global Priorities Institute?
How controversial is the multiverse interpretation of quantum physics?
What’s the best argument against academics just doing whatever they’re interested in?
How strong is the case for long-termism? What are the best opposing arguments?
Are economists getting convinced by philosophers on discount rates?
Given moral uncertainty, how should population ethics affect our real life decisions?
How should we think about archetypal decision theory problems?
The value of exploratory vs. basic research
Person affecting views of population ethics, fragile identities of future generations, and the non-identity problem
Is Derek Parfit’s repugnant conclusion really repugnant? What’s the best vision of a life barely worth living?
What are the consequences of cluelessness for those who based their donation advice on GiveWell style recommendations?
How could reducing global catastrophic risk be a good cause for risk-averse people?
What’s the core difficulty in forming proper credences?
The value of subjecting EA ideas to academic scrutiny
The influence of academia in society
The merits of interdisciplinary work
The case for why operations is so important in academia
The trade off between working on important problems and advancing your career
Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.
The 80,000 Hours Podcast is produced by Keiran Harris.
Blog post by Robert Wiblin · Published October 22nd, 2018
We’ve published a new article about how to avoid accidentally causing harm through your career:
“We encourage people to work on problems that are neglected by others and large in scale. Unfortunately those are precisely the problems where people can do the most damage if their approach isn’t carefully thought through.
If a problem is very important, then setting back the cause is very bad. If a problem is so neglected that you’re among the first focused on it, then you’ll have a disproportionate influence on the field’s reputation, how likely others are to enter it, and many early decisions that could have path-dependent effects on the field’s long-term success.
We don’t particularly enjoy writing about this admittedly demotivating topic. Ironically, we expect that cautious people – the folks who least need this advice – will be the ones most likely to take it to heart.
Nonetheless we think cataloguing these risks is important if we’re going to be serious about having an impact in important but ‘fragile’ fields like reducing extinction risk.
In this article, we’ll list six ways people can unintentionally set back their cause. You may already be aware of most of these risks, but we often see people neglect one or two of them when new to a high stakes area – including us when we were starting 80,000 Hours.”
I’ve probably spent more time reading Tyler Cowen – Professor of Economics at George Mason University – than any other author. Indeed it’s his incredibly popular blog Marginal Revolution that prompted me to study economics in the first place. Having spent thousands of hours absorbing Tyler’s work, it was a pleasure to be able to question him about his latest book and personal manifesto: Stubborn Attachments: A Vision for a Society of Free, Prosperous, and Responsible Individuals.
Tyler makes the case that, despite what you may have heard, we can make rational judgments about what is best for society as a whole. He argues:
Our top moral priority should be preserving and improving humanity’s long-term future
The way to do that is to maximise the rate of sustainable economic growth
We should respect human rights and follow general principles while doing so.
We discuss why Tyler believes all these things, and I push back where I disagree. In particular: is higher economic growth actually an effective way to safeguard humanity’s future, or should our focus really be elsewhere?
In the process we touch on many of moral philosophy’s most pressing questions: Should we discount the future? How should we aggregate welfare across people? Should we follow rules or evaluate every situation individually? How should we deal with the massive uncertainty about the effects of our actions? And should we trust common sense morality or follow structured theories?
After covering the book, the conversation ranges far and wide. Will we leave the galaxy, and is it a tragedy if we don’t? Is a multi-polar world less stable? Will humanity ever help wild animals? Why do we both agree that Kant and Rawls are overrated?
Today’s interview is released on both the 80,000 Hours Podcast and Tyler’s own show: Conversation with Tyler.
Tyler may have had more influence on me than any other writer but this conversation is richer for our remaining disagreements. If the above isn’t enough to tempt you to listen, we also look at:
Why couldn’t future technology make human life a hundred or a thousand times better than it is for people today?
Why focus on increasing the rate of economic growth rather than making sure that it doesn’t go to zero?
Why shouldn’t we dedicate substantial time to the successful introduction of genetic engineering?
Why should we completely abstain from alcohol and make it a social norm?
Why is Tyler so pessimistic about space? Is it likely that humans will go extinct before we manage to escape the galaxy?
Is improving coordination and international cooperation a major priority?
Why does Tyler think institutions are keeping up with technology?
Given that our actions seem to have very large and morally significant effects in the long run, are our moral obligations very onerous?
Can art be intrinsically valuable?
What does Tyler think Derek Parfit was most wrong about, and what was he was most right about that’s unappreciated today?
How should we think about animal suffering?
Do self-aware entities have to be biological in some sense?
What’s the most likely way that the worldview presented in Stubborn Attachments could be fundamentally wrong?
During ‘underrated vs overrated’, should guests say ‘appropriately rated’ more often?
Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.
The 80,000 Hours podcast is produced by Keiran Harris.
Update April 2019: We think that our use of the term ‘talent gaps’ in this post (and elsewhere) has caused some confusion. We’ve written a post clarifying what we meant by the term and addressing some misconceptions that our use of it may have caused. Most importantly, we now think it’s much more useful to talk about specific skills and abilities that are important constraints on particular problems rather than talking about ‘talent constraints’ in general terms. This page may be misleading if it’s not read in conjunction with our clarifications.
What are the most pressing needs in the effective altruism community right now? What problems are most effective to work on? Who should earn to give and who should do direct work? We surveyed managers at organisations in the community to find out their views. These results help to inform our recommendations about the highest impact career paths available.
Our key finding is that for the questions that we asked 12 months ago, the results have not changed very much. This gives us more confidence in our survey results from 2017.
Research isn’t the only way academics can have a large impact
When we think of academic careers, research is what first comes to mind, but academics have many other pathways to impact which are less often considered. Academics can also influence public opinion, advise policy-makers, or manage teams of other researchers to help them be more productive.
If any of these routes might turn out to be a good fit for you, then it makes the path even more attractive. We’ll sketch out some of these other paths:
1. Public outreach
Peter Singer’s career began in an ordinary enough way for a promising young academic, studying philosophy at Oxford University. But he soon started moving in a different direction from his peers,
Blog post by Robert Wiblin · Published October 3rd, 2018
Our job board now lists 128 vacancies, with 45 additional opportunities since last month.
If you’re actively looking for a new role, we recommend checking out the job board regularly – when a great opening comes up, you’ll want to maximise your preparation time.
The job board remains a curated list of the most promising positions to apply for that we’re currently aware of. They’re all high-impact opportunities at organisations that are working on some of the world’s most pressing problems:
Paul Christiano is one of the smartest people I know and this episode has one of the best explanations for why AI alignment matters and how we might solve it. After our first session produced such great material, we decided to do a second recording, resulting in our longest interview so far. While challenging at times I can strongly recommend listening – Paul works on AI himself and has a very unusually thought through view of how it will change the world. Even though I’m familiar with Paul’s writing I felt I was learning a great deal and am now in a better position to make a difference to the world.
A few of the topics we cover are:
Why Paul expects AI to transform the world gradually rather than explosively and what that would look like
Several concrete methods OpenAI is trying to develop to ensure AI systems do what we want even if they become more competent than us
Why AI systems will probably be granted legal and property rights
How an advanced AI that doesn’t share human goals could still have moral value
Why machine learning might take over science research from humans before it can do most other tasks
Which decade we should expect human labour to become obsolete, and how this should affect your savings plan.
—
Here’s a situation we all regularly confront: you want to answer a difficult question, but aren’t quite smart or informed enough to figure it out for yourself. The good news is you have access to experts who are smart enough to figure it out. The bad news is that they disagree.
If given plenty of time – and enough arguments, counterarguments and counter-counter-arguments between all the experts – should you eventually be able to figure out which is correct? What if one expert were deliberately trying to mislead you? And should the expert with the correct view just tell the whole truth, or will competition force them to throw in persuasive lies in order to have a chance of winning you over?
In other words: does ‘debate’, in principle, lead to truth?
According to Paul Christiano – researcher at the machine learning research lab OpenAI and legendary thinker in the effective altruism and rationality communities – this question is of more than mere philosophical interest. That’s because ‘debate’ is a promising method of keeping artificial intelligence aligned with human goals, even if it becomes much more intelligent and sophisticated than we are.
It’s a method OpenAI is actively trying to develop, because in the long-term it wants to train AI systems to make decisions that are too complex for any human to grasp, but without the risks that arise from a complete loss of human oversight.
If AI-1 is free to choose any line of argument in order to attack the ideas of AI-2, and AI-2 always seems to successfully defend them, it suggests that every possible line of argument would have been unsuccessful.
But does that mean that the ideas of AI-2 were actually right? It would be nice if the optimal strategy in debate were to be completely honest, provide good arguments, and respond to counterarguments in a valid way. But we don’t know that’s the case.
According to Paul, it’s clear that if the judge is weak enough, there’s no reason that an honest debater would be at an advantage. But the hope is that there is some threshold of competence above which debates tend to converge on more accurate claims the longer they continue.
Most real world debates are set up under highly suboptimal conditions; judges usually don’t have a lot of time to think about how to best get to the truth, and often have bad incentives themselves. But for AI safety via debate, researchers are free to set things up in the way that gives them the best shot. And if we could understand how to construct systems that converge to truth, we would have a plausible way of training powerful AI systems to stay aligned with our goals.
This is our longest interview so far for good reason — we cover a fascinating range of topics:
What could people do to shield themselves financially from potentially losing their jobs to AI?
How important is it that the best AI safety team ends up in the company with the best ML team?
What might the world look like if several states or actors developed AI at the same time (aligned or otherwise)?
Would artificial general intelligence grow in capability quickly or slowly?
How likely is it that transformative AI is an issue worth worrying about?
What are the best arguments against being concerned?
What would cause people to take AI alignment more seriously?
Concrete ideas for making machine learning safer, such as iterated amplification.
What does it mean to say that a crow-like intelligence could be much better at science than humans?
What is ‘prosaic AI’?
How do Paul’s views differ from those of the Machine Intelligence Research Institute?
The importance of honesty for people and organisations
What are the most important ways that people in the effective altruism community are approaching AI issues incorrectly?
When would an ‘unaligned’ AI nonetheless be morally valuable?
What’s wrong with current sci-fi?
Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.
The 80,000 Hours podcast is produced by Keiran Harris.
Here’s your regular reminder of everything 80,000 Hours has released over the last four months, since our last roundup. If you’d like to get these updates more regularly, you can join our newsletter.
We’ve done a major redesign of our job board, increasing the number of vacancies listed there from ~20 to over 100. It has doubled its traffic since the first half of the year, and is now one of the five most popular pages on the whole site.
Blog post by Robert Wiblin · Published September 30th, 2018
We recently published a number of new articles that you might have missed if you don’t follow us on social media (Facebook and Twitter) or our research newsletter.
Probably our most important release for this year is this article summarising many of our key findings since we started in 2011:
It outlines our new suggested process anyone can use to generate a short-list of high-impact career options given their personal situation.
It then describes the top five key categories of career we most often recommend, which should produce at least one good option for almost all graduates, and why we’re enthusiastic about them.
It goes on to list and explains the top 10 “priority paths” we want to draw attention to, because we think they can enable to right person to do a particularly large amount of good for the world.
Second, if you’re trying to figure out which job is the best fit for you, or how to coordinate with other people – for example the effective altruism community – you will want to read:
Third, if you’d like to influence government or work in politics, you should check out our comprehensive review of the pros and cons of being a Congressional Staffer and how to become one:
In Stanley Kubrick’s iconic film Dr. Strangelove, the American president is informed that the Soviet Union has created a secret deterrence system which will automatically wipe out humanity upon detection of a single nuclear explosion in Russia. With US bombs heading towards the USSR and unable to be recalled, Dr Strangelove points out that “the whole point of this Doomsday Machine is lost if you keep it a secret – why didn’t you tell the world, eh?” The Soviet ambassador replies that it was to be announced at the Party Congress the following Monday: “The Premier loves surprises”.
Daniel Ellsberg – leaker of the Pentagon Papers which helped end the Vietnam War and Nixon presidency – claims in his new book The Doomsday Machine: Confessions of a Nuclear War Planner that Dr. Strangelove might as well be a documentary. After attending the film in Washington DC in 1964, he and a military colleague wondered how so many details of the nuclear systems they were constructing had managed to leak to the filmmakers.
The USSR did in fact develop a doomsday machine, Dead Hand, which probably remains active today.
If the system can’t contact military leaders, it checks for signs of a nuclear strike. Should its computers determine that an attack occurred, it would automatically launch all remaining Soviet weapons at targets across the northern hemisphere.
As in the film, the Soviet Union long kept Dead Hand completely secret, eliminating any strategic benefit, and rendering it a pointless menace to humanity.
You might think the United States would have a more sensible nuclear launch policy. You’d be wrong.
As Ellsberg explains based on first-hand experience as a nuclear war planner in the early stages of the Cold War, the notion that only the president is able to authorize the use of US nuclear weapons is a carefully cultivated myth.
The authority to launch nuclear weapons is delegated alarmingly far down the chain of command – significantly raising the chance that a lone wolf or communication breakdown could trigger a nuclear catastrophe.
The whole justification for this is to defend against a ‘decapitating attack’, where a first strike on Washington disables the ability of the US hierarchy to retaliate. In a moment of crisis, the Russians might view this as their best hope of survival.
Ostensibly, this delegation removes Russia’s temptation to attempt a decapitating attack – the US can retaliate even if its leadership is destroyed. This strategy only works, though, if you tell the enemy you’ve done it.
Instead, since the 50s this delegation has been one of the United States most closely guarded secrets, eliminating its strategic benefit, and rendering it another pointless menace to humanity.
Even setting aside the above, the size of the Russian and American nuclear arsenals today makes them doomsday machines of necessity. According to Ellsberg, if these arsenals are ever launched, whether accidentally or deliberately, they would wipe out almost all human life, and all large animals.
Strategically, the setup is stupid. Ethically, it is monstrous.
If the US or Russia sent its nuclear arsenal to destroy the other, would it even make sense to retaliate? Ellsberg argues that it doesn’t matter one way or another. The nuclear winter generated by the original attack would be enough to starve to death most people in the aggressor country within a year anyway. Retaliation would just slightly speed up their demise.
So – how was such a system built? Why does it remain to this day? And how might we shrink our nuclear arsenals to the point they don’t risk the destruction of civilization?
Daniel explores these questions eloquently and urgently in his book (that everyone should read), and this conversation is a gripping introduction. We cover:
Why full disarmament today would be a mistake
What are our greatest current risks from nuclear weapons?
What has changed most since Daniel was working in and around the government in the 50s and 60s?
How well are secrets kept in the government?
How much deception is involved within the military?
The capacity of groups to commit evil
How Hitler was more cautious than America about nuclear weapons
What was the risk of the first atomic bomb test?
The effect of Trump on nuclear security
What practical changes should we make? What would Daniel do if he were elected president?
Do we have a reliable estimate of the magnitude of a ‘nuclear winter’?
What would be the optimal number of nuclear weapons for the US and its allies to hold?
What should we make of China’s nuclear stance? What are the chances of war with China?
Would it ever be right to respond to a nuclear first strike?
Should we help Russia get better attack detection methods to make them less anxious?
How much power do lobbyists really have?
Has game theory had any influence over nuclear strategy?
Why Gorbachev allowed Russia’s covert biological warfare program to continue
Is it easier to help solve the problem from within the government or at outside orgs?
What gives Daniel hope for the future?
Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.
The 80,000 Hours podcast is produced by Keiran Harris.
Blog post by Niel Bowerman · Published September 18th, 2018
Within just four years of finishing her PhD in biophysics, Jessica Tuchman Mathews was Director of Global Issues for President Carter’s National Security Council. In her first year in the role she helped put together a nuclear non-proliferation pact among 15 countries including the US and the Soviet Union.
Later in her career, Jessica served as Deputy to the Undersecretary of State for Global Affairs, wrote a weekly column for the Washington Post, and most recently served as President of the Carnegie Endowment for International Peace, an influential Washington-based foreign policy think tank.
What launched such an successful career? In our conversation with Jessica, she argued it was the AAAS Science & Technology (S&T) Policy Fellowship. Jessica was selected as one of their inaugural fellows in 1973.
At 80,000 Hours, we think the AAAS Policy Fellowship is one of the best routes into the US Government for people with a STEM or social science PhD, or an engineering masters and three years of industry experience.
Policy fellows work within the US Government for one year in policy-related roles relevant to science and technology. Nearly 300 fellows are accepted each year, and almost all of them take assignments within the executive branch,
Consider two familiar moments at a family reunion.
Our host, Uncle Bill, is taking pride in his barbequing skills. But his niece Becky says that she now refuses to eat meat. A groan goes round the table; the family mostly think of this as an annoying picky preference. But were it viewed as a moral position rather than personal preference – as they might if instead Becky were avoiding meat on religious grounds – it would usually receive a very different reaction.
An hour later Bill expresses a strong objection to abortion. Again, a groan goes round the table: the family mostly think that he has no business in trying to foist his regressive preferences on other people’s personal lives. But if considered not as a matter of personal taste, but rather as a moral position – that Bill genuinely believes he’s opposing mass-murder – his comment might start a serious conversation.
Amanda Askell, who recently completed a PhD in philosophy at NYU focused on the ethics of infinity, thinks that we often betray a complete lack of moral empathy. Across the political spectrum, we’re unable to get inside the mindset of people who expresses views that we disagree with, and see the issue from their point of view.
A common cause of conflict, as above, is confusion between personal preferences and moral positions. Assuming good faith on the part of the person you disagree with, and actually engaging with the beliefs they claim to hold, is perhaps the best remedy for our inability to make progress on controversial issues.
One seeming path to progress involves contraception. A lot of people who are anti-abortion are also anti-contraception. But they’ll usually think that abortion is much worse than contraception – so why can’t we compromise and agree to have much more contraception available?
According to Amanda, a charitable explanation is that people who are anti-abortion and anti-contraception engage in moral reasoning and advocacy based on what, in their minds, is the best of all possible worlds: one where people neither use contraception nor get abortions.
So instead of arguing about abortion and contraception, we could discuss the underlying principle that one should advocate for the best possible world, rather than the best probable world. Successfully break down such ethical beliefs, absent political toxicity, and it might be possible to actually figure out why we disagree and perhaps even converge on agreement.
Today’s episode blends such practical topics with cutting-edge philosophy. We cover:
The problem of ‘moral cluelessness’ – our inability to predict the consequences of our actions – and how we might work around it
Amanda’s biggest criticisms of social justice activists, and of critics of social justice activists
Is there an ethical difference between prison and corporal punishment? Are both or neither justified?
How to resolve ‘infinitarian paralysis’ – the inability to make decisions when infinities get involved.
What’s effective altruism doing wrong?
How should we think about jargon? Are a lot of people who don’t communicate clearly just trying to scam us?
How can people be more successful while they remain within the cocoon of school and university?
How did Amanda find her philosophy PhD, and how will she decide what to do now?
Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.
The 80,000 Hours podcast is produced by Keiran Harris.
With 698 inmates per 100,000 citizens, the U.S. is the world’s leader in incarcerating people. But what effect does this actually have on crime?
According to David Roodman, Senior Advisor to Open Philanthropy, the marginal effect is zero.
This stunning rebuke to the American criminal justice system comes from the man Holden Karnofsky called “the gold standard for in-depth quantitative research”. His other investigations include the risk of geomagnetic storms, whether deworming improves health and test scores, and the development impacts of microfinance – all of which we also cover in this episode.
Does having tougher sentences deter people from committing crime? After reviewing studies on gun laws and ‘three strikes’ in California, David concluded that the effect of deterrence is zero.
Does imprisoning more people reduce crime by incapacitating potential offenders? Here he says yes, noting that crimes like motor vehicle theft have gone up in a way that seems pretty clearly connected with recent Californian criminal justice reforms (though the effect on violent crime is far lower).
Finally, do the after-effects of prison make you more or less likely to commit future crimes?
This one is more complicated.
His literature review suggested that more time in prison made people substantially more likely to commit future crimes when released. But concerned that he was biased towards a comfortable position against incarceration, David did a cost-benefit analysis using both his favoured reading of the evidence and the devil’s advocate view; that there is deterrence and that the after-effects are beneficial.
For the devil’s advocate position David used the highest assessment of the harm caused by crime, which suggests a year of prison prevents about $92,000 in crime. But weighed against a lost year of liberty, valued at $50,000, and the cost of operating prisons, the numbers came out exactly the same.
So even using the least-favourable cost-benefit valuation of the least favourable reading of the evidence — it just breaks even.
The argument for incarceration melts further when you consider the significant crime that occurs within prisons, de-emphasised because of a lack of data and a perceived lack of compassion for inmates.
In today’s episode we discuss how to conduct such impactful research, and how to proceed having reached strong conclusions.
We also cover:
How do you become a world class researcher? What kinds of character traits are important?
Are academics aware of following perverse incentives?
What’s involved in data replication? How often do papers replicate?
The politics of large orgs vs. small orgs
How do you decide what questions to research?
How concerned should a researcher be with their own biases?
Geomagnetic storms as a potential cause area
How much does David rely on interviews with experts?
The effects of deworming on child health and test scores
Is research getting more reliable? Should we have ‘data vigilantes’?
What are David’s critiques of effective altruism?
What are the pros and cons of starting your career in the think tank world? Do people generally have a high impact?
How do we improve coordination across groups, given our evolutionary history?
Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.
The 80,000 Hours podcast is produced by Keiran Harris.
Levitt collected tens of thousands of people who were deeply unsure whether to make a big change in their life. After offering some advice on how to make hard choices, those who remained truly undecided were given the chance to use a flip of a coin to settle the issue. 22,500 did so. Levitt then followed up two and six months later to ask people whether they had actually made the change, and how happy they were out of 10.
People who faced an important decision and got heads – which indicated they should quit, break up, propose, or otherwise mix things up – were 11 percentage points more likely to do so.
It’s very rare to get a convincing experiment that can help us answer as general and practical a question as ‘if you’re undecided, should you change your life?’ But this experiment can!
Experts believe that artificial intelligence will be better than humans at driving trucks by 2027, working in retail by 2031, writing bestselling books by 2049, and working as surgeons by 2053. But how seriously should we take these predictions?
Katja Grace, lead author of ‘When Will AI Exceed Human Performance?’, thinks we should treat such guesses as only weak evidence. But she also says there might be much better ways to forecast transformative technology, and that anticipating such advances could be one of our most important projects.
There’s often pessimism around making accurate predictions in general, and some areas of artificial intelligence might be particularly difficult to forecast.
But there are also many things we’re now able to predict confidently — like the climate of Oxford in five years — that we no longer give ourselves much credit for.
Some aspects of transformative technologies could fall into this category. And these easier predictions could give us some structure on which to base the more complicated ones.
One controversial debate surrounds the idea of an intelligence explosion; how likely is it that there will be a sudden jump in AI capability?
And one way to tackle this is to investigate a more concrete question: what’s the base rate of any technology having a big discontinuity?
A significant historical example was the development of nuclear weapons. Over thousands of years, the energy density of explosives didn’t increase by much. Then within a few years, it got thousands of times better. Discovering what leads to such anomalies may allow us to better predict the possibility of a similar jump in AI capabilities.
Katja likes to compare our efforts to predict AI with those to predict climate change. While both are major problems (though Katja and 80,000 Hours have argued that we should prioritise AI safety), only climate change has prompted hundreds of millions of dollars of prediction research.
That neglect creates a high impact opportunity, and Katja believes that talented researchers should strongly consider following her path.
Some promising research questions include:
What’s the relationship between brain size and intelligence?
How frequently, and when, do technological trends undergo discontinuous progress?
What’s the explanation for humans’ radical success over other apes?
What are the best arguments for a local, fast takeoff?
In today’s interview we also discuss:
Why is AI impacts one of the most important projects in the world?
How do you structure important surveys? Why do you get such different answers when asking what seem to be very similar questions?
How does writing an academic paper differ from posting a summary online?
When will unguided machines be able to produce better and cheaper work than humans for every possible task?
What’s one of the most likely jobs to be automated soon?
Are people always just predicting the same timelines for new technologies?
How do AGI researchers different from other AI researchers in their predictions?
What are attitudes to safety research like within ML? Are there regional differences?
Are there any other types of experts we ought to talk to on this topic?
How much should we believe experts generally?
How does the human brain compare to our best supercomputers? How many human brains are worth all the hardware in the world?
How quickly has the processing capacity for machine learning problems been increasing?
What can we learn from the development of previous technologies in figuring out how fast transformative AI will arrive?
What are the best arguments for and against discontinuous development of AI?
Comparing our predictions of climate change and AI development
How should we measure human capacity to predict generally?
How have things changed in the AI landscape over the last 5 years?
How likely is an AI explosion?
What should we expect from a post AI dominated economy?
Should people focus specifically on the early timeline scenarios even if they consider them unlikely?
How much influence can people ever have on things that will happen in 20 years? Are there any examples of people really trying to do this?
Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.
The 80,000 Hours podcast is produced by Keiran Harris.
“Do the job that’s your comparative advantage” might sound like obvious advice, but it turns out to be more complicated.
In this article, we sketch a naive application of comparative advantage to choosing between two career options, and show that it doesn’t apply. Then we give a more complex example where comparative advantage comes back into play, and show how it’s different from “personal fit”.
In brief, we think comparative advantage matters when you’re closely coordinating with a community to fill a limited number of positions, like we are in the effective altruism community. Otherwise, it’s better to do whatever seems highest-impact at the margin.
In the final section, we give some thoughts on how to assess your comparative advantage, and some mistakes people might be making in the effective altruism community.
The following are some research notes on our current thoughts, which we’re publishing for feedback. We’re pretty uncertain about many of the findings, and even how to best define the terms, and could easily see ourselves changing our minds if we did more research.
Reading time: 10 minutes
When does comparative advantage matter? A simple example where it doesn’t
Here’s a case where you might think comparative advantage applies, but it actually doesn’t. (We’ll define terms more carefully in the next section.)
Imagine there are two types of role, research and outreach. There are also two people, Carlie and Dave,
Will Trump be re-elected? Will North Korea give up their nuclear weapons? Will your friend turn up to dinner?
Spencer Greenberg, founder of ClearerThinking.org, has a process for working out such real life problems.
Let’s work through one here: how likely is it that you’ll enjoy listening to this episode?
The first step is to figure out your ‘prior probability’: your estimate of how likely you are to enjoy the interview before getting any further evidence.
Other than applying common sense, one way to figure this out is ‘reference class forecasting’. That is, looking at similar cases and seeing how often something is true, on average.
Spencer is our first ever return guest (Dr Anders Sandberg appeared on episodes 29 and 33 – but only because his one interview was so fascinating that we split it into two).
So one reference class might be, how many Spencer Greenberg episodes of the 80,000 Hours Podcast have you enjoyed so far? Being this specific limits bias in your answer, but with a sample size of just one – you’ll want to add more data points to reduce the variance of the answer (100% or 0% are both too extreme answers).
Zooming out, how many episodes of the 80,000 Hours Podcast have you enjoyed? Let’s say you’ve listened to 10, and enjoyed 8 of them. If so 8 out of 10 might be a reasonable prior.
If we want a bigger sample we can zoom out further: what fraction of long-form interview podcasts have you ever enjoyed?
Having done that you’d need to update whenever new information became available. Do the topics seem more interesting than average? Did Spencer make a great point in the first 5 minutes? Was this description unbearably self-referential?
In the episode we’ll explain the mathematically correct way to update your beliefs over time as new information comes in: Bayes Rule. You take your initial odds, multiply them by a ‘Bayes Factor’ and boom – updated probabilities. Once you know the trick it’s even easy to do it in your head. We’ll run through several diverse case studies of updating on evidence.
Speaking of the Question of Evidence: in a world where Spencer was not worth listening to, how likely is it that we’d invite him back for a second episode?
Also in this episode:
How could we generate 20-30 new happy thoughts a day? What would that do to our welfare?
What do people actually value? How do EAs differ from non EAs?
Why should we care about the distinction between intrinsic and instrumental values?
What types of activities are people generally under-confident about? Why?
When should you give a lot of weight to your existing beliefs?
When should we trust common sense?
Does power posing have any effect?
Are resumes worthless?
Did Trump explicitly collude with Russia? What are the odds of him getting re-elected?
What’s the probability that China and the US go to War in the 21st century?
How should we treat claims of expertise on nutrition?
Why were Spencer’s friends suspicious of Theranos for years?
How should we think about the placebo effect?
Does a shift towards rationality typically cause alienation from family and friends? How do you deal with that?
Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.
The 80,000 Hours podcast is produced by Keiran Harris.
