What is this problem?

Our ability to solve problems in the world relies heavily on our ability to understand it and make high-quality decisions. We need to be able to identify what problems to work on, to understand what factors contribute towards these problems, to predict which of our actions will have the desired outcomes, and to respond to feedback and change our minds.

There’s plenty of reason to think that our decision-making competence is currently less than perfect, as psychology research over the past few decades has documented a whole host of cognitive biases affecting judgements and decisions.

For example, when we try to judge our chances of success, we focus too much on all the reasons why our case will be different from average: despite the fact most startups fail, most prospective entrepreneurs are convinced they will be the unusual case that succeeds.²

We’re also often overconfident in our predictions³ — it’s been argued that unwarranted confidence contributed to the explosion of the space shuttle Challenger, for example, where NASA overruled the safety concerns expressed by an engineer.⁴

Many of the most important problems in the world are incredibly complicated, requiring an understanding of complex interrelated systems, an ability to predict the outcome of different actions, and an ability to balance competing considerations. That means there’s all the more room for errors in judgement to slip in. Even experts in political forecasting often do worse than simple actuarial predictions, when estimating the probabilities of events up to five years in the future.⁵ The organisations best placed to solve the world’s most important problems are also often highly bureaucratic, meaning that decision-makers face many constraints and competing incentives, not always aligned with better decision-making.

We think that improving the decision-making competence of key institutions may be particularly crucial, as the risks we face as a society are substantial, and institutions often have a large role in managing the risks.

With technological developments in nuclear weapons, autonomous weapons, bioengineering, and artificial intelligence, our destructive power is quickly increasing. Crises resulting from war, malicious actors, or even accidents could claim billions of lives or more.⁶

It’s not clear that individual decision-making, or the structure of key institutions, has evolved at anything near the pace needed to manage these potential crises — our institutions and decision-making processes look pretty similar to those that failed in the First and Second World Wars, and yet the worst-case scenarios they need to mitigate are several orders of magnitude larger.

But there’s some good news. Researchers are studying how to improve our ability to make predictions about the future, how to better think probabilistically, and how to think about complex problems in a more structured way.

It seems like developing and applying strategies that improve human judgement and decision-making could be very valuable — especially if focused on institutions working on particularly important problems, and combined with a thorough understanding of how such institutions operate.

Why work on this problem?

Improving decision-making could help us to solve many other problems

Improving humanity’s ability to gather information, as well as our judgement and decision-making, improves our ability to solve almost all other problems. It is thus an extremely robust and high-leverage way of solving problems. Some of the most important revolutions in decision-making include the inventions of the scientific method, probability theory, and Bayesian reasoning.

But these tools — as well as more modest but still impactful innovations such as better forecasting methods — aren’t always at the fingertips of key decision-makers, like those high up in governmental, corporate, or philanthropic institutions. Developing reasoning tools, making them available, and promoting their use in a variety of settings seems like it could help with a large range of issues.

This also means it’s a particularly good approach to focus on if you’re uncertain about what specific problems are most important to work on.

There are several ways to make progress on improving decision-making

Research so far has made some progress identifying techniques that reliably improve judgements and decision-making — and at least where there is good evidence, there does seem to be growing interest in getting these techniques implemented in practice.

Philip Tetlock’s work on forecasting,⁷ for example, has identified a number of ways to improve the accuracy of predictions on real-world events, which have been tested in large-scale randomised control trials (RCTs). A separate research programme on prediction markets suggests getting people to ‘bet’ on their predictions can increase the accuracy of political forecasts, corporate predictions, and statistical weather forecasts.⁸

(Incidentally we interviewed Tetlock about his research, when to trust experts, and his career advice for people working on improving institutional decision-making on our podcast.)

With increasingly solid evidence for these effects, getting them practically implemented seems achievable — there is already interest in forecasting techniques in the intelligence community, for example.⁹ One promising next step here would be to take findings that we have solid evidence for, and run smaller-scale pilots in specific organisations or parts of government.

For instance, people’s judgements can be dramatically overconfident, leading to poorly informed decisions — unjustified confidence in Iraq’s possession of WMDs had dramatic consequences, as we’ve seen. Part of the problem here is that people aren’t very well calibrated — our brains don’t seem to have a very good intuitive sense of what it means to say something is, say, “70% likely” — and as a consequence, the statements we make with 70% confidence turn out to be true much less than 70% of the time.

But there’s some evidence that it’s possible to improve people’s calibration through training. Hubbard Decision Research has trained over 1,000 people in calibration and found that 80% of participants were ideally calibrated after five exercises.¹⁰

More research here — particularly larger-scale RCTs, and studies that look at the effects of calibration training on real-world judgements (rather than just trivia questions) — could help strengthen the case for implementing these techniques in practice.

Other techniques that might be promising include structured analytic techniques (SATs)¹¹ for reducing biases in judgement, and the Delphi method¹² for aggregating different viewpoints — we discuss these and other promising approaches in more detail later.

Work in this area is relatively neglected

Work on ‘improving decision-making’ very broadly isn’t all that neglected. There are a lot of people, in both industry and academia, trying out different techniques to improve decision-making. The British government spends a lot of money on management consulting¹³, for example, and there are researchers working on questions related to decision-making at all major universities across a range of different disciplines — including psychology, economics, business, marketing, and political science.¹⁴

However, there seems to be very little work focused on rigorously testing different techniques to get strong evidence of effectiveness, or putting the best proven techniques into practice in the most influential institutions.

For example, there has been a lot of work trying out ‘scenario analysis’ techniques, with thousands of papers discussing various methods of correcting for overconfidence by considering a wider variety of possible scenarios. But it wasn’t until 2005 that someone published an experiment actually testing whether scenario analysis improved prediction accuracy — and as it turned out, it didn’t.¹⁵ This illustrates how, without good evidence on the effectiveness of different techniques, a lot of well-intentioned effort on ‘improving decision-making’ might be wasted.

Similarly, though a lot of effort goes into gathering knowledge generally, much less goes into figuring out the best ways to gather, aggregate, and use knowledge.

Improving epistemics and decision-making also seems more neglected than other ways of trying to ‘improve the system,’ such as general science and education, suggesting this work is more effective. People often argue for investing in science, education, or for certain kinds of political reform, for similar reasons we’ve given here: because these things will help us better tackle all kinds of problems.

For instance, the US government spends around 4.6% of GDP on education ($800 billion),¹⁶ and in a survey of the top 100 US foundations by GiveWell, US education accounted for 15% of spending, beaten only by healthcare.¹⁷ By contrast, there doesn’t seem to be much government funding or charitable efforts explicitly directed at improving institutional decision-making processes in the ways we’ve discussed. And, despite the potential importance of artificial intelligence in the 21st century, we could only identify a handful of people working on systematic methods to forecast its speed of development and likely impacts (we interviewed one of those researchers on our podcast).

What’s more, there’s reason to think that focusing on institutions directly might be a more effective way to improve decision-making than a broad approach to improved education, as it targets a smaller set of people who already have a lot of influence, and focuses more on institutional processes which can often have a big impact on how high-stakes decisions actually get made.

It’s worth considering why this work hasn’t received a lot of attention so far, and whether it might be neglected for good reason. One such reason might be if it really is incredibly difficult to innovate in this area or get key actors and large institutions to adopt new practices — we discuss some of these potential barriers in the next section. In addition, the kinds of large-scale controlled trials we need to conduct to rigorously test techniques can be expensive and time-consuming.

What are the major arguments against this being pressing?

It’s difficult to get change in practice

Perhaps the main concern with this area is that it’s not clear how easy it is to actually get better decision-making strategies implemented in practice — especially in bureaucratic organisations, and where incentives are not geared towards accuracy.

It’s often hard to get groups to implement practices that are costly to them in the short term — requiring training, resources, and changes from the status quo — and only promise abstract or long-term benefits. For example, there’s been some resistance to getting prediction markets implemented in practice, as running prediction markets with ‘real money’ is probably an illegal form of gambling (and even if you could get around this, it’s legally complex).¹⁸

However, these problems may be surmountable if we can find ways to show decision-makers that the techniques will help them achieve the objectives they care about.

You might also be able to help shift decision-making practices by setting up institutional incentives that favour deliberation and truth-seeking.

But recognising the difficulties of getting change in practice also means it seems especially valuable for people thinking about this issue to develop an in-depth understanding of how important groups and institutions operate, and the kinds of incentives and barriers they face. It seems plausible that overcoming bureaucratic barriers to better decision-making may be even more important than developing better techniques.

You might want to work on a more specific problem

Suppose you think that climate change is the most important problem in the world today. You might believe that a huge part of why we’re failing to tackle climate change effectively is that people have a bias towards working on concrete, near-term problems over those that are more likely to affect future generations. And so you might consider doing research on how to overcome this bias, with the hope that you could make important institutions more likely to tackle climate change.

However, if you think the threat of climate change is especially pressing compared to other problems, this might not be the best way for you to make a difference — even if you discover something useful about reducing the bias to work on immediate problems, it might be very hard to get that implemented in a way that’s going to directly make a difference to climate change.

But it’s likely better to focus your efforts on climate change more directly — for example by working for a think tank doing research into the most effective ways to cut carbon emissions. In general, more direct interventions seem more likely to move the needle.

That said, if you can’t implement solutions to a problem without improving the decision-making processes involved, it may be a necessary step.

The advantage of broad interventions like improving decision-making is that they can be applied to a wide range of problems. The disadvantage of working in this area is that it might be harder to target your efforts towards a specific problem. So if you think one specific problem is significantly more urgent than others, and you have an opportunity to work on that problem more directly, then it is likely more effective to do the direct work.

There might be better ways to improve our ability to solve the world’s problems

One of the biggest arguments for working in this area is that if you can improve the productivity or judgement of people working on important problems, then this increases the effectiveness of everything they do to solve those problems.

But you might think there are better ways to increase the speed or effectiveness of work on the world’s most important problems.

For example, perhaps the biggest bottleneck on solving the world’s problems isn’t poor decision-making, but simply lack of information: people may not be working on the world’s biggest problems because they’re lacking crucial information about what those problems are. Being more rational won’t help them if they don’t have that information.

A lot of work on promoting effective altruism might fall in this category: giving people better information about the effectiveness of different causes, interventions, and careers.

What can you do in this area?

We can think of work in this area as falling into several broad categories:

1. More rigorously testing existing techniques that seem promising.
2. Doing more fundamental research to identify new techniques.
3. Fostering adoption of the best proven techniques in high-impact areas.
4. Directing more funding towards all of the above.

All of these areas seem pressing and seem to have room to make immediate progress (we already know enough to start trying to implement better techniques, but stronger evidence will make adoption easier, for example). This means that which area to focus on will depend quite a lot on your personal fit and opportunities — we discuss each in more detail below.

1. More rigorously testing existing techniques that seem promising

The idea here would be to take techniques that seem promising, but haven’t been rigorously tested yet, and try to get stronger evidence of where and whether they are effective. Some techniques or areas of research that fall into this category:

• Calibration training — improving the accuracy of probability judgements — has a reasonable amount of evidence suggesting it is effective. However, most calibration training focuses on trivia questions — testing whether this training actually improves judgement in real-world scenarios could be promising, and could help to get these techniques applied more widely.
• Structured Analytic Techniques (SATs) are a number of techniques developed for reducing cognitive biases in intelligence analyses. Examples of SATs include checking key assumptions and challenging consensus views. These seem to be grounded in an understanding of the psychological literature, but few have been tested rigorously (i.e. with a control group and looking at the impact on accuracy). It could be useful to select some of these techniques that look most promising, and try to test which are actually effective at improving real-world judgements.¹⁹ It might be particularly interesting and useful to try to directly pitch some of these techniques against each other and compare their levels of success.
• Methods of aggregating expert judgements, including Roger Cooke‘s classical model for structured expert judgement (which scores different judgements according to their accuracy and informativeness and then uses these scores to combine them),²⁰ and the Delphi method (a method for building consensus in groups by using multiple iterations of questions to collect data from different group members).
• Better expert surveying in important areas — for example, on the model of the IGM Economic Experts Panel.

Academic researchers and groups working in this area:

• Professor Philip Tetlock at the University of Pennsylvania, who has pioneered research in improving forecasting.
• Professor Robin Hanson, an economist at George Mason University who has researched and advocated for the use of prediction markets.
• Professor George Wright at Strathclyde University in Glasgow, a psychologist who does research into applying the Delphi method and scenario thinking to improve judgements.
• Stephen Coulthart at the University of Albany has done some research looking at the effectiveness of different SATs in intelligence analysis.²¹
• There are some groups focused on more applied decision-making research, which may have money to run large field experiments, particularly in business schools, including: the Behavioural Insights Group at the Harvard Kennedy School; the Operations, Information and Decisions Group at the Wharton School, University of Pennsylvania; the Decision Sciences Group at Duke’s Fuqua School of Business; the Center for Decision Research at The University of Chicago Booth School of Business; and the Behavioural Science Group at Warwick Business School.

There may also be some non-academic organisations with funding for, and interest in running, more rigorous tests of known decision-making techniques:

• Intelligence Advanced Research Projects Activity (IARPA) is probably the biggest funder of research in this area right now, especially with a focus on improving high-level decisions.
• Consultancies with a behavioural science focus, such as the Behavioural Insights Team, may also have funding and interest in doing this kind of research. These organisations generally focus on improving lots of small decisions, rather than on improving the quality of a few very important decisions, but they may do some work on the latter.²²

2. Doing more fundamental research to identify new techniques

You could also try to do more fundamental research: developing new techniques and approaches to improved epistemics and decision-making, and then testing them. This is more pressing if you don’t think the existing techniques are very good.

One example of an open question in this area is: how do we judge ‘good reasoning’ when we don’t have objective answers to a question? That is, when we can’t just judge answers or contributions based on whether they lead to accurate predictions or answers we know to be true?²³ Two examples of current research programmes related to this question are IARPA’s Crowdsourcing Evidence, Argumentaion, Thinking and Evaluation (CREATE) programme and Philip Tetlock’s Making Conversations Smarter, Faster (MCSF) project. You could try to get involved with one of the teams working on these projects.

The academics and institutions listed above might also be promising places to work if you’re interested in developing new decision-making techniques.

3. Fostering adoption of the best proven techniques in high-impact areas

Alternatively, you could focus more on implementing those techniques we currently think are most likely to improve collective decision-making (such as the research on forecasting by Tetlock, prediction markets, or calibration training).²⁴ If you think one specific problem is particularly important, you might prefer to focus on the implementation of techniques (rather than developing new ones), as this is easier to target towards specific areas.

As mentioned above, a large part of ‘fostering adoption’ might first require better understanding the practical constraints and incentives of different groups working on important problems, in order to understand what changes are likely to be feasible. For this reason, working in any of the organisations or groups listed below — with the aim of better understanding the barriers they face — might be valuable, even if you don’t expect to be in a position to change decision-making practices immediately.

These efforts might be particularly impactful if focused on organisations that control a large number of resources, or organisations working on important problems. Here are some examples of specific places that might be good to work if you want to do this:

• Any major government, perhaps especially in the following areas: (1) intelligence/national security and foreign policy; (2) parts of government working on technology policy (the Government Office for Science in the UK, for example, or the Office of Science and Technology Policy in the US); (3) development policy (DFID in the UK, USAID in the US); or (4) defence (the Department of Defense in the US, or the Ministry of Defence in the UK). Many parts of government are also now hiring for ‘behavioural science’ specialist roles,²⁵ which might be a good option if you’re qualified for them (which generally means a master’s or PhD in psychology or a related discipline, and ideally some policy experience).
• Organisations that direct large amounts of money towards solving important world problems — such as foundations or grants agencies like the Gates Foundation or Open Philanthropy. See also our profile on working as a foundation grantmaker.
• Think tanks and academic institutions focused on questions about the long-run future of humanity, which require making judgements about speculative long-term scenarios, such as the Future of Humanity Institute in Oxford and the Centre for the Study of Existential Risk in Cambridge.

You could also try to test and implement improved decision-making techniques in a range of organisations as a consultant. Some specific organisations where you might be able to do this, or at least build up relevant experience, include:

• Good Judgment is an organisation founded on the basis of Tetlock’s successful research project on forecasting, and now runs training for individuals and organisations to apply these findings to improve predictions.
• Working at a specialist ‘behavioural science’ consultancy, such as the Behavioural Insights Team or Ideas42. Some successful academics in this field have also set up smaller consultancies — such as Hubbard Decision Research (which has worked extensively on calibration training).²⁶
• HyperMind, an organisation focused on wider adoption of prediction markets.
• Going into more general consultancy, with the aim of trying to specialise in behavioural science or decision-making — see our profile on management consulting for more details.

Finally, you could also try to advocate for the adoption of better practices across government and organisations, or for improved decision-making more generally — if you think you can get a good platform for doing so — working as a journalist, speaker, or perhaps an academic in this area.

Julia Galef is a good example of someone who has followed this kind of path. Julia worked as a freelance journalist before cofounding the Center for Applied Rationality, she co-hosts the podcast Rationally Speaking, and has a YouTube channel with hundreds of thousands of followers. She’s also written a book on improving your own judgement, while running the Update Project, which focuses on helping decision-makers improve their models of the world and resolve disagreements more productively. More broadly, she’s aiming to build an intellectual community of influential people across a range of different fields, who are genuinely trying to be truth-seeking and resolve disagreements in a productive way. You can learn more about Julia’s career path by checking out our interview with her.

4. Directing more funding towards research in this area

One challenge for all of the above areas is that it may be difficult to get funding for the kinds of work and research involved. Therefore, another approach might be to move a step backwards in the chain and try to direct more funding towards work in all of the aforementioned areas: developing, testing, and implementing better decision-making strategies.

The main place we know of that seems particularly interested in directing more funding towards improving decision-making research is IARPA in the US.²⁷ Becoming a programme manager at IARPA — if you’re a good fit and have ideas about areas of research that could do with more funding — is therefore a very promising opportunity. There’s also some chance Open Philanthropy will invest more time or funds in exploring this area (they have previously funded some of Tetlock’s work on forecasting). Otherwise you could try to work at any other large foundation with an interest in funding scientific research where there might be room to direct funds towards this area.

If you work at any of the organisations listed above, you might also try to advocate for more funds to be directed towards testing and implementing improved decision-making practices, even if you’re not in a position to do the work yourself.

If you don’t have the relevant background to do research or implement better practices yourself, but you think this is important and you’re in (or think you could work up to) an influential position in an important organisation, then you might be able to allocate more funding towards improving decision-making practices where you work (e.g. by funding tests or hiring someone who would be able to do this work). If you’re in a position to do this, this could be even higher impact than working somewhere like IARPA, where there already seems to be a lot of motivation to direct funding towards these problems.

We don’t currently think that there are many great direct donation opportunities in this area, so this probably isn’t the best way to have an impact — at least not for relatively minor donations. If you’re a larger donor, though, you might consider funding academics to do the sort of research outlined in points 1 and 2, or even trying to set up an organisation to conduct and/or fund more of this kind of research.

Learn more

• Listen to our podcast episodes on forecasting with Philip Tetlock: Prof Tetlock on predicting catastrophes, why keep your politics secret, and when experts know more than you and Accurately predicting the future is central to absolutely everything. Professor Tetlock has spent 40 years studying how to do it better.
• We also recommend our podcast with Mushtaq Khan on using institutional economics to predict effective government reforms.
• See also this video interview with 80,000 Hours advisor Alex Lawsen on forecasting and AI progress.
• For other angles, check out our podcast episodes on voting reform with Aaron Hamlin, Taiwan’s experiments with democracy with Audrey Tang, and misaligned incentives in international development with Karen Levy.
• Learn about Our World in Data’s strategy for making important information about the biggest issues out there available to decision-makers everywhere.
• See a talk by Tyler John on ways governments can take into account the interests of future generations.
• And check out a talk from Max Stauffer on strategies to support better political decision-making.
• Read’s Lizka’s Effective Altruism Forum post, Disentangling “Improving Institutional Decision-Making.”
• If you’re interested in doing research in this area, our career profile on valuable academic research gives more detail on that general career path.
• If you think you might be better placed to get techniques implemented, or to direct funding, check out our profiles on policy-oriented civil service and working as a foundation grantmaker.
• To learn more about existing research in this area, consider reading: (1) Philip Telock’s Superforecasting, (2) Robin Hanson’s work promoting prediction markets, (3) some of the other research emerging from IARPA’s relevant grants, in particular: ACE, HFC, FUSE, and SHARP.
• See also the work of Andy Matuschak, Michael Nielsen, and Bret Victor on "tools for thought", new mediums for science and engineering, and "programmable attention". Also this blog post about their work, by a former Chief Advisor to the former UK Prime Minister Boris Johnson.