AI policy and strategy research

Summary

In a nutshell: The world needs concrete policies to manage the risks from advanced AI — and the people who develop those policies rely heavily on researchers to figure out what would actually work. Researchers at the best organisations have real influence over what gets implemented, including in government. That said, it’s hard to know whether your research is making a difference, and a lot of policy research has little impact. We think this is a strong path for people with solid research skills who want to engage with AI governance.

Pros:

  • Policy research that reaches the right people at the right time can be very influential.
  • The field is growing and there’s considerable demand for people who take AI risk seriously.
  • This path offers Iinteresting and challenging intellectual work.

Cons:

  • Long feedback loops: it’s often hard to know whether your research influenced anything, even after the fact.

Key facts on fit:

  • The most important qualification is prior research experience — try writing up an analysis and sharing it for feedback to test your fit before committing to this path.
  • You’ll need to be comfortable making judgement calls with limited information, since many of the most important questions in this field aren’t yet well-defined.
  • You don’t need to be a technical AI researcher, but some quantitative ability and knowledge of how AI systems work is useful in many roles.

Recommended

If you are well suited to this career, it may be the best way for you to have a social impact.

Review status

Based on an in-depth investigation 

Why work on research on AI policy and strategy?

In 2019, when researchers at the Center for Security and Emerging Technology (CSET) started working on AI policy research, it was a fairly niche topic. 2019 AIs couldn’t do much, and most policymakers weren’t thinking about the technology. But CSET started researching AI export controls, data protection, and cybersecurity. When AI development sped up over the next few years and the government needed expertise, they were ready. CSET researchers testified to Congress about how useful hardware export controls might be, their reports were cited in major news coverage about the need for AI regulation, and CSET’s founding director ended up serving as a technology and national security advisor to the president.

Over the next few years, AI will keep developing quickly, and policy research can help us keep up. The world needs concrete policies that could actually address the largest risks from advanced AI, and we need to start working on those policies even before we know what those risks will be.

The people who develop and deploy new policies face hard questions: How should frontier AI development be regulated? What evaluations should be required before a model is deployed? How should compute be governed? What should the international architecture for AI governance look like? Policymakers can’t answer these questions alone; they often look to outside researchers for help, and those researchers have real influence over what gets enacted.

Research that contributes to better AI governance can take a lot of forms:

  • Policy development: Work out what specific policies would actually reduce the largest risks, and what the tradeoffs are between different approaches.
  • Strategy and feasibility: Figure out which policies are politically and technically achievable, what coalitions would be needed to pass them, and how they would be implemented and enforced.
  • Technical research that informs policy: This includes research on compute governance, information security standards for frontier AI, and methods for evaluating dangerous capabilities in AI systems.
  • Forecasting and measurement: Research on the trajectory of AI capabilities, compute, and deployment (conducted by organisations like Epoch AI) helps decisionmakers anticipate developments before they need to react.

Would you be a good fit?

Skills that will help you succeed

AI policy and strategy is a lot like any other research field. Our advice on becoming a researcher covers the requisite skills. Most importantly, you’ll need to:

  • Identify research questions that matter and that you can make progress on.
  • Write clear, structured prose that reaches the relevant audience(s), helping them understand and take action.
  • Make good decisions with limited information, since many of the most important questions in this field are not yet well-defined.
  • Seek and incorporate feedback, especially from people who disagree with you.

Beyond general research skills, and background knowledge on recent developments in AI, your other needs depend on the research you want to do, and what type of institution you’ll work in:

  • If your research interests are in a technical domain, you’ll need technical AI research skills.
  • Some quantitative ability and facility with data goes a long way in some AI governance projects, even ones that don’t focus on technical AI research — though many roles don’t require it.
  • Political and bureaucratic skills are critical for certain roles in government, especially ones where you’ll interact with elected officials and their staff members, but less important in roles where you’ll interact primarily with other researchers.

What experience is useful?

This work doesn’t require one specific background, but certain kinds of experience tend to be especially useful:

  • Prior research experience, whether through full-time roles, university coursework, internships, or independent projects. This is the single most important qualification for research work, and it’s also how you find out whether research suits you.
  • Policy experience. Time spent inside government, in a think tank, on a campaign, or at a policy-focused nonprofit gives you a sense of how policy work actually happens and what might be useful to do next.
  • Subject-matter expertise. Deep knowledge of either the technical side of AI or another area related to research you plan to pursue (like administrative law, economic policy, or biosecurity).

Downsides to working on AI policy and strategy research

Several experts we’ve talked to warn that a lot of research on AI governance may prove to be useless. It takes a long time for results from policy research to start coming out, and even after the fact, it can be hard to evaluate how useful the research was. If a policy is enacted, it’s usually hard to know whether a given piece of research about that policy really made a difference, or even whether the policy was overall good or bad.

So it’s important to be reflective and seek input from others in the field about which areas have the greatest potential. We list several organisations below that are doing strong work in the field and be good places to seek mentorship.

Top organisations

The following organisations are doing research we think is especially likely to contribute to good AI governance. This list isn’t exhaustive, and the field is changing quickly; talk to us if you want more specific guidance on where to apply.

Governance research roles at AI companies may offer real influence on how they approach risk, but they also come with tradeoffs, including the possibility that working at a frontier AI company contributes to capabilities advancement. Read our career review discussing the pros and cons of working at a top AI company before applying.

Examples of people pursuing this path

Next steps

If you’re ready to apply for jobs

Our job board features opportunities in AI policy and governance research:

    View all opportunities

    If you need to build career capital

    Get research experience in academia, via a fellowship, or on your own. If you’re a student, choose research-heavy courses or pursue extracurricular research programs at your university.

    An advanced degree (master’s or PhD) can be useful for both career capital and research experience. Some research roles require one, and it can be helpful even for roles that don’t — though it also requires a heavy time commitment. Our article on US policy master’s degrees covers some of the considerations around applying to graduate programs.

    Get feedback on research. One approach for testing your fit — especially when starting out — is to write up analyses and responses to existing work on AI policy, or to investigate questions that haven’t received much attention. But you don’t need a formal publication process to start sharing your work: setting up a blog or Substack is simple. Share the link with people whose judgment you respect, and in spaces where other researchers will find it, and request feedback. If reading this made you think of something you’re excited to research, that’s a good sign that you should try it!

    If the results are promising, and you enjoyed the process, that’s an excellent sign.

    If you aren’t getting positive feedback after a month or so, and you don’t feel like you’re making much progress, don’t sink much more time into researching on your own: many researchers need mentors and colleagues to do their best work, especially early in their careers, so you might do better research as part of a fellowship or graduate program. On the other hand, most research is fairly independent, so if you’ve learned that you need to work very closely on a team to be productive, most research careers won’t be a good fit for you.

    If you’ve done research as part of a course, you can also request feedback from people whose opinions you value (beyond your professor).

    Fellowships are among the best ways to enter policy work. They offer first-hand policy experience, funding, training, mentoring, and networking. While many require an advanced degree, some are open to college graduates.

    Here are some fellowships and resources we recommend looking into:

    Get jobs where you’ll learn a lot about a relevant topic. Even if a role isn’t directly in AI policy research, time spent in a related field — machine learning research, security policy, regulatory work, technical AI safety — can build a foundation for later work.

    Talk to people. Reach out to people who do the kind of work you’re interested in and/or work at some of the organisations listed above. The AI governance research world is small, and personal conversations can be the best way to make connections, learn about job opportunities, get feedback on ideas, and hear what people are really working on. See our Substack for tips on finding people to connect with, even if you don’t have an existing network.

    Speak with us

    If you think this path might be a great option for you, but you need help deciding or thinking about what to do next, our team might be able to help.

    We can help you compare options, make connections, and possibly even help you find jobs or funding opportunities.

    APPLY TO SPEAK WITH OUR TEAM

    Learn more

    Top recommendations

    • BlueDot Impact’s selective Frontier AI Governance course can be taken full time (six days in total) or part-time over a longer period, so that it fits alongside your job or studies. It covers the frontier AI landscape, key governance debates, and how to find your own path into the field, and is aimed at people seriously considering AI governance work. Alumni have gone on to work at places like RAND, CSET, IAPS, GovAI, CAISI, and frontier lab policy teams.
    • See Horizon’s list of US AI policy resources, think tanks, fellowships, and more.

    Further recommendations

    Resources from 80,000 Hours:

    Other resources:

    Read next:  Learn about other high-impact careers

    Want to consider more paths? See our list of the highest-impact career paths according to our research.

    Plus, join our newsletter and we’ll mail you a free book

    Join our newsletter and we’ll send you a free copy of a book focused on how to tackle the greatest threats facing humanity. T&Cs here.