AI policy and strategy research
Review status
Based on an in-depth investigation
Table of Contents
Why work on research on AI policy and strategy?
In 2019, when researchers at the Center for Security and Emerging Technology (CSET) started working on AI policy research, it was a fairly niche topic. 2019 AIs couldn’t do much, and most policymakers weren’t thinking about the technology. But CSET started researching AI export controls, data protection, and cybersecurity. When AI development sped up over the next few years and the government needed expertise, they were ready. CSET researchers testified to Congress about how useful hardware export controls might be, their reports were cited in major news coverage about the need for AI regulation, and CSET’s founding director ended up serving as a technology and national security advisor to the president.
Over the next few years, AI will keep developing quickly, and policy research can help us keep up. The world needs concrete policies that could actually address the largest risks from advanced AI, and we need to start working on those policies even before we know what those risks will be.
The people who develop and deploy new policies face hard questions: How should frontier AI development be regulated? What evaluations should be required before a model is deployed? How should compute be governed? What should the international architecture for AI governance look like? Policymakers can’t answer these questions alone; they often look to outside researchers for help, and those researchers have real influence over what gets enacted.
Research that contributes to better AI governance can take a lot of forms:
- Policy development: Work out what specific policies would actually reduce the largest risks, and what the tradeoffs are between different approaches.
- Strategy and feasibility: Figure out which policies are politically and technically achievable, what coalitions would be needed to pass them, and how they would be implemented and enforced.
- Technical research that informs policy: This includes research on compute governance, information security standards for frontier AI, and methods for evaluating dangerous capabilities in AI systems.
- Forecasting and measurement: Research on the trajectory of AI capabilities, compute, and deployment (conducted by organisations like Epoch AI) helps decisionmakers anticipate developments before they need to react.
Would you be a good fit?
Skills that will help you succeed
AI policy and strategy is a lot like any other research field. Our advice on becoming a researcher covers the requisite skills. Most importantly, you’ll need to:
- Identify research questions that matter and that you can make progress on.
- Write clear, structured prose that reaches the relevant audience(s), helping them understand and take action.
- Make good decisions with limited information, since many of the most important questions in this field are not yet well-defined.
- Seek and incorporate feedback, especially from people who disagree with you.
Beyond general research skills, and background knowledge on recent developments in AI, your other needs depend on the research you want to do, and what type of institution you’ll work in:
- If your research interests are in a technical domain, you’ll need technical AI research skills.
- Some quantitative ability and facility with data goes a long way in some AI governance projects, even ones that don’t focus on technical AI research — though many roles don’t require it.
- Political and bureaucratic skills are critical for certain roles in government, especially ones where you’ll interact with elected officials and their staff members, but less important in roles where you’ll interact primarily with other researchers.
What experience is useful?
This work doesn’t require one specific background, but certain kinds of experience tend to be especially useful:
- Prior research experience, whether through full-time roles, university coursework, internships, or independent projects. This is the single most important qualification for research work, and it’s also how you find out whether research suits you.
- Policy experience. Time spent inside government, in a think tank, on a campaign, or at a policy-focused nonprofit gives you a sense of how policy work actually happens and what might be useful to do next.
- Subject-matter expertise. Deep knowledge of either the technical side of AI or another area related to research you plan to pursue (like administrative law, economic policy, or biosecurity).
Downsides to working on AI policy and strategy research
Several experts we’ve talked to warn that a lot of research on AI governance may prove to be useless. It takes a long time for results from policy research to start coming out, and even after the fact, it can be hard to evaluate how useful the research was. If a policy is enacted, it’s usually hard to know whether a given piece of research about that policy really made a difference, or even whether the policy was overall good or bad.
So it’s important to be reflective and seek input from others in the field about which areas have the greatest potential. We list several organisations below that are doing strong work in the field and be good places to seek mentorship.
Top organisations
The following organisations are doing research we think is especially likely to contribute to good AI governance. This list isn’t exhaustive, and the field is changing quickly; talk to us if you want more specific guidance on where to apply.
- Think tanks:
- Center for Security and Emerging Technology (CSET)— produces policy analysis on AI and national security
- RAND Corporation — Broad public-policy research nonprofit; AI work focuses on security, governance, and societal impacts
- Centre for Long-Term Resilience — think tank advising the UK government on extreme risks from AI, biosecurity, and risk management
- Research nonprofits:
- Institute for AI policy and Strategy — Researches AI policy at the intersection of national security, compute governance, and international strategy
- Epoch AI — Tracks and forecasts trends in AI development
- Centre for the Governance of AI (GovAI) — Conducts AI governance research and fieldbuilding
- Center for AI Safety (CAIS) — Conducts technical AI safety research, field-building, and advocacy to reduce societal-scale AI risks
- Institute for Law and AI — Researches and advises on the legal challenges posed by advanced AI
- Palisade Research — empirically studies dangerous AI capabilities (cyber, deception, shutdown resistance) and communicates findings to policymakers
- Secure AI Project — Advocates for AI safety policy at the state and federal level (e.g. enforceable safety protocols for frontier developers).
- Coefficient Giving (disclaimer: our main funder) — Philanthropic funder and advisor focused on high-impact giving across causes including AI risk
- Research roles within government
- Center for AI Standards and Innovation (CAISI) — Evaluates frontier AI models for national security risks (cyber, bio, chem) and develops standards for measuring AI capabilities
- Cybersecurity and Infrastructure Security Agency (CISA) — Federal cybersecurity agency (under DHS) that researches and coordinates defence of critical infrastructure, including from AI-related cyber threats
- Office of Critical and Emerging Technologies (CET, DOE) — Coordinates DOE’s work on AI, including AI governance and infrastructure, across the department and its national labs
- AI companies:
Governance research roles at AI companies may offer real influence on how they approach risk, but they also come with tradeoffs, including the possibility that working at a frontier AI company contributes to capabilities advancement. Read our career review discussing the pros and cons of working at a top AI company before applying.
Examples of people pursuing this path
Next steps
If you’re ready to apply for jobs
Our job board features opportunities in AI policy and governance research:
If you need to build career capital
Get research experience in academia, via a fellowship, or on your own. If you’re a student, choose research-heavy courses or pursue extracurricular research programs at your university.
An advanced degree (master’s or PhD) can be useful for both career capital and research experience. Some research roles require one, and it can be helpful even for roles that don’t — though it also requires a heavy time commitment. Our article on US policy master’s degrees covers some of the considerations around applying to graduate programs.
Get feedback on research. One approach for testing your fit — especially when starting out — is to write up analyses and responses to existing work on AI policy, or to investigate questions that haven’t received much attention. But you don’t need a formal publication process to start sharing your work: setting up a blog or Substack is simple. Share the link with people whose judgment you respect, and in spaces where other researchers will find it, and request feedback. If reading this made you think of something you’re excited to research, that’s a good sign that you should try it!
If the results are promising, and you enjoyed the process, that’s an excellent sign.
If you aren’t getting positive feedback after a month or so, and you don’t feel like you’re making much progress, don’t sink much more time into researching on your own: many researchers need mentors and colleagues to do their best work, especially early in their careers, so you might do better research as part of a fellowship or graduate program. On the other hand, most research is fairly independent, so if you’ve learned that you need to work very closely on a team to be productive, most research careers won’t be a good fit for you.
If you’ve done research as part of a course, you can also request feedback from people whose opinions you value (beyond your professor).
Fellowships are among the best ways to enter policy work. They offer first-hand policy experience, funding, training, mentoring, and networking. While many require an advanced degree, some are open to college graduates.
Here are some fellowships and resources we recommend looking into:
- List of US policy fellowships
- Presidential Management Fellowship
- Presidential Innovation Fellowship
- Horizon Fellowship
- TechCongress Fellowship
- STPI Science Policy Fellowship
- AAAS Science & Technology Policy Fellowships (STPF)
Get jobs where you’ll learn a lot about a relevant topic. Even if a role isn’t directly in AI policy research, time spent in a related field — machine learning research, security policy, regulatory work, technical AI safety — can build a foundation for later work.
Talk to people. Reach out to people who do the kind of work you’re interested in and/or work at some of the organisations listed above. The AI governance research world is small, and personal conversations can be the best way to make connections, learn about job opportunities, get feedback on ideas, and hear what people are really working on. See our Substack for tips on finding people to connect with, even if you don’t have an existing network.
Speak with us
If you think this path might be a great option for you, but you need help deciding or thinking about what to do next, our team might be able to help.
We can help you compare options, make connections, and possibly even help you find jobs or funding opportunities.
Learn more
Top recommendations
- BlueDot Impact’s selective Frontier AI Governance course can be taken full time (six days in total) or part-time over a longer period, so that it fits alongside your job or studies. It covers the frontier AI landscape, key governance debates, and how to find your own path into the field, and is aimed at people seriously considering AI governance work. Alumni have gone on to work at places like RAND, CSET, IAPS, GovAI, CAISI, and frontier lab policy teams.
- See Horizon’s list of US AI policy resources, think tanks, fellowships, and more.
Further recommendations
Resources from 80,000 Hours:
- Article: The US AI policy landscape: where to work to have the biggest impact
- Podcast: Ajeya Cotra on whether it’s crazy that every AI company’s safety plan is ‘use AI to make AI safe’
- Podcast: Eileen Yam on how we’re completely out of touch with what the public thinks about AI
- Podcast: Helen Toner on the geopolitics of AI in China and the Middle East
- Podcast: Allan Dafoe on why technology is unstoppable and how to shape AI development anyway
- Podcast: Carl Shulman on the economy and national security and government and society after AGI
- Podcast: Lennart Heim on the compute governance era and what has to come after
- Podcast: Sella Nevo on who’s trying to steal frontier AI models, and what could they do with them
- Podcast collection: The 80,000 Hours Podcast on Artificial Intelligence
Other resources:
- Emerging Tech Policy Careers
- Think tank reports from CSET, CNAS, and CSIS
- The White House’s 2023 US national artificial aintelligence R&D strategic plan
- NIST’s 2023 AI risk management framework
Read next: Learn about other high-impact careers
Want to consider more paths? See our list of the highest-impact career paths according to our research.