Scaling organisations making AI go well

By · Published July 15th, 2026 ·

Matthew Henry matthewhenry, CC0, via Wikimedia Commons

Summary

In a nutshell: Many organisations working on making AGI go well plan to grow rapidly in the coming years, driven by the urgency of the problems humanity faces and the significant increase in philanthropic funding available to solve them. Many of these organisations are struggling to fill senior management and operations roles like chief operating officers, chiefs of staff, recruiting leads, and programme managers. If you have experience managing teams or complex projects, especially at a growing startup or nonprofit, you could lead and execute projects meant to bring about a better future.

Pros:

  • A great operator can multiply the output of an entire team.
  • Your expertise will be in high demand; few senior candidates combine the right career experience with deep AI safety knowledge and interest.
  • You can see the results of your actions and know you made a difference on projects.
  • The field is growing fast, so you can advance quickly if you perform well.

Cons:

  • Even with an impressive CV, you’ll likely need to develop AI safety knowledge and understand an organisation’s strategic priorities before you secure a role.
  • The talent pipelines for management and operations roles are less well-developed than those for paths like research or policy.
  • While these organisations are growing fast, many are still small. Senior hires may occasionally be required to take on administrative tasks.

Key facts on fit: If you have strategy and management experience in a fast-growing startup or nonprofit, or in consulting, finance, tech, or recruiting — and you have deep AI safety knowledge and motivation — you might be a great fit. To help scale an AI safety organisation you need to understand its goals, reason in a transparent and analytical way about how it can achieve them, and be open to others’ viewpoints on how to solve problems.

Recommended

If you are well suited to this career, it may be the best way for you to have a social impact.

Review status

Based on an in-depth investigation 

Frontier AI companies have scaled their revenues, headcounts, and AI capabilities at a staggering rate in recent years. Since 2023, OpenAI and Anthropic have grown their staff by roughly 5–10x, revenue by over 30x, and AI models from simple chatbots to autonomous agents capable of expert-level cyberattacks and software engineering.

If progress continues at this pace — or is accelerated by powerful AI systems automating AI research and development itself — the world will not be prepared. Few government, nonprofit, or private-sector institutions are equipped to handle the risks of transformative AI.

To keep up, many AI safety, security, and governance organisations will have to scale at a similar pace. Some philanthropists hope to deploy tens of billions of dollars in the coming years to address these urgent threats. To use this much funding well, organisations currently working on these issues will need to plan and execute ambitious projects — which in turn means professionalising and growing their teams.

However, these organisations’ growth is already bottlenecked by a lack of senior management and operational leadership talent. In two AI safety talent surveys (one by us and another by a grantmaker), the positions organisations said they most struggled to fill were in senior management and operations: chief operating officers, chiefs of staff, directors of operations, and senior programme or people managers.

ALT DESCRIPTION

It’s rare for hiring managers to find applicants who combine real-world management or leadership experience with AI safety knowledge and motivation, as well as the analytical reasoning and strategic taste required to make good decisions in this field. Unlike technical or policy researchers, managers and operators can’t easily point to tangible outputs that demonstrate their judgment. And the best potential candidates are often later in their careers, without as much time to read, take courses, network in the community, etc.

For the AI safety field to rise to the challenge humanity is facing, it will need seasoned executors with knowledge of the field and good strategic judgement. If you have experience in a senior position scaling nonprofits or startups, we strongly encourage you to apply for advising or to jump straight to the following sections:

What do senior strategy, management, and operations professionals in AI safety do?

This career review covers a broad category of jobs focused on scaling the size and impact of AI safety organisations. Experts we’ve interviewed identified these jobs as significant bottlenecks right now: they report needing applicants who can manage people, build scalable systems, multiply others’ outputs, and reason well about strategy. Senior generalists need to think in a transparent and open-minded way to help make and execute organisational strategy decisions.

Here are a few examples of positions that use this skill set:

Chief operating officer

A COO works alongside other senior leaders to make an organisation’s strategic decisions, then builds the systems that turn those decisions into reality. They’re typically responsible for internal management structures, establishing a team’s culture, and setting up processes for hiring, compensation, events, etc. In short, a COO multiplies the output of the entire organisation and makes growth possible.

This job requires especially strong analytical reasoning and strategic judgement. As one expert told us, it often resembles what strategy consultants are brought in to do in other industries — figure out what’s happening, how things work, and how to improve processes — but the COO stays to lead the implementation.

What this looks like in practice depends heavily on the organisation’s size. In smaller teams, COOs may take on hands-on projects like leading hiring or running events — though ‘hard ops’ like payroll and taxes are almost always delegated to staff or external consultants. As an organisation grows, the COO might hire a director of operations or other direct reports to manage and oversee specific functions.

Chief of staff

A chief of staff’s responsibilities depend heavily on the principal they work for (usually a CEO or other executive) and the needs of the organisation. This title can describe many types of positions. However, all chiefs of staff enable leaders to do more and higher quality work.

Some chiefs of staff work across their principal’s portfolio — sitting in on most meetings and decisions, owning their inbox, and acting as an executor or thought partner across many projects. Others specialise, freeing up their principal’s time by owning a section of the portfolio themselves. This might involve directly managing some employees, running a hiring round, leading the CEO’s communications, etc. A chief of staff aims to eventually lead these projects with minimal supervision, deeply understanding the way their principal thinks and how they’d handle a given situation.

On either end of this spectrum, the value is the same: the opportunity cost of leadership time is enormous, and a good chief of staff can dramatically free up a leader’s time or multiply their output. One chief of staff told us they felt their work was impactful not just because they led an important hiring round and drafted public communications for their CEO, but because this allowed their CEO to focus on more urgent strategic issues. All the roles described in this career review have this ‘multiplier’ mindset, but it’s particularly useful for chiefs of staff.

Talent director or recruiting lead

As the thousands of roles we post on our job board each year demonstrate, finding new talent is a big task for the field of AI safety. Several leaders in the field told us that they need expert recruiters to help grow their work. A strong recruiter builds deep networks in AI safety, develops good taste about who would excel in a role and why, oversees active outreach to promising candidates, and serves as the project manager for hiring rounds.

In AI safety, a recruiting lead needs to be especially knowledgeable about the organisation’s work and its priorities. They need to know how to identify talented candidates, including how their views line up with the organisation’s mission. They also need to notice when an otherwise promising candidate might not be a good fit, which can be hard to recognise and requires working closely with staff to design hiring criteria.

This work also requires applied strategic decision-making. Skilled recruiters are constantly building mental models of candidates and the field as a whole, mapping out where someone’s skills could be useful and how to acquire the information they need about potential candidates (such as designing and running a work test). Recruiters can also help many organisations outside of their own, by referring second-place candidates or noticing when a candidate might be well-suited for a different role elsewhere. If you care a lot about AI safety and have strong interpersonal skills, you could make a sizable impact by helping an organisation build their team.

Programme or research manager

Programme managers design and run structured programmes like events, workshops, publications, or fellowships. They research and scope out new projects end-to-end, proposing a strategy and success criteria, as well as how they’ll track their impact over time. Programme managers often work with in-house or external experts to design new initiatives. Then, they execute. One expert told us they have a much harder time finding good programme managers than researchers because fewer candidates are interested.

As we’ve covered elsewhere, the growth of the AI safety field has led to an increased demand for research managers to support full-time researchers and fellows. They plan and oversee research projects, set goals, motivate others, assist in logistics and project management, review papers, and network with researchers across the field. Research managers typically need some knowledge of the research behind the projects they oversee, but not to the same depth as the researchers themselves.

Why work on scaling AI safety organisations?

There is significant and growing demand for these roles.

As mentioned above, in two recent surveys of hiring managers in AI safety, senior management and operations roles were identified as the most important bottleneck to growth. We suspect this will be even more true in the future, as philanthropists hope to move tens of billions of dollars towards efforts to make the future of AI go well for humanity, presenting organisations with many opportunities to scale. We know of at least one AI safety nonprofit that plans to double its headcount over the next year (from about 40 to about 80 staff).

These roles can multiply an entire team’s output.

Strategic decisions like salary or hiring policies can shape an organisation’s ability to recruit and retain top talent. A great manager can bring out the best in their reports, helping them meet clear goals and get unstuck. A well-run organisation can absorb new resources and turn them into results, while a poorly run organisation squanders them. Even very talented people sometimes get little done because they work somewhere that isn’t well-organised or pursuing sensible goals. One AI safety expert told us that the field has entered its “mid-game,” where the ability to execute on specific projects has become more urgent.

You’ll likely find intellectual stimulation and growth opportunities.

These roles put you close to senior leadership, letting you strategise alongside interesting thinkers and develop a deep understanding of the field — one you can use to shape AI’s chaotic and transformative impact on the world. Success in these positions requires making judgement calls under uncertainty, reasoning about tradeoffs, and quickly updating on the real-world results of your actions.

And because these roles are relatively neglected, you can also build a world-class skill set very quickly — “how do I run one of the world’s top AI safety events?” or “how do I incorporate AI progress forecasts into our annual growth plans?” are paths to mastery that few others are considering.

Would you be a good fit?

If your bookshelf looks like this, or you’d enjoy it if it did, you should consider this path.

Some AI safety knowledge is critical

We often hear from readers asking how much AI safety knowledge they need for these roles, and why it’s so essential for getting hired.

Experienced professionals are (understandably) sometimes confused or frustrated when they apply for an AI safety job that fits their background, but are rejected early in the process. They may have written a few short answer responses in the application or had an interview that tested their knowledge of and interest in AI safety, only to get a vague rejection. Often, they insufficiently communicated an understanding of the organisation’s priorities, or knowledge about AI safety in general. This is sometimes referred to as having a lack of ‘context.’

We think there are (at least) three reasons why hiring managers often prioritise preexisting AI safety knowledge and motivation in their senior operations and strategy hires:

  1. Understanding why decisions are made, not just how: growing an organisation involves many judgement calls about org-wide priorities, and sometimes requires standing in for existing leadership. The AI safety field’s strategic aims don’t map cleanly onto normal nonprofit or startup logic. Teams who believe humanity could go extinct in the next ten years due to an intelligence explosion, or that the main beneficiaries of their work could be future digital minds, will probably allocate resources and make decisions in different ways than others. Even organisations that work on more immediate AI security or misuse issues often have small teams, so want their senior staff to have a good internal model of the field as a whole to be able to take on strategic responsibilities.
  2. Culture fit and enjoyment of the work: mission-oriented organisations often build high-trust environments around shared values. For example, if you want to join a climate nonprofit or political party, you should decide whether you’d enjoy the culture you’d be joining. You’ll do better work and be happier if you believe in an organisation’s mission, and they want to hire people who do.
  3. This takes time to train: the field is still relatively small, with a lot of tacit knowledge that takes time to absorb. Many staff feel crunched by urgent problems in their daily work, so are hesitant to invest extra time into training. Still, some organisations are building this into their onboarding process by running reading and discussion groups, sending staff to conferences, and using more internal staff mentorship.

In practice, applying this knowledge could look like: deciding which organisations or media outlets to partner with to achieve a shared goal; designing and executing a fundraising strategy for potential donors interested in AI safety; leading high-touch hiring rounds for top talent; or determining what the organisation’s risk tolerance should be for concerns like public relations, opportunity costs of moving too slowly, budgeting given your organisation’s estimates around AI progress, etc.

Of course, the kind of knowledge and motivation required will differ across organisations. Some organisations have detailed worldviews and theories of change that affect all of their work (MIRI, for example), while others work across many research areas and have less of a defined ‘house view’ (CSET, for example). The organisations on our job board have different approaches to making AI go well, so we’d encourage you to read their websites carefully before applying. In addition, the more well-scoped a position with an organisation is, such as building HR or IT systems, the lower the bar will typically be for this.

Beyond knowledge and motivation, many AI safety organisations also highly value candidates with strong analytical reasoning skills. This is partly a general skill that exists in other industries — one expert recommended Jeff Bezos’ annual shareholder letters, the New England Journal of Medicine’s “Clinical Problem Solving” series, or books like Good Strategy, Bad Strategy as examples of sharp analytical reasoning.

But AI safety organisations often prioritise a particular kind of analytical reasoning. You may be asked to express arguments in a probabilistic, transparent, and open-minded way. For example, this post by Trevor Levin demonstrates a method of probabilistic and analytical reasoning to plan for (part of) AI’s future impact. AI safety organisations often work on neglected problems with few feedback loops, which makes it harder to learn from similar past work or iterate, so doing this analysis early and throughout a project is important.

What other skills are needed to succeed?

  • Strategic thinking and prioritisation. You think clearly about the value of an outcome compared to the resources it would take to achieve it and its opportunity cost. You can roughly measure and compare this before making a decision and after seeing the result.
    • This includes analytical reasoning skills, as described in the previous section.
  • Ability to delegate well. Management is about getting things done through other people. You can magnify a team’s impact by setting clear goals and staying involved throughout the process, including giving feedback and holding people accountable.
  • Execution. When taking up an area of responsibility, you can own and improve it even if it isn’t well-defined. You proactively notice problems and solve them, directly or by enabling others to do so. You’re organised and comfortable with task management.
  • Building operating systems. You like thinking in processes and have an optimisation mindset. You’re excited to build systems that scale to save staff time (i.e. a ticketing system or employee handbook that can handle 10 questions or 1000). Similarly, you help set org-wide cultural norms like goal-setting, written pre-meeting proposals, documenting processes, etc.
  • Developing positive relationships and buy-in from staff. You can listen to employee needs and get them bought in, since even the best systems only work if they’re used.
  • Hiring great people. Not all management and leadership roles involve hiring, but as organisations scale, being able to find great people is hugely important — whether you’re doing it yourself or creating clear hiring criteria to empower the recruiters you work with.

What experience is useful?

  • Experience in a fast-growing startup or nonprofit, particularly if you managed people, made strategic decisions about growth, or built internal systems for the org. A top candidate might have led operations as staff grew from a handful of people to 30 or more, or from 30 employees to 100 or more.
  • Senior roles in strategy consulting, tech companies, recruiting, or other nonprofits where you utilised the skills described above. Since AI safety organisations are mostly small-to-medium, it’s especially helpful if you’ve worked in smaller teams or managed your own programmes in an entrepreneurial way within a larger organisation.

Why is it so hard for hiring managers to find and attract top candidates?

Hiring senior generalists in AI safety has somewhat of a ‘chicken and egg’ problem. Recruiters want to find candidates who can demonstrate their ability to make good strategic decisions in applied AI safety contexts, but there are few ways to practice and showcase this skill from outside the field.

For this reason, trusted referrals and in-person connections at conferences are often used to identify candidates. But experienced professionals also have a high financial opportunity cost for their time, and can be busy with existing careers, families, or other responsibilities — making it difficult to invest time in conferences, part-time side projects, courses, etc.

Experienced professionals often need an efficient way to learn about the field and get connected in a few hours, rather than a few weeks. For this reason, we offer career advising calls to help you fast-track your AI safety career. Our team is particularly excited to speak with people who have the backgrounds described in this post. We’ve helped many mid-career professionals pivot into AI safety over the years.

Potential downsides of this path

  • Founding vs scaling: If you have scaling experience and a strong understanding of the AI safety field, you may also be well suited to founding or co-founding an organisation. This is especially true if you have experience building infrastructure for a fast-growing startup or nonprofit. To decide whether to start something new instead of building on existing work, you’ll have to form your own models of gaps in the field by speaking with experts, reading, reviewing grantmakers’ requests for proposals, and considering your comparative advantage.
  • Status and recognition: Object-level contributions like research often get more acclaim than operations or strategy work (especially outside of your organisation, where your contributions may be invisible). We think this is a mistake, since these roles are tremendously impactful and can multiply the output of an entire team. Still, if external recognition matters a lot to you, you might not enjoy some aspects of these jobs.
  • A different kind of work: Because they involve a lot of everyday, mundane decisions, these roles may at first seem less intellectually stimulating than other career paths. While this might be true in some respects, these roles still require a lot of analytical reasoning and systems-thinking. You also get the pleasure of seeing what actually happens: experts told us it’s satisfying to see how well your plans survive contact with reality.
  • Compensation: While salaries at AI safety organisations have risen and are often good by nonprofit standards, you might have higher paid opportunities elsewhere if you have the experience described above. Still, if you want a highly impactful job, this could be one of your best options. You might also find this work much more fulfilling than most private-sector work.1 Still, consider checking our job board to see how well different jobs are compensated.

How to get started if you have some relevant experience

  • Talk to us. Given the demand for this talent profile and the fact that people sometimes just need a few introductions or ways to learn more, if you have some experience, you should apply to speak with an advisor at 80,000 Hours. We’ll help you work through your uncertainties, and we can usually recommend a few people to meet.
  • Develop your foundational AI safety knowledge. Most candidates with existing career capital will need to spend some time learning about the field. Our 11 essential readings on AI safety, risk, and alignment page is a good place to start. You could also apply for a career grant or take a course from BlueDot Impact, make a career pivot with the Center for Effective Altruism’s four-day accelerator, or review AISafety.com‘s map of the field. If you prefer to listen, The 80,000 Hours Podcast has a curated series of episodes on AI, and most of our problem profiles and career reviews have audio narrations available.
  • Apply for roles on our job board. While this career review has focused on building up your knowledge, we think people with the experience to scale these organisations should still apply directly for open positions; you can learn as you go rather than waiting until you feel like an expert. The process of applying will help you test your fit and gain context on the organisations you’re interested in, since you can learn a lot from work tests and interviews. Our advisors have also seen experienced candidates underrate the value of their skills.
  • Network. AI safety is a small world with a lot of tacit knowledge. You could try attending an Effective Altruism Global conference (which can involve dozens of 1-on-1 conversations over a single weekend) or speaking with professionals currently working in the field in co-working spaces like Constellation or Mox in the Bay Area, or the DC or London EA communities. Even cold emails can be a great way to meet experts and learn about your fit — just make sure you concisely communicate that you’ve engaged with their work and why you reached out to them in particular. 80,000 Hours advisors can also help you get connected.
    • You can read our longer networking guide here.
  • Write. It’s hard to demonstrate substantive knowledge through a resume. Even if you’ve done the work to learn about AI, you might get overlooked — we’ve seen that happen to plenty of talented candidates. One way to overcome this: we’ve heard from several hiring managers that they have an easier time advancing candidates who’ve written about AI safety online. You could use what you’ve learned to write a book review or summary, respond to recent AI news, share your thoughts on an open question, or write about an organisation.
    • If you’re worried about publishing your ideas as a relative amateur, you could write pseudonymously, or attach a private writing sample to your applications — though this approach makes it harder for you to network and tighten your feedback loops.
  • Try a side project or part-time consulting. A full pivot isn’t the only option. Several hiring managers told us that contracting or consulting roles, while sometimes hard to initially find, are an underused and low-risk option for applicants and organisations. You build context, relationships, and a track record while the organisation gets to see your work, and might hire you or recommend you to others. Conferences, cold outreach, or job postings can sometimes lead to these opportunities.2

How to get started if you’re earlier in your career

This review has mostly focused on professionals with several years of management and strategy experience. However, junior readers should also be optimistic about considering this path.

Because AI safety organisations struggle to hire seasoned professionals who deeply understand their worldview and thinking, many have prioritised training and promoting their early-career employees into senior positions rather than hiring externally. One expert told us they would sacrifice several years of management experience for a candidate that could clearly stand in for their organisation’s leaders in matters of strategic taste, like designing a budget given short AI timelines, or communicating ideas to important stakeholders.

Great junior operations staff can often “eat their boss’s job,” as MIRI’s CEO Malo Bourgon said in our previous operations management career review. That review is a great place for more junior operations candidates to start learning about how to get into and excel in the field.

Next steps

Our job board features opportunities for scaling AI safety organisations:

    View all opportunities

    Speak with us

    If you think this path might be a great option for you, but you need help deciding or thinking about what to do next, our team might be able to help.

    We can help you compare options, make connections, and possibly even help you find jobs or funding opportunities.

    APPLY TO SPEAK WITH OUR TEAM

    Learn more:

    Read next:  Learn about other high-impact careers

    Decorative post preview

    Want to consider more paths? See our list of the highest-impact career paths according to our research.

    Plus, join our newsletter and we’ll mail you a free book

    Join our newsletter and we’ll send you a free copy of a book focused on how to tackle the greatest threats facing humanity. T&Cs here.

    Notes and references

    1. Our advisors sometimes describe this as an “impact sabbatical,” where you agree to take a lower salary for a few years before returning to the private sector. If you believe the development of transformative AI models is possible in the next few years, this might be one of the most important times in human history to work directly on these problems. We’d hate to see great talent sitting at their desks watching the world change outside their window, rather than working to positively influence it themselves.↩

    2. For example, in this post Aaron Gertler describes how his first role (helping to run an event) led to a referral to a full-time job. This post also tells Luca’s story of turning his side-project building a talent database led to him getting hired at 80,000 Hours.↩