The idea this week: AI governance careers present some of the best opportunities to change the world for the better that we’ve found.

Last week, US Senator Richard Blumenthal gave a stark warning during a subcommittee hearing on artificial intelligence.

He’s become deeply concerned about the potential for an “intelligence device out of control, autonomous, self-replicating, potentially creating diseases, pandemic-grade viruses, or other kinds of evils — purposely engineered by people, or simply the result of mistakes, no malign intention.”

• Read our review of AI governance and coordination career paths to see how you could get involved.

We’ve written about these kinds of dangers — potentially rising to the extreme of an extinction-level event — in our problem profile on preventing an AI-related catastrophe.

“These fears need to be addressed, and I think can be addressed,” the senator continued. “I’ve come to the conclusion that we need some kind of regulatory agency.”

And the senator from Connecticut isn’t the only one:

• The White House has led a coalition of the top AI companies to coordinate on risk-reducing measures, and they recently announced a joint voluntary commitment to some key safety principles. President Joe Biden and Vice President Kamala Harris have been directly involved in these efforts, with the president himself saying the technology will require “new laws, regulation, and oversight.”
• Four top companies developing advanced AI systems — Anthropic, Google, Microsoft, and OpenAI — launched a new industry body called the Frontier Model Forum, which they say is “focused on ensuring safe and responsible development of frontier AI models.”
• Other countries are moving to take substantial AI policy steps as well, including the United Kingdom, the European Union, and China.

What does all this mean?

There’s a lot of interest in AI governance at the moment, and we think it could play a crucial role in reducing catastrophic risks.

But while an opportunity for progress has opened up with the increased interest in AI governance, it’s far from guaranteed that the development of policy in this area will go well. The world still needs much more work and many more thoughtful and talented people to produce policy measures that adequately mitigate the risk and increase the chances of beneficial outcomes for all. If you’re a good fit for this career, you may be able to help.

To learn more, look at our brand new career review of AI governance and coordination, which we list among the most potentially impactful paths we’ve researched. It details key facts about:

• How you can assess your personal fit
• How to gain the necessary career capital
• What kinds of roles might be impactful

We discuss roles in global governments, international institutions, think tanks, nonprofits, and third-party organisations — as well as working to develop good corporate policy and governance in the private sector.

We also updated our older article specifically on US AI policy paths to reflect recent developments and thinking.

Learn more:

• Working in US AI Policy
• Anthropic CEO Dario Amodei’s statement to the US Senate on AI risk
• What could an AI-caused existential catastrophe actually look like?
• Lennart Heim on the compute governance era and what has to come after