Enjoyed the episode? Want to listen later? Subscribe here, or anywhere you get podcasts:

We should not have a single point of failure.

If this is really part of a shared global effort to safeguard the future of humanity, we need to intervene at every point possible — prevention, detection, and response — to reduce the risk as close to zero as we can that we would face something catastrophic from a biological release in the future.

Jaime Yassif

If a rich country were really committed to pursuing an active biological weapons program, there’s not much we could do to stop them. With enough money and persistence, they’d be able to buy equipment, and hire people to carry out the work.

But what we can do is intervene before they make that decision.

Today’s guest, Jaime Yassif — Senior Fellow for global biological policy and programs at the Nuclear Threat Initiative (NTI) — thinks that stopping states from wanting to pursue dangerous bioscience in the first place is one of our key lines of defence against global catastrophic biological risks (GCBRs).

It helps to understand why countries might consider developing biological weapons. Jaime says there are three main possible reasons:

  1. Fear of what their adversary might be up to
  2. Belief that they could gain a tactical or strategic advantage, with limited risk of getting caught
  3. Belief that even if they are caught, they are unlikely to be held accountable

In response, Jaime has developed a three-part recipe to create systems robust enough to meaningfully change the cost-benefit calculation.

The first is to substantially increase transparency. If countries aren’t confident about what their neighbours or adversaries are actually up to, misperceptions could lead to arms races that neither side desires. But if you know with confidence that no one around you is pursuing a biological weapons programme, you won’t feel motivated to pursue one yourself.

The second is to strengthen the capabilities of the United Nations’ system to
investigate the origins of high-consequence biological events — whether naturally emerging, accidental or deliberate — and to make sure that the responsibility to figure out the source of bio-events of unknown origin doesn’t fall between the cracks of different existing mechanisms. The ability to quickly discover the source of emerging pandemics is important both for responding to them in real time and for deterring future bioweapons development or use.

And the third is meaningful accountability. States need to know that the consequences for getting caught in a deliberate attack are severe enough to make it a net negative in expectation to go down this road in the first place.

On top of this, Jaime also thinks it’s vitally important to get better at anticipating threats. She thinks governments around the world should be investing more in biosecurity intelligence — to find out early if other states or non-state actors are developing a fledgling interest in developing biological weapons.

But having a good plan and actually implementing it are two very different things, and today’s episode focuses heavily on the practical steps we should be taking to influence both governments and international organisations, like the WHO and UN — and to help them maximise their effectiveness in guarding against catastrophic biological risks.

Jaime and Rob explore NTI’s current proposed plan for reducing global catastrophic biological risks, and discuss:

  • The importance of reducing emerging biological risks associated with rapid technology advances
  • How we can make it a lot harder for anyone to deliberately or accidentally produce or release a really dangerous pathogen
  • The importance of having multiples theories of risk reduction
  • Why Jaime’s more focused on prevention than response
  • Multiple intervention points for reducing risks throughout the bioscience R&D lifecycle: funders, research oversight committees, suppliers of goods and services, and publishers
  • The history of the Biological Weapons Convention
  • How much we can rely on traditional law enforcement to detect terrorists
  • Jaime’s disagreements with the effective altruism community
  • And much more

And if you think you might be interested in dedicating your career to reducing GCBRs, stick around to the end of the episode to get Jaime’s advice — including on how people outside of the US can best contribute, and how to compare career opportunities in academia vs think tanks, and nonprofits vs national governments vs international orgs.

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

Producer: Keiran Harris
Audio mastering: Ryan Kessler
Transcriptions: Katy Moore

Highlights

NTI's vision

Jaime Yassif: The way that we talk about it is, we want to reduce emerging biological risks associated with rapid technology advances. And we want to move quickly to keep up with the pace of these technology developments and develop meaningful solutions that reduce risks. We’re thinking about tackling this, really, at an international level, because fundamentally, it’s an international problem. And unless countries around the world tackle this in a comprehensive way, it’s going to be hard to meaningfully address the risk.

Jaime Yassif: We see a gap in the current biosecurity architecture that we’re hoping to fill. There are a couple reasons for this. First of all, countries around the world, as we saw in our Global Health Security Index, for the most part really don’t have the governance, tools, and regulations in place to provide effective oversight of bioscience and biotechnology, and in particular, dual-use bioscience research and development.

Jaime Yassif: Our data showed that fewer than 5% of countries around the world really had evidence that they were effectively doing this. And even in the United States, which has done a lot, there’s a lot of room for improvement in the US as well. And so that was one driver for us thinking about this.

Jaime Yassif: At an international level, there’s work going on at various institutions that’s really valuable, but there’s no international entity existing right now that’s dedicated to reducing this set of risks that I’ve been talking about and taking on these specific approaches that we’re discussing.

Jaime Yassif: The two most visible and, in my view, valuable players in this space are the World Health Organization, which has a science office that’s relatively new that’s tackling some of these issues. And that’s fantastic and they’re doing really valuable work. And likewise, the Biological Weapons Convention, the part of the UN that supports that and serves as secretariat for that convention. That convention is really important for upholding the norm against bioweapons development and use. And they’re thinking about technology issues that are associated with that convention, and they’re trying to build up capabilities to be smarter about that and better informed.

Jaime Yassif: And that’s fabulous, but I still think that there’s a gap in the system and that these existing institutions aren’t really designed or set up to address the set of issues that I’ve been describing. And so, we at NTI are envisioning setting up a new international organization that’s dedicated to biosecurity and that would work very closely with the UN System, including the BWC and including WHO, but would be independent and would really be the home for all these different kinds of projects, in an ideal future. And it would work to meaningfully reduce risks.

Jaime Yassif: The mission that we’re envisioning is that it would work to reduce catastrophic biological events that could be caused by a deliberate attack or an accidental release event. And that this entity would promote stronger global norms about biosecurity and also develop tools and incentive structures to incentivize people and institutions to actually adhere to those norms. So that’s the big picture, and that’s a lot to tackle — it’s a very broad mandate. We envision that it would start pretty small, with a pretty narrow focus. And if we can demonstrate that that works well, for example, with DNA synthesis screening, then we can expand from there. That’s the vision.

The layered defence approach

Jaime Yassif: We don’t believe that there’s a single silver bullet. There’s no one action we can take that will reduce the risk to zero or near zero. We really believe in a layered defense and that there are multiple intervention points throughout the bioscience and biotechnology research and development lifecycle, and we want to go as upstream as possible.

Jaime Yassif: So it begins at the funding and project conception stage. It progresses through research execution, either at an academic facility or in a company. It involves the acquisition of goods and services like DNA and getting pathogen samples from a pathogen repository. These are all things that you need to get in order to do your work, if you’re a scientist. Then it goes on to publication and/or commercialization. So there are different groups that control these different points and each of those points is an opportunity for risk reduction that I think is meaningful.

Jaime Yassif: In recent years and recent decades, as our community has struggled to find really effective ways to reduce risks and threats in the bio space — anytime you come up with any solution, people can poke holes in it. And it’s very tempting to say, “Oh, well, it’s not a 100% solution” or “There are big holes in it and therefore that’s not an answer and we should drop it.”

Jaime Yassif: I think that that’s a mistake. Having tried to develop multiple solutions and continuing to go through that pattern, I’ve decided that just because something has holes or it doesn’t reduce the risk by 100% doesn’t mean we should drop it. And then in terms of this layered defense, I absolutely believe that that’s the way to go.

Jaime Yassif: When I was working at Open Phil as a program officer, we had a conversation that I thought was a really helpful way to talk about it in simple terms, which is just: you find all the biggest holes in the system, and you try to plug the biggest holes first. So I’m not making an argument that we shouldn’t prioritize. We absolutely should, and there are a priori ways to figure out what are more and less impactful things we can do. But I think we shouldn’t reduce the list down to one or two actions. It should be a longer list.

Biosafety vs biosecurity

Jaime Yassif: Biosafety, as I would define it and I think as we would define it at NTI, is preventing accidental release of biological agents or pathogens from the lab — either through some sort of physical failure of equipment, or through some human that gets infected and spreads it by human-to-human contact. So biosafety is meant to prevent that.

Jaime Yassif: Biosecurity is preventing the exploitation of the legitimate enterprise by malicious actors. And so, in my view it’s preventing the dissemination of info hazards that could create a roadmap to make it easier for people to engineer or create a pathogen. Or preventing people from getting the materials that they could use in order to make a pathogen. It’s preventing them from getting access to facilities where they might be able to use those facilities to make something nasty. It’s those kinds of things. So it’s preventing exploitation for weapons purposes.

Rob Wiblin: So labs and institutions and universities more often think about biosafety — they think about how their staff could get sick more than the broader global picture of “How do we prevent terrible things from happening?”

Jaime Yassif: Well, biosafety encompasses accidental release that could spread into the community, in addition to occupational safety issues. Biosafety is an idea that has an institutional history — there’s a process, it has broader international support, and it’s very clearly defined. Biosecurity just isn’t as well-developed — what it means to do that and what are the control measures you put in place to reduce those risks. That just doesn’t have as broad international support, it doesn’t have the institutional history, and it doesn’t have the institutional infrastructure.

Jaime Yassif: So in terms of what it means to strengthen biosecurity at various institutions, I think one way to think about it is if you take the existing biosafety infrastructure at universities, and you make sure that those groups of people have the expertise and resources to also do biosecurity review. And you try to get that to be shared internationally. There’s work to be done here in developing what those practices are.

Jaime Yassif: And as part of our efforts at NTI, we’re also working with the group at Stanford, with leadership from Megan Palmer. She’s thinking a lot about these questions — how do we share information across different communities to figure out how to do this better? Because I think there’s a lot of learning we need to do as a community to do that well. But my understanding is that a lot of this is going on in the United States and the West, and in other countries I’m less sure — and that we have to help our partners in other countries build more robust infrastructure. NTI has engaged a lot of projects to help train people, experts in other countries, to do this well and to build that infrastructure. And there’s a lot of willingness but again, these people are allies and partners and they need help and they need resources.

Jaime Yassif: In terms of companies, we definitely want to see similar infrastructure there, but I don’t have as much visibility into what’s going on right now. I would assume they’re taking some precautions, at the very least, to meet their legal requirements. And probably going above and beyond that, because their enlightened self-interest is to avoid catastrophic accidents or other mishaps.

Constraining capabilities

Jaime Yassif: I think the way that we need to keep up with rapid advances in bioscience and biotechnology is by partnering with the very organizations that are driving those advances and move at that pace. I think the reason we can succeed is if we are working with the same scientists, and academic research institutions, and companies, and funders, and publishers that are driving the advances — if we build in biosecurity into the very processes by which the technology’s developed — we have a real chance at staying ahead of the curve.

Jaime Yassif: I think it’s a combination of working with these more agile, fast-moving groups, and then it’s also being proactive and not reactive. So instead of responding to a new threat after it emerges and preparing a patch, if we look over the horizon and see what’s coming and proactively build in new biosecurity systems before new capabilities come online, I think we can keep up.

Jaime Yassif: A classic example of this is in the DNA synthesis screening domain. The community of DNA synthesis providers, most of which are companies, voluntarily screen DNA synthesis orders in order to make sure the pieces of DNA — which are the building blocks of all living things, the building blocks of dangerous pathogens — don’t fall into the hands of malicious actors. So they’re pretty technologically sophisticated. They’re able to evolve their screening capabilities as technology advances.

Jaime Yassif: An example of looking over the horizon is benchtop synthesis. That’s this new mode where instead of centralized DNA synthesis, you have this future — which we think is coming soon — where people might be able to print DNA on their benchtop. Not everyone’s going to do that, but a part of the market is going there. What we are thinking about at NTI, and what the benchtop synthesis development community is thinking about, is how do we build in biosecurity by design, and make that a reality before these tools are disseminated widely. That’s a classic example of how you stay ahead of the curve and you don’t let the technology get out ahead of you before the security’s built in. And we need to do that across the board.

Jaime Yassif: I think it’s good to zoom out and talk about the big picture before we get into the nitty-gritty details of the actions. So what we’re trying to do is prevent two types of big risks. One is we want to prevent malicious actors, like rogue actors, from exploiting the legitimate global bioscience and biotechnology research and development enterprise that they could use to make it easier for them to develop a biological weapon and carry out an attack. That’s one huge piece of this.

Jaime Yassif: The other piece is we also want to prevent catastrophic lab accidents. So we want to reduce the possibility or reduce the risk that there could be an accidental laboratory release of an engineered pathogen that could have catastrophic consequences. So those are the two big pieces that we’re after in our work to strengthen governance of bioscience and biotechnology.

Biological weapons arms races

Jaime Yassif: One [motivation for pursuing a biological weapons program] is just fear of what their adversary might be up to. So recognizing that right now, there’s not as much transparency among states about what is going on with regard to biodefense and bioscience and biotechnology development. There is a risk that there could be misperceptions, that there could be escalating suspicion, and that could lead to an undesirable future where you have arms racing behavior.

Jaime Yassif: So that’s a future we really want to avoid. And a way that you address that is through increased transparency. Transparency is helpful in the sense that if, in fact, the vast majority of states aren’t in violation of the Biological Weapons Convention, that would just help clarify that. And to the extent that transparency could actually bring to light where there are violations, that’s also information we want to know.

Rob Wiblin: You’ve raised this issue of a potential arms race with biological weapons and our past guests have raised that as well. But I’m just imagining that scenario, where country A and country B do not have amicable relations, and A suspects that B has a biological weapons program — would A really benefit that much from establishing their own bioweapons program? How does that really help them to defend themselves? They could just invest in all kinds of other military stuff that isn’t related to bioweapons programs. I guess I’m not quite understanding how it exactly assists them in the arms race situation.

Jaime Yassif: Sure. I think that’s a fair question. And I’m certainly not trying to justify the development of biological weapons or encourage any state to seriously consider it — that’s definitely not my position, just to be very clear. But I think if you want to really understand where the rationale might come from, I think some people are concerned that bioweapons could be viewed as an asymmetric weapon.

Jaime Yassif: So if you’re a superpower and you’ve got lots of resources, then it’s easy to build up very strong conventional capabilities that are very effective in deterring adversaries from attacking you, and actually can be used in warfare. And there’s no moral or humanitarian norm against using conventional weapons. But those are expensive and hard to develop, and having an edge in conventional terms is hard.

Jaime Yassif: A nuclear weapon is an asymmetric weapon. And there have been some discussions about how that dynamic works, and we need to work to disincentivize development of nuclear weapons. But I would say the threshold for developing a biological weapon is even lower, because dual-use bioscience and biotechnology, knowledge, tools, resources — they’re widely distributed. And so there’s just a lower barrier to entry, so you could imagine a country saying, “Well, I can develop this asymmetric weapon at lower cost, and it’s more accessible to me than these other means of military dominance or deterrence.”

Why the US walked away from negotiating a verification mechanism for the BWC

Jaime Yassif: So something that’s important to understand generally about the Biological Weapons Convention (BWC) is that it’s broken into these three blocs that are really a historical artifact of the Cold War. You’ve got the Western group, which is comprised of Western countries — many of which are allied with the US — you’ve got Europe. You’ve got the Eastern bloc, which is a group that’s associated with the former Soviet Union and countries that are closely allied to them. And then you’ve got the Non-Aligned Movement, which is a lot of countries from the Global South, developing countries. And they all have slightly different interests, and those interests came forth in this discussion.

Jaime Yassif: So here were the challenges: the Non-Aligned Movement, fundamentally, as part of these protocol negotiations, they were really advocating for development assistance. They wanted to weaken some of the export controls, including the Australia Group, so they could have access to technology. Technology transfer is important to them, and they wanted also financial assistance to build their own capability —

Rob Wiblin: Biotech industries.

Jaime Yassif: Exactly, exactly. And so —

Rob Wiblin: What’s the Australia Group?

Jaime Yassif: The Australia Group is a group of like-minded countries, mostly Western, that have agreed upon export controls of key technologies that are associated with potential weapons of mass destruction. And so it constrains the export of certain knowledge or goods to certain countries that could otherwise lead to proliferation of weapons of mass destruction.

Rob Wiblin: Some poorer countries, some developing countries, weren’t keen on this, because they saw it as limiting their scientific research development or their economic development. And so they wanted to see those kind of export controls reduced and maybe even countries like Australia or the US paying to build a biotech R&D industry or sector within their countries.

Jaime Yassif: Yeah. So they’re emphasizing economic development assistance and weakening of certain export controls as part of their positioning in the context of these discussions. And so that was what they were pushing for, which was viewed by others as a threat to the nonproliferation regime and could actually exacerbate proliferation and weaken the BWC overall. And so that was a dynamic that was underway.

Jaime Yassif: And there were some other countries that had other agendas. I’m not going to name names. But there were certain countries that were trying to create a list-based approach to what is considered a biological threat, instead of having a general-purpose criterion and saying, “If you have intention to weaponize this, it’s already crossing the line.” Redefining what it means to violate the BWC to a list-based approach, where you basically define “Within these quantities and these categories of pathogens, this is a violation” — and therefore implicitly things that fall outside this specific list have what it was perceived to be a quote unquote “safe harbor.” And so again, to some other countries, including in the West, there was a concern that this could create big holes in the BWC in terms of what is considered a violation, which could undermine the Convention.

Jaime Yassif: And then a third thing is… Part of what would be an effective verification mechanism would include challenge inspections — that’s commonly found in other regimes. So some countries were trying to weaken the challenge inspections provisions by saying they would have to pass through the UN Security Council, in which certain countries have veto power. So all of these things were viewed by others as really weakening what was possible, and weakening the BWC, and untenable.

Jaime Yassif: It was a really challenging negotiation, and really entrenched positions in different political groupings. It wasn’t clear that they were ever going to arrive at some sort of consensus statement about what was going to work that was going to be meaningfully strengthening BWC instead of weakening it.

Articles, books, and other media discussed in the show

NTI’s work:

Other publications:

Opportunities to get involved:

Other 80,000 Hours Podcast episodes:

Related episodes

About the show

The 80,000 Hours Podcast features unusually in-depth conversations about the world's most pressing problems and how you can use your career to solve them. We invite guests pursuing a wide range of career paths — from academics and activists to entrepreneurs and policymakers — to analyse the case for and against working on different issues and which approaches are best for solving them.

The 80,000 Hours Podcast is produced and edited by Keiran Harris. Get in touch with feedback or guest suggestions by emailing [email protected].

What should I listen to first?

We've carefully selected 10 episodes we think it could make sense to listen to first, on a separate podcast feed:

Check out 'Effective Altruism: An Introduction'

Subscribe here, or anywhere you get podcasts:

If you're new, see the podcast homepage for ideas on where to start, or browse our full episode archive.