New releases

#132 – Nova DasSarma on why information security may be critical to the safe development of AI systems

Podcast by Robert Wiblin and Keiran Harris · Published June 14th, 2022

If a business has spent $100 million developing a product, it’s a fair bet that they don’t want it stolen in two seconds and uploaded to the web where anyone can use it for free.

This problem exists in extreme form for AI companies. These days, the electricity and equipment required to train cutting-edge machine learning models that generate uncanny human text and images can cost tens or hundreds of millions of dollars. But once trained, such models may be only a few gigabytes in size and run just fine on ordinary laptops.

Today’s guest, the computer scientist and polymath Nova DasSarma, works on computer and information security for the AI company Anthropic with the security team. One of her jobs is to stop hackers exfiltrating Anthropic’s incredibly expensive intellectual property, as recently happened to Nvidia. As she explains, given models’ small size, the need to store such models on internet-connected servers, and the poor state of computer security in general, this is a serious challenge.

The worries aren’t purely commercial though. This problem looms especially large for the growing number of people who expect that in coming decades we’ll develop so-called artificial ‘general’ intelligence systems that can learn and apply a wide range of skills all at once, and thereby have a transformative effect on society.

If aligned with the goals of their owners, such general AI models could operate like a team of super-skilled assistants, going out and doing whatever wonderful (or malicious) things are asked of them. This might represent a huge leap forward for humanity, though the transition to a very different new economy and power structure would have to be handled delicately.

Continue reading →

The case for reducing existential risks

Article by Benjamin Todd · Last updated June 2022 · First published October 2017

In 1939, Einstein wrote to Roosevelt:

It may be possible to set up a nuclear chain reaction in a large mass of uranium…and it is conceivable — though much less certain — that extremely powerful bombs of a new type may thus be constructed.

Just a few years later, these bombs were created. In little more than a decade, enough had been produced that, for the first time in history, a handful of decision-makers could destroy civilisation.

Humanity had entered a new age, in which we faced not only existential risks from our natural environment, but also the possibility that we might be able to extinguish ourselves.

In this new age, what should be our biggest priority as a civilisation? Improving technology? Helping the poor? Changing the political system?

Here’s a suggestion that’s not so often discussed: our first priority should be to survive.

So long as civilisation continues to exist, we’ll have the chance to solve all our other problems, and have a far better future. But if we go extinct, that’s it.

Why isn’t this priority more discussed? Here’s one reason: many people don’t yet appreciate the change in situation, and so don’t think our future is at risk.

Social science researcher Spencer Greenberg surveyed Americans on their estimate of the chances of human extinction within 50 years. The results found that many think the chances are extremely low,

Continue reading →

#131 – Lewis Dartnell on getting humanity to bounce back faster in a post-apocalyptic world

Podcast by Robert Wiblin, Luisa Rodriguez and Keiran Harris · Published June 3rd, 2022

“We’re leaving these 16 contestants on an island with nothing but what they can scavenge from an abandoned factory and apartment block. Over the next 365 days, they’ll try to rebuild as much of civilisation as they can — from glass, to lenses, to microscopes. This is: The Knowledge!”

If you were a contestant on such a TV show, you’d love to have a guide to how basic things you currently take for granted are done — how to grow potatoes, fire bricks, turn wood to charcoal, find acids and alkalis, and so on.

Today’s guest Lewis Dartnell has gone as far compiling this information as anyone has with his bestselling book The Knowledge: How to Rebuild Civilization in the Aftermath of a Cataclysm.

But in the aftermath of a nuclear war or incredibly deadly pandemic that kills most people, many of the ways we do things today will be impossible — and even some of the things people did in the past, like collect coal from the surface of the Earth, will be impossible the second time around.

As Lewis points out, there’s “no point telling this band of survivors how to make something ultra-efficient or ultra-useful or ultra-capable if it’s just too damned complicated to build in the first place. You have to start small and then level up, pull yourself up by your own bootstraps.”

So it might sound good to tell people to build solar panels — they’re a wonderful way of generating electricity. But the photovoltaic cells we use today need pure silicon, and nanoscale manufacturing — essentially the same technology as microchips used in a computer — so actually making solar panels would be incredibly difficult.

Continue reading →

Let’s get serious about preventing the next pandemic

Blog post by Benjamin Todd · Published May 31st, 2022

I recently argued that it’s time for our community to be more ambitious.

And when it comes to preventing pandemics, it’s starting to happen.

Andrew Snyder-Beattie is a programme officer at Open Philanthropy, a foundation which has more than $1 billion available to fund big pandemic-prevent projects. He and Ethan Alley (co-CEO at Alvea) recently wrote an exciting list of projects they’d like to see get founded, including:

An international early detection centre
Actually good PPE
Rapid and broad-spectrum antivirals and vaccines
A bioweapons watchdog
Self-sterilising buildings
Refuges

One of these ideas is already happening. Alvea aims to produce a cheap, flexible vaccine platform using a new type of vaccine (DNA vaccines), starting with an Omicron-specific shot. In two months, they hired 35 people and started preclinical trials.

The above are technical solutions, which make it possible for a relatively small number of people to make a significant difference to the problem. But policy change is also an important angle.

Guarding Against Pandemics was an effort to lobby the US government for $30 billion in funding for pandemic prevention. Unfortunately the relevant bill didn’t pass, but the sum at stake made it clearly worth trying.

Efforts in the UK might be more successful —

Continue reading →

Clay Graubard and Robert de Neufville on forecasting the war in Ukraine

Podcast (80k After Hours) by Robert Wiblin and Keiran Harris · Published May 2022

In this episode of 80k After Hours, Rob Wiblin interviews Clay Graubard and Robert de Neufville about forecasting the war between Russia and Ukraine.

They cover:

Their early predictions for the war
The performance of the Russian military
The risk of use of nuclear weapons
The most interesting remaining topics on Russia and Ukraine
General lessons we can take from the war
The evolution of the forecasting space
What Robert and Clay were reading back in February
Forecasters vs. subject matter experts
Ways to get involved with the forecasting community
Impressive past predictions
And more

Who this episode is for:

People interested in forecasting
People interested in the war in Ukraine
People who prefer to know how likely they are to die in a nuclear war

Who this episode isn’t for:

People who’d hate it if a friend said they were 65% likely to come out for drinks
People who’d prefer if their death from nuclear war was a total surprise

Continue reading →

#130 – Will MacAskill on balancing frugality with ambition, whether you need longtermism, and mental health under pressure

Podcast by Robert Wiblin and Keiran Harris · Published May 23rd, 2022

Imagine you lead a nonprofit that operates on a shoestring budget. Staff are paid minimum wage, lunch is bread and hummus, and you’re all bunched up on a few tables in a basement office.

But over a few years, your cause attracts some major new donors. Your funding jumps a thousandfold, from $100,000 a year to $100,000,000 a year. You’re the same group of people committed to making sacrifices for the cause — but these days, rather than cutting costs, the right thing to do seems to be to spend serious money and get things done ASAP.

You suddenly have the opportunity to make more progress than ever before, but as well as excitement about this, you have worries about the impacts that large amounts of funding can have.

This is roughly the situation faced by today’s guest Will MacAskill — University of Oxford philosopher, author of the forthcoming book What We Owe The Future, and founding figure in the effective altruism movement.

Years ago, Will pledged to give away more than 50% of his income over his life, and was already donating 10% back when he was a student with next to no income. Since then, the coalition he founded has been super successful at attracting the interest of donors who collectively want to give away billions in the way Will and his colleagues were proposing.

While surely a huge success, it brings with it risks that he’s never had to consider before:

Will and his colleagues might try to spend a lot of money trying to get more things done more quickly — but actually just waste it.
Being seen as profligate could strike onlookers as selfish and disreputable.
Folks might start pretending to agree with their agenda just to get grants.
People working on nearby issues that are less flush with funding may end up resentful.
People might lose their focus on helping others as they get seduced by the prospect of earning a nice living.
Mediocre projects might find it too easy to get funding, even when the people involved would be better off radically changing their strategy, or shutting down and launching something else entirely.

Continue reading →