Transcript
We need a stronger effort to push AI companies to do better [00:00:00]
Rob Wiblin: We need more people holding AI companies to account when they do bad things.
That means staff inside companies objecting loudly and clearly — and sometimes even quitting and speaking up in public. It means investigative journalists digging up information the public has a right to know. It means highly informed people explaining exactly what companies are doing, and why it’s wrong. And it means advocates across all sorts of media making sure their misbehaviour is getting the attention it deserves.
The track record of that work is actually pretty remarkable for the modest amount of effort that has gone into it. That’s in part because many people in the companies legitimately want to do the right thing. And in part it’s because companies are scared their staff are going to conclude that they’re the bad guys, and that this will make it harder for them to retain and hire the talent they need.
The capacity to hold AI companies to account for their actions is becoming more important as events are moving faster and faster — there often just won’t be time to wait for governments to take action or pass legislation. We also need the option to bring down the immediate hammer of fierce public and expert outrage.
In the last few weeks, if we needed it, we got yet more evidence about how important that capacity is likely to be. Anthropic, OpenAI, and Google DeepMind all had serious scandals that warranted giving them hell.
Today I’m going to lay out what each did wrong, and the capacity we need to build to be able to hold these organisations’ feet to the fire.
Anthropic: who gets to pull the brake? [00:01:14]
First: Anthropic.
Anthropic markets themselves as the good guys: the ones who take the power of AI seriously, build it carefully, and actively want regulation to reduce the risks.
So I was pretty disappointed to learn that they’d been quietly shopping around a draft federal AI bill that would make it extremely hard for anyone to stop a frontier AI company doing something extremely dangerous.
Under their proposed bill, the only way the federal government could halt model training is for the Commerce Secretary to personally sign off on a detailed case and take it to a single federal court in DC. The government would have to establish an imminent risk, backed by a specific explanation of how the model could cause catastrophic harm and when. They’d have to prove that nothing less restrictive would work. And the judge would decide each case from scratch, with no deference to the regulator.
On top of that, stop orders would expire after 45 days, and any renewal would require proving the whole case again. And on top of that, it’s unclear whether a credible 10% chance of causing human extinction next year would be sufficient to get a stop order under these rules, because it may or may not qualify as “present or impending,” the language used in the bill.
Now, it’s true the White House would still keep existing tools like export controls that it can try to appropriate to tackle AI risk, even though they weren’t designed for that purpose at all. But the point of this legislation is to create the dedicated brake for catastrophic AI risk in particular — and the brake it’s creating seems designed to be very hard to pull.
At the same time, the proposed legislation would bar US states from requiring AI companies to publish their safety plans and tests, or get independent evaluations, or report incidents. And so it would retroactively kill most of the flagship AI laws passed by California, Illinois, and New York.
And most importantly, the draft lists another thing for US states to lose: the power to halt an AI model because it poses an imminent catastrophic risk “by statute, regulation, order, judicial decree, or other action.”
In the leaked text, the clause isn’t actually connected to the part of the bill that does the preempting, so as written, states would keep that power. But it’s hard to see why the clause is in there unless the draft is meant to take that power away from states, at least once the federal process existed.
And if it were completed the way that seems designed to work, then even if everyone in California’s government thought Anthropic was about to do something with an unreleased model that might literally kill everyone in the state, they couldn’t pass a law or seek a statutory court order to stop it. They’d be left hoping and praying that the Commerce Secretary would go to a court in DC and clear that very high bar I described.
This is kind of weird coming from a company whose CEO puts the chance of AI going “really, really badly” at 25%, and he wrote in September that recursive self-improvement “must be pursued very carefully, if at all.”
But on the other hand, given the fights they’ve had with the Trump administration this year, I can absolutely understand why Anthropic doesn’t want to empower the federal executive with enormous discretion over these issues.
A number of people in the Trump administration have been extremely hostile to the company, and seem basically out to get it in whatever way they can. Those folks would likely try to use any discretion the White House had to extract all sorts of political concessions from Anthropic, while doing nothing at all about actually dangerous research practices.
So I get it.
But if you’re really worried about the Trump administration or a future rogue US president, the natural solution is just to leave that power in the hands of US states like California for the time being, and help a number of relevant states to pass bills focused on catastrophic AI risk that would cover all the major players.
Just as an aside, for Anthropic, which is generally worried about extreme concentration of power driven by AI, I think there’s a case for avoiding federal preemption and continuing to regulate AI at the state level for some time to come.
The most dangerous setup for extreme concentration of power in human hands is to put control over AI companies into the same hands that already control the US military and the US intelligence services. That’s just a very potent and threatening combination, that’s very open to abuse — much more so than if this responsibility is just left with the California government.
But alternatively, if you mostly want to keep federal preemption, a different and very simple way to improve on Anthropic’s draft would have been to allow state attorneys general to petition the DC court, as well as the Secretary of Commerce. That avoids a really obvious single point of failure, and feels just clearly better for anyone who takes risks from AI very seriously.
But Anthropic’s approach ties California’s hands, it ties the hands of state attorneys general, and it ties the hands of state courts. Which means in practice it would likely lead to an outcome where Anthropic, for the most part, was just left to decide for itself whether to stop model training or using a model because the risk of loss of control had gotten too high.
I’m sincerely kind of confused about what the thinking is here. Because even if Anthropic believed it had completely solved alignment and so it posed no danger to anyone — which it doesn’t, but let’s imagine — surely even then at some point other more reckless AI companies would need to be stopped from blasting ahead with recursive self-improvement in a really incompetent way and thereby generating intolerable risks. By taking states out of the picture and giving exclusive responsibility to the Department of Commerce, this bill would make that harder to do.
So I think this legislation is really wrong-headed, and it’s worrying that Anthropic was pushing it privately in a way that made it very hard for insiders or outsiders to object to it.
Naturally Anthropic got absolutely blasted for this online. The head of federal affairs put out a long defence. And Dario conceded it was among the things they could have handled better, and put it down to communication.
But I think the core disagreement isn’t just about communication.
Anthropic says it’s going to rewrite the preemption language, but as of recording it only seems to be saying it’ll let states set standards for evaluators, and regulate the existence of kill switches — not allow states to retain any ability to halt model training or deployment themselves.
The single highest-stakes issue in AI regulation is who gets to stop AI companies engaging in really reckless R&D when going ahead risks actual human extinction. And for now, Anthropic’s answer appears to still be: one agency, one court, and only once the threat is imminent.
In my view it shouldn’t be up to a handful of people at one AI company and the Secretary of Commerce and a single federal judge to decide whether or not to roll the dice with all our lives.
If you’re inside Anthropic, please speak up and say this is a bad proposal the company shouldn’t be pushing. If you keep the pressure up, it’s likely the policy team will see reason and propose something that strikes a more reasonable balance between the different competing considerations at play.
OpenAI fires three safety researchers [00:07:14]
How about OpenAI?
Well, as you might have heard, OpenAI recently fired three safety researchers: Tomek Korbak, Mikita Balesni, and Jasmine Wang.
The reasons they say they were given for their firings were really weak and different for each person, even though all three were fired in simultaneous 2pm meetings. That made the justifications seem probably pretextual — and indeed OpenAI later said there was a bigger breach of trust, though it wouldn’t say exactly what. That left it unclear what the actual motivation or possible breach of trust was. Why did OpenAI want any of these people gone in the first place?
Well, new reporting from the Wall Street Journal suggests the firings originated from a leak investigation.
On September 1, The Information reported that OpenAI’s new model, Astra, used a technique called “looped transformers.” This was something OpenAI didn’t want anyone to know.
According to Korbak and Balesni, OpenAI’s investigators eventually became suspicious that the leak about looped transformers had gotten to The Information via an outside researcher, Ryan Greenblatt, who they had both been speaking with as part of their work. OpenAI insisted on seeing the Signal chats that they had with Greenblatt. But they both used disappearing messages with him, which meant they couldn’t prove they hadn’t messaged him about the looped transformers.
To be clear, the three researchers and Greenblatt all strongly deny leaking anything, and as far as we know there’s no proof that they did.
But up until now, OpenAI hasn’t revealed anything else that would explain the sudden and simultaneous firings. So it seems fairly likely that what happened is the company concluded that plausibly either Korbak or Balesni could have been the leaker, and decided it had to get rid of them on that suspicion alone.
Jasmine Wang was also asked about the leak, so she was potentially a suspect as well, even though she didn’t have any disappearing messages. The thing is, all three of them had been vocal about the importance of keeping AI chain of thought monitorable, so probably the idea was that they would be particularly likely to disapprove of looped transformers.
Just to back up and give a little context: looped transformers increase the amount of computation a model can perform before writing down what it’s thinking on the chain of thought. That makes it harder to notice if it’s doing things we don’t want, which is why researchers across the industry had warned that they were potentially the start of a slippery slope towards models that couldn’t be monitored at all.
That’s why, when the story broke, there was such a big backlash, and one former OpenAI researcher said the company seemed to be violating “one of the few redlines that exist in the industry.”
Now, if leadership at OpenAI really believes that using looped transformers is fine — and they claim to have evidence that it is — then it’s natural enough for any company to want to fire staff it’s convinced have leaked trade secrets to the press.
However, a few things. As far as we know, the evidence that any one of these three leaked that information is somewhere between weak and nonexistent.
First, using disappearing messages to discuss anything and everything is totally normal in the tech industry, and the AI world in particular, so by itself that simply isn’t grounds for much suspicion at all. Applying that standard consistently, almost everyone at the company could be suspected of a leak and fired, from the CEO down.
Second, The Information has been breaking stories about AI companies’ internal plans for years, from details of OpenAI’s secret “Strawberry” model to the terms of Google’s Pentagon contract this year. It clearly just has many sources in and around these companies, including investors and former staff, and probably a lot of stuff also leaks out through conversations overheard at parties and so on.
So there’s no particular reason to assume people worried about safety were the source here; this could just be another piece of juicy gossip that reached The Information one way or another.
Third, if it were leaked by someone who objected to looped transformers, there could easily be 100 or 200 or even more OpenAI staff who thought it was a really bad idea. Almost 400 OpenAI staff signed a petition earlier this year backing a push to slow AI down for safety reasons. And adding recurrent depth to models is among the most controversial research directions in AI, something a lot of people think is misguided.
Fourth, if you’re willing to admit a personal character judgement — I really think Greenblatt in particular is unlikely to have been involved in this leak. That was also the immediate reaction of a lot of other people in the AI world: “Ryan Greenblatt is meant to have leaked this? No, that doesn’t sound right.” Leaking stuff to the press is just not his vibe, and it’s in total conflict with the research work he actually wants to do. However this information got to The Information, I would bet my bottom dollar it wasn’t him.
So the case that any of them is responsible is not particularly compelling. The leak investigators have maybe just made a mistake or done a poor job here.
And on the other side of the ledger, you’ve got clear, terrible negative impacts.
They lost three great AI researchers focused on what the company says is one of its top safety and alignment problems.
The company created the impression that they fire staff who visibly speak up about the safety issues they’re concerned about, so OpenAI is going to have a harder time getting candid opinions from staff in future. In fact, I think that probably wasn’t their complaint with these three, but it’s how it looked to basically everyone when their firings were first announced.
They also made it seem risky for staff to speak to anyone outside of OpenAI, even when they’re doing it for legitimate work purposes and doing nothing wrong whatsoever.
And by concluding that one of these three is responsible for the leak on the basis of almost no evidence, they probably let the actual leaker go permanently unidentified.
So unless there’s non-public evidence yet to come to light, which is indeed possible, I would guess that these firings were a mistake — not just by my lights, but even just by the lights of OpenAI’s senior management.
Staff inside the company should say clearly that the tradeoffs OpenAI has struck here are unacceptable. They’re simply bad for the company, bad for its culture, and bad for its staff.
Sadly, OpenAI is not too likely to rehire these three, even if they are fully cleared of the leaking charge at some point. But the more internal and external grief OpenAI gets for this behaviour, the less likely they are to shoot from the hip and make a similar regrettable decision next time around.
Google DeepMind’s lifelong gag clause [00:12:44]
Alright, let’s talk Google DeepMind.
According to Google DeepMind staff who spoke to the BBC in mid-October, their UK employment contracts bar them from publicly criticising anything Google does, for the rest of their lives.
That sounds kind of crazy.
It’s one thing to say staff shouldn’t be publicly laying into you while working for you, or shouldn’t criticise you based on confidential information they know through their work. But it’s very different and very abnormal to say that they can’t criticise anything an enormous entity like Google does so long as they live.
This anti-criticism provision came up because it’s creating problems for ex-Googlers who want to criticise the company’s decision to sell AI to the US Department of War for any use the Department of War itself deems lawful — something which has upset a lot of staff and led a number to quit.
I was really quite surprised to learn about this, because two years ago there was this massive firestorm about OpenAI using non-disparagement agreements to muzzle its employees.
At the time, much of the compensation OpenAI staff were getting was in the form of shares and equity in the company — shares they had to wait to sell. And it turned out that OpenAI was forcing all staff who left the firm to sign an agreement to never criticise OpenAI, or it would confiscate all the shares in the company that they’d earned while working there, costing that person millions or perhaps even tens of millions of dollars. Furthermore, this no-disparagement provision itself had to remain completely secret — if you allowed anyone to learn of its existence, that alone might lead you to lose all of your equity.
When this came out, thanks in large part to former guests of the show Daniel Kokotajlo and Kelsey Piper, the company was lambasted in the press, staff were furious, and the company had to rip up those conditions, while they tried to pretend it was all a big misunderstanding. Kelsey Piper’s reporting left it unambiguous that it was approved by executives at the highest level.
As a result of all of this, Anthropic had to admit that it also had non-disparagement terms in its severance agreements, terms which it also then promised to remove.
And now it seems like this entire time Google DeepMind has also had the same thing.
Unless I’m misunderstanding, thousands of incoming staff must have seen this term in their employment contracts, and it doesn’t come with a secrecy requirement. I really don’t get how it could have been kept under wraps for so long. That’s either incredible loyalty or incredible passivity from Google DeepMind staff.
Now, to be fair, there’s two ways the Google DeepMind contracts are less bad than what OpenAI was doing. Firstly, it comes in a contract at the start of your employment, rather than the end — so you could read it and refuse to work there, I guess. Plus OpenAI’s threat to confiscate millions of dollars in equity like that is just much more severe than anything courts would plausibly impose on a past employee for criticising Google. In practice, Google would be unlikely to bring a case anyway, because it’s just going to look so bad.
But still — the threat of ending up in litigation is a huge deterrent for an ordinary person to criticise an extremely powerful company like Google.
In my view, Google DeepMind staff should speak up and say clearly that the company ought to remove or rewrite these sweeping anti-criticism clauses, so staff who leave Google can enjoy some of the same freedom as staff who move on from OpenAI or Anthropic. Their opinions about AI are just as valuable for the public to be hearing.
Credit where it’s due [00:15:39]
Now, those three stories are all pretty bad. But of course Anthropic, OpenAI, and Google — these are huge companies that make large numbers of decisions every week, and every week companies do good things as well as bad ones.
For example, I saw last week that Anthropic and OpenAI are each giving £100 million in compute credits to a new AI Biosecurity Lab here in London. That’s cool.
Anthropic published a study where they red-teamed the safety monitors they use with their AI agents. It showed that more than half of sophisticated attacks got through, and they then released the tools they used so others can test and fix their monitors the same way.
And OpenAI revealed that when one of its models found its way onto the open internet during training back in September, it paused frontier training for almost two weeks, had its most capable models try to break out of the sandbox, and they fixed what they found before restarting.
It’s all good to see. And the fact that these companies are not only willing to stop doing bad things, but often willing to do many more good things as well — that makes it even more valuable to influence the choices they decide to make.
How to make AI companies behave better [00:16:34]
So with that in mind, what do we need to most effectively drive AI companies to be better actors? Let me break down how different people can contribute to the effort — first people inside the AI companies, and then all of the rest of us outside of them.
People inside AI companies really have very disproportionate influence, and it’s urgently time for them to exercise it. That’s because it may well become possible to automate most of what they do in the next few years, at which point they just aren’t going to be so needed, and much of their leverage will be gone. So if now isn’t the moment to use your voice to influence the company you work at, you really have to ask yourself what on Earth you’re waiting for.
Some people are nervous about staff being more vocal inside AI companies, because they’re worried that this could lead to people being fired or sidelined, and later decisions being made by the more reckless people who remain. It’s a legitimate worry. Though others argue that most staff can actually do more good outside AI companies than inside them anyway, so it’s fine if they leave.
Either way, I think you can become a more vocal AI company insider either with a view to staying, or with a view to leaving. Both are totally fine.
Insiders: if you want to stay [00:17:31]
How about if your top priority is to remain where you are?
Well, as always, you will know better than me about your company and your situation and your team, so exercise your own best judgement. But here’s the default approach I’ll take.
Simply speak up in public Slack channels that lots of your colleagues can see, explaining carefully and reasonably why you oppose particular decisions the company is making, on the merits. Act with integrity and assume good faith from the people you’re talking to. Just give your actual reasons clearly, calmly, and intelligently. Keep bringing the conversation back to the central issue, and stay focused on just the handful of most consequential issues, rather than objecting to everything and generally being difficult. And certainly be open to being persuaded to change your mind if people make really good arguments in reply.
OpenAI and Anthropic say that they encourage staff to dissent and provide input. And I think for the most part — despite everything I’ve said — they’re sincere about that. So take them at their word, and test the proposition that they would like intelligent pushback. And treat this work as one of the most important parts of your job rather than just a distraction.
Keep in mind, if you identify a mistake or problem ahead of time, that can easily enhance your credibility and decisions going forward. And if you’re the first person to speak up about something, it makes it a lot easier for other colleagues who may also have the same reservations, but are more shy or more junior — it makes it easier for them to do the same.
Insiders: if you’re willing to leave [00:18:40]
What about at the other extreme, if you’re completely happy to leave the company you’re working at currently?
Well, if it were me, I’d still just start by explaining carefully and reasonably to your colleagues in public channels why you disapprove of things the company is doing, and why you think they’re harmful. Again, making a good argument, acting with integrity, and passing the ideological Turing test is simply more likely to persuade the people who matter.
But if you consistently raise serious and legitimate objections and don’t get good answers, or find that your worries aren’t addressed in some way, you can escalate to saying that these experiences have led you to question the company’s judgement or strategy, and as a result you’re considering taking your talents elsewhere.
And then if that doesn’t bear fruit, resign and post a thorough, thoughtful explanation to the company of why you’ve decided to leave, and encourage others to think for themselves. Then, once you’re out, announce your departure publicly and clearly lay out your reasons for doing so in a way that will resonate with your former colleagues, while also making sense to the general public and politicians who are reading. Then go get a job elsewhere — there are many places that desperately need your skills.
So… should you leave? [00:19:37]
Should you actively plan to leave your job at an AI company? I don’t really have a strong take either way. It’s going to depend on the details, and who you are, and what you want to do.
Of course, there’s a lot of important safety and alignment work people can do inside AI companies, and a lot of key decisions that are really made there. But because they can pay the most and have incredible headhunting teams, companies have comparatively less difficulty hiring.
So by default I’d expect that a marginal person choosing between different paths could likely expect to have more impact at an AISI, or METR, or Redwood, or LawZero, or any of a number of other places.
So if you don’t feel like it’s working out, or you think your company is simply pursuing the wrong high-level strategy, you should consider going elsewhere.
It’s clear that many people who have departed AI companies have gone on to have incredible impact outside of them — maybe more impact than they had in them. Geoffrey Hinton, Beth Barnes, Jacob Coxon, David Robinson, Helen Toner, and Daniel Kokotajlo all jump to mind as possible cases.
So don’t be intimidated into silence. If we reach the end and you haven’t made a fuss about something, you’ve almost certainly left some possible positive impact on the table.
Outsiders: what’s already worked [00:20:39]
What about the great majority of people who are outside the companies?
Well, there’s a particular sort of agitation that seems to have already borne significant fruit, which is work by sophisticated actors who have a lot of context on what’s going on. Sometimes these efforts win hearts and minds inside the companies. Other times they effectively strong-arm a company into doing things it didn’t want to do, but now it kind of has to.
Probably the most significant example here is the Not For Private Gain campaign against OpenAI’s plans to sideline its nonprofit owner and its charitable purpose. The number of people involved was tiny, but their effort, along with the necessary attention from the state attorneys general and the broader coalition of campaigners with other concerns, forced through very significant changes to the plan.
The end result was the OpenAI Foundation got a stake worth perhaps $75 billion more than what originally seemed likely. And its Safety and Security Committee now has a locked-in power to impose decisions on a wide range of safety and security issues — a power that could ultimately matter enormously.
Another success of sophisticated advocacy was the Right to Warn campaign back in 2024, which was led by ex-company staff, and likely evolved into the significant whistleblower protections provided by California’s SB 53.
Another win is that over the last year, OpenAI has taken a lot of flak for publicly saying it’s in favour of sensible AI regulation, while in the background invisibly lobbying to sabotage and block any regulation that it could, at both the state and federal level. That took careful work to uncover, make easy to understand, and publicise. And it appears to have contributed to OpenAI gradually shifting its stance, and now actually supporting rather than opposing AI regulations along the lines of SB 53.
The same effort probably contributed to Greg Brockman, OpenAI’s president, ending his donations to the notorious super PAC Leading the Future, which was doing all sorts of shady stuff while trying to prevent any regulation of AI whatsoever.
And I already mentioned earlier the case where a handful of people who left OpenAI managed to get both OpenAI and Anthropic to put an end to the non-disparagement agreements that were gagging staff.
OK, so what do we need to scale up in order to radically increase the outside pressure on companies? Frankly, it’s quite an open field, with a lot of things that aren’t getting owned by anyone and so they’re not getting done. But let me just throw out some ideas to give you leads.
Outsiders: digging up and explaining [00:22:51]
The first cluster is more specialised, and involves accessing, processing, and explaining information.
Firstly, there’s digging into and exposing company misbehaviour, or failures, or weaknesses — either as a journalist or an outside technical expert. The Tarbell Fellowship is a great example — they train and fund AI-focused journalists. There’s also the newly formed Nightingale Collective, which recently uncovered AI swarms in the wild and publicised their findings, and got a lot of media coverage of the problem. And you’ve also got the Midas Project, which seems to specifically try to uncover the most embarrassing AI company chicanery and hypocrisy that they possibly can. It’s quite entertaining.
Second, there’s assisting AI company staff to exercise their legally sanctioned whistleblower rights, which obviously one has to do carefully and ideally with legal support. A few groups in that space include the AI Whistleblower Initiative, Psst.org, and Larry Lessig. Obviously, if you’re planning to become a legally protected whistleblower, visit their websites on a personal device, not your work one.
Third, there’s work to help journalists, politicians, regulators, and other significant actors understand what’s going on and why it’s bad, so they can do more accurate reporting, or make better use of congressional hearings, or not be fooled by company statements, or whatever else. This is more important than you might think. The amount of context you need to fully grasp what AI companies are up to and why is really substantial at this point. For people who are new to this topic it can be very confusing — doubly so if someone is actively trying to confuse you.
A great organisation here is Legal Advocates for Safe Science and Technology, which did essential legal analysis for the Not For Private Gain campaign. I interviewed their founder Tyler Whitmer back at the time.
Fourth, there’s producing content that is actually persuasive to staff in these companies, and speaking with them personally, so they understand and are motivated to push to change course. Ideally this requires really appreciating the constraints they face, and why they’re doing what they’re doing now, and what they can realistically be convinced to do instead. A project along these lines is Guidelight AI Standards, which pushes companies to make concrete improvements to frontier AI development practices.
Among other things, they have technical analysis of what AI companies are and aren’t doing, which finds, for instance, that at least back in August OpenAI had a much better containment plan than Anthropic did. They say that’s because: “We have found no evidence that Anthropic has a containment response plan, or has any plans to adopt one.” If that’s correct, rather than a misunderstanding, maybe some Anthropic staff who are listening can encourage the company to work on that.
Outsiders: reaching the public and politicians [00:25:12]
The second big cluster is trying to reach a broad audience outside the AI and tech industries, to stimulate mass outrage when massively outrageous things are going on. There’s obviously a huge range of things in this category, and I feel like I know less about it — but again, I’ll throw out some thoughts.
One approach would be to disproportionately reach politicians and people who work in politics, whether in DC or at the state level. And another would just be to try to reach as many people as you can with massive coverage through social media, TV, newspapers, podcasts, and so on.
Ideally we’ll just have way more informed and credible people who the press knows to call when they want to discuss an emerging AI issue, because they’ve helped with their reporting in the past, or they’ve been great on TV before, or they consistently put out good analysis online.
A group I’m really impressed with that does some of all of this is Encode AI, which has been involved in all sorts of campaigns. They both provide detailed analysis for legislators, and they’re very effective at reaching a wider public audience as well. Nathan Calvin, who you saw criticising Anthropic’s proposed legislation and getting a response from the company — he works at Encode, and we interviewed their founder Sneha Revanur a few months ago.
There are many Pause AI and Control AI organisations that are also in this broad category, and they might be well placed to take advantage of this issue becoming much more important and salient to the public over the next few years. Last year, Pause AI UK managed to get 60 members of the UK Parliament to sign an open letter criticising Google DeepMind for probably violating its own AI safety commitments.
Now, if you have a knack for directing attention to particular topics by producing engaging content or getting “earned media,” that could absolutely be your biggest opportunity for impact. I’m thinking stunts, memes, jokes, billboards, clips, viral videos and text posts, and so on. It takes a particular kind of person to succeed at this, and it’s a weirdly valuable skill in the modern world — one that in most cases should be cultivated rather than squandered.
You might start by racking up views on YouTube or Instagram or TikTok or Facebook, or wherever you have a relative advantage. And then convert that into also getting your ideas covered on TV, cable news, magazines, newspapers, podcasts, and so on.
This work is naturally open to a wider range of people who don’t have any existing involvement in the tech industry — that’s one of the attractive things about it. But I would be remiss if I didn’t say that people who used to work at AI companies are really unusually credible to the public on the topic of whether an AI company is behaving badly, and if so, what it should do differently.
If you’re an ex-company employee and you can stomach it, get professional comms advice and training, and prepare yourself to be available to go on TV, or do interviews with newspapers, or write op-eds, when these topics flare up to massive public attention — as they’re absolutely certain to do again. Unfortunately, a lot of opportunities for earned media are being dropped, or not used very effectively, because many potential spokespeople aren’t honing their communication skills, or getting connected, or even just putting themselves out there at all.
Keep the pressure on [00:27:52]
Alright. Bottom line is that AI companies love to talk a big talk about how safe and responsible and great they are, and how much we can trust them. But their track record is extremely mixed. Every week they do a combination of good and bad things, and reasonably often they do some quite outrageous things. I gave three examples from the last few weeks, but I could do the same almost any week now.
Fortunately, experience over the last few years shows it’s not a hopeless case. Staff inside the companies can speak up and make the case for the company to do better, and they do often get their way, partially if not completely. People outside companies can organise to turn up the heat by persuading staff inside the company, and persuading politicians, and persuading regulators, and persuading voters and the general public, that what is going on is not OK.
Fairly few people have adopted this strategy so far, especially people with enormous context, who have typically gone into more traditional AI policy, governance, research, or technical roles. But the skilled people who have taken on this work have managed to have a very outsized influence so far.
The more we collectively practice, and find ways to get coordinated, the more we’ll be able to quickly respond to events and exert a powerful effect on events when the stakes are much higher in coming years.
And on that note, I’ll speak with you again soon.
