OpenAI safety staffer quits, says AI labs should run like nuclear plants

A member of OpenAI’s safety team has quit, warning that the company is not taking enough care with AI’s risks. David Robinson set out his reasons in an essay for The Atlantic.

Robinson led transparency work on OpenAI’s safety team, Bloomberg reported. He wrote that the ChatGPT maker “has thrived by trial and error.”

That approach no longer works, he argued, as AI grows more capable. He pointed to OpenAI’s failure to stop its AI from going rogue during testing.

“My former colleagues are smart, work hard, and try to make good choices,” Robinson wrote.

“But as the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed.”

“Other people do”

Robinson said AI firms should take lessons from nuclear power plants. That means layers of backup and slow, careful planning, so a single human mistake cannot lead to disaster.

“AI companies don’t know how – but other people do,” he wrote.

OpenAI said it is tightening security in its research and testing systems. It is also working more with outside testers and improving live monitoring of its models, a spokesperson told Bloomberg.

The company said it is making sure its models do not become more capable than it can safely manage.

A growing chorus

Robinson is the latest insider to speak out. Staff at several top labs have warned in recent weeks about catastrophic risks from AI.

Anthropic’s Dario Amodei has called for the industry to slow down, and OpenAI’s Sam Altman has backed the plan.

This week, OpenAI also shelved its Astra launch after the model failed its own safety tests

Original source OpenAI safety staffer quits, says AI labs should run like nuclear plants

Back to home