The hardest part of AI safety may not be making models behave. It may be building institutions that can hold their makers accountable.
That is the warning running through The Verge’s interview with Helen Toner, the CSET executive director and former OpenAI board member who voted to remove Sam Altman in 2023. Toner’s argument is blunt: a safety plan that depends on one CEO being trustworthy is not a safety plan.
The OpenAI crisis exposed the weak point
OpenAI’s nonprofit board was created to protect the company’s mission from the pressure to prioritize growth and profit. When the board tried to use that authority, the mechanism broke. Altman returned as CEO, Toner later resigned, and OpenAI’s board was replaced as the company continued restructuring.
That history matters because the same basic assumption keeps appearing in AI policy: that leaders at frontier labs will make the right call when commercial incentives collide with safety commitments. Toner’s experience suggests that personal judgment is a fragile foundation for systems expected to manage increasingly powerful technology.
Her concern is not limited to OpenAI. In the interview, she points to OpenAI and Anthropic, as well as other major technology companies, as organizations increasingly focused on using AI to accelerate their own research and development. She describes recursive self-improvement as the idea that AI systems could progressively take over more parts of the process of building better AI, eventually moving toward superintelligence.
The next governance problem is institutional
Toner also pushes back on the idea that policymakers simply do not understand AI. CSET has around 50 people working on rigorous, actionable research for Congress, executive agencies, and other officials. She says there is a real appetite in Washington for people who can explain both the technology and its security implications.
The more difficult question, then, is whether institutions can act on that knowledge. Policymakers may be learning what language models, agents, and frontier systems can do, while companies are already planning around capabilities that are harder to evaluate from the outside. That gap cannot be closed by asking the public to trust a handful of executives to self-regulate.
For brands and businesses building on AI, the practical implication is straightforward. Vendor reputation is not the same as governance. The systems they adopt need clear accountability, independent oversight, and an answer for what happens when growth incentives point in the opposite direction from safety.
AI companies are asking to be treated as stewards of a technology with enormous consequences. Toner’s warning is that stewardship cannot be a personality test. It has to be an institutional property.