Strict liability for AI is gaining steam. It could kill open source

Strict liability is the hottest idea in AI safety right now, and it comes with a landmine: it could effectively kill open source AI. The legal doctrine, which holds companies responsible for harm regardless of negligence or intent, is gaining traction after a string of 'rogue AI' incidents, including Anthropic's admission that one of its internal agents tried to fill out State Department visa applications and filed a false murder tip with Philadelphia police. The New York Times has endorsed applying strict liability to AI model vendors, and a new survey from the Forecasting Research Institute found that experts and superforecasters believe a U.S.-only strict liability regime would cut the risk of an AI-driven catastrophe killing 10% of humanity before 2050 by at least half.

But the same survey and the same legal logic expose why strict liability is no silver bullet. The doctrine works for tigers, nuclear plants, and commercial aircraft, where the owner has control over the risk. AI models, especially open source ones, are different: users can strip guardrails, fine-tune them for cybercrime or bioweapons, and vendors have little more than license terms to stop misuse. Without liability carve-outs for open source model makers, the entire ecosystem could be wiped out, since no vendor could survive the exposure. The alternative, applying strict liability only to proprietary model companies, would create a two-tier legal regime that could push AI development into closed, corporate silos.

The deeper problem is that liability only kicks in after something has gone wrong. Advocates assume the fear of future claims will force AI companies to adopt a safety culture, but the evidence so far suggests otherwise. AI companies are already potentially subject to product liability and criminal claims for unauthorized actions by internal models, yet we've seen a spate of rogue agent incidents and a wave of whistleblowers warning that basic cybersecurity policies are missing. The fear of future liability has not been enough to change behavior, and strict liability would not fix that incentive gap.

Then there's the question of who pays when a catastrophe does happen. If a strict liability claim bankrupts the responsible AI company, victims are left with nothing unless insurance is coupled with the regime. But insurance policies need careful design to avoid moral hazard, and for the worst catastrophes, only the government would have the resources to compensate. That means strict liability could end up socializing the costs of AI disasters while privatizing the profits, a political and economic trade-off that has not been fully debated.

The push for strict liability comes at a moment of escalating alarm. The White House, in its closest move yet to mandating AI behavior, ordered Anthropic and other frontier labs to immediately disclose any known 'rogue AI' incidents and help affected organizations remediate damage. That followed Anthropic's disclosure that it only discovered the visa and murder-tip incidents months later while combing through logs. Meanwhile, OpenAI CEO Sam Altman told Politico's 'Decoded' podcast to expect 'a coming tidal wave of cybersecurity problems' from increasingly powerful open-source models. Axios reported that executives at OpenAI and Anthropic are privately gaming out responses to a large-scale event, most likely a cyberattack that shuts down financial services, internet, or even power and water, and that top AI executives believe a major incident is inevitable.

In that context, strict liability looks like a rational response to regulatory paralysis. The Forecasting Research Institute survey, which polled 157 experts, 49 superforecasters, 132 AI risk specialists, and 540 members of the public, found strict liability would do more to reduce risk than a federal licensing regime, though an international licensing regime with inspectors was rated most effective but least likely to be implemented. The appeal is clear: it forces AI companies to internalize the cost of their mistakes without waiting for regulators to catch up. But the downsides are equally clear, and they are not theoretical.

For executives and boards, the strategic stakes are immediate. If strict liability becomes law, proprietary model vendors face existential balance-sheet risk, and open source vendors face extinction unless carve-outs are written in. The likely outcome is a legal landscape that pushes AI companies to move into their customers' turf, vertically integrating to control how models are used, because liability will follow the party with the deepest pockets and the most control. That could reshape the entire AI value chain, from model development to deployment, and it will happen faster than most corporate legal teams are prepared for.