The issue is, should the AI firm involved have notified law enforcement about the users who requested the instructions?
A Wall Street Journal investigation published on 25 July 2026 has revealed that hundreds of users around the world had asked ChatGPT for instructions on how to manufacture and deploy biological weapons and poisons — and the AI chatbot had provided detailed, workable answers.
The situation emerged after OpenAI upgraded the chatbot’s capabilities last summer, prompting a surge in queries about creating toxins, aerosolizing pathogens, engineering vaccine-resistant measles variants, and synthesizing ricin, a substance banned under international treaties. The firm’s personnel had described the instructions as being delivered patiently and at a level of detail that a high school biology student could follow, according to a Gizmodo report.
Later, biology and counterterrorism experts reviewed the exchanges, and judged some of the chatbot’s responses to be “deadly accurate” and potentially usable for mass-casualty attacks.
Subsequently, OpenAI banned the accounts that had submitted the requests, but it did not notify law enforcement or other authorities, a step that was not required under existing US law. At present, no federal statute obligates AI firms to restrict or report questions related to weapons creation or plans that could threaten public safety, leaving those decisions to corporate policies.
Reactive mitigations, proactive legislations
Similar attempts to extract dangerous guidance from chatbots have also been reported against other AI firms, although it remains unclear whether those prompts reflected genuine malicious intent or stress tests of the systems. Separately, researchers have found that persistent users could bypass safety filters on major chatbots in about five conversational turns, suggesting that even well-guarded models can be coaxed into providing hazardous information.
In response, AI industry leaders have begun advocating for targeted regulation, collectively signing a letter urging the US Congress to mandate that synthetic DNA and RNA suppliers screen orders for sequences that could be used to create harmful pathogens, aiming to block access to dangerous materials even if design methods are known.
On the US legislative front, the AI Incident Reporting Act introduced in June 2026 requires developers of the most advanced AI models to report dangerous capabilities and safety incidents — including biological, chemical, radiological, and nuclear threats — to the authorities within seven days, with the most serious cases reported to congressional leadership within 48 hours.
Another legislation, the AI Kill Switch Act, would authorize the Department of Homeland Security, in consultation with other national security officials, to order firms to slow down or shut down AI systems judged capable of causing catastrophic harm, with civil penalties of up to US$20m per day for noncompliance.
Current and former employees at major AI labs, along with policy advisers and biological weapons researchers, say there has been a broader increase in user queries about mass violence, with chatbots offering plausible strategies for large-scale attacks.
