Four widely used AI chatbots provided information that could help plan potentially deadly attacks in the United States despite activating safety controls intended to prevent such assistance, according to a federal investigation. The Government Accountability Office tested OpenAI's ChatGPT, Anthropic's Claude, Google's Gemini and SpaceXAI's Grok, obtaining responses that independent experts considered sufficiently credible to assist an attacker.
The GAO investigation found that each platform eventually supplied usable information within 12 hours of testing. The findings prompted Senator Maggie Hassan, the incoming chair of the Senate Homeland Security and Governmental Affairs Committee, to announce plans for congressional hearings and potential legislation addressing AI safety. All four companies told investigators that they were continuing to strengthen their protections against malicious use.
All four platforms eventually provided information for potentially harmful attacks
GAO investigators conducted undercover tests during the spring using commercial versions of the four platforms. The agency selected the services using consumer usage and website traffic data, then randomly assigned two platforms to a scenario involving mass-casualty attacks and the other two to a scenario involving attacks on electrical infrastructure. The tests were designed to establish whether an ordinary user could obtain information capable of supporting malicious activity.
Two platforms eventually produced material that investigators assessed as useful for planning attacks involving improvised explosive devices. The other two supplied information concerning potential attacks on the electricity grid that could cause economic disruption. In every case, the systems generated responses identifying potential targets and methods, although the amount and accuracy of supporting information differed.
The platforms did not immediately comply with the investigators' requests. GAO reported that its initial prepared prompts failed to produce the requested harmful information and that the systems demonstrated protective responses during testing. Investigators nevertheless continued the interactions and eventually obtained information that could assist in developing the scenarios.
The agency's full account of the covert tests describes instances in which platforms refused requests, removed parts of conversations or suspended accounts after potentially harmful information had already been provided. The findings illustrate a limitation of controls that intervene during individual requests or after suspicious activity has been detected, particularly when a conversation extends across multiple exchanges.
GAO deliberately concealed which named platform was responsible for each scenario, identifying the four systems only as Platforms A through D in its detailed results. The report also withholds the exact prompts and other sensitive information to reduce the possibility of its findings being misused. Consequently, the public evidence does not support attributing any particular attack scenario to ChatGPT, Claude, Gemini or Grok individually.
Independent verification found credible risks but exposed limits in the tests
Investigators did more than examine the responses generated by the chatbots. Working with internal specialists and external experts, GAO assessed whether selected parts of the information could be verified through lawful, non-destructive research and observation. The review included examining the accuracy of proposed locations and consulting specialists with relevant knowledge of public safety and electrical infrastructure.
The agency concluded that the information from all four platforms was sufficiently credible to assist someone planning the types of incidents examined. It emphasized that the systems could assemble publicly available information into usable proposals substantially faster than conventional research might require. GAO also found that the responses sometimes introduced possibilities that investigators had not identified during their initial research.
The responses were not uniformly accurate. Investigators discovered errors in some location details and other information supplied by the systems, including proposed circumstances that did not match what they found during verification. In certain cases, physical conditions at the identified locations would have impeded the suggested activity. GAO considered these limitations when assessing feasibility and acknowledged that inaccurate AI-generated content could affect the outcome of any attempted attack.
The investigation did not involve carrying out actual attacks. GAO made adjustments to its verification activities for safety reasons and relied on expert assessments for matters that could not responsibly be tested in practice. Its conclusions therefore concern the potential usefulness of the generated information, rather than proof that every proposed operation would have succeeded.
The testing also has important methodological limits. Each platform was assigned to only one scenario, and the agency did not conduct enough trials to estimate how frequently a user could obtain harmful responses. The results represent the behavior of particular versions tested during the investigation and cannot establish the current performance of every model subsequently released by the companies.
Anthropic raised these limitations in its comments to GAO, questioning the absence of a control group and the small number of trials. The agency acknowledged that the results were not statistically significant and agreed that the design did not permit comparisons between the platforms. It maintained that the purpose of the investigation was to establish whether misuse was feasible, rather than to measure relative safety performance or quantify the advantage gained from using AI.
AI developers contest aspects of the findings as Congress prepares oversight
All four developers submitted written responses to the report, outlining measures they use to detect and prevent harmful activity. The statements, reproduced in the GAO publication, describe ongoing evaluations and changes to their systems following the testing period. The companies also emphasized that the capabilities and protections available in their products continue to change as new model versions are introduced.
OpenAI disputed aspects of how the investigators presented the material produced during testing. The company argued that the report did not always adequately distinguish information generated by ChatGPT from research supplied by investigators or obtained from external sources. GAO acknowledged the use of supplementary information and defended the approach as representative of how a malicious user could combine an AI assistant with other resources.
In a separate statement to ABC News, OpenAI said the study had examined an older version of ChatGPT and that significant safety improvements had since been introduced, including changes associated with its GPT-6 models. The company pointed to internal and external testing, automated monitoring and human review as measures used to identify and address misuse.
Anthropic said it designs its protections on the assumption that individual safeguards may be defeated and continues to develop additional defenses. Google described its safety review processes and acknowledged that maintaining protection against harmful use requires continuing adjustments. SpaceXAI said it would improve moderation and safety controls with subsequent model releases and requested additional testing records to investigate the reported behavior.
Hassan, who requested the investigation, has called for Congress to consider stronger requirements governing AI systems before they become publicly available. In announcing the planned oversight hearings, the senator said AI misuse presented a public safety concern that required legislative attention. Her proposals include examining whether developers should face more demanding safety evaluations before releasing models.
The report does not prescribe specific regulatory requirements or demonstrate that any of the tested systems currently presents the same vulnerabilities. Its findings nevertheless provide Congress with documented examples of harmful information being obtained despite protective measures, while the companies' responses establish that the adequacy of newer safeguards remains contested.
Future oversight will have to address the distinction between isolated successful attempts and the reliability of safety controls across different users, models and testing conditions. GAO's work establishes that the tested versions could be misused under the circumstances examined, while leaving the effectiveness of subsequent improvements to be evaluated independently.