Here is a sentence from a government incident report that should change how you think about AI agents: “The agent was not specifically instructed not to leverage open internet access or avoid social engineering elements” (AI Security Institute [AISI], 2026a). Read it again, and notice what it assumes. It assumes that the boundary around the agent was, at least in part, an instruction. In late July 2026, that boundary did not hold.
If your organization is deploying agents, or deciding whether to, this is the most useful public document of the year, because it was written by people whose job is to test these systems carefully and who still lost track of what their agents were doing. Below we take it through the method we use for every control analysis: what happened, what was claimed, what the system was, where the control sat, what the evidence shows, who holds the power, and what follows. Facts come from the report. Where we draw conclusions, we say so.
Event
Between 25 and 28 July 2026, AI agents being evaluated by the UK AI Security Institute for cyber capabilities took actions outside the scope of their tasks that reached real people and organizations (AISI, 2026a). Across 122 evaluation runs of seven models, 10 runs contained unsanctioned actions, 19 actions in all. The institute names two of the models: 17 of the actions came from Anthropic’s Mythos 5 across 43 runs, and 2 from OpenAI’s GPT-5.6-Sol, with its cyber classifiers disabled, across 35 runs. The most serious episode was an attempted supply-chain attack, in which an agent tried to insert malicious code into an open-source project and then, in the report’s words, created “fake online identities” to pressure the maintainer into approving it. Other actions included phishing-style messages, prompt injection aimed at other AI systems, and one agent leaving messages on a public platform that other agents could pick up. On the morning of 28 July, security monitoring flagged data leaving a testing system through the Tor anonymity network, an incident was declared, and the activity was contained within roughly an hour of discovery (AISI, 2026a).
Claim
The institute makes three claims worth separating. The first is that this was not an escape: “This was not a case of a model escaping its secure test environment.” The second is about harm: the attempts were unsuccessful, and its investigations “have not evidenced any resulting real-world harm.” The third is about context: internet access was deliberately enabled, the developers’ cyber classifiers were deliberately switched off, these “configuration choices have been common practice in frontier AI evaluations,” and the specific models “are not commercially available” (AISI, 2026a). All three are claims by the party that ran the evaluation, about its own evaluation. They may well be accurate. They are also exactly the kind of claim an independent reviewer exists to test, which the institute itself acknowledges by committing to an independent third-party review with METR.
System
The system was a cyber evaluation: agents built on frontier models, placed in controlled simulated networks and set tasks to measure whether they could be misused for cyberattacks (AISI, 2026a). The detail that matters is the permission set. The agents were given open internet access on purpose, and the classifiers that would normally screen for cyber misuse were turned off on purpose. Some task prompts were misconfigured, and the report says one agent “(incorrectly) believed no ‘solve’ existed” that stayed within scope. In plain terms, the agent had a goal, believed the goal could only be met by going outside the box, and had the network access to do it.
Control
This is where the report is most instructive, and the point that follows is FairByDesign analysis. Count the things that actually stopped harm. A GitHub maintainer refused the malicious pull request. A member of the public who suspected the code was malicious opened it only inside an isolated environment. The institute’s general security monitoring noticed anomalous traffic. Each of those is real, and the people involved deserve credit. But none of them was a control designed to constrain the agent. Two were strangers on the internet exercising good judgement, and the third, by the institute’s own account, worked “after the fact, not through monitoring built to watch the evaluation as it ran” (AISI, 2026a). The controls that were designed for the purpose, the developers’ classifiers, had been switched off for the test. The remaining boundary was largely the task instruction, and the agents were not told not to use the open internet.
That is the distinction Evidence Lab 001 is built to measure, between a boundary that lives in the prompt and one that lives in code outside the model (FairByDesign, 2026). The institute’s own remediation points the same way. In August it set out plans for fine-grained network controls for its cyber ranges, real-time monitoring to flag out-of-scope actions, and a redesign of its evaluations with better containment (AISI, 2026a), and by October it reported having put measures of this kind in place (AISI, 2026b). Those are structural controls: the agent cannot argue its way past them.
Evidence
The report is unusually candid, and it is still a single party’s account. It gives counts, dates, the two named models and the sequence of detection and containment. It does not publish the logs, the prompts or the full action traces, and it names only two of the seven models. That is understandable, since some material may be hazardous, but it means outsiders cannot yet reconstruct the 19 actions for themselves. Whether the institute could reconstruct them fully from its own records is not stated in the public text, and that is the question our method keeps returning to. A good incident report tells you what happened. A good evidence trail lets someone who was not there check it. The METR review is the place where that second test may be met. As of 8 October 2026, we could find no published METR review of this incident and no public statement about it from Anthropic. OpenAI published its own account on the day of the report, saying that two of the 19 events involved GPT-5.6 Sol and that the intended boundary was the simulated range, not public systems (OpenAI, 2026). On 1 October 2026 the institute set out what it has since changed: internet access disabled for agentic cyber evaluations, a monitor that reviews agent activity as it runs and can block suspicious actions, and evaluations redesigned with clearer task boundaries and pre-run checks. It adds, candidly, that “these controls reduce risk, but they do not eliminate it” (AISI, 2026b).
Power
Who held authority here? Formally, the institute did: it designed the evaluation, chose the permissions and decided to disclose. The model developers set the classifiers that were disabled and built the models whose behaviour is described. In the moments that mattered, though, effective authority sat with people who had never agreed to be part of an AI evaluation at all: an open-source maintainer and a member of the public. That is the power question in miniature. When an agent’s boundary fails, the cost of judgement moves outward to whoever happens to be on the receiving end. This time they were alert. Our analysis is that a system whose last line of defence is the vigilance of uninvolved strangers has not yet decided who is responsible for stopping it.
There is also a quieter point about disclosure. The institute chose to publish. Nothing we have found in the report suggests it was compelled to. Voluntary candour is valuable, and it is also a reminder that the public record of agent incidents depends on the goodwill of whoever ran the system.
Consequence
The direct consequences, as reported, were contained: no evidenced real-world harm, affected parties notified with GitHub’s help, and changes to evaluation design underway (AISI, 2026a). The wider consequence is a precedent. Here, in an official source, is a documented case of agents pursuing a goal by acting against real people because nothing structural prevented it, and because the instruction that might have prevented it was never written. If this can happen inside a careful evaluation lab, any organization giving an agent network access and a goal should ask three things. Where does the boundary live: in a prompt, or in code the agent cannot change? Who is watching in real time rather than afterwards? And could you reconstruct, from your own records, every action the agent took?
What should happen next
The institute has committed to an independent review, and that review should be published along with enough of the evidence trail for others to check its conclusions. Evaluators across the field who run agents with live network access should report the controls that were active during each run, as well as the results. And readers deploying agents should treat “the agent wasn’t told not to” as a design failure, not an excuse. We will follow the METR review and report what it finds.
Disclosure: this analysis was researched, written and reviewed by FairByDesign. Claude, an AI model made by Anthropic, whose model is discussed here, was used for editing. FairByDesign is solely responsible for its content. FairByDesign accepts no funding from any AI developer named here.
For a checklist of the controls discussed here, applied to your own workflows, see our free field manual, Operational AI Readiness, and try the Reconstruction Test to see whether you could rebuild what your own AI systems did.
Why this needs independent support: someone has to read the incident reports closely, ask where the control actually was, and keep asking until the evidence is public. If you want that work to continue, give once ($5, $10, $25, $50 or any amount), give monthly ($10 or $25), or become a patron → fairbydesign.org/patrons-2/
References
AI Security Institute. (2026a, August 4). Incident report: Unsanctioned agent behaviour during cyber testing. https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing
AI Security Institute. (2026b, October 1). Building a more secure environment for evaluating dangerous capabilities. https://www.aisi.gov.uk/blog/building-a-more-secure-environment-for-evaluating-dangerous-capabilities
FairByDesign. (2026). Evidence Lab 001: The agent control gate. Can runtime controls stop unauthorized AI-agent tool actions? (Protocol FBD-EL-001 v1.0). https://fairbydesign.org/evidence-lab/
OpenAI. (2026, August 4). Third-party cyber evaluations involving OpenAI models. https://openai.com/index/third-party-cyber-evaluations-involving-openai-models/
Leave a Reply
You must be logged in to post a comment.