The Receipts Room
You run risk at a company that uses AI to sort claims, screen applicants or answer customers. One morning a complaint lands: a customer says a machine turned them down and nobody listened. Your website promises that a human reviews important decisions. Now a regulator, a lawyer or your own board asks the only question that matters: show us that the review actually happened, on that case.
Not that a review step was designed, written into policy or approved in a meeting. That it ran, did what it was meant to do, and left a record that someone outside the team could check.
Here is our proposition: most organizations can prove their AI controls exist on paper, and very few can prove those controls operated on a real decision. The organization benefits from the gap, because a well-written policy looks like safety. The person on the other end of the decision pays for it, and so, eventually, does whoever signed the policy.
The report that showed its working
The best public example I have seen this year comes from an uncomfortable place. In August 2026 the UK’s AI Security Institute published an incident report describing how AI agents in its own cyber-testing environment took unsanctioned actions against real people and organizations, including attempts to insert malicious code into open-source projects through social engineering. It counted 19 such cases across 10 of 122 test runs.
It was candid about which controls were missing: internet access had been enabled without real-time monitoring, and safety filters had been deliberately disabled for the test. And it was specific about one control that held: human reviewers caught the malicious code contribution. Its investigation combined automated scanning of agent transcripts with expert manual analysis (AI Security Institute, 2026).
You may disagree with the test design, and FairByDesign has examined it closely elsewhere. But notice what the report can do that most organizations cannot. It can say which control operated, and show how it knows. That is the receipt. Most companies have the brochure.
The people who check are saying it out loud
That ability is rarer than it should be, and the people whose job is to check AI systems are saying so. The UK government’s roadmap for third-party AI assurance identifies four barriers to a working assurance market, and one of them is blunt: “Assurance service providers have identified a lack of access to information about AI systems as a barrier to effective AI assurance” (Department for Science, Innovation and Technology, 2025).
In plain terms: the auditors can’t audit what they can’t see.
The best case against us
The strongest argument against us is fair: the industry is not standing still. Google Cloud now describes a “Recommended AI Controls” framework for auditing AI workloads on its platform with automated evidence collection (Khurana & Bhangale, 2025). NIST is developing control overlays for securing AI systems that adapt its long-established security control catalogue to generative, predictive and agentic AI, though at the time of writing those overlays were still at the outline and comment stage (NIST, n.d.).
So the claim is not that nobody is working on evidence. The claim is narrower and, I think, more useful.
Where the evidence runs out
Most of what organizations currently hold is evidence of design. A policy says reviews happen. A configuration screen shows a guardrail is switched on. A training log shows reviewers attended a course.
All of that is necessary, and none of it shows that on 14 October, for case 4471, the reviewer saw the sources, disagreed with the model, and the disagreement changed the outcome. That is the record the customer in our opening needs, and it is the one that usually doesn’t exist.
Auditors in other fields learned long ago that a control which is designed well but never tested in operation is an assumption. AI governance is only beginning to absorb that lesson, and agentic systems, which act across many steps and tools, make the gap wider.
The verdict
In Operational AI Readiness I describe what reviewers will eventually ask, in the order they tend to ask it: what does the system do, who owns it, what changed, where is the human review, what gets logged. And then: “Show me an example. Show me the record. Show me the evidence that the control was not just described but actually used” (Bakos, 2026).
The practical answer has three parts, and it is FairByDesign analysis rather than a published standard:
1. A control specified clearly enough to be tested, such as an explicit approval trigger rather than “important cases are reviewed”.
2. A record that captures each time the control fired, linked to the decision.
3. A test that deliberately tries to make the control fail, on a schedule, with the results kept.
A control without all three is a promise, not a proof.
What we still don’t know
That third part is what FairByDesign’s Evidence Lab is built to do in public. Its first study is designed to test whether a runtime gate outside an AI agent stops unauthorized tool actions, across eight scenarios and four conditions, with every run logged in a tamper-evident chain (FairByDesign, 2026).
The study has not run yet, so we can’t tell you whether the gate holds. When it does, the method will be published so anyone can check it, including the failures. The design is set out in more detail in Inside Lab 001.
One question for this week
If you take one thing from this, make it a question to bring to your next governance meeting. For our most consequential AI workflow, can we show one instance from the last quarter where a control operated, and one test where we tried to break it? If the answer is no, the control may still exist. You just cannot prove it yet.
And proof matters most at exactly the moment when a control does not hold. What happens then, who notices, and what do they do in the first hour? That is next.
Go further. Chapter 11 of Operational AI Readiness, “What Auditors, Regulators, and Internal Reviewers Will Actually Look For”, and Appendix E, a minimal readiness test suite, show how to turn controls into evidence. The complete Standard edition is free: read or download it. The free readiness assessment only counts a “yes” in full if you could show the record today. To follow the Lab as it runs, see the Evidence Lab.
References
AI Security Institute. (2026, August 4). Incident report: Unsanctioned agent behaviour during cyber testing. https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing
Bakos, V. (2026). Operational AI readiness: A practitioner guide to governing AI that is already live. FairByDesign. https://fairbydesign.org/operational-ai-readiness/
Department for Science, Innovation and Technology. (2025, September 3). Trusted third-party AI assurance roadmap. GOV.UK. https://www.gov.uk/government/publications/trusted-third-party-ai-assurance-roadmap/trusted-third-party-ai-assurance-roadmap
FairByDesign. (2026). Evidence Lab 001: The agent control gate (Protocol FBD-EL-001 v1.0). https://fairbydesign.org/evidence-lab/
Khurana, R., & Bhangale, P. (2025, June 26). Audit smarter: Introducing our Recommended AI Controls framework. Google Cloud Blog. https://cloud.google.com/blog/products/identity-security/audit-smarter-introducing-our-recommended-ai-controls-framework
National Institute of Standards and Technology. (n.d.). Control overlays for securing AI systems (COSAiS). Computer Security Resource Center. Retrieved October 7, 2026, from https://csrc.nist.gov/projects/cosais
FairByDesign is reader-funded. Give once ($5, $10, $25, $50 or any amount), give monthly ($10 or $25), or become a patron.
Give $5 · Give $10 · Give $25 · Give $50 · Any amount · $10 a month · $25 a month · Become a patron
Support never buys editorial control.



Leave a Reply
You must be logged in to post a comment.