From the Field Manual. Each week we take one chapter of our free book Operational AI Readiness and turn it into something you can use. This week: Chapter 4, the smallest set of controls that keeps a live AI system governable.
In October 2023, New York City announced a chatbot to help people starting and running businesses. Five months later, reporters at The Markup tested it. Asked whether an employer could take a share of workers’ tips, the bot answered, “Yes, you can take a cut of your worker’s tips.” Asked whether landlords must accept tenants with housing vouchers, it told ten separate Markup staffers no, although source-of-income discrimination is illegal in the city. It also said restaurants could refuse cash, despite a 2020 city law requiring them to accept it (Lecher, 2024).
The bot was built on Microsoft’s Azure AI services and carried a disclaimer that it “may occasionally produce incorrect, harmful or biased content.” A city spokesperson said it had already given “thousands of people” timely, accurate answers and that the city would keep upgrading the tool (Lecher, 2024). Later versions added warnings not to treat its answers as legal or professional advice and appeared to narrow the questions it would take on, while readers kept reporting failures. In January 2026 the new mayor called it “functionally unusable,” and by early February the city had taken it down (Lecher & Honan, 2026).
That is more than two years of a public-facing AI system giving business owners advice, some of it wrong in ways that could cost workers and tenants real money. The useful question is not whether a chatbot can make mistakes. Every one can. It is what the smallest set of controls looks like that would have made those mistakes visible, owned and fixable early.
The chapter’s core idea: not everything, just enough
Chapter 4 of Operational AI Readiness starts from a familiar failure in two directions. Some teams overbuild, creating what Viktoria Bakos calls “a cathedral of forms, committees, and slow-moving abstraction.” Others underbuild, treating one reviewer, one log stream and one vague policy sentence as discipline (Bakos, 2026, p. 34). Her answer is the minimum viable control stack: seven components, each of which closes a blind spot the others cannot.
Key idea: the minimum viable control stack
The smallest set of working controls that lets a consequential AI system stay owned, reviewable, observable and interruptible once it is live (Bakos, 2026, p. 15). It has seven parts: inventory, named ownership, effective human oversight, runtime visibility, approval gates, change control, and validation with incident readiness. It matters to you because when an AI system gives you a wrong answer about your rights or your money, these are the things that decide whether anyone notices, and whether anyone fixes it.
The seven layers, in plain language
Here is the stack, with the question each layer forces. We have used the MyCity record only to frame questions, because the public record does not show the city’s internal controls.
- Inventory. Do you know the system exists, what it is for and what it depends on? Without that, the book says, an organization “is merely surrounded by” AI rather than governing it (Bakos, 2026, p. 34).
- Named ownership. When an answer is wrong, who owns the operational explanation? If the honest reply is a meeting invite with six people on it, ownership is weak (Bakos, 2026, p. 35). The Markup’s reporting quotes a technology office spokesperson; it does not show who owned the bot’s answers.
- Oversight with timing and authority. Review has to happen at a point where it can still change the outcome. “A reviewer who sees only the final text and lacks authority to stop the action is not much of a control” (Bakos, 2026, p. 36). A public chatbot answers instantly, so oversight has to live in what it is allowed to answer and how quickly bad answers are caught.
- Runtime visibility. Can you see what the system said, from which sources, and spot patterns? Logs that show a request happened are not enough (Bakos, 2026, p. 36).
- Approval gates. Some outputs should not go straight through. Friction belongs where consequences justify it, not everywhere (Bakos, 2026, p. 37). Questions about wages, housing or legal duties might be one such category.
- Change control. Prompt edits, new sources and topic restrictions all change live behavior, not just model upgrades (Bakos, 2026, p. 37). MyCity changed over its life; a change record is what lets you show whether each change helped.
- Validation and incident readiness. Test under realistic use, and know in advance how the system is paused, who is told and what evidence is kept. “A system that cannot fail coherently is not ready” (Bakos, 2026, p. 38). Ten reporters asking the same question is a test any team can run before launch.
The chapter’s sharpest point is that removing any single layer leaves a specific hole: no inventory and you lose the boundary, no owner and responsibility diffuses, no change control and the workflow “evolves without memory” (Bakos, 2026, p. 38). It also changes the conversation inside a team, from “does the feature work?” to “is the workflow owned, reviewable, observable and interruptible?” (Bakos, 2026, p. 38).
What your team can do this month
- Pick one consequential workflow (Chapter 3 helps you choose honestly) and write seven lines, one per layer, saying what exists today. Blank lines are your plan.
- Name one owner for that workflow’s outputs, by role and by person, and tell the people who use it.
- Run a repeat-question test. Ask the system the same ten consequential questions, several times, as different users. Inconsistent answers are a validation finding, not a curiosity.
- Start a one-page change log today: date, what changed (prompt, source, model, topic rules), why, and who agreed.
- Write the pause plan: who can switch it off or narrow it, and what gets preserved when they do.
What this means for ordinary people
A disclaimer is not a control. If a public or commercial chatbot tells you something about your rights, your pay or your rent, treat it as a starting point and check it against the official rule or a human. And if it gets something wrong, report it. In the MyCity case, the errors became public because outsiders tested the tool and wrote down what they found.
What the record does not show
We do not know what internal testing, logging, ownership or change records New York City kept for the chatbot, how many people acted on wrong answers, or whether anyone was harmed as a result. Our walk through the seven layers is an illustration of the book’s framework, not a finding about the city’s controls.
Questions you can ask
- If you are affected: “Where does this chatbot get its answers, and how do I reach a person who can confirm them?”
- If you are affected: “If I acted on a wrong answer from your chatbot, how do I report it, and who will respond?”
- If you run AI at work: “For our most consequential AI workflow, which of the seven layers could we show a record for today?”
- If you run AI at work: “Who has the authority to narrow or pause it this afternoon, and do they know that?”
Go further. Operational AI Readiness is FairByDesign’s free field manual for AI that is already live. Chapter 4, “The Minimum Viable Control Stack,” explains each of the seven layers with worked examples from finance, hiring, procurement, customer disputes and security, and the later chapters take the layers one by one. Read the chapter free, or test one workflow with the free readiness assessment.
Your next step: read Operational AI Readiness free. The complete Standard edition is available to read and download, with no account needed. Start with Chapter 4, then use the appendices to put the stack to work.
FairByDesign is reader-funded. If this helped, get the free Mercury Brief or see ways to support. Support never buys editorial control.
References
Bakos, V. (2026). Operational AI readiness: A practitioner guide to governing AI that is already live (Free ed.). FairByDesign. https://fairbydesign.org/operational-ai-readiness/
Lecher, C. (2024, March 29). NYC’s AI chatbot tells businesses to break the law. The Markup. https://themarkup.org/news/2024/03/29/nycs-ai-chatbot-tells-businesses-to-break-the-law
Lecher, C., & Honan, K. (2026, January 30). Mamdani to kill the NYC AI chatbot we caught telling businesses to break the law. The Markup. https://themarkup.org/artificial-intelligence/2026/01/30/mamdani-to-kill-the-nyc-ai-chatbot-we-caught-telling-businesses-to-break-the-law
Leave a Reply
You must be logged in to post a comment.