The Vanishing Human
From the Field Manual. Each week we take one chapter of our free book Operational AI Readiness and turn it into something you can use. This week: Chapter 9, on deciding where AI must wait for a human yes.
The letter says your insurer will not pay for a test your doctor ordered. A medical director signed off on it, so you assume a doctor looked at your case. (That letter is an illustration. What follows is not.)
In March 2023, ProPublica and The Capitol Forum reported that at the health insurer Cigna, that “look” averaged 1.2 seconds per claim.
Here is my proposition: “a human approves it” is not a safeguard. It is a claim, and it only becomes a safeguard when you can say approves it how, with what information, with what choices, and in how long. Many of the most reassuring sentences in AI governance contain those four words. Chapter 9 of our free field manual argues that they don’t settle anything. They open the question.
Inside the 1.2 seconds
The reporters described a claims system at Cigna called PXDX. According to their reporting, it flagged claims where a patient’s diagnosis did not match the tests or procedures Cigna accepts for it, and company medical directors then denied the flagged claims in batches without opening patient files. Corporate documents showed that over two months in 2022, Cigna doctors denied more than 300,000 requests this way, spending an average of 1.2 seconds on each. “We literally click and submit,” one former Cigna doctor told the reporters (Rucker et al., 2023).
Who benefits from a step like that? The organization gets speed, and it gets to say a doctor decided. Who pays? The patient who receives the letter, especially one who doesn’t fight it. ProPublica reported that a Cigna document estimated only 5% of people would appeal a PXDX denial (Rucker et al., 2023). A weak gate counts on people not asking.
Cigna’s side of the story
Cigna called the reporting “biased and incomplete” and said the description of doctors rejecting claims without examining them was “incorrect.” It said the system was built to “accelerate payment of claims for certain routine screenings,” and that its reviews happen after care is given and do not result in denials of care (Rucker et al., 2023). Cigna has also said PXDX does not use artificial intelligence (Pifer Parduhn, 2023). A lawsuit filed in federal court in California in July 2023 alleges the insurer “wrongfully delegated” its duty to evaluate claims to the system (Pifer Parduhn, 2023). Those are allegations, not findings.
Cigna is right that PXDX is a rules-based algorithm, not a chatbot. I use it anyway because it shows, with real numbers, what happens to “a human approves it” when the human step is not designed to be a decision. And as AI systems take on more recommending and drafting, the same pattern becomes easier to build and harder to see.
There is a second objection worth taking seriously: put a human gate on everything and nothing gets done. The field manual agrees. Approval gates are not a licence for bureaucracy everywhere. They belong at the points where reversibility is low, harm is material, or consequences are expensive to unwind (Bakos, 2026, p. 63). A denied claim is one of those points.
A gate is a design, not a person in the room
Where a gate exists, “a human looks at it” is too vague to count. A real gate answers five questions: what triggers review, who is authorized to approve, what that person must see, what decisions they can make, and what record remains afterward. If any answer is hand-waving, the gate is weak; many organizations discover that what they call approval is merely “ambient awareness” (Bakos, 2026, p. 63).
Key idea: approval gates and automation laundering
An approval gate is a named point where a consequential AI-influenced action must get explicit human review and authorization before it goes ahead, defined by its trigger, reviewer, information, choices and record (Bakos, 2026, p. 15). Automation laundering is what happens when a human step exists on paper but volume, design or pressure means the human simply ratifies the machine. It matters to you because “a person reviewed your case” is only a protection if that person had the time, information and freedom to say no.
How to tell a real yes from a rubber stamp
Clear triggers. “Sensitive cases get extra scrutiny” breeds inconsistent practice. Strong gates name the categories: payment release, access revocation, employment decisions, adverse customer treatment, security containment, and so on, and encode them in the workflow (Bakos, 2026, p. 64).
Real choices. Weak gates offer approve or escalate. Strong ones allow approve, reject, revise, request more evidence, escalate or pause. As the chapter puts it, if approval is the only practical path, “the gate teaches compliance, not judgment” (Bakos, 2026, pp. 64–65).
A recorded reason. Not an essay, but enough structured rationale to show why an action went ahead, changed or stopped. Those records prove the control worked, and patterns in them reveal where the AI is weak (Bakos, 2026, p. 65).
The right level of approver. A routine dispute may need a trained frontline reviewer; a large payment or a security action on critical systems may need a senior or second approver. “The gate should fit the consequence” (Bakos, 2026, pp. 65–66).
Workable conditions. Reviewers need the context to judge, clear flags for anomalies, real alternatives, and enough time. Rejection must not be socially punished. “An elegant policy that creates impossible reviewer conditions is still a bad gate” (Bakos, 2026, p. 66). A second per case is a reviewer condition.
The verdict
On the reported numbers, a signature at that speed tells you a person was present, not that a person decided. Cigna disputes the account, and the lawsuit has not proved anything. But the design lesson does not depend on who wins: if a review step has no time, no information and no real option to say no, calling it “human review” launders the machine’s answer. The test of a gate is not whether a human touches it. It is how often, and how easily, that human can say no.
If a decision lands on you
If an insurer, employer or agency tells you a person reviewed a decision against you, it is fair to ask what that review involved. Ask for the reasons in writing, ask who reviewed it, and use the appeal route. The 5% estimate is exactly why appeals matter.
- If you are affected: “Who reviewed this decision, what information did they have, and how do I appeal?”
- If you are affected: “Was an automated system involved in flagging or recommending this outcome?”
- If you run AI at work: “For each place we say a human approves AI output, how long does approval take and how often is the answer no?”
- If you run AI at work: “Can our reviewers revise, request evidence or pause, and are those choices recorded with a reason?”
If you run the gate: five fixes this month
- List your “a human approves it” claims and answer the five gate questions for each. Any blank answer is your first fix.
- Measure the gate. Pull approval time per item and the approve, reject and revise rates for one month. A near-100% approval rate at a few seconds per case is a finding, not a success.
- Add at least one option beyond approve. “Request more evidence” or “revise” is often the cheapest way to turn a stamp into a judgment.
- Write the triggers down in plain categories, and check that the system enforces them rather than relying on memory.
- Capture a one-line reason on every rejection and override, and review the patterns monthly.
What we still don’t know
The core figures come from company documents described by ProPublica and The Capitol Forum, which Cigna disputes. We have not seen those documents ourselves. We do not know the current status or outcome of the California lawsuit or of any regulatory review, and nothing here establishes that any individual patient was wrongly denied. Cigna says PXDX is not AI; our point is about the design of the human step, which applies whatever produces the recommendation.
Which leaves the question I can’t stop asking: if a gate that approves almost everything in seconds counts as “human review,” how many other human reviews in your life would pass that test?
Go further. Operational AI Readiness is FairByDesign’s free field manual for AI that is already live. Chapter 9, “Approval Gates for High-Impact Actions,” works through trigger logic, decision options, approval levels and automation laundering with examples from payments, hiring, procurement, customer disputes and security. Appendix C gives a human-in-the-loop specification you can adapt. Read the chapter free, or check your gates with the free readiness assessment.
Help keep the field manual free. FairByDesign gives this book away, reads the public record so you do not have to, and stays independent of the companies and agencies we write about. That only works because readers fund it. Give once ($5, $10, $25, $50 or any amount), give monthly ($10 or $25), or become a patron: fairbydesign.org/patrons-2. Support never buys editorial control.
References
Bakos, V. (2026). Operational AI readiness: A practitioner guide to governing AI that is already live (Free ed.). FairByDesign. https://fairbydesign.org/operational-ai-readiness/
Pifer Parduhn, R. (2023, July 25). Cigna sued over algorithm allegedly used to deny claims. Healthcare Dive. https://www.healthcaredive.com/news/cigna-lawsuit-algorithm-claims-denials-california/688857
Rucker, P., Miller, M., & Armstrong, D. (2023, March 25). How Cigna saves millions by having its doctors reject claims without reading them. ProPublica; The Capitol Forum. https://www.propublica.org/article/cigna-pxdx-medical-health-insurance-rejection-claims




Leave a Reply
You must be logged in to post a comment.