Two of Five: The IRS AI Tools That Went Live Without an Impact Assessment

If you have ever filed a tax return, you have probably wondered, at least once, how anyone decides which returns get a closer look. For years, part of the answer at the US Internal Revenue Service has involved artificial intelligence. The Treasury Inspector General for Tax Administration, the IRS’s independent watchdog, notes that the agency began using AI to improve tax return classification and issue selection as far back as 2017. By December 2025 its inventory listed 225 AI use cases (Treasury Inspector General for Tax Administration [TIGTA], 2026, p. 1).

On 24 September 2026, the inspector general published a short audit with a pointed finding. It looked at five AI tools the IRS itself presumed to be “high-impact,” all of them already deployed and used for what the report calls workload selection. Two of the five, 40 percent, had no documented impact assessment (TIGTA, 2026, p. 2). The report, Assessment of Artificial Intelligence Risk Management Practices (Report 2026-208-059), is worth reading closely, because it shows how a basic safeguard can go missing even in an agency that has been using AI for nearly a decade.

What the auditors looked at

The audit set out to test whether data in the IRS’s Compliance Data Warehouse is reliable enough to support AI. The auditors did not check the accuracy of the data itself. Instead they examined how the IRS documented its assessments of that data (TIGTA, 2026, transmittal memo; Appendix I). They identified seven deployed high-impact use cases that draw on the warehouse and reviewed the five used for workload selection (Appendix I, p. 5). Fieldwork ran from October 2025 to April 2026.

“High-impact” has a specific meaning in US federal government. Under the Office of Management and Budget’s memorandum M-25-21, issued in April 2025, it covers AI whose output serves as a principal basis for decisions with a legal, material, binding or significant effect on things like civil rights, privacy, safety or access to critical government services (Office of Management and Budget [OMB], 2025, p. 19). The auditors put the stakes plainly: these use cases “may have a material effect on taxpayer outcomes” (TIGTA, 2026, p. 3).

Two of five, and the deadline had passed

M-25-21 gave agencies 365 days to put minimum risk management practices in place for high-impact AI, which meant April 2026 (OMB, 2025, p. 14; TIGTA, 2026, p. 2). One of those practices is an impact assessment, and the memo is clear about timing: agencies must complete one “before deploying any high-impact AI use case” (OMB, 2025, p. 16). The memo is equally clear about the consequence of non-compliance. If a high-impact use case does not meet the minimum practices, “the agency must safely discontinue use of the AI functionality” (OMB, 2025, p. 15).

Against that standard, the auditors found that two of the five deployed tools had no impact assessment on file. The other three did, although those assessments were carried out before M-25-21 took effect, under earlier guidance (TIGTA, 2026, p. 2, note 8). The report does not say whether the two tools without assessments were paused after the April deadline.

Part of the story is a gap between agencies. M-25-21 places the obligation on federal agencies, and in this case that means the Department of the Treasury, of which the IRS is a part. IRS managers told the auditors that as of July 2026, they had not received any guidance or instructions from Treasury on implementing the minimum practices. In February 2026, the IRS developed and began implementing its own AI governance policy and risk management process to align with the memo (TIGTA, 2026, p. 3). That is a reasonable thing to do while waiting. It does not change the fact that two tools were already running without the one document meant to justify them.

The data question

The second finding goes to the foundation. An impact assessment under M-25-21 must cover “the quality and appropriateness of the relevant data” and its fitness for the AI’s intended purpose (OMB, 2025, p. 16). The auditors asked IRS program managers and data scientists how they had checked that the data was fit for use. The answer was that checks had been done and the data was fine. The documentation told a less tidy story. One of the five use cases had detailed records of its testing processes, including change management, oversight and validation. The other four had no testing documentation at all (TIGTA, 2026, p. 3).

There is a detail here that a careful reader should not miss. According to the IRS’s own internal web page for the data warehouse, its data should not be used as a substitute for source systems for operational decision-making (TIGTA, 2026, p. 3). That does not mean the warehouse data was unsuitable for these tools. It does mean that the question of fitness was a real one, which is exactly why it needed to be written down.

Managers in the IRS’s Research, Applied Analytics and Statistics organization agreed that standard documentation would be valuable, but said they had not prioritized it (TIGTA, 2026, p. 3). The auditors cited a long-standing federal principle from OMB Circular A-123: proper documentation is a necessary part of effective internal control, because it preserves the evidence behind decisions (p. 3). Without it, they warned, the result may be “inaccurate, biased, or unreliable AI outputs” (p. 4).

This is the gap I find most instructive, because it is so ordinary. The people closest to the models said the work was done, and they may well be right. But a check that lives only in someone’s memory cannot be reviewed, repeated by a successor, or shown to a taxpayer, a court or an auditor. From the outside, an undocumented check and a skipped check look the same.

What the IRS agreed to

The inspector general made two recommendations to the IRS’s Chief Data and Analytics Officer: complete impact assessments for high-impact use cases in line with M-25-21, and develop and implement standard processes for evaluating data quality and documenting data fitness. The IRS agreed with both. It said it would complete impact assessments for currently deployed high-impact use cases as of November 2026, and that it has already implemented standard data quality processes (TIGTA, 2026, p. 4).

What the record does not show

The report does not name the five tools or describe exactly what each one does, beyond workload selection. It does not show that any taxpayer was wrongly selected, treated or harmed, and it should not be read that way. The auditors did not test the accuracy of the data or the performance of the models. It does not say whether the two tools without impact assessments kept running after April 2026, or whether any were paused as the memo requires for non-compliant uses. And it covers five use cases out of 225, chosen because they use one data source, so it cannot tell us how widespread the gap is across the IRS. Those are questions for the IRS and Treasury, not conclusions we can draw.

Questions you can ask

• If you are a taxpayer: if you are selected for review, you can ask how your return was selected and whether an automated tool was involved. Write down the answers you get.
• If you run AI in an organization: pick your three most consequential AI tools and ask for each one’s impact assessment. Check the date. Was it written before the tool went live?
• If you own the data behind a model: can you show, on paper, how you decided the data was fit for that specific use? “We checked” is not evidence until it is written down.
• If you sit on a board or oversight body: ask what happens to a high-impact tool that misses its assessment. Is there a rule to pause it, and has anyone ever used it?

If you answer for AI at work, this is the kind of finding your own auditors will ask about next. Mercury Intelligence ($10 a month) gives you our full investigations and a governance impact analysis of each one: what the finding means for your controls, the evidence a reviewer will expect, and the questions to put to your team this quarter. See what is included, or save with the annual plan.

---

FairByDesign is reader-funded. If this helped, get the free Mercury Brief or see ways to support. Support never buys editorial control.

References

Bakos, V. (2026). Operational AI readiness: A practitioner guide to governing AI that is already live. FairByDesign. https://fairbydesign.org/operational-ai-readiness/

Office of Management and Budget. (2025, April 3). Accelerating federal use of AI through innovation, governance, and public trust (Memorandum M-25-21). Executive Office of the President. https://www.whitehouse.gov/wp-content/uploads/2025/02/M-25-21-Accelerating-Federal-Use-of-AI-through-Innovation-Governance-and-Public-Trust.pdf

Treasury Inspector General for Tax Administration. (2026, September 24). Assessment of artificial intelligence risk management practices (Report No. 2026-208-059). U.S. Department of the Treasury. https://oversight.gov/sites/default/files/documents/reports/2026-09/2026208059fr.pdf