Who Is Responsible When AI Decides? Providers, Deployers and Owners Explained

Imagine applying for a part-time online tutoring job and never hearing back. Not because of your qualifications, but because you typed in your date of birth. According to the US Equal Employment Opportunity Commission, that is what happened to more than 200 qualified applicants to iTutorGroup, a set of companies providing English-language tutoring to students in China. The EEOC said the companies’ application software automatically rejected women aged 55 or older and men aged 60 or older. In 2023 iTutorGroup agreed to pay $365,000 and provide other relief to settle the agency’s lawsuit (EEOC, 2023).

Notice who paid. Not the software. The company that used it.

A second case shows the same principle from another angle. In December 2023, the US Federal Trade Commission announced that the pharmacy chain Rite Aid would be banned from using facial recognition for surveillance for five years. Between 2012 and 2020, the FTC said, Rite Aid used the technology in stores to flag shoppers it believed had shoplifted, and the system falsely tagged customers, with women and people of color at higher risk of misidentification. Rite Aid had bought the technology from two outside companies. That did not move the responsibility. The FTC’s list of alleged failures was almost entirely about the buyer: it did not test accuracy before deployment or ask its vendors how they had tested, did not track false matches afterwards, did not adequately train staff, and did not adequately oversee its service providers (FTC, 2023).

Whenever an automated system makes or shapes a decision about a person, the question “who is responsible?” has three layers. It helps to have words for them.

The provider: whoever builds it and puts it out

The European Union’s AI Act gives the clearest vocabulary. A provider is the person or organization that develops an AI system, or has one developed, and places it on the market or puts it into service under its own name or trademark, whether for payment or free of charge (Regulation (EU) 2024/1689, art. 3(3)). Think of the company selling a facial recognition product, a hiring screening tool or a chatbot platform.

Providers carry the heaviest obligations for high-risk systems: designing for accuracy and human oversight, documenting how the system works, and supplying instructions for use. But a provider cannot see how every customer uses its product, which is why the law does not stop there.

The deployer: whoever uses it on real people

A deployer is any person, company, public authority or other body “using an AI system under its authority”, except for purely personal, non-professional use (art. 3(4)). An employer screening applicants, a store scanning faces, a bank scoring loan applications and a council prioritising benefit claims are all deployers.

For high-risk systems, the AI Act gives deployers their own duties. They must assign human oversight to people with “the necessary competence, training and authority”, monitor how the system is operating and suspend it if they identify a serious risk, keep the logs the system generates for at least six months, tell workers before using such a system on them, and, for certain decisions about people, tell those people that a high-risk system is involved (art. 26(2), (5), (6), (7), (11)). People affected by such decisions also gain a right to “clear and meaningful explanations of the role of the AI system” (art. 86(1)). For the stand-alone high-risk uses listed in the Act, such as hiring and access to essential services, these rules apply from 2 December 2027, after an amendment that took effect in July 2026 (European Commission, n.d.).

The line between the two roles can move. A deployer that puts its own name on a high-risk system, substantially modifies it, or uses a general tool for a new high-risk purpose can become a provider in the eyes of the law, with the provider’s obligations (art. 25(1)). Building a hiring screener on top of a general chatbot is not just “using” it.

The owner: the name that answers the phone

“Owner” is not a term in the AI Act. It is the missing piece inside the deployer. An organization can be legally responsible and still have nobody who actually knows what its system did last week.

Picture the usual arrangement. A vendor supplies the model. An engineering team maintains the integration. A business unit uses it daily. Legal appears after a complaint. Each group owns a slice, and when something goes wrong, the answer to “who owns this?” is a meeting invitation. Chapter 1 of Operational AI Readiness sets the bar plainly: an operationally ready team assigns ownership and review responsibilities “clearly enough that the answer does not change depending on who is asked” (Bakos, 2026).

Chapter 12 adds an uncomfortable observation about where that ownership often lands. AI governance is frequently handed to legal or compliance teams, while prompts, data sources, tool permissions and rollback switches are changed day to day by engineering, product and operations. When the people closest to the system treat controls as someone else’s paperwork, “the controls may survive formally while rotting operationally”. The fix the book proposes is cultural as much as technical: the people shaping the live system accept that ownership includes the ability to constrain it, observe it and explain it (Bakos, 2026).

Read the Rite Aid allegations again with that lens. Testing, monitoring false matches, training staff and checking vendors are not jobs for a provider in another office. They are the everyday work of an owner.

Why it matters

For people on the receiving end, the practical upshot is simple: direct your questions to the organization that made the decision about you. It is the deployer, and in many places it is the party regulators hold to account.

For organizations, buying AI from a reputable vendor is not a transfer of responsibility. Contracts can share costs and information, but they cannot make the decision about your customer, applicant or patient someone else’s decision. The question to ask is not “is the vendor compliant?” but “who here can show what this system did, and who here can stop it?”

What the record does not show

Both cases were resolved by settlement, so the agencies’ descriptions are allegations rather than findings after a trial. The EEOC release describes “software”, not artificial intelligence, and does not explain how the age rule was configured or by whom. The FTC did not name Rite Aid’s two vendors. Neither case was decided under the EU AI Act, whose deployer duties for high-risk uses are not yet in force; we use its vocabulary because it is the clearest available, not because it governed these events.

Questions you can ask

• If a decision affects you: Did software screen, score or flag me, and can a person review the result?
• If a decision affects you: Who in your organization is responsible for that system, and how do I raise a concern with them?
• If your organization uses AI: For each system, can we name one owner, not a department, who could explain last week’s outputs?
• If your organization uses AI: What did we ask the vendor about testing before we switched it on, and where is the answer written down?

Your next step: pick one AI system your organization relies on and take the free readiness assessment. It takes about ten minutes, stores nothing, and its second domain, ownership and authority, asks exactly the questions above. A “yes” only counts in full if you could show the record today.

---

FairByDesign is reader-funded. If this helped, get the free Mercury Brief or see ways to support. Support never buys editorial control.

References

Bakos, V. (2026). Operational AI readiness: A practitioner guide to governing AI that is already live. FairByDesign. https://fairbydesign.org/operational-ai-readiness/

European Commission. (n.d.). AI Act [Regulatory framework for AI]. Shaping Europe’s digital future. Retrieved October 15, 2026, from https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai

Federal Trade Commission. (2023, December 19). Rite Aid banned from using AI facial recognition after FTC says retailer deployed technology without reasonable safeguards [Press release]. https://www.ftc.gov/news-events/news/press-releases/2023/12/rite-aid-banned-using-ai-facial-recognition-after-ftc-says-retailer-deployed-technology-without

Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). (2024). Official Journal of the European Union, L, 2024/1689. http://data.europa.eu/eli/reg/2024/1689/oj (articles consulted via https://artificialintelligenceact.eu/article/26/)

U.S. Equal Employment Opportunity Commission. (2023, September 11). iTutorGroup to pay $365,000 to settle EEOC discriminatory hiring suit [Press release]. https://www.eeoc.gov/newsroom/itutorgroup-pay-365000-settle-eeoc-discriminatory-hiring-suit