Synthetic Consensus

The Evidence Chain, Part 8 of 10. Part 7 showed how machine output becomes the evidence that later machines learn from. This part looks at the most persuasive way that evidence can mislead you: by agreeing with itself.

Not long before late January 2026, the disinformation researcher Nina Jankowicz found a quotation attributed to her in a story by a large news outlet. She had never said it. She wrote to the outlet, and it took the quote down. End of story, you would think. It wasn’t. As she told the Guardian that month, AI models went on citing the invented quote as hers for some time afterwards (Down, 2026). The correction reached the source. It never reached the copies.

Hold that picture, because it is about you. The next time you ask a chatbot a question that matters (whether a medicine interacts with another, what a new law requires of your employer, who is telling the truth about a war) it will probably hand you an answer with several links underneath, all agreeing. Your brain will do what brains have always done with agreement: it will relax. Five sources saying the same thing feels like five people who looked and saw the same thing. The question this part asks is how often that feeling is now wrong, and whether anyone, including you, can tell.

An old failure with a new price

Intelligence analysts have a name for agreement that is not independent: circular reporting. The textbook case is Curveball, the defector whose account of Iraqi mobile weapons laboratories underpinned the most consequential intelligence judgment of the early 2000s. A presidential commission later found that almost a hundred reports about the laboratories had come from that one source, through a foreign service that never let the Americans question him, and that the judgment had been presented as though it rested on several (Commission on the Intelligence Capabilities of the United States Regarding Weapons of Mass Destruction [WMD Commission], 2005). Volume produced the feeling of corroboration. The corroboration was hollow.

Manufacturing that illusion used to be expensive. It took a fabricator, an intelligence service and years. My argument in this part is that generative AI has made it cheap, fast and almost invisible, and that it now happens both on purpose and by accident.

How one voice becomes a chorus

Start with the deliberate version. In March 2025 NewsGuard reported that a Moscow-based network of websites known as Pravda had published some 3.6 million articles in 2024, and that when its analysts tested ten leading chatbots on false narratives the network had pushed, the chatbots repeated those narratives about a third of the time (Sadeghi & Blachez, 2025). The sites barely need human readers. Days later, researchers at the Atlantic Council’s Digital Forensic Research Lab counted 1,907 hyperlinks to Pravda-affiliated sites across 1,672 Wikipedia pages in 44 languages, and found content from the network in answers from ChatGPT, Gemini, Copilot and Perplexity (Châtelet & Lesplingart, 2025). In October 2025 the Institute for Strategic Dialogue put 300 questions about the war in Ukraine to four chatbots in five languages and found that nearly a fifth of the answers cited Russian state-attributed sources, with the problem worst where reliable information on a topic was thin (Maristany de las Casas, 2025).

Look at the shape of that chain. One origin publishes under many mastheads. Some of its pages get cited by an encyclopedia that millions trust. A chatbot retrieves the encyclopedia and the mastheads and presents them as separate sources. By the time the claim reaches you it has three or four apparently independent supports, and every one of them leads back to the same desk.

Now the version nobody planned. In January 2026 the Guardian tested OpenAI’s GPT-5.2 and found it cited Grokipedia, an encyclopedia written by an AI model that does not allow direct human editing, nine times across more than a dozen questions, mostly on obscure topics where the encyclopedia’s claims went further than Wikipedia’s (Down, 2026). That is a machine citing a machine, presented to a human as a source. And even without any bad actor, shared tools pull writing together: in controlled studies, people writing with the same AI assistance produced work that was individually better but collectively more alike (Doshi & Hauser, 2024; Padmakumar & He, 2024). Synthetic consensus does not need a liar. It needs only a shared upstream.

Why the new labels do not reach it

Europe’s answer to machine-made content is now law. Since 2 August 2026, Article 50 of the AI Act has required providers of generative systems to mark their output in a machine-readable way and deployers to disclose AI-generated text published to inform the public, unless a human has reviewed it and someone holds editorial responsibility (Regulation (EU) 2024/1689, art. 50). In June 2026 the European Commission published a final code of practice telling signatories how to mark and label such content (European Commission, 2026). These are real advances, and I want them enforced.

But notice which question they answer. A label tells you whether a machine was involved. It does not tell you whether the five sources under your answer are five looks at reality or one look repeated five times. A Pravda article can be written by a person; an analyst’s report can be edited by a human and so carry no label at all; a citation to a human-written encyclopedia can carry a claim laundered from somewhere else. Authenticity tells you who signed the document. It does not tell you how many independent routes back to the world stand behind it, and nothing in the new rules asks.

Who decides what counts as agreement

This is where the technical problem becomes a power problem. Whoever controls the retrieval layer, the handful of systems that choose which sources sit under an answer, now decides what consensus looks like for hundreds of millions of people. Those who can publish at industrial scale, whether a state network or a billionaire’s encyclopedia, get to look like many voices. Those who publish slowly and carefully look like one. The ISD finding that manipulation thrives where reliable information is scarce (Maristany de las Casas, 2025) points somewhere uncomfortable. My own inference, which the studies above do not test directly, is that the people most exposed are those asking about places and languages that wealthy newsrooms cover least, across much of the Global South, and the generation that increasingly starts its research by asking a machine. That is not a reason to tell young people to stop using these tools. It is a reason to hand them the one question that cuts through: how many origins are there, really?

The prediction

My prediction is that consequential decision systems, in intelligence, finance, journalism, science, health and law, will have to measure the independence of their evidence, not only its authenticity, and that institutions which do not will produce a new generation of Curveball-style failures in which the corroboration was real on paper and hollow underneath. The research question for this part is simple to state and hard to answer: can a decision system, or a person, tell the difference between five sources and one source repeated five times?

For Mercury Intelligence members: the section below walks the Pravda case link by link as an origin count, maps the rules that touch it clause by clause, sets out an independence audit you can run on any AI-assisted research, and takes the strongest objection seriously.

---

Support independent investigation. Every exposure in this part came from someone outside the platforms counting origins by hand. Nobody who sells answers at scale will pay for that question to be asked, so we ask it with readers’ support instead. Give once ($5, $10, $25, $50 or any amount), give monthly ($10 or $25), or become a patron: fairbydesign.org/patrons-2.

References

Cahill, J. (2026, September 10). EU AI Act Article 50 transparency obligations go live. Baker Botts. https://www.bakerbotts.com/thought-leadership/publications/2026/september/eu-ai-act-article-50-transparency-obligations-go-live

Châtelet, V., & Lesplingart, A. (2025, March 12). Russia-linked Pravda network cited on Wikipedia, LLMs, and X. Digital Forensic Research Lab, Atlantic Council. https://dfrlab.org/2025/03/12/pravda-network-wikipedia-llm-x/

Coalition for Content Provenance and Authenticity. (2025). Content Credentials: C2PA technical specification (Version 2.2). https://spec.c2pa.org/specifications/specifications/2.2/specs/C2PA_Specification.html

Commission on the Intelligence Capabilities of the United States Regarding Weapons of Mass Destruction. (2005). Report to the President of the United States. https://www.globalsecurity.org/intell/library/reports/2005/wmd_report_25mar2005_chap01.htm

Doshi, A. R., & Hauser, O. P. (2024). Generative AI enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 10(28), Article eadn5290. https://doi.org/10.1126/sciadv.adn5290

Down, A. (2026, January 24). Latest ChatGPT model uses Elon Musk’s Grokipedia as source, tests reveal. The Guardian. https://www.theguardian.com/technology/2026/jan/24/latest-chatgpt-model-uses-elon-musks-grokipedia-as-source-tests-reveal

European Commission. (2026, June 10). Code of practice on transparency of AI-generated content. Shaping Europe’s Digital Future. https://digital-strategy.ec.europa.eu/en/policies/code-practice-ai-generated-content

Lorenz, J., Rauhut, H., Schweitzer, F., & Helbing, D. (2011). How social influence can undermine the wisdom of crowd effect. Proceedings of the National Academy of Sciences, 108(22), 9020–9025. https://doi.org/10.1073/pnas.1008636108

Maristany de las Casas, P. (2025, October 27). Talking points: When chatbots surface Russian state media. Institute for Strategic Dialogue. https://www.isdglobal.org/digital_dispatches/talking-points-when-chatbots-surface-russian-state-media/

National Institute of Standards and Technology. (2024). Reducing risks posed by synthetic content: An overview of technical approaches to digital content transparency (NIST AI 100-4). https://doi.org/10.6028/NIST.AI.100-4

Office of the Director of National Intelligence. (2015a). Intelligence Community Directive 203: Analytic standards (as amended 2022). https://archive.dni.gov/files/documents/ICD/ICD-203.pdf

Office of the Director of National Intelligence. (2015b). Intelligence Community Directive 206: Sourcing requirements for disseminated analytic products. https://archive.dni.gov/files/documents/ICD/ICD-206.pdf

Padmakumar, V., & He, H. (2024). Does writing with language models reduce content diversity? In Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2309.05196

Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market for Digital Services and amending Directive 2000/31/EC (Digital Services Act). (2022). Official Journal of the European Union, L 277, 1–102. http://data.europa.eu/eli/reg/2022/2065/oj

Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). (2024). Official Journal of the European Union, L, 2024/1689. http://data.europa.eu/eli/reg/2024/1689/oj

Sadeghi, M., & Blachez, I. (2025, March 6). A well-funded Moscow-based global “news” network has infected Western artificial intelligence tools worldwide with Russian propaganda. NewsGuard. https://www.newsguardtech.com/special-reports/moscow-based-global-news-network-infected-western-artificial-intelligence-russian-propaganda/