AI hallucination reaches the courtroom: a global look at emerging cases

From private chatbox to public dispute

The post, written by Wenlong Li, Research Professor at Zhejiang University Guanghua Law School and 2026 PPPM Awardee, reflects the author's own opinion and not necessarily the views of FPF. Li argues that hallucination was long treated as an amusing or embarrassing limitation. That framing is wearing thin, he writes, because by 2026 false or unsupported generated claims have surfaced in search interfaces, customer-service chatbots, public-facing AI accounts, self-published media, professional commentary and data protection complaints.

The author says no centralised database exists for this kind of dispute. Existing hallucination databases mainly document fake citations or unreliable AI material submitted in court proceedings, which fall outside his scope. The cases were instead gathered through monitoring of legal and policy developments, targeted searches and tracking of professional and social-media feeds. Li stresses that this is not a systematic review and does not claim to capture every relevant dispute. A table in the post covers 11 judicial and regulatory matters, and inclusion does not imply that liability was established.

For the purposes of the post, "AI hallucination" means machine-generated text that presents a verifiable factual claim as though it were grounded, even though the claim is false, unsupported or not reliably traceable to evidence. Li notes the term is contested and does not attribute human belief or deceptive intent to the model.

China: users and providers held to account

Five of the matters arose in China, though the author warns this should not be read as evidence that hallucination causes more disputes there than elsewhere. Two decisions put responsibility on users who turned generated text into public content.

• In a Hangzhou Binjiang District People's Court case, a self-media operator prompted an AI system to write an article presenting an unrelated company as an Alibaba subsidiary, added Alibaba branding and published it without a prominent AI-content label. The court held the operator responsible for verifying the article and found that the conduct constituted unfair competition, a finding Li calls somewhat puzzling.
• In a Guangzhou Internet Court case, a finance commentator gave an AI application an unverified 1,700-Chinese-character document, asked it to expand the material to more than 10,000 characters, and published the resulting allegations of corporate misconduct as "original". The court found reputational infringement and stressed a heightened, full-chain duty of care arising from the publisher's professional identity, control over the prompts and decision to disseminate.

The provider-facing cases ended differently. In a Hangzhou Internet Court case, an AI application gave a user inaccurate information about a college campus and then generated statements promising compensation if the answer was wrong. The court rejected the claim: the promise was not attributable to the provider as a binding expression of intention, the provider had given warnings and adopted reasonable mitigation measures, and the user proved neither reliance nor loss.

In a Nanjing case, Baidu's AI answer falsely associated an identifiable lawyer with imprisonment and an explosion-related offence. The courts distinguished the synthesized AI answer from search suggestions and related-search terms, treated it as a direct platform-generated presentation, and ordered Baidu to apologize for the reputational harm.

Germany: output under an institutional banner

Li writes that Germany currently offers the clearest examples of public-facing AI statements delivered with an air of factual authority. In the Google AI Overview proceedings (26 O 869/26), the Munich I Regional Court granted a preliminary injunction after an AI-generated search summary linked two publishers to fraud, subscription traps and other dubious practices. The court treated the summary as Google's own content because it reorganised source material into a newly worded answer and asserted connections the cited sources did not make.

In Campact v X.AI LLC (324 O 461/25), the Hamburg Regional Court restrained X.AI from repeating Grok's false statement that Campact depended heavily on federal funding. Publication through the public @grok account made the output attributable to its operator. According to the post, both proceedings focus on the authority created by the provider's design and presentation of the output.

What the patchwork means

Li describes a patchwork of responses across legal fields, shaped by the interface, the actor who deploys or adopts the statement, the nature of the harm and the remedies available. He identifies several fault lines: private interaction versus public externalisation, user misuse versus institutional adoption, warning versus reasonable reliance, individual correction versus system-level intervention, and model output versus the design of the surrounding service.

In his view, a hallucination becomes legally salient when it is externalised, amplified, institutionalised, relied upon or made difficult to challenge and correct. The emerging law asks who put the statement into circulation, how it was framed, what expectations of reliability it created and who could verify it. For companies that deploy generative tools and for people who publish their output, the cases suggest that warnings, labelling and verification practices may matter when responsibility is decided.