Limiting the blast radius of AI hallucinations

Firms must put AI controls and audit trails in place to reduce hallucination impacts, writes iManage VP of product management, AI services Jan Van Hoecke
Prefer the Global Legal Post on Google
Limiting the blast radius of AI hallucinations

AI hallucinations have not gone away. By October 2026, Damien Charlotin’s public database had recorded more than 2,100 court decisions involving fabricated or misrepresented authority, and it is still adding several a day. What has changed is the reaction, because each new case now surprises almost no one. That quiet acceptance reflects a shift in thinking, with hallucination no longer seen as a model setting that the next release will fix, but as a risk to be contained.

For firms today, the question isn’t whether AI will get something wrong, but how far the damage could spread when it does and how that impact should be managed.

The factors that determine the fallout

Cloud and security engineers stopped chasing zero failure long ago. Instead, they design systems so that when something breaks, the damage stays contained – a principle known as limiting the blast radius.

For hallucinations, five main factors determine the size of the blast radius:

  • Audience – who sees the error? Does it reach only the author’s eyes, or does it extend out to the team, a client, a court or the public?
  • Reversibility – can that error still be corrected or is it already on the record?
  • Propagation – will the error be reused for additional decisions? For instance, has it been saved as a precedent?
  • Detectability – how easy is the error to detect? A non-existent case is easy to catch, but a subtle misreading of a real one is not.
  • Accountability – who answers for the error once it surfaces?

Some examples help illustrate how these different factors can play out to create either a manageable problem with a small blast radius – or a very serious problem with a highly damaging impact.

For instance, a fabricated citation that a reviewing associate spots and fixes has a small radius. A customer-facing chatbot that invents a refund policy has a slightly larger one, as Air Canada discovered when a chatbot gave faulty advice to a customer about plane fares.

Meanwhile, delayed propagation can turn a ‘small’ error into a potentially large blast radius. Case in point: a made-up clause in a precedent bank at a law firm does not do any direct damage – until the next matter draws on it, and the next matter after that, and so on.

At perhaps the furthest end of the blast radius spectrum sits a case like Withers v City of Aberdeen, in which a US federal judge removed every lawyer from the case, barred two of them from the district for two years and cancelled the trial after their briefs cited cases that didn’t exist.

Agents push the radius further still. A person reviewing a generated draft provides built-in containment, but an agent that can automatically update records, send emails or file documents can turn a single hallucination into a chain of consequences before anyone notices.

Given these challenges, how best for firms to proceed?

Building in the guardrails

For starters, set controls at the points where AI-generated work leaves the author. The radius grows when work is filed, sent to a client or saved to a shared repository. Those are the points to put checks in place, much like egress filtering in network security.

As part of these checks, it’s worth defining what ‘verified’ means. Many AI hallucination cases had a secondary review, and they still missed errors. Firms should specify, for instance, that checking that a case exists is not enough – a reviewer has to open the primary source and confirm it says what the sentence claims and that it is still good law.

Additionally, firms should measure where errors are caught. Note whether each error was caught by the author, by the reviewer, by opposing counsel or by the court.

Another practical recommendation is to keep the audit trail with the matter in a system of record. For high-radius work, the matter file should record the sources consulted (including what each connected system was asked and returned), which passage supports which claim, where retrieval came back empty, the model and prompt version, and who reviewed, changed and signed off on the output. Firms must be able to reconstruct what happened in the event of an error.

While the above takes care of humans, what about AI agents? The advisory here is to limit what agents can do. They should not be able to file, send or overwrite a shared record without human approval. Accountability still has to stop with an actual person.

Focus on what you can control

Verifying citations isn’t just good business practice for law firms – it is becoming mandatory. In the US, for instance, California Senate Bill 574, which takes effect on 1 January 2027, amends California’s sanctions statute so that a filing can’t contain any citation the responsible attorney hasn’t personally verified. Commentators read this as covering every citation, whether it came from AI, an associate, a paralegal, a research service or an old brief – which means the pressure is on. In effect, the law puts a mandatory check at the point where the blast radius is largest – the court filing – and the audit trail is how a firm proves the check took place.

Models will keep making mistakes. What firms control is how far those mistakes travel and whether they can show exactly what happened when one slips through. Firms that build those controls now will be able to use AI on high-stakes work, because they will be able to defend it.

Jan Van Hoecke is VP of product management, AI services at iManage. He is a highly experienced computer scientist with a passion for technology and problem-solving.

Email your news and story ideas to: [email protected]

The Global Legal Post

© 2026 The Global Legal Post. All Rights Reserved

Top