The two failure modes
Before the gate, we tried to solve hallucinations with prompt engineering. "Do not invent facts." "Only use information present in the brief." "Be careful with comparative claims." Each instruction worked for a while and then the model would find an edge case.
The problem is that an LLM cannot reliably tell, from inside its own output, what it knew from the brief versus what it filled in from training. Both feel equally true to the model. The solution is not to make the model more careful — it is to remove the question from the model entirely.
The whitelist
For every client, we maintain a single Markdown file we call the fact whitelist. It is short and boring on purpose.
# McMahon Auto Service — fact whitelist
# Last verified: 2026-04-12 by Sarah (account manager)
[BUSINESS]
- Founded: 2009
- Locations: Carlsbad (1 store)
- Services: brake repair, oil change, AC service,
diagnostic, tire rotation
- Hours: Mon-Fri 8-6, Sat 8-2
[OFFERS]
- Free brake inspection (no expiry)
- Same-day appointments (Mon-Fri, depends on bay)
- Loaner car: NO (do not claim)
[BANNED CLAIMS]
- Any price comparison ("lowest", "cheapest")
- ASE certified (one tech only, cannot generalize)
- "Family owned" (owned by a small holding)
- Any specific warranty length
- Any wait time guaranteeThat is the whole file. Every line is verified by the account manager, dated, and signed. When the AI generates copy, the validator parses every claim in the output against this file. Anything not on the whitelist gets rejected and the model is asked to rewrite without that claim.
How the gate works
The flow is the same for RSAs, sitelinks, callouts, descriptions, and landing-page copy.
- Model generates output.
- Validator extracts every factual claim. (Years, prices, comparisons, certifications, services, locations, hours, warranties.)
- Each claim is matched against the whitelist. Match → pass. No match → fail.
- Failed claims are returned to the model with the rejected sentence and the reason.
- Model rewrites only the failing lines. Loop.
- After three loops with no full pass, the run halts and a human is paged.
The validator does not need to be smart. It needs to be strict. We use a small classifier to detect claim-like phrases and a literal substring match against the whitelist for verification. False positives are fine — they cost a rewrite. False negatives are not — they cost a client.
The trace UI
The other half of the framework is visibility. Every piece of AI-generated copy in actcenter is shipped with a trace that the reviewer can expand. It looks like this:
And, on a failure:
The reviewer sees the trace, not just the verdict. That changes the relationship between the human and the AI. The reviewer no longer has to ask "how do I know this is true?". They can see the proof or the failure, line by line.
The day it actually mattered
About four months in, the gate caught something a human would not have. The model wrote "open 7 days a week" for a roofing client. The reviewer was about to approve — the phrase felt right, every roofing client in the back of their head was open 7 days. The trace flagged it: the whitelist for that client said Monday to Friday only.
The client had updated their hours three weeks earlier. The reviewer had not internalized the change. The whitelist had.
That is when we stopped thinking of the gate as a hallucination filter and started thinking of it as institutional memory. The whitelist is not a guardrail for the AI. It is a guardrail for everyone — including the senior reviewer who has 22 accounts in their head and cannot hold every detail.
The reframe
Source validation is not about the AI being unreliable. It is about any writer — human or model — being unable to hold every client fact in working memory across a portfolio. The gate is a memory system that happens to also catch hallucinations.
What we did not do
A few things that look obvious in hindsight and that we deliberately avoided:
1. We did not use retrieval over the open web
Letting the AI "look up" facts from the open web is worse than letting it guess. The web contains plausible-looking but wrong information about every business. The whitelist is curated by a human who has been on the client's site, on a phone call, or has read the contract. That is the only acceptable source.
2. We did not let the AI write its own whitelist
The whitelist must be written by an account manager. We tried the shortcut where the model parses the client's website and proposes a draft. The drafts were close enough to be dangerous — they confidently included claims the client did not want to make. Manual write only.
3. We did not let the AI argue with the gate
When a claim fails, the model rewrites. It does not get to defend the original. We have seen LLMs build elaborate justifications for hallucinated facts. The gate does not engage. It says reject and asks for a rewrite.
What this costs to set up
Per client, the whitelist takes 30 to 60 minutes to write the first time. It gets updated about once a quarter as the client's services or offers change. The validator is the same across all clients — it lives in our skill stack, not in the per-client folder.
The ongoing cost is roughly zero. The cost of not having it would have been at least one bad ad shipping per quarter across the portfolio, which is the number we used to see when we relied on careful reviewers. Across 39 accounts, that is enough avoided embarrassment to justify the framework on day one.
You can build this in an afternoon
The Source Validation Gate is not a vendor. It is a pattern. The whitelist is Markdown, the validator is a Python script with about 80 lines, and the trace UI is whatever HTML you can stand up. We use it across every AI-generated artifact in actcenter — RSAs, descriptions, sitelinks, talking points, weekly client emails.
If you are letting any kind of LLM write client-facing copy in 2026, this is the cheapest possible safety net. Build it before the day you would have wished you had.