AI search engines can sound sure while citing dead or low-quality sources, a quick test shows
A simple provenance check on three popular AI-powered search tools revealed that two of them either pointed to dead links or backed their answers with low-credibility sites, even though all three displayed the same confident prose and a row of source “chips.”
The test that sparked the surprise
I asked a factual, recent question: “What changed in the EU AI Act in 2026?” The answer exists in the official amendment text, so it either is there or isn’t. I fed the query to three AI search engines that surface citations: Perplexity, Google’s AI mode, and Brave Search.
All three returned identical facts—same dates, same description—so they tied on pure correctness. The divergence appeared only when I examined the cited sources.
How I judged source quality
I created a three-tier scoring scheme that weights each citation by host type:
- Primary (weight 3) – the site that actually publishes the law (e.g., the official EU register).
- Official (weight 2) – the institution that issues or oversees the law (government domains, agency sites).
- User-generated content (UGC, weight 0) – platforms such as YouTube or Reddit.
Each engine’s overall score is the average weight of the URLs it displayed.
| Engine | Reported sources | Score |
|---|---|---|
| Perplexity | Official government domains | 2.00 |
| Google AI mode | Consulting firms and a YouTube video | 1.00 |
| Brave Search | Primary source | 3.00 |
On paper Brave looked perfect, Perplexity was decent, and Google lagged behind.
The hidden failure: dead links
The tier map only looks at the domain name; it does not verify whether the linked page actually loads. I followed every URL. Brave’s “primary” citation returned an empty body—the link was dead. The engine claimed to point to the law but delivered nothing.
Perplexity used official government domains.
Google used consultancies and a YouTube video.
Two distinct problems emerged:
- A high-quality domain does not guarantee a reachable page.
- A well-crafted summary can be backed by low-credibility sources.
The user interface hides both issues. All three tools present the same confident paragraph and a uniform row of source chips, without any visual cue that one chip links to a dead page or that another points to a YouTube tutorial rather than the statutory text.
Why provenance matters for developers
Developers can turn provenance into a testable metric:
- Score every answer using a tiered weighting similar to the one above.
- Validate link health automatically; flag empty responses or HTTP errors.
- Trigger alerts when an answer’s provenance score falls below a configurable threshold.
This approach is more concrete than chasing “hallucinations” – the model may generate a plausible sentence, but the provenance check tells you exactly which citation (or lack thereof) caused the problem, allowing a targeted fix.
Takeaway
A brief provenance test shows that AI search engines can appear authoritative while citing dead or low-quality sources. Developers who rely on these engines should treat citation quality as a measurable property, not an assumed guarantee, and build automated checks into their pipelines. Verifying that a “primary” source actually loads can be the difference between a trustworthy answer and a silent misinformation trap.