4 Million Responses Show Hallucination Is a Retrieval Problem

For two years the industry has treated hallucination as a knowledge problem and prescribed a knowledge solution: bigger models and more data. The measurement says the shelves are nearly full. The keys are lost.

This is the most genuinely surprising research I have read this year, and unusually for AI news, it is a peer-reviewed measurement rather than a vendor claim or a benchmark press release.

It also has an immediate, practical consequence for anyone running AI in a business—one that most teams are currently getting backwards, at a real cost.

What the Study Measured

Google Research published Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality, with the full paper on arXiv.

The core insight is methodological and, once stated, obvious in the way good ideas are. Standard factuality evaluations treat every error identically. A model gets a fact wrong; the score goes down. That tells you nothing about why — whether the fact was never learned, or was learned and could not be retrieved.

So the researchers built a framework that profiles knowledge at the level of individual facts rather than questions. For each fact they ask, in sequence: is it encoded in the model at all? If so, can the model recall it directly? If not directly, can it recall it with inference-time computation — what the paper calls thinking?

They introduced a benchmark called WikiProfile, built through an automated pipeline grounded in web search, and ran it across 13 large language models producing roughly four million responses.

The Five Profiles

Every fact lands in one of five categories, and the taxonomy is the useful part:

• Encoding Failure. The model never learned it. Genuinely absent.
• Recall Failure. Encoded, but unreachable even with thinking.
• Direct Recall. Answered correctly, straight away.
• Recall with Thinking. Wrong on a direct question, correct once the model reasons first.
• Inference without Encoding. Derived rather than remembered.

The Findings

The headline result is that encoding is nearly saturated in frontier models, with GPT-5 and Gemini-3 encoding 95 to 98% of the tested facts.

Almost everything is in there. The shelves are stocked.

But frontier models fail to directly recall 26 to 34% of encoded facts without thinking, and 11 to 12% even with it. Put differently, only about two-thirds of facts are known directly; the remainder need reasoning or stay lost.

So when a frontier model gets a Wikipedia-grade fact wrong, the most likely explanation is not that it never knew. It is that it could not reach what it knows.

A third of what these models know is not available on demand. It is available on request, if you ask in a way that gives the model a route back to it — which is a completely different engineering problem from the one most teams think they have.

Thinking as a Recovery Mechanism

The second finding is the actionable one. Thinking recovers 40 to 65% of encoded-but-not-directly-known facts in thinking-optimised models, cutting the genuinely lost category to a few percent.

A companion paper from the same group, Thinking to Recall — full paper here — to be presented at COLM 2026, isolates why this happens — and the puzzle they start from is a good one.

Consider a question like the year a particular inventor entered a hall of fame. Either the model has that fact in its weights or it does not. No arithmetic, no deduction, no multi-step logic. So why would a reasoning trace help at all?

Their answer, from a series of controlled experiments: generating a reasoning trace unlocks correct answers that are otherwise effectively unreachable. Reasoning is not only a mechanism for working things out. It is a mechanism for exposing the model's own internal memory and expanding what the paper calls its parametric knowledge boundary.

The human analogy the researchers reach for is exact: tip-of-the-tongue. You know the name. You cannot produce it. You talk around it — where you met them, what they did — and it surfaces.

The Language Finding You Should Know About

Adjacent research on factual recall degradation adds a dimension that matters if you operate outside English, and it is not comfortable reading.

Multilingual models systematically show degraded factual recall on non-English queries, and mechanistic probing suggests why: the model encodes the query, switches into an English-centric concept space to perform the recall, and only converts the answer into the target language at the end.

Recall is happening in one language regardless of which one you asked in. If your business runs in Arabic, Urdu, Spanish or anything else, the retrieval bottleneck this article describes is likely worse for you than the headline figures suggest — and the practical mitigation is the same one, applied harder: ground the facts rather than trusting recall, and consider whether retrieval cues in English improve results on your own tests.

Small Models Behave Differently

One more result worth carrying, because it affects model selection.

Smaller models lean much harder on recognition. GPT-5 mini knows more than a quarter of facts only in multiple-choice settings — it can pick the right answer from a list but cannot generate it — a pattern nearly absent in frontier models.

That is a concrete difference in failure mode, not just in quality. A small model given options performs far better than the same model asked an open question. If you are running a cheap model on classification-style work, this is good news. If you are asking one to recall facts unaided, it is a warning.

The Caveats, Stated Properly

I have been enthusiastic, so let me bound the claim before drawing conclusions from it.

WikiProfile is built from Wikipedia-grade facts. That is a reasonable proxy for general knowledge and it is not your proprietary data, your client records or your internal documentation. Nothing here says a model has encoded anything about your business.

Encoding is measured behaviourally — inferred from whether the model can recognise a fact among alternatives — rather than read directly out of the weights. That is a sound experimental design and it is still an inference.

The benchmark was constructed with an automated, LLM-assisted pipeline grounded in web search, which is efficient and introduces its own selection characteristics.

And these are specific models at a specific moment. The models tested will be superseded within months, as everything in this field is.

What survives all four caveats is the structural finding: for general factual knowledge in frontier models, retrieval failure dominates absence. That is the part I would build on.

Why This Should Change What You Do

Here is the practical inversion, and it is where most teams are currently spending money in the wrong direction.

The standard response to a model getting facts wrong is to reach for a bigger model. More parameters, more training data, more knowledge. That instinct assumes an encoding problem.

If encoding is already at 95 to 98%, scaling buys you the remaining two to five percent — and leaves the 26 to 34% recall gap exactly where it was. You would be paying frontier prices to solve a problem the frontier does not solve.

The paper's own conclusion points the same way: further gains in factuality may come less from brute-force scaling of pre-training and more from post-training and inference-time strategies.

Which is, I would gently note, the most direct research endorsement I have yet seen of the argument I have been making since I started writing here: stop chasing the biggest model. The gains you want are not where the marketing says they are.

Upgrading your model to fix hallucination is like buying a bigger library because you cannot find a book. The book is on the shelf. What you need is a better index.

What To Do Instead

1. Turn Thinking On for Factual Work

This is the cheapest available improvement and many teams have it switched off, because reasoning modes cost more per call and the benefit was assumed to be limited to genuinely hard problems.

The research says otherwise. Reasoning recovers a large fraction of otherwise-inaccessible facts even on simple single-hop questions where there is nothing to reason about. If your workload involves factual recall, the thinking budget is buying accuracy, not just capability.

Worth pricing carefully, though — as I wrote yesterday, more output tokens is exactly how a model with an unchanged headline rate becomes 2.32 times more expensive per completed task. Reasoning is a real cost. It is now also a demonstrated factuality intervention, which changes the calculation.

2. Give the Model Better Keys

If retrieval is the bottleneck, prompt design is a retrieval intervention rather than a stylistic one.

Practically: supply surrounding context before asking for the specific fact. Name the domain, the period, the adjacent entities. Ask the model to lay out what it knows about the topic before answering the narrow question. Each of those is a cue that gives the model a route back to something it already holds.

This is the same move as the tip-of-the-tongue trick. You do not try harder. You approach from a different direction.

3. Use Recognition Where You Can

Given that smaller models perform substantially better when identifying a correct answer among alternatives than when generating one unaided, restructure the question where the task allows.

Classification beats open recall. Verification beats generation. If you can turn "what is the X for this?" into "which of these is the X for this?", you have moved the task into the regime where cheap models are strongest — which is also a task-model matching decision, made at the level of the question rather than the model.

4. Ground Anything That Matters

None of the above makes parametric recall reliable enough for claims you will publish or bill for. An 11 to 12% residual failure rate even with thinking is not a number you build a client deliverable on.

For anything consequential, supply the source rather than relying on memory. That is the verification argument I have made repeatedly this month, and this research strengthens it: the failure mode is not the model being ignorant and knowing it. It is the model being confidently unable to reach something it holds — which produces exactly the fluent, plausible, wrong output that is hardest to catch.

5. Re-Test Before You Conclude a Model Cannot Do Something

This one matters for evaluation discipline. If a model fails a factual task in your testing, the correct conclusion is not necessarily that it lacks the knowledge.

Re-ask with thinking enabled. Re-ask with surrounding context. Re-ask as a multiple choice. If any of those succeed, the knowledge was there and your prompt was the problem — and you have just avoided an unnecessary upgrade.

The Bigger Reframe

I think this research changes the mental model of what a language model is, in a way worth sitting with.

The common picture is a very well-read entity that sometimes has gaps and confidently fills them in. The measurement suggests something closer to an entity with a nearly complete library and an unreliable catalogue — one that, when it cannot find the book, writes a plausible summary of what the book probably said.

Those two pictures imply completely different interventions. The first says: add knowledge. The second says: improve access, and never trust unaided recall for anything that matters.

It also reframes a statistic I have cited repeatedly. Why do 80% of executives report no measurable AI ROI? Partly because organisations diagnosed a knowledge problem and bought a knowledge solution — larger models, longer contexts, more expensive tiers — when the constraint was retrieval and prompting, which cost almost nothing to fix.

And it explains something about why agent deployments get rolled back. An agent chaining ten factual steps at a two-thirds direct-recall rate compounds retrieval failure across every hop. The individual model is not the problem. The architecture that assumed reliable unaided recall is.

Gartner still projects $206 billion of AI agent spending this year. A meaningful slice of that will go to bigger models bought to fix a problem bigger models do not fix.

Which is, once more, the systems problem in its purest form. The capability is already in the box. What is missing is the structure around it that reliably gets it out — and that structure is prompting discipline, grounding, and knowing which questions to ask in which shape. None of it is purchasable.

Frequently Asked Questions

Does this mean hallucinations are fixable?

Substantially reducible rather than fixable. The research shows a large share of factual errors come from retrieval failure, and that reasoning recovers 40 to 65% of those. But a residual 11 to 12% of encoded facts stay unreachable even with thinking, and genuine encoding gaps still exist. Treat it as a large, cheap improvement available now — not as a solved problem.

Should I turn on thinking or reasoning modes for everything?

For factual work, test it — the accuracy case is now evidence-based rather than intuitive. For classification, routing and transformation, the benefit is smaller and the token cost is real. The right answer is a measurement on your own workload rather than a blanket setting.

Does this apply to my company's internal data?

No, and this is the most important limit. The study measures Wikipedia-grade general facts that a model plausibly saw in training. Your internal documents, client records and proprietary processes are almost certainly not encoded at all — that is a genuine empty shelf, and the answer there is retrieval augmentation, not better prompting.

Is a smaller model good enough then?

For the right shape of task, often yes. Smaller models are notably stronger at recognition than at unaided generation — GPT-5 mini knows over a quarter of facts only in multiple-choice settings. Structure the work as choosing among options rather than recalling from nothing and cheap models become far more viable.

How do I know if my model knows something but cannot recall it?

Ask it three ways: directly, with reasoning enabled, and as a multiple choice with the correct answer among plausible alternatives. If it fails the first and passes either of the others, the knowledge is encoded and your prompt was the constraint. That is a five-minute test and it will change how you write prompts.

What is the single most useful takeaway?

That the next time your AI gets a fact wrong, upgrading the model is probably the wrong response. Re-ask with reasoning on and more surrounding context first. It costs nothing to try and, on this evidence, works most of the time.

Related Reading

Stop Chasing the Biggest Model — the argument this research directly supports: the gains you want are not where scaling puts them.

You Don't Have an AI Problem, You Have a Systems Problem — capability already in the box, structure missing around it.

The Automation Ratio Is the Only Metric That Predicts Survival — the measure that tells you whether your prompting changes actually improved anything.

80% of Executives Report No Measurable AI ROI — the spending that went to a knowledge solution for a retrieval problem.

74% of AI Agent Deployments Get Rolled Back — what compounding retrieval failure looks like across a ten-step chain.

You Don't Need an Agent, You Need a Rule — the four-category classification, and why deterministic steps never have a recall problem.

The No-Code Automation Workflow Guide — building workflows that ground factual claims instead of trusting unaided recall.

About the Author

Hamza Baig is the founder of Hexona Systems, an AI automation agency serving clients across six continents, and the AI Automation Institute, a community of more than 40,000 entrepreneurs building with AI.

He has been featured in the GHL Top 50, Yahoo Finance and Brainz Magazine, and writes regularly on automation architecture, agent governance and the operational realities of AI deployment.

Read more analysis on the Hamza Automates blog, or get in touch to discuss an automation build.

Follow @hamza_automates on Instagram for daily automation breakdowns.

Note: this article summarises published academic research for a general business audience; figures are drawn from the papers and the authors' own summaries, and technical detail is simplified. Consult the primary papers linked above for precise claims and methodology.