Factuality Evaluation
Factuality evaluation is the measurement of how often a language model's statements are true and supported, checked against known answers, source documents or trusted references by test sets, automated grading and human review. It turns "the model sometimes makes things up" into a rate that can be tracked, compared between models and used as a release threshold.
- Also known as
- Factuality eval, Factual accuracy evaluation, Hallucination evaluation
- Used at
- model development, vendor selection, pre-launch testing of AI products and ongoing monitoring of deployed assistants
- Two kinds
- world-knowledge factuality (is the claim true?) and grounded factuality, often called faithfulness (is the claim supported by the documents the model was given?)
- Public benchmarks
- TruthfulQA (2021, 817 questions in 38 categories built around common misconceptions); FActScore (2023, splits long answers into atomic facts and scores the share a reliable source supports); SimpleQA (2024, short questions graded correct, incorrect or not attempted); FACTS Grounding (2025, long answers checked against a supplied document by judge models)
- Grading
- exact-match answers, human reviewers, and increasingly "judge" models that compare each claim with a reference, themselves checked against human labels
- Scoring
- a 2025 paper argued that evaluations rewarding only correct answers train models to guess rather than admit uncertainty
How factuality evaluation works in practice
A team deploying a model builds its own test set before trusting any public score. For a fintech support assistant that means several hundred real customer questions with answers verified against the fee schedule, product terms and policies, plus questions the assistant ought to decline. Each run of the system produces answers; each answer is broken into individual claims; each claim is checked against the source — by a person for a sample, by a judge model for the rest. The result is a set of rates: share of claims supported, share contradicted, share of out-of-scope questions correctly declined.
Those rates are only useful if they are stable and specific. A general benchmark says little about whether a model quotes the right FX margin from a firm's own tariff. A test set that never changes can be overfitted by repeated prompt tweaks. A judge model has its own error rate, so teams calibrate it against human labels and re-check a sample at each release.
The results feed decisions: which model to use, whether a release goes ahead, which question types go to a human. A mature team treats a drop in factuality like a failed software test — it blocks the release.
For a buyer of an AI product, the evidence is the evaluation itself: the test set and how it was built, the rates over time and the controls they trigger, not a leaderboard score quoted in a pitch deck.
Example
An insurer's AI assistant answers policyholders' questions about their cover. The team builds 600 test questions from past enquiries, each with an answer verified against the policy wording, including 80 the assistant should decline. The first version has 91% of its claims supported, contradicts the policy in 4% and correctly declines 55% of out-of-scope questions. After adding retrieval over the policy wordings and an instruction to cite a clause for every statement about cover, supported claims rise to 98.2%, contradictions fall to 0.6% and correct refusals reach 90%. A judge model grades each run; humans re-grade a 100-answer sample and agree with it 94% of the time. The release rule: no version ships if contradictions exceed 1%.
Common mistakes
- Treating a public benchmark score as evidence of accuracy on the firm's own documents. - Scoring only correct answers, which rewards confident guessing over "I don't know". - Relying on a judge model without measuring how often it agrees with human reviewers. - Reusing one fixed test set until the system is tuned to it. - Checking whether a claim is true but not whether the cited source actually says it. - Evaluating once before launch and never again as models and documents change.