An AI answer can include a convincing explanation and a row of links while still getting the central claim wrong. Checking the answer means opening those links, identifying what each source actually establishes and comparing it with the sentence that relies on it. This guide provides a practical way to do that for reports, product comparisons and everyday research.
The problem is recognised in NIST’s Generative AI Profile, published in July 2024. It discusses confidently presented errors, including invented logic and citations. A source list is therefore a starting point for verification. Its presence alone cannot establish the accuracy of the text above it.
Turn the answer into claims you can check
Begin with the statements that could change a decision: a product supports a feature, a fee applies, a deadline has moved, or a study found a particular result. Separate these from suggestions and interpretation. A sentence saying a product is suitable for a team may combine a factual feature claim with a judgement about that team’s needs.
Write the important statements in a short checklist. Preserve their conditions: location, date, subscription tier, version and the group being discussed. Removing a condition while summarising can create a broader claim than the source supports. A feature available to enterprise customers in one country may not be available on the plan being evaluated.
For a long report, checking every sentence with equal effort is rarely practical. Prioritise claims whose failure would change the recommendation, then names, dates, quantities and quotations. That prioritisation is a proposed workflow, not a claim that a fixed number of checks guarantees accuracy.
Find the original page behind the citation
Open the cited document and identify who published it. A vendor’s documentation can establish what the vendor says its product supports. A regulator’s notice can establish the content and date of that notice. An independent measurement may support a performance claim, provided its method and conditions are clear.
A news story quoting a release can help locate the original announcement, but several stories based on the same release do not become several independent measurements. Follow the source chain until you can see where the material assertion originated. If the original is unavailable, keep that limitation visible in your working notes.
This is especially useful for charts and tables. The source might describe a percentage increase while the answer treats it as an absolute share. It might report transactions while the answer calls them users. Our UPI analysis includes the underlying numbers and calculation method precisely because a chart’s shape does not explain its definitions.
Match the sentence to the evidence
Finding the document is only the first check. Locate the passage, table or figure that is supposed to support the claim. Read enough around it to identify exclusions and caveats. A search result excerpt may omit the sentence that limits the finding to a pilot or a narrow population.
A practical review sequence. The diagram describes a checking method, not a benchmark result or a guarantee of correctness.
Suppose an answer says a service offers instant withdrawals. The documentation might instead describe an instant internal transfer followed by a bank payout. Both can appear under a broad payments heading, but they answer different questions. The stablecoin redemption explainer shows why an on-chain confirmation and a bank receipt need separate evidence.
Quotation marks deserve particular care. Check that the quoted words appear in the cited source and have not been assembled from different passages. If you cannot locate them, do not retain them as a quotation merely because the surrounding idea seems plausible. Paraphrasing also needs accurate support; removing quotation marks does not solve a factual mismatch.
Check when the source was true
A product page can change while keeping its address. A notice can announce a future effective date. A research paper may have a later correction. Record the publication or update date where available, and distinguish it from the date of the event being discussed.
For a historical question, use evidence that was available by the relevant time. Reading today’s documentation cannot by itself prove what a product supported last year. An archived version, dated release note or original announcement may be needed. Where the history cannot be established, narrow the statement to what the accessible record supports.
For a current decision, an old source can still explain the mechanism while failing to establish today’s limits or price. The age of the source is not a universal quality score. Its usefulness depends on whether the particular fact is likely to have changed.
Keep a compact evidence record
| Item | What to record |
|---|---|
| Claim | The precise statement, including conditions |
| Source | Publisher, page title and original URL |
| Evidence | Section, passage or table supporting the claim |
| Time | Publication or update date and relevant event date |
| Result | Supported, partly supported, contradicted or unresolved |
An unresolved result is useful. It tells the next reader where more work is needed. A partly supported result often suggests a smaller, more accurate sentence: a rollout has started, for example, rather than a feature being available to everyone. Keep the distinction between what the source states and what you infer from it.
Avoid copying private work documents into a tool simply to speed up this process. The review method can be applied to public sources and locally held notes, and the permissions for sharing internal material are a separate consideration. A colleague who can read the original document may be better placed to check a sensitive claim.
Revise the answer after checking
Replace unsupported precision with the level of detail the evidence permits. Remove claims that cannot be established, correct numbers and dates, and make important qualifications part of the sentence. If the corrected evidence changes the recommendation, revise the recommendation too. Fixing footnotes while keeping an unsupported conclusion leaves the main problem intact.
An AI tool can help organise the claims or suggest places to look, but asking it whether its earlier answer was correct does not independently verify that answer. The decisive work is the comparison with the underlying evidence. The finished result should let another reader retrace the important steps without having to trust the confidence of the prose.
Questions
Are AI answers with citations always reliable?
No. A citation can be invented, point to the wrong document or support a narrower statement. Open the source and compare it with the claim.
What should I check first in a long answer?
Start with claims that affect the decision, then quantities, dates, names and quotations. Preserve conditions such as country, product version and eligibility.
Can the AI tool verify its own answer?
It can help organise a review, but its confidence is not independent evidence. Verification requires examining the original sources and resolving mismatches.
Sources
- NIST: Artificial Intelligence Risk Management Framework, Generative Artificial Intelligence Profile, NIST AI 600-1, July 2024, particularly sections on confabulation and information integrity.
- NIST publication record, 26 July 2024.
The checking sequence and examples are this article’s practical guidance. Continue with AI & Work, or send the editors a correction.
