An AI answer cites a document containing the same keywords, but the passage does not support the sentence the model wrote. Citation presence is not the same as citation support.
For teams evaluating knowledge assistants, review the exact claim against the actual cited source. A plausible document title or nearby topic can create confidence without supplying the necessary evidence.
What does a supporting citation establish?
It establishes that an authorised source passage supports the relevant claim, including its conditions, scope and qualifications. The support can be partial, absent or contradicted even when the document is topically relevant.
Break multi-part answers into reviewable claims. One passage may support a service policy but not the additional response-time promise placed in the same sentence.
Use a claim-to-source review matrix
Create fictional policies with known statements and exceptions. The proposed evaluation categories separate different failure types.
| Citation condition | Review result |
|---|---|
| Exact relevant rule with conditions | Supported within stated scope |
| Same topic, different rule | Relevant document but unsupported claim |
| Rule missing its exception | Overstated or incomplete support |
| Source contradicts answer | Contradicted claim |
| Source unavailable to the user | Access or provenance problem |
Keep the actual cited version and location. A later document update should not silently replace the evidence used for an earlier evaluation result.
Review retrieval and generation separately
If the supporting passage was never retrieved, inspect the search and ingestion path. If it was available but the answer added an unsupported claim, inspect answer construction and source-use rules.
A language model can help flag potential mismatches, but it should not be the only unreviewed authority certifying its own output. Use competent human review for consequential claims and record disagreement.
The source-linkage guide helps reviewers reach the exact evidence. The footnote extraction guide covers qualifications that can change the claim.
Test difficult support relationships deliberately
Include answers with several claims, an outdated rule and a passage that sounds relevant but lacks the requested fact. Inspect whether the system narrows the answer or states the evidence gap rather than attaching a convenient citation.
My knowledge platform contribution provides related system context, not a claimed citation-support score. A new evaluation needs its own reviewed examples and stated limitations.
Report supported, unsupported and unresolved claims with clear denominators. A high share of answers containing links says little about whether those links justify the actual statements.
Does a citation prove the answer is correct?
No. Inspect the exact source relationship, relevant qualifications and authorised version.
Can one source support every sentence in a long answer?
Only when the content actually establishes each claim. Review multi-part answers at a useful claim level.
What should we evaluate first?
Use a plausible but non-supporting passage in the knowledge answer review.