The knowledge assistant answers from last year's policy even though a replacement is available. Both passages contain the right keywords, but only one is authoritative for the current question.
For teams testing document assistants, version and effective-date rules belong in the evaluation. Text similarity alone cannot decide which policy applies.
What makes a policy document outdated?
A document can be superseded, expired or applicable only to a past period or audience. The newest upload is not necessarily the newest effective policy, and an older document may still matter for a historical question.
Have the organisation define source precedence and effective scope. The ingestion system should preserve the metadata and relationships required to apply that rule.
Create a policy-version test family
Use fictional policies with deliberately different dates and conditions. The proposed cases check meaning rather than a simple latest-file shortcut.
| Question or source state | Expected evidence |
|---|---|
| Current-policy question | Applicable approved current source |
| Historical-period question | Relevant historical version and clear scope |
| New upload with future effective date | Existing rule remains applicable until defined transition |
| Older file uploaded again | Upload time does not override policy authority |
| Conflicting versions without precedence | Uncertainty surfaced for owner review |
Check the answer's qualifications and citation. Selecting the current document while quoting a superseded cached passage still produces the wrong result.
Test invalidation across derived content
Updates may affect search entries, summaries, cached passages and stored answer context. Verify which derivatives are refreshed or retired and how the system behaves while the change is still processing.
The source version guide helps preserve provenance. The company evaluation-set guide helps retain both current and historical cases without relabelling them inconsistently.
Separate content authority from model confidence
A model may express the old rule fluently because it appears in several passages. Repetition and confident wording do not establish the organisation's approved source hierarchy.
My knowledge platform contribution provides related system context, not a claim that this policy lifecycle is implemented in every integration. A new organisation needs its own authority rules and reviewed fixtures.
After a document update, rerun the relevant questions before relying on a generic ingestion-complete status. Record the source version used for each result so a later reviewer can explain why an answer changed.
Should the latest uploaded file always win?
No. Effective date, approval, audience and supersession rules can differ from upload order.
Are old documents always irrelevant?
They can support historical questions when their scope is explicit and access is appropriate.
What should we test first?
Use a future-dated replacement and an older applicable policy in the knowledge versioning review.