01 /Tag — Knowledge Retrieval
13 articles in this topic.
Build knowledge-assistant release gates around permission boundaries, critical task failures, reproducible source evidence, and a clear rollback decision.
Choose RAG benchmark documents by workflow, format, age, permissions, and failure risk so clean policy pages do not hide retrieval weaknesses.
Compare knowledge-assistant versions with paired cases, fixed source snapshots, repeated runs, and separate reporting for critical regressions.
Build a citation-version audit that checks document identity, effective dates, source spans, and the actual destination a reader opens.
Evaluate knowledge assistants with claim-level evidence, task completion, permission tests, and explicit abstention cases rather than polished language.
Evaluate table-based retrieval by preserving row labels, column headers, units, footnotes, and revisions before judging the final numeric answer.
Keep human evaluation decisions reproducible with stable test identifiers, immutable run records, source revisions, and explicit reviewer disagreement.
Measure completeness with required evidence-backed facts, explicit optional context, and penalties for unsupported additions using a worked policy example.
Interpret citation reopens using task context and reason codes so healthy verification is not confused with incomplete or untrustworthy AI answers.
Review multi-document AI answers by separating supported facts from inferred joins, mismatched scopes, and conclusions that no cited source establishes.
Test permission changes across retrieval, cached answers, conversation history, and citation destinations with a repeatable before-and-after access exercise.
Test conflicting knowledge sources with explicit authority rules, effective dates, and unresolved cases instead of allowing the assistant to pick a convenient passage.
Turn unanswered knowledge questions into owned documentation work by distinguishing source gaps from retrieval, permission, and ambiguity failures.