Removing access to a document is incomplete if a knowledge assistant can still quote it from a cached answer. Test permission changes across the entire answer path, including old conversations and the destination behind a citation.
This is an engineering acceptance exercise, not a substitute for your organisation's access policy. Start by deciding what revocation means for previously displayed material, retained conversation history, and new requests. Different products may make different retention decisions, but those decisions need explicit owners.
Enumerate the places protected text can persist
Trace the source connector, search index, retrieval cache, answer cache, conversation store, export path, and source viewer. Write down which component checks the current user and which consumes a previously computed decision. A document identifier is not proof that the current reader remains authorised.
The Next.js authentication guide separates authentication from authorisation. Apply that distinction here: a valid session establishes identity, while access to a particular source still needs its own decision.
Run a fictional department-transfer exercise
Create a test user who can initially read an invented finance memo. Ask a question whose answer includes a distinctive, non-sensitive canary phrase from that memo. Record the source revision, conversation identifier, and answer timestamp.
Move the user to a department without access. Repeat the question in a fresh chat, an existing chat, another browser session, and after a cache refresh. Try opening the old citation directly. The expected result for new retrieval is no protected passage and no answer derived from that passage.
For old conversation history, compare behaviour with the policy you chose. If the product deliberately retains historical answers, make the exposure visible in the risk review. Do not report revocation as complete simply because fresh search now returns nothing.
Add a user who still has access. Their successful result distinguishes a targeted revocation from an outage that blocks everyone. Add a second restricted document to check that a fix has not accidentally broadened access elsewhere.
Make propagation delay a release decision
Measure the time between the source permission change and each relevant boundary enforcing it. Define the maximum acceptable interval before launch. Where immediate revocation is required, a periodic synchronisation job alone may not satisfy the policy.
Shorter cache lifetimes increase repeated work; long-lived caches increase the amount of state that can outlast a permission decision. Cache invalidation and request-time permission checks are design choices to test together. Neither a hidden link nor a polite refusal instruction proves protected text was excluded from model context.
The company document integration guide describes the broader workflow. Related experience appears in the Nucleus case study.
Questions about revocation tests
Is checking the answer text enough?
No. Inspect retrieved context and citation access as well. A model may receive restricted text without repeating the exact canary phrase.
Should tests use actual finance documents?
Prefer controlled fixtures with realistic permission relationships. Keep sensitive production content out of shared evaluation reports.
What evidence should block release?
Any protected passage reaching an unauthorised request should trigger investigation under your agreed policy. Preserve the trace and bring the boundary map to a technical review.