Removing a document from its source should stop it supporting new AI answers under the defined policy. Delete or invalidate its chunks, metadata, previews, and dependent caches rather than removing only the main file record.
Index deletion is a propagated state change
A source removal can mean deletion, withdrawal, lost connector access, or a permission change. Those events are not interchangeable. The connector should preserve the source’s documented semantics before deciding what derived content to remove.
For a proposed pipeline, use stable source identity and a durable removal marker or equivalent state. Apply the change to all derived resources and prevent delayed imports from recreating the removed content without checking the current source state.
Microsoft Graph’s delta overview describes discovering changed and deleted entities for supported resources. Use the actual connector’s documented change contract; do not assume every source exposes identical deletion evidence.
Replay a fictional removal race
A policy document is deleted while an older extraction job is running. The removal event clears its current index entries, but the extraction finishes later. The worker must not reinsert the obsolete chunks solely because its own job succeeded.
The test tracks source version, removal state, job revision, and final eligible passages. A second query should show that the assistant cannot cite the removed source as current support, while respecting any approved historical-evidence policy.
| Removal path | Required assertion |
|---|---|
| Normal source deletion | Derived search evidence removed or invalidated |
| Late extraction completes | No unintended resurrection |
| Repeated removal event | Idempotent cleanup |
| Permission removed, file still exists | Correct scoped treatment |
| Cache contains prior answer | Dependency invalidation follows policy |
Use a unique synthetic passage marker and inspect search, previews, suggestions, and answer caches. Absence from one query result does not prove comprehensive removal.
Keep uncertain source states explicit
If the connector cannot tell deletion from temporary access loss, apply the defined safe policy and route the discrepancy to its owner. Do not claim permanent source deletion from one failed fetch.
Full cleanup can take time across storage layers. State the supported propagation boundary and monitor oldest unresolved removals. Faster indexing is not useful if it repeatedly restores withdrawn or inaccessible content.
This deletion design is proposed. The voice integration case study describes related retrieval work without claiming these guarantees. Pair it with deleted-content cache tracking.
Removed-document FAQ
Is deleting the vector row enough?
Not necessarily. Review metadata, alternate indexes, previews, caches, and pending jobs.
Does a failed source fetch prove deletion?
No. Distinguish temporary outage and permission loss using the source’s supported evidence.
What proves completion?
Verify the approved derived paths and record any unresolved copies or propagation limits explicitly.