An employee asks “How do I get approval?” and the assistant selects equipment checkout when the employee meant time off. A top-ranked result is not useful when the question's intended workflow remains unresolved.
For teams testing internal search, include questions whose terms can refer to different workflows. The assistant should use appropriate context or ask a focused clarification rather than confidently select a convenient interpretation.
What makes an employee question ambiguous?
Ambiguity can come from shared vocabulary, missing department context, several possible time periods or an unclear resource. The relevant distinction depends on the organisation's actual documents and roles.
User context can help, but it should come from appropriate trusted information and remain within the intended privacy and permission boundaries. Do not expose restricted policies merely to list every possible interpretation.
Build a purposeful ambiguity set
Use fictional company documents with deliberately overlapping terms. The proposed matrix distinguishes useful clarification from unnecessary questioning.
| Question condition | Expected behaviour |
|---|---|
| Several plausible workflows | Ask the clarification that changes retrieval |
| Trusted relevant context available | Apply it with appropriate scope |
| One interpretation is access-restricted | Avoid disclosing protected content |
| Question names the workflow clearly | Answer without redundant clarification |
| Caller or user corrects interpretation | Invalidate stale results appropriately |
Review the clarification itself. Asking five broad questions can be less useful than one concise question about the missing distinction.
For a concrete fixture, give the authorised test user access to two fictional policies: requesting time off and borrowing equipment. Ask only “How do I get approval?” without additional context. A useful clarification is “Is this about time off or borrowing equipment?” Then repeat with “How do I request time off?” The second question should select the time-off policy without the same clarification.
Separate search failure from language uncertainty
Inspect the source set and retrieved candidates. If the question is clear and the correct policy is available, returning the wrong document is a retrieval failure, not evidence that the user should always provide more detail.
The company question-set guide helps balance ambiguous, supported and unsupported cases. The missing-evidence guide prevents another common confusion: no available answer is different from several possible meanings.
Test the final answer after clarification
Keep the clarified intent associated with the subsequent search and answer. A late result for the first interpretation should not reappear after the user selected another workflow.
My knowledge platform contribution supplies related system context, not a guarantee of behaviour on every company's terminology. A new deployment needs its own vocabulary, role and source fixtures.
Record what information resolved the ambiguity and whether the final citation supports the scoped claim. This gives the team a useful repair target without turning every failed search into a longer mandatory form.
Should internal search always ask for the department?
Only when that information is necessary and appropriate for the task. Clear questions should not receive redundant clarification.
Can the assistant list restricted policies as possible meanings?
Preserve access rules during clarification as well as final retrieval and answer generation.
What should we test first?
Use one term shared by two fictional workflows in the internal search review.