Identify approved sources and owners
List active documents, storage locations, owners, and update schedules. A process owner must resolve conflicting procedures instead of leaving the model to choose arbitrarily. Preserve versions and effective dates. Removed documents must also be removed from indexes and relevant caches. Start with a limited collection. Retrieval quality depends on readable, maintained material and a clear distinction between active documents and archives.
Apply permissions during retrieval
A user who cannot read a document should not obtain its summary through AI. Map identities and source permissions to retrieval and filter before content reaches the model. Hiding a source link after generating an answer is insufficient. Microsoft documents document-level controls and retrieval filtering for RAG; implementation depends on the platform, data structure, and available identity integration.
Distinguish missing answers from denied access
Missing information can prompt clarification or escalation to a process owner. Unauthorized requests should not reveal protected titles or excerpts. Design useful messages without leaking source details. Test different teams, changed permissions, and sessions with older cached context. Permissions are not only an interface feature: retrieval and output must follow the same boundary.
Check references and answer content together
A linked answer may still lack support. Check whether the cited material supports the statement, uses an applicable version, and contains a locatable relevant passage. Test single and multiple sources, ambiguous questions, and absent answers. Record whether the assistant acknowledges uncertainty. Assess retrieval separately from answer writing to identify what needs improvement.
Treat documents as sources rather than system instructions
Retrieved material can contain text that attempts to change assistant behavior. It must not grant new authority or override application rules. Limit available tools, separate data from instructions, and test out-of-scope commands embedded in sources. Read-only access often meets an initial knowledge-assistant need. System changes and message sending are additional scope requiring explicit authorization design.
Prepare an evaluation matrix
Record the question category, test user, allowed sources, expected answer, and response to access denial. Repeat the matrix after document or model changes. KODE can help design retrieval, access controls, evaluation, and update guidance. An initial discussion only needs document volume, user categories, and retrieval needs, without confidential sources.