This project explores how to design a conversational AI system when sensitive information has to remain within a client’s environment.
The architectural question is how to separate orchestration from client-owned storage, embeddings, and model processing while keeping the system practical to deploy and operate.
The constraints
Data boundaries affect the whole design: where documents are stored, where retrieval runs, what enters a model request, and what ends up in logs. Cost and deployment complexity also need to be considered alongside those boundaries.
Decisions to document
- Which components need access to which data.
- Where retrieval and model processing should run.
- What information can cross a service boundary.
- How logging and debugging can work within those limits.
- How deployment options change cost and operational effort.
Next for this write-up
I’ll describe the architectural problem using a generic diagram, explain the alternatives considered, and identify which decisions were proposed or implemented.