Jazmine RAG Architecture — Technical Deep Dive

This memo describes the architecture behind a private enterprise RAG assistant built for secure document search, grounded answers, and department-aware access.

Context

The system needed to answer internal knowledge questions without exposing sensitive documents to uncontrolled external services. The product goal was fast knowledge access with enough source visibility for users to trust and verify the answer.

System architecture

1. 01Document intake
2. 02Parser
3. 03Chunker
4. 04Metadata enrichment
5. 05Embedding worker
6. 06Vector store
7. 07Retriever
8. 08Access filter
9. 09Context builder
10. 010Local LLM
11. 011Cited response
12. 012Feedback capture

The architecture separated offline ingestion from query-time answer generation. This kept expensive parsing and embedding work out of the user path and made it possible to reprocess documents as metadata improved.

Key design decisions

• Keep source citations visible in the answer UI.
• Treat department metadata as part of retrieval authorization.
• Store document metadata with each chunk.
• Prefer conservative refusal when context is insufficient.
• Design for local model latency rather than assuming cloud-model speed.

AI workflow design

The workflow used semantic retrieval to collect candidate chunks, filtered by metadata and access scope, then assembled a bounded context for the local model. The prompt asked the model to answer only from supplied context and to expose uncertainty when context was weak.

Data flow

Documents entered the ingestion pipeline through a controlled folder or upload path. Parsers extracted text, the chunker split content into retrieval-sized segments, and each chunk received metadata such as document name, source path category, department, and timestamp.

At query time, the retriever embedded the user question, selected candidate chunks, applied filters, and passed the final context to the LLM for response generation.

Evaluation approach

Evaluation focused on answer grounding and retrieval usefulness:

• Did the retrieved chunks contain the information needed?
• Did the answer cite the correct source?
• Did the model avoid unsupported claims?
• Did users accept the answer as useful in the workflow?
• Did latency remain acceptable for repeated use?

Failure modes

• Relevant documents exist but are not retrieved.
• Retrieved chunks contain partial context and cause a misleading answer.
• Source documents are stale or duplicated.
• Department metadata is missing or wrong.
• The model combines sources in a way the documents do not support.
• Latency makes users abandon the workflow.

Security/governance

The system was designed around data boundary control, department-aware filtering, and auditable retrieval behavior. In a hardened deployment, authentication and RBAC would be enforced before retrieval, and query logs would avoid storing sensitive raw prompts unless explicitly approved.

Deployment constraints

Local deployment changed the tradeoffs. Model size, hardware availability, container resources, and response latency mattered. The design favored a modular pipeline so model, vector store, and worker choices could be changed without rewriting the product.

Limitations

The system is only as good as the source corpus and metadata. RAG reduces hallucination risk but does not eliminate it. Users still need source visibility and review for important answers.

What I would improve next

• Add a dedicated retrieval evaluation set.
• Add reranking for high-value queries.
• Build document freshness and duplicate detection.
• Track citation coverage and unsupported-answer rates.
• Add observability for empty retrieval, slow answers, and repeated weak questions.