ProdAIStack
Hands-on lab

Production RAG Architecture Lab

Build and test a production RAG architecture with versioned ingestion, hybrid keyword and vector retrieval, RRF fusion, authorization filtering, reranking, context assembly, citation validation, evaluation metrics, and release gates.

TypeScriptNode.jsadvancedTechnically verified
View on GitHub Read companion article
Production AI architecture showing an application service, model gateway, LLM provider, reliability controls, observability and evaluation

What you will build

Build and test a production RAG architecture with versioned ingestion, hybrid keyword and vector retrieval, RRF fusion, authorization filtering, reranking, context assembly, citation validation, evaluation metrics, and release gates.

01

RAG Application Contracts

Provider-neutral document, chunk, evidence, citation, and grounded-answer contracts.

02

Idempotent Ingestion and Versioning

Content hashes, source versions, replay safety, upserts, and deletes.

03

Versioned Chunker

Deterministic chunking interface that emits version metadata.

Prerequisites

  • Production AI System Design: From Model Call to Reliable Product
  • Comfort with APIs and backend services
  • Basic understanding of embeddings and vector similarity
  • TypeScript familiarity for the companion Lab

Capabilities you will leave with

  • RAG Application Contracts
  • Idempotent Ingestion and Versioning
  • Versioned Chunker
  • Evidence Provenance Metadata
  • Keyword Retriever
  • Vector Retriever
Build sequence

Project modules

RAG Application Contracts

Provider-neutral document, chunk, evidence, citation, and grounded-answer contracts.

src/contracts.ts
View source

Idempotent Ingestion and Versioning

Content hashes, source versions, replay safety, upserts, and deletes.

src/ingestion/idempotency.ts
View source

Versioned Chunker

Deterministic chunking interface that emits version metadata.

src/ingestion/chunker.ts
View source

Evidence Provenance Metadata

Stable source, chunk, URI, location, version, and authorization metadata.

src/ingestion/provenance.ts
View source

Keyword Retriever

Deterministic lexical retrieval abstraction for the local harness.

src/retrieval/keyword.ts
View source

Vector Retriever

Vector retrieval abstraction with a deterministic local implementation.

src/retrieval/vector.ts
View source

Reciprocal Rank Fusion

Fuse keyword and vector ranked lists with deterministic RRF.

src/retrieval/rrf.ts
View source

Hybrid Retrieval Pipeline

Generate lexical/vector candidates and combine them before reranking.

src/retrieval/hybrid.ts
View source

Authorization-Aware Retrieval

Tenant and ACL filtering before candidates can enter answer generation.

src/retrieval/authorization.ts
View source

Deterministic Reranker Boundary

Reranker interface with a deterministic local scoring implementation.

src/retrieval/reranker.ts
View source

Deterministic Context Builder

Deduplicate, diversify, order, and enforce a context budget while preserving evidence IDs.

src/context/context-builder.ts
View source

Citation Builder

Construct citation contracts from retrieved evidence metadata.

src/citations/citation-builder.ts
View source

Citation Validator

Reject citations that do not map to evidence in the current retrieval context.

src/citations/citation-validator.ts
View source

End-to-End RAG Pipeline

Provider-neutral orchestration for filtering, retrieval, fusion, reranking, context assembly, generation, and citation validation.

src/pipeline/rag-pipeline.ts
View source

Retrieval Evaluation Metrics

Recall@k, MRR, and nDCG helpers for deterministic retrieval evaluation.

src/eval/retrieval-metrics.ts
View source

Citation Metrics

Citation precision and coverage calculations.

src/eval/citation-metrics.ts
View source

RAG Evaluation Release Gate

Fail CI when retrieval, citation, authorization, freshness, latency, or other configured thresholds regress.

src/eval/release-gate.ts
View source
Measured behavior

Experiment results

Experiment 01

Keyword vs Vector vs Hybrid Retrieval

PASSED

Compare keyword, vector, and RRF-fused hybrid retrieval on the same evaluation queries.

Keyword
Recall At3: 1 · Mrr: 1 · Ndcg At3: 1
Vector
Recall At3: 1 · Mrr: 1 · Ndcg At3: 1
Hybrid
Recall At3: 1 · Mrr: 1 · Ndcg At3: 1
Local test harness

Strategies were evaluated on the same query set.

Experiment 02

Chunk Size and Overlap Sensitivity

PASSED

Evaluate multiple deterministic chunk configurations on the same source/query set.

Recall At3
1
Duplicate Context Rate
0
Average Context Size
38
Local test harness

No single chunk configuration is claimed as universal.

Experiment 03

Incremental Value of Reranking

PASSED

Compare fused ranking with deterministic reranking over the same candidate set.

Fusion Mrr
1
Reranked Mrr
1
Rerank Latency Ms
0.16
Local test harness

Incremental quality may be unchanged.

Experiment 04

Context Budget Trade-off

PASSED

Vary candidate count and context budget while keeping the evaluation corpus fixed.

Variants
Budget: 20 · Context Units: 20 · Evidence Count: 2, Budget: 40 · Context Units: 38 · Evidence Count: 3, Budget: 80 · Context Units: 38 · Evidence Count: 3
Duplicate Context Rate
0
Local test harness

Units are whitespace-separated estimates, not provider billing tokens.

Experiment 05

Authorization Leakage Test

PASSED

Use multi-tenant test data and queries that would otherwise match unauthorized content.

Unauthorized Retrieval Count
0
Local test harness

Authorization filtered candidates before ranking and context.

Experiment 06

Citation Precision and Coverage

PASSED

Validate citations against retrieved evidence and expected claim-source relationships.

Citation Precision
1
Citation Coverage
1
Local test harness

Metrics are deterministic evidence-ID checks.

Experiment 07

Freshness and Delete Propagation

PASSED

Update and delete source documents, rerun ingestion, and verify serving-index behavior.

Stale Result Count
0
Deleted Result Count
0
Ingestion Lag Ms
0.409
Local test harness

Timing is local harness timing.

Local setup

Run locally

terminal
git clone https://github.com/sarvab-dev/prodai-stack-labs
cd prodai-stack-labs/production-rag-architecture
npm install
npm test
Technically verified
Continue learning

Connect the implementation to the architecture

Continue in Production AI Foundations.

Review the decisions in Production RAG Architecture: Ingestion, Retrieval, Citations, and Evaluation.