A scalable and reliable RAG system built with intelligent vector search, advanced embeddings, precision
reranking,
and modern LLMs to deliver accurate and consistent knowledge
retrieval and generation, even under high workloads.
High-performance modules engineered for maximum speed, security, and contextual accuracy.
Ultra-fast offline vector database operating directly on persistent local disk for sub-20ms semantic similarity searches.
Mitigates context loss. Retrieves Top candidates and reranks precisely utilizing Jina AI multilingual cross-encoder transformers.
Powered by high-speed commercial LLMs natively orchestrated via LangChain pipelines delivering fast, grounded responses.
Experience zero-latency UX. Generator tokens stream asynchronously from the FastAPI API into the UI in real time.
Seamlessly toggle between local disk storage and Cloud Chroma DB clusters via a single configuration variable.
Built for production. Docker-ready, lightweight Python ASGI web-servers designed for Azure Container Apps & cloud VMs.
Comprehensive developer guides and technical specifications for interacting with our RAG and Semantic Search infrastructure. Below you will find structured walkthrough details designed to streamline ingestion, vector search, cross-encoder reranking, and generation.
Multi-format document loading (PDF, DOCX, TXT) with automatic text extraction, 500-token character chunking, and 50-token overlap.
Document chunks are embedded via Jina AI embeddings (`retrieval.passage`) and indexed directly in local ChromaDB storage.
Vector similarity search retrieves candidate chunks, followed by Jina Multilingual Cross-Encoder reranking for top relevance.
Context-grounded response generation via LLM streamed token-by-token directly to the browser via Server-Sent Events (SSE).
Open the interactive chat interface to query uploaded documents in real time.
FastAPI Router specifications, custom endpoints, schema validation logic, and SSE controller details. All endpoints accept and return structured JSON schemas with full CORS support.
Synchronous RAG pipeline query execution returning context sources and generated response.
Server-Sent Events (SSE) stream returning metadata and tokens as they are generated.
Uploads documents (PDF, DOCX, TXT), extracts text instantly, and indexes in background.
Fetch, rename, or delete chat sessions and message history stored in SQLite WAL database.
LLM configurations, system prompt context rules, generation parameters, and model routing.
High-speed, low-cost LLM engine configured with `temperature=0.1` for factual grounding and minimal hallucination. Receives retrieved document context snippets dynamically injected into the system prompt.
Vector representations, asymmetric task optimization hints, chunk sizes, and embedding LRU caching parameters.
Multilingual dense vector bi-encoder. Text chunks are indexed with `task="retrieval.passage"`, while search queries use `task="retrieval.query"`. Includes an in-memory LRU query cache for 0ms repeat query hits.
Cross-encoder similarity scoring that evaluates query and chunk pairs together to prevent lost-in-the-middle context issues.
Evaluates candidate chunks retrieved from vector search and re-orders them by true semantic relevance. Short queries (≤ 5 words) automatically bypass the reranker to save 500-1500ms network latency.
Hybrid Vector Storage Architecture featuring persistent Local ChromaDB for low-latency queries alongside enterprise Chroma Cloud cluster integration with dynamic runtime failover.
Vector embeddings & metadata are stored on local disk in `data/db/`. Sub-20ms instant similarity scoring without internet latency or third-party cloud billing overhead.
Hosted ChromaDB cloud cluster connected over HTTPS using `CHROMA_CLOUD_API_KEY` and tenant endpoints. Ideal for multi-node production deployment & distributed scaling.
If Local ChromaDB encounters disk full errors, permission issues, missing dependencies, or runtime failures (in local or Azure Container App production), the RAG pipeline automatically & seamlessly fails over to Chroma Cloud. Synchronization & search requests route without throwing 500 errors to the client.
Frontend and backend modules forming the user experience and data pipeline.
Manages SSE streaming, auto-resizing textarea, in-place session renames, and requestAnimationFrame render throttling.
Integrated speech recognition and ElevenLabs / Edge-TTS audio playback engine.
Parses PDF, DOCX, and TXT files using PyMuPDF, python-docx, and LangChain splitters.
Persistent connection SQLite WAL database for chat sessions and message persistence.
Core programming languages, frameworks, web servers, and UI toolkits.
Asynchronous runtime
ASGI web server
LCEL pipeline logic
Hybrid vector store with auto-failover
Modern UI styling
Clean SVG iconography
Data processing sequences, dual-stage retrieval pipeline, and real-time generation routing.
System configuration parameters, performance tunings, and database defaults.