RAG Chat Assistant

A multi-turn conversational RAG (Retrieval-Augmented Generation) application for querying PDF documents. It combines hybrid vector + keyword search via Weaviate, a structured 5-node LangGraph reasoning pipeline with intelligent query routing, and a real-time streaming interface over WebSockets.

Features

Multi-turn conversations — full conversation history with context-aware reformulation
Intelligent query routing — an LLM classifies every turn before any work happens, so "what did I just ask?" and out-of-scope questions skip retrieval entirely instead of searching the book for nothing
Hybrid search — combines dense vector similarity and BM25 keyword search (50/50 alpha) for best-of-both-worlds retrieval
Streaming responses — tokens stream in real time to the browser via WebSocket
User accounts — email/password sign-in via Supabase; each user sees only their own conversations
Persistent sessions — conversations live in Supabase Postgres and survive redeploys; the sidebar lists them and restores any one on click
REST fallback — non-streaming HTTP endpoint available alongside WebSocket
Tracing and retrieval inspection — optional OpenTelemetry tracing to Arize Phoenix: every turn becomes a trace showing the routing decision, the chunks retrieved with their scores and pages, and the generated answer — and eval scores attach back to the trace that earned them

Architecture

User (Streamlit UI)  ──sign in──▶  Supabase Auth
    │
    │  WebSocket  ws://localhost:8000/ws/chat/{session_id}   (+ access token)
    ▼
FastAPI Backend  (api_service.py)
    │  verifies the token, checks the session belongs to the caller
    ▼
Use cases  (application/)  ──▶ ports ──▶ adapters (infrastructure/)
    │  the API depends on interfaces; only the adapters know the frameworks
    ▼
LangGraph RAG Pipeline
    ├── 0. Route Query     — classify the turn: retrieve | history | both | none
    │        ├── retrieve / both ──▶ 1 → 2 → 3 → 4
    │        └── history / none  ─────────────▶ 4   (no search runs at all)
    ├── 1. Contextualize   — analyze history, extract current intent
    ├── 2. Extract Terms   — distill 1–3 precise search queries via LLM
    ├── 3. Retrieve        — hybrid search (vector + BM25) against Weaviate
    └── 4. Generate        — stream the answer from the LLM, prompt chosen by route
    │
    ▼
Conversation Memory  (Supabase Postgres — or local SQLite when unconfigured)

Tech Stack

Layer	Technology
Frontend	Streamlit
Backend	FastAPI + WebSocket
RAG Pipeline	LangGraph
Vector Database	Weaviate (local Docker)
LLM & Embeddings	OpenAI-compatible API (`gpt-5.6-luna` / `text-embedding-3-large`)
Auth & User Data	Supabase (Auth + Postgres)
Conversation Memory	Supabase Postgres via LangGraph `PostgresSaver` (SQLite fallback)
Document Loader	PyPDF
Tracing (optional)	OpenTelemetry / OpenInference → Arize Phoenix

Layering

The codebase follows a ports-and-adapters (hexagonal) split:

Layer	Contains	May import
`domain/`	Events, value objects, exceptions	nothing
`application/`	Use cases and the ports they depend on	`domain`
`infrastructure/`	Adapters for LangGraph, Postgres, Supabase + the container	anything
`app/api/`, `ui/`	HTTP/WebSocket surface and the Streamlit client	anything

The practical payoff is that the pipeline is swappable and testable in isolation: routes call use cases, use cases call ports, and only infrastructure/adapters/ knows that the pipeline is LangGraph or that memory is Postgres. infrastructure/container.py wires it all together on first request — lazily, because importing the graph builds it and opens the database pool.

Project Structure

.
├── config.py                        # All configuration constants (reads from .env)
│
├── domain/                          # Framework-free core — imports nothing
│   ├── events.py                    # Status/StreamStart/Token/Complete answer events
│   ├── values.py                    # Routes, default title, SearchResult
│   └── exceptions.py                # SessionNotFound, NotSessionOwner
│
├── application/                     # Use cases + the interfaces they depend on
│   ├── ports/
│   │   ├── rag_port.py              # "Answer this question" — hides the pipeline
│   │   ├── session_repository.py    # Conversation storage and ownership
│   │   └── auth_port.py             # Token → identity
│   └── use_cases/
│       ├── send_message.py          # Answer a turn and record both messages
│       ├── create_session.py        # Mint a conversation
│       ├── list_sessions.py         # The caller's conversations
│       ├── get_transcript.py        # Restore a conversation
│       └── delete_session.py        # Delete transcript *and* pipeline memory
│
├── infrastructure/
│   ├── container.py                 # Composition root — builds adapters lazily
│   └── adapters/
│       ├── rag/langgraph_rag_adapter.py       # Drives the graph; owns the streaming bridge
│       ├── persistence/sessions_adapter.py    # Postgres storage, no-ops without Supabase
│       └── auth/supabase_auth_adapter.py      # Token verification off the event loop
│
├── app/
│   ├── observability.py             # OpenTelemetry tracing → Phoenix; a no-op when unconfigured
│   │
│   ├── tools/
│   │   ├── embeddings.py            # Embedding client wrapper
│   │   ├── embedding_cache.py       # On-disk cache of precomputed vectors + manifest
│   │   ├── weaviate_client.py       # Single connect_weaviate() used by every path
│   │   ├── vectorstore.py           # PDF loading, chunking, and Weaviate indexing
│   │   ├── build_embedding_cache.py # Utility: (re)generate embeddings_cache.npz
│   │   └── reset_weaviate.py        # Utility: wipe and re-index Weaviate collection
│   │
│   ├── graph/
│   │   ├── state.py                 # GraphState TypedDict
│   │   ├── nodes.py                 # Pipeline nodes + generate_stream
│   │   └── builder.py               # Graph wiring, routing branch, compilation
│   │
│   ├── api/
│   │   ├── lifespan.py              # FastAPI startup/shutdown + thread pool + DB pool
│   │   └── routes.py                # HTTP and WebSocket endpoints (thin — calls use cases)
│   │
│   ├── auth/
│   │   └── verify.py                # Supabase token verification + FastAPI dependency
│   │
│   └── db/
│       ├── pool.py                  # Shared psycopg connection pool
│       └── sessions.py              # Session ownership + transcript storage
│
├── ui/
│   ├── styles.py                    # Custom CSS
│   ├── auth.py                      # Sign-in / sign-up screen
│   ├── api_client.py                # Authenticated REST calls to the backend
│   ├── websocket_client.py          # WebSocket communication helpers
│   ├── sidebar.py                   # Conversation list + settings sidebar
│   └── chat.py                      # Chat history and input handling
│
├── supabase/
│   └── schema.sql                   # Tables, indexes, and RLS lockdown
│
├── evals/                           # Answer-quality evaluation (RAGAS + DeepEval)
│   ├── dataset.py                   # 5 questions with reference answers
│   ├── artifact.py                  # Run-artifact schema + context parsing
│   ├── judge.py                     # Judge model adapted to both frameworks
│   ├── run.py                       # Utility: pipeline → evals/runs/<timestamp>.json
│   ├── score.py                     # Utility: grade a saved run
│   ├── phoenix_sync.py              # Attach scores to the Phoenix spans that produced them
│   └── runs/                        # Run artifacts (gitignored)
│
├── tests/                           # 188 pytest tests — no live services needed
│   ├── conftest.py                  # Env priming + GraphState/Weaviate fixtures
│   ├── test_graph_nodes.py          # Routing, contextualize, retrieval, streaming
│   ├── test_api.py                  # REST + WebSocket endpoints, wire protocol, auth
│   ├── test_use_cases.py            # Use cases against fake ports — no framework imports
│   ├── test_auth.py                 # Token verification
│   ├── test_db.py                   # Session ownership + transcript storage
│   ├── test_config.py               # Env trimming, connection-string validation
│   ├── test_embedding_cache.py      # Cache manifest staleness rejection
│   ├── test_embeddings.py           # Embedding client batching
│   ├── test_evals.py                # Eval dataset + run artifact (no eval deps needed)
│   └── test_observability.py        # Tracing no-ops cleanly; score-annotation payloads
│
├── docker/
│   └── entrypoint.sh                # Boot order: Weaviate → FastAPI → Streamlit
├── Dockerfile                       # Single-container image (app + embedded Weaviate)
│
├── .hf/space.yml                    # HF Space config, spliced into README.md at deploy
├── .github/workflows/ci-cd.yml      # test → docker-build → deploy to HF Spaces
│
├── hybrid_search_graph_history.py   # Entry point — run pipeline from terminal
├── api_service.py                   # Entry point — FastAPI server
├── streamlit_app.py                 # Entry point — Streamlit UI
│
├── computer_architecture.pdf        # Source document (Git LFS)
├── embeddings_cache.npz             # Precomputed vectors, ~47 MB (Git LFS)
│
├── .env.example                     # Environment variable template
├── pytest.ini                       # Test discovery config
├── requirements.txt                 # Dependencies
├── requirements-eval.txt            # Evaluation-only dependencies (not in the image)
└── requirements-observability.txt   # Tracing-only dependencies (not in the image)

Quick Start

1. Prerequisites

Python 3.10+
Docker (for Weaviate)
An OpenAI-compatible API key

2. Start Weaviate

docker run -d \
  -p 8080:8080 \
  -p 50051:50051 \
  cr.weaviate.io/semitechnologies/weaviate:latest

3. Install dependencies

pip install -r requirements.txt

Two optional extras are kept separate so neither lands in the deployed image: requirements-eval.txt for Evaluation and requirements-observability.txt for Observability.

4. Configure environment

Copy .env.example to .env and add your API key and provider endpoint:

cp .env.example .env
# then edit .env:
# LLM_API_KEY=sk-...
# LLM_API_BASE=https://api.openai.com/v1

LLM_API_BASE and LLM_API_KEY belong together — the endpoint only works with a key issued by that provider. LLM_API_BASE defaults to OpenAI proper if you leave it out.

All other settings (model names, ports, PDF path) are in config.py.

4b. (Optional) Enable user accounts

Skip this and the app runs as it always did: no sign-in, one shared conversation space, history in a local SQLite file. Set all three variables and it gains accounts, per-user conversation ownership, and history that survives redeploys.

Create a project at supabase.com.
Project Settings → API — copy the Project URL and the anon public key into SUPABASE_URL and SUPABASE_ANON_KEY.
Project Settings → Database → Connection string → Session pooler — copy that into SUPABASE_DB_URL.

Use the Session pooler, not the direct connection and not the transaction pooler on port 6543. LangGraph's PostgresSaver uses prepared statements, which transaction-mode PgBouncer rejects — and it fails at query time with prepared statement "_pg3_0" already exists rather than at startup, which is a confusing way to find out.
Authentication → Providers → Email — toggle Confirm email off if you want sign-up to log people straight in.
Run supabase/schema.sql in the SQL Editor. Start the backend once, then run Part 2 of that file to enable RLS on the checkpoint tables LangGraph creates — without it, anyone holding the (public) anon key can read every conversation over PostgREST.

On Hugging Face Spaces, add the same three as Settings → Variables and secrets.

Watch the trailing newline. Pasting a secret into a dashboard often appends one, and in a Postgres URL it becomes part of the database name — the server then rejects the connection with FATAL: database "postgres\n" does not exist, which names neither the variable nor whitespace as the cause. config.py trims defensively and logs a warning when it has to, but it's worth saving the secret cleanly.

Note: a browser refresh signs you out — Streamlit's session_state doesn't survive a page reload. Your conversations are safe in Postgres; you just sign in again. Persisting the refresh token in a cookie would need a third-party Streamlit component.

5. Add your PDF

Place your PDF in the project root. By default the app looks for computer_architecture.pdf. To use a different file, update PDF_PATH in config.py.

6. Start the backend

uvicorn api_service:app --reload --host 0.0.0.0 --port 8000

On first startup the backend populates the Weaviate index. If embeddings_cache.npz is present (it is, in this repo) it restores the precomputed vectors in a few seconds; otherwise it loads the PDF, chunks it, embeds it, and writes the cache. Subsequent starts skip this step entirely if the collection is already populated.

7. Start the frontend

streamlit run streamlit_app.py

Open http://localhost:8501 in your browser.

Testing

pip install pytest httpx     # test-only dependencies
pytest                       # 188 tests
pytest tests/test_graph_nodes.py                                # one file
pytest tests/test_api.py::TestHealthEndpoint                    # one class

The suite needs no live services — no Weaviate, no database, no API key, no trace collector. tests/conftest.py primes a dummy LLM_API_KEY and blanks the three SUPABASE_* variables and PHOENIX_COLLECTOR_ENDPOINT before any project module is imported, so AUTH_ENABLED and TRACING_ENABLED are false throughout and the Weaviate/OpenAI clients are mocked at the module boundary.

tests/test_use_cases.py covers the application layer against hand-written fake ports and imports no framework at all. tests/test_api.py drives whole turns through the real route, use case and adapter with only the graph run itself faked, pinning the WebSocket event sequence documented under "WebSocket protocol" below.

Evaluation

The test suite checks that the pipeline works; evals/ measures how well it answers. A fixed set of five computer-architecture questions with reference answers is run through the real pipeline and graded by two independent frameworks, RAGAS and DeepEval, on the same four metrics.

pip install -r requirements-eval.txt

python -m evals.run                    # pipeline → evals/runs/<timestamp>.json
python -m evals.score                  # grade the newest run with both frameworks

Unlike pytest, this needs a live Weaviate with the collection indexed and a working LLM endpoint, and it costs real API calls — which is why the dependencies live in requirements-eval.txt and are installed by neither the Dockerfile nor CI.

Two phases on purpose. evals.run only produces a run artifact: the question, the reference answer, the generated answer, every retrieved chunk, the routing decision and the latency. evals.score grades that artifact. Adding a metric or swapping the judge therefore re-reads a JSON file instead of re-running retrieval and generation, and the artifact records the model, chunk size and collection that produced it so a score sheet can't be misattributed to the wrong configuration.

Metric	Question it answers	RAGAS	DeepEval
Faithfulness	Is every claim in the answer supported by the retrieved chunks?	`Faithfulness`	`FaithfulnessMetric`
Answer relevancy	Does the answer actually address the question?	`ResponseRelevancy`	`AnswerRelevancyMetric`
Context precision	Are the useful chunks ranked ahead of the noise?	`LLMContextPrecisionWithReference`	`ContextualPrecisionMetric`
Context recall	Do the retrieved chunks cover the reference answer?	`LLMContextRecall`	`ContextualRecallMetric`

Two frameworks over one artifact is the point: where they disagree on the same metric, the number is measuring the judge as much as the pipeline.

Judge concurrency is capped on purpose. RAGAS defaults to 16 parallel judge calls, which a shared proxy endpoint answers by queueing them until they time out — a full run then returns empty cells rather than scores. evals.score caps it at 4 with a 300s ceiling; --max-workers 1 serialises everything, and DeepEval switches to synchronous metrics at that setting. Raise it if the judge endpoint is one you control.

Useful flags:

python -m evals.run --dry-run              # print the dataset, call nothing
python -m evals.run --only locality        # one question (repeatable)
python -m evals.score --framework ragas    # or deepeval, or both (default)
python -m evals.score --max-workers 1      # serialise judge calls on a fragile endpoint
python -m evals.score --run evals/runs/20260731T101500Z.json
python -m evals.score --out scores.json --fail-under 0.7
python -m evals.score --no-phoenix         # skip attaching scores to their traces

Scores link back to traces. With tracing switched on (see Observability), evals.run records each question's span id in the artifact and evals.score attaches every metric to that span as a <framework>:<metric> annotation. A faithfulness of 0.42 in the terminal is then one click in Phoenix away from the prompt, the retrieved chunks and the completion that earned it — and because both frameworks annotate the same span, you can see exactly where RAGAS and DeepEval disagree. The push is best-effort and happens after the table prints, so a collector that is down never costs you a score sheet that took real judge calls to produce.

The judge is configured separately from the model under test, through the optional EVAL_* variables in Configuration. Each falls back to its LLM_* counterpart, so the harness runs with no extra setup — but a model grading its own answers grades them generously, and both frameworks depend on structured JSON output much more heavily than the pipeline does, so pointing the judge at a provider with dependable JSON mode gives more trustworthy numbers. Run artifacts land in evals/runs/ and are gitignored.

Observability

Metrics tell you that retrieval got worse. Traces tell you which chunks came back and why. Tracing is optional and off by default; switching it on turns every turn into an OpenTelemetry trace, exported to a local Arize Phoenix container.

docker run -d -p 6006:6006 arizephoenix/phoenix:latest   # collector + UI on :6006
pip install -r requirements-observability.txt

echo 'PHOENIX_COLLECTOR_ENDPOINT=http://localhost:6006' >> .env

Restart the backend and ask a question — the trace appears in the comparch-app project at http://localhost:6006:

chat_turn                       CHAIN      session.id, rag.route, rag.route_reasoning
├── route_query                 CHAIN      the routing LLM call
├── contextualize               CHAIN
├── extract_terms               CHAIN
├── retrieve                    RETRIEVER  10 documents, each with id / text / page / score
└── chat.completions.create     LLM        the full prompt and the streamed answer

What it is actually good for:

Retrieval quality — the retrieve span lists every chunk that survived the top-10 cut with its hybrid-search score and page number, so "the answer was wrong" becomes "chunk 3 outranked the one that mattered".
Routing — rag.route and rag.route_reasoning are on the trace. A conversational turn simply has no retrieve span, which is the routing skip made visible; route_reasoning reading "keyword fallback" on every turn means the provider stopped honouring structured output.
Sessions — traces are grouped by session.id, so a multi-turn conversation reads as one thing rather than n unrelated traces.
Cost and latency — token counts and per-node timings come from the instrumentors for free.

Notes:

Both instrumentors are required. This project's LLM calls are split between a LangChain client (routing, contextualization, key-term extraction) and a raw openai client (the answer itself and every embedding — generate_stream bypasses LangChain). Instrumenting only LangChain would trace the cheap classification calls and miss the generation entirely.
Off means off. With PHOENIX_COLLECTOR_ENDPOINT unset, app/observability.py imports nothing beyond the standard library and every hook is a no-op — which is how CI and the Hugging Face Space run, on requirements.txt alone. If the endpoint is set but the packages are missing, it logs one actionable warning and continues rather than failing to start.
It is plain OTLP. Nothing in the pipeline is Phoenix-specific; pointing PHOENIX_COLLECTOR_ENDPOINT at Jaeger, Grafana Tempo or any other OTLP collector works. Only the UI and the eval score annotations use Phoenix's own API.
Traces from evals.run go to a separate comparch-eval project, so a benchmark never mixes into the traffic you are debugging.

Docker deployment

The Dockerfile builds a single self-contained image: the Weaviate binary is copied from the official image into a python:3.11-slim layer, so one container runs the vector database, the API, and the UI.

docker build -t rag-chat .
docker run -p 7860:7860 -e LLM_API_KEY=sk-... rag-chat

docker/entrypoint.sh enforces the boot order: it starts embedded Weaviate, polls /v1/.well-known/ready for up to 60 seconds, starts the FastAPI backend, then execs Streamlit as PID 1 on port 7860 (the port Hugging Face Spaces expects). Weaviate persists to /data/weaviate, which a Spaces rebuild wipes — which is exactly why the embedding cache exists.

CI/CD

.github/workflows/ci-cd.yml runs on every push and pull request:

Job	Runs on	Does
`test`	all pushes/PRs	`pytest` — no live services required
`docker-build`	all pushes/PRs	Builds the `Dockerfile` as a smoke test (no push)
`deploy-hf`	pushes to `main` only	Force-pushes the repo to the Hugging Face Space remote

Deployment needs an HF_TOKEN repository secret, plus LLM_API_KEY, LLM_API_BASE and the Supabase variables set as Space secrets. Both the build and deploy jobs check out with lfs: true — without it the deploy would push LFS pointer files and the Space would find neither the PDF nor the embedding cache.

Space configuration

Hugging Face reads a Space's build settings — sdk, app_port, title, thumbnail colors — only from a YAML block at the very top of README.md, with no alternative config file. To keep that deployment metadata out of this README, the fields live in .hf/space.yml and the deploy-hf job splices them onto README.md immediately before pushing to the Space. The commit exists only on the Space; the repo's README.md is never modified.

Edit .hf/space.yml to change Space settings — not the README, and not the HF dashboard (the dashboard writes back into the Space's README.md, which the next force-push overwrites). sdk: docker and app_port: 7860 are the load-bearing pair: they tell HF to build the Dockerfile instead of looking for a Gradio app, and to route traffic to the port docker/entrypoint.sh serves Streamlit on.

Configuration

All settings live in config.py. Anything read through _env() can be overridden by an environment variable or .env entry; the rest are plain constants you edit in the file.

Read from the environment:

Variable	Default	Description
`LLM_API_KEY`	(required)	API key for the chat + embedding provider
`LLM_API_BASE`	`https://api.openai.com/v1`	Provider endpoint — swap for Azure OpenAI, a local Ollama, or any OpenAI-compatible proxy
`SUPABASE_URL`	(unset)	Supabase project URL
`SUPABASE_ANON_KEY`	(unset)	Supabase anon public key
`SUPABASE_DB_URL`	(unset)	Postgres session pooler connection string
`CONVERSATIONS_DB_PATH`	`conversations.db`	SQLite checkpointer path (used only when Supabase is unconfigured)
`EMBEDDING_CACHE_PATH`	`embeddings_cache.npz`	Precomputed vector cache
`WEAVIATE_HOST`	`localhost`	Weaviate host
`WEAVIATE_LOCAL_PORT` / `WEAVIATE_GRPC_PORT`	`8080` / `50051`	Weaviate HTTP and gRPC ports
`WEAVIATE_URL`	`http://localhost:8080`	Derived from host + port unless set explicitly
`API_HOST` / `API_PORT`	`localhost` / `8000`	FastAPI server binding
`EVAL_LLM_MODEL`	(falls back to `LLM_MODEL`)	Judge model for `evals/` — see Evaluation
`EVAL_EMBEDDING_MODEL`	(falls back to `EMBEDDING_MODEL`)	Embeddings the judge uses for answer relevancy
`EVAL_API_BASE`	(falls back to `LLM_API_BASE`)	Endpoint the judge calls
`EVAL_API_KEY`	(falls back to `LLM_API_KEY`)	Key for that endpoint
`PHOENIX_COLLECTOR_ENDPOINT`	(unset)	Base URL of an OTLP trace collector — setting it switches tracing on, see Observability
`PHOENIX_PROJECT_NAME`	`comparch`	Prefix for the two trace projects, `<name>-app` and `<name>-eval`

Constants in config.py:

Constant	Value	Description
`PDF_PATH`	`computer_architecture.pdf`	Path to the source PDF
`LLM_MODEL`	`gpt-5.6-luna`	Chat model
`EMBEDDING_MODEL`	`text-embedding-3-large`	Embedding model
`EMBED_BATCH_SIZE`	`128`	Chunks per embedding request
`CHUNK_SIZE` / `CHUNK_OVERLAP`	`800` / `100`	Chunking — part of the embedding cache's identity
`WEAVIATE_COLLECTION`	`BookChunk_hist`	Weaviate collection name

AUTH_ENABLED is derived, not set: it is true only when all three SUPABASE_* variables are present. Everything auth- and Postgres-related keys off it, so a partial configuration cleanly falls back to the original single-user behaviour rather than half-working. TRACING_ENABLED works the same way off PHOENIX_COLLECTOR_ENDPOINT.

Every variable is read through config._env(), which strips surrounding whitespace — a secret pasted into a dashboard often picks up a trailing newline, and in a Postgres URL that newline ends up inside the database name.

How It Works

RAG Pipeline (LangGraph)

The pipeline is a directed graph with five nodes and one branch:

Route Query — Before anything expensive happens, the LLM classifies the turn into one of four labels using structured output:

Route	Meaning	Path
`retrieve`	Needs the textbook; the conversation doesn't already contain the answer	full pipeline
`history`	Answerable from the conversation alone ("what did I just ask?")	straight to Generate
`both`	A technical follow-up that needs the book and the earlier turns	full pipeline
`none`	Outside computer architecture, or smalltalk ("thanks!")	straight to Generate

A history or none turn therefore costs one classification call plus the answer — no contextualization, no term extraction, no embedding calls, and no Weaviate queries. It also can't hallucinate a connection to textbook chunks it was handed for no reason.

Classification degrades in three tiers, because the configured API base is an OpenAI-compatible proxy whose tool-calling support isn't guaranteed: structured output → a one-word text answer → defaulting to both. Every fallback biases toward doing the retrieval, so a routing failure behaves exactly like the pipeline did before routing existed.

Contextualize — The LLM reads the last 8 messages and the current question, then produces a CONTEXT_SUMMARY describing what prior context is needed. For standalone questions it passes through unchanged.
Extract Terms — The LLM distills the reformulated question into 1–3 precise search terms optimized for retrieval.
Retrieve — For each search term, a hybrid search runs against Weaviate combining:
- Dense vector similarity (text-embedding-3-large)
- BM25 keyword search
- Equal 50/50 alpha weighting
Results are deduplicated by chunk_id and the top 10 chunks by score are kept.
Generate — The LLM streams the answer, using whichever prompt matches the route: grounded in the retrieved chunks for retrieve/both, answering from the transcript alone for history, or politely declining and naming what it can help with for none.

Conversation Memory

Each conversation has a session_id (e.g. session_a3f2c1b0), which is also the LangGraph thread_id.

With Supabase configured, three things are stored in Postgres:

Table	Holds
`chat_sessions`	Who owns a conversation, its title, when it was last used
`chat_messages`	The readable transcript the sidebar restores (with sources and search queries)
`checkpoints` & friends	LangGraph's own state, written by `PostgresSaver`

Two stores rather than one because LangGraph's checkpoints are opaque serialized blobs — you cannot cheaply query them for "this user's conversations, titled by their first question."

Conversation titles are derived from the first question asked, and ownership is enforced on the server: every route that takes a session_id verifies the caller's token and checks chat_sessions.user_id before touching the pipeline. Requesting someone else's conversation returns 403, whether over REST or WebSocket.

Without Supabase configured, state falls back to conversations.db (SQLite) via SqliteSaver, exactly as before — convenient locally, but the file does not survive a container rebuild on Hugging Face Spaces, which is the reason the Postgres path exists.

Document Indexing

On first run, the PDF is:

Loaded page-by-page with PyPDFLoader
Split into ~800-token chunks with 100-token overlap
Each chunk is assigned a deterministic chunk_id, derived from a hash of its source, page, and text content, and used as the Weaviate object UUID
Embedded in batches and stored in the Weaviate collection BookChunk_hist

Because the chunk_id is content-derived rather than random, re-running indexing against the same content upserts existing objects in place instead of appending duplicates — indexing is safe to re-run.

Embedding Cache

Embedding 6,136 chunks takes minutes and costs money, and Weaviate's data directory does not survive a container rebuild — so the vectors are cached on disk in embeddings_cache.npz (~47 MB, tracked by Git LFS and baked into the Docker image).

When the collection is empty, startup restores the index straight from this file: no PDF parsing, no embedding API calls, roughly 8 seconds. Only a cache miss falls back to embedding from the PDF, and that path writes the cache afterwards.

A manifest travels with the vectors recording the embedding model, the chunk size/overlap, and the PDF's sha256. If any of them no longer matches config.py, the cache is rejected (with the reason logged) and the app re-embeds — so you can't accidentally serve vectors from the wrong model.

After changing the PDF, the chunking, or the embedding model, rebuild and commit the cache:

python -m app.tools.build_embedding_cache --force
git add embeddings_cache.npz && git commit -m "Rebuild embedding cache"

By default this exports the vectors out of a locally indexed Weaviate collection (free), and only calls the embedding API if the collection is missing or its contents no longer match the current PDF. Use --pdf-only to always re-embed.

CI note: the docker-build and deploy-hf jobs check out with lfs: true. Without it, the deploy would push LFS pointer files and the Space would find neither the cache nor the PDF.

API Reference

Every endpoint except /, /health, and /docs requires Authorization: Bearer <supabase-access-token> when Supabase is configured.

Method	Path	Description
`GET`	`/`	API info and available endpoints
`GET`	`/health`	Weaviate connectivity check
`POST`	`/sessions`	Create a conversation owned by the caller
`GET`	`/sessions`	List the caller's conversations
`GET`	`/sessions/{id}/messages`	Full transcript of one conversation
`DELETE`	`/sessions/{id}`	Delete a conversation and its memory
`WS`	`/ws/chat/{session_id}`	Streaming chat (WebSocket)
`POST`	`/chat/{session_id}`	Non-streaming chat (REST)
`GET`	`/debug/history/{session_id}`	Inspect LangGraph state for a session
`GET`	`/docs`	Interactive Swagger UI

WebSocket protocol

Client → Server — the first frame is the auth handshake, then questions:

{ "token": "<supabase-access-token>" }
{ "question": "What is pipelining?" }

Authentication is a first message rather than a connect header on purpose: the client kwarg for headers is extra_headers in websockets 13 but additional_headers in 14+, and requirements.txt pins websockets>=13,<15 for unrelated reasons. A protocol-level handshake works across the whole range. A failed handshake closes the socket with code 4403 before any question is read.

Server → Client (in order)

{ "type": "connected",    "session_id": "...", "message": "Connected to RAG chat service" }
{ "type": "processing" }
{ "type": "status",       "message": "Route: retrieve" }
{ "type": "stream_start", "search_queries": ["pipelining", "instruction hazards"] }
{ "type": "token",        "content": "P" }
{ "type": "token",        "content": "ipelining" }
{ "type": "complete",     "answer": "...", "sources": [...], "search_queries": [...] }

stream_start is emitted by the Extract Terms node, so a turn routed to history or none never sends one — it goes from status straight to tokens, and its complete frame carries empty sources and search_queries. Clients must not wait for stream_start before rendering tokens.

REST endpoint

curl -X POST http://localhost:8000/chat/my-session \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  -d '{"question": "What is cache coherence?"}'

Utilities

Force re-indexing (deletes the Weaviate collection so the next startup rebuilds it):

python -m app.tools.reset_weaviate                 # rebuild from the embedding cache (seconds, free)
python -m app.tools.reset_weaviate --purge-cache   # also drop the cache, forcing a full re-embed

Targets the collection configured via WEAVIATE_COLLECTION in config.py (BookChunk_hist by default).

Rebuild the embedding cache (see Embedding Cache):

python -m app.tools.build_embedding_cache --force

Run the pipeline from the terminal (no UI):

python hybrid_search_graph_history.py

Measure answer quality (see Evaluation):

python -m evals.run      # needs a live Weaviate and LLM; writes evals/runs/<timestamp>.json
python -m evals.score    # grades the newest run with RAGAS and DeepEval

Troubleshooting

Weaviate connection refused Confirm Docker is running and the container is up:

docker ps
curl http://localhost:8080/v1/.well-known/ready

LLM_API_KEY not found Make sure .env exists in the project root with LLM_API_KEY=... set, or export the variable in your shell.

PDF not found on startup Check that your PDF file name matches PDF_PATH in config.py and that it is placed in the project root.

Want to use a different LLM provider Set LLM_API_BASE in .env to any OpenAI-compatible endpoint (e.g. Azure OpenAI, local Ollama with openai compatibility), put that provider's key in LLM_API_KEY, and update LLM_MODEL in config.py to a model that endpoint serves. A 401 usually means the key and the endpoint came from different providers.

Every question is being routed the same way The query router asks the model for structured output first and falls back to parsing a one-word answer if the provider rejects that. Both tiers log at WARNING when they fail, and the fallback tier reports keyword fallback as its reasoning. If routing fails entirely it defaults to running the full retrieval pipeline, so the symptom is lost savings, never a wrong answer.

No traces appearing in Phoenix Check the three things in order: PHOENIX_COLLECTOR_ENDPOINT is set (without it, tracing is off by design and silent); the packages are installed (pip install -r requirements-observability.txt — otherwise startup logs one warning naming this file); and the collector is reachable (curl http://localhost:6006/healthz). On startup with tracing on, the backend log prints the resolved endpoint and project name.

Acknowledgments

Claude (Anthropic) was used as an assistant for designing the Streamlit UI components in this project.

Navigation Menu

Repository files navigation

RAG Chat Assistant

Features

Architecture

Tech Stack

Layering

Project Structure

Quick Start

1. Prerequisites

2. Start Weaviate

3. Install dependencies

4. Configure environment

4b. (Optional) Enable user accounts

5. Add your PDF

6. Start the backend

7. Start the frontend

Testing

Evaluation

Observability

Docker deployment

CI/CD

Space configuration

Configuration

How It Works

RAG Pipeline (LangGraph)

Conversation Memory

Document Indexing

Embedding Cache

API Reference

WebSocket protocol

REST endpoint

Utilities

Troubleshooting

Acknowledgments

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 59 Commits
.github/workflows		.github/workflows
.hf		.hf
app		app
application		application
docker		docker
domain		domain
evals		evals
infrastructure		infrastructure
supabase		supabase
tests		tests
ui		ui
.env.example		.env.example
.gitattributes		.gitattributes
.python-version		.python-version
Dockerfile		Dockerfile
README.md		README.md
api_service.py		api_service.py
computer_architecture.pdf		computer_architecture.pdf
config.py		config.py
embeddings_cache.npz		embeddings_cache.npz
hybrid_search_graph_history.py		hybrid_search_graph_history.py
main.py		main.py
pyproject.toml		pyproject.toml
pytest.ini		pytest.ini
requirements-eval.txt		requirements-eval.txt
requirements-observability.txt		requirements-observability.txt
requirements.txt		requirements.txt
streamlit_app.py		streamlit_app.py

Folders and files

Latest commit

History

Repository files navigation

RAG Chat Assistant

Features

Architecture

Tech Stack

Layering

Project Structure

Quick Start

1. Prerequisites

2. Start Weaviate

3. Install dependencies

4. Configure environment

4b. (Optional) Enable user accounts

5. Add your PDF

6. Start the backend

7. Start the frontend

Testing

Evaluation

Observability

Docker deployment

CI/CD

Space configuration

Configuration

How It Works

RAG Pipeline (LangGraph)

Conversation Memory

Document Indexing

Embedding Cache

API Reference

WebSocket protocol

REST endpoint

Utilities

Troubleshooting

Acknowledgments

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages