RAG Microservice
Solo - API, queue, indexing pipeline, retrieval and prompt
Python · FastAPI · LangChain
OpenAI embeddings + GPT-4 · Qdrant
Redis Queue (RQ) · Docker Compose
Built July 2025
Reviewed again November 2026
Long PDFs are hard to search and easy to misquote. This service lets you ask a document a question and get an answer built only from the passages that matched, with page numbers you can check.
It’s the classic retrieval-augmented generation (RAG) loop, split into services: an indexing script that chunks and embeds the PDF into Qdrant, a FastAPI layer that queues every question instead of waiting on the model, and a worker that retrieves, prompts and answers.
It’s also the project I learned the most from re-reading. The last section is an honest review of what I’d fix now.
HOW IT WORKS
Two paths: index once, ask often.
A PDF is loaded page by page, so every chunk keeps its page label. LangChain’s recursive splitter cuts it into 1,000-character chunks with 200 characters of overlap, so a sentence split across a boundary still appears whole in one of them.
Each chunk is embedded with OpenAI’s text-embedding-3-large (3,072 dimensions) and stored in a Qdrant collection that uses cosine distance.
POST /chat puts the question on a Redis queue and returns a job id immediately. A separate RQ worker embeds the question, pulls the five closest chunks from Qdrant, and sends them to GPT-4 with the question. The client polls GET /result/{id} until the answer is ready.
DECISIONS
Why a queue in front of the model.
Embedding a question, searching and generating a GPT-4 answer takes seconds. Doing that inside the request would tie up an API worker for every question and turn any model slowness into timeouts.
With a queue, the API stays fast and stateless, and workers scale separately from the API: more questions means more workers, not a bigger web server.
Redis (Valkey), Qdrant, the API and the worker are four services in one docker-compose file, with volumes so the vector index and the queue survive restarts.
A YEAR LATER
Reviewing my own code like a stranger's PR.
I built this in July 2025 and called it production-ready in the README. Reading it again in November 2026, after running FetchJobs in production, I’d block that claim in review. These are the findings, each one checked against the code.
