how i built handbook-rag
Every growing organization has an employee handbook. Most of them are dense 80+ page documents containing nuanced policies on vacation accruals, remote work stipends, equipment ownership, and internal compliance.
In practice, finding answers is surprisingly painful. Standard keyword search breaks when your phrasing doesn't match the legal wording, and asking a generic LLM leads to hallucinations because company policies are internal and highly specific.
I built handbook-rag to solve this: a lightweight CLI assistant that takes company handbook
documents, vectorizes them, and answers natural language questions with source-grounded policy citations.
the architecture
The system is built with Python, LangChain, OpenAI text embeddings, and FAISS for fast local vector storage. When a user queries the terminal interface, the system computes the query embedding, searches the index for high-similarity policy passages, and passes them to a reasoning LLM to synthesize the final answer.
the chunking challenge
The most important lesson in building RAG for policy documents is that chunking quality dictates retrieval quality. If you chunk naively by character length, sentences get sliced in half and crucial caveats—like "provided manager approval is obtained"—get detached from their parent rules.
I structured document ingestion with recursive semantic splitting, prioritizing headers and double line breaks while maintaining overlap so that contextual prerequisites never get lost across chunk boundaries.
source grounding & zero hallucination
For internal policies, an incorrect answer is worse than no answer. I engineered the prompt to enforce strict citation bounds: the model is explicitly instructed to only synthesize facts present in the retrieved passages and cite the section reference. If an answer isn't in the handbook, it states so transparently rather than guessing.
why cli-first?
Starting with a terminal-first CLI kept iteration speed fast. Instead of fiddling with frontends, I could instantly benchmark retrieval accuracy, test chunking thresholds, and inspect prompt outputs in real time.
The entire source code, setup instructions, and architecture are open source on GitHub.