Gmail RAG Pipeline
The problem
My inbox had become a useful but unsearchable archive. Investment newsletters, job threads, guitar deals, company research I’d saved for later. All there, but keyword search only returns what you already know to search for. And sending email content to an external API wasn’t an option. Too much context in there that I didn’t want leaving the machine.
The fix was local semantic search: whole-email BGE embeddings, ChromaDB, no external API calls, nothing leaving the device.
See it work
A query whose words never appear in the target email. Keyword search returns nothing useful; the semantic pipeline puts the right result first.
What was built
A local ingestion-and-query pipeline: pull inbox email over the Gmail API, embed each message with a locally-run model, store the vectors in ChromaDB, and answer natural-language queries against them. There is no LLM generation step and no external API call anywhere in the pipeline.
What it shows
The retrieval quality here is a measured number. An evidence pack in the repo (eval/) benchmarks the semantic pipeline against a BM25 bag-of-words baseline on a 60-email synthetic corpus with hand-verified ground truth, over 16 queries at k = 5.
The gap shows up where semantic search is supposed to help: queries whose wording does not match the correct email’s wording (the grant-deadline case above is one). Latency is hardware-dependent, so it comes with the machine it was measured on: on a MacBook Pro (M1 Pro, 16GB, CPU only, no GPU), query latency was p50 60.7ms and p95 67.9ms over 48 samples, and ingestion ran at 7.7 to 13.5 emails/sec. The point is less the exact figure than that a local CPU-only pipeline is fast enough for interactive search. Because the model runs locally, marginal cost is $0.
Where it fails documented
- Most low scores come from a benchmark ceiling: 12 relevant emails per category and k = 5 caps recall at 0.417, so a perfect retriever still scores low here.
- One genuine slip: a date-bound deadline reminder ranked alongside event invites because the query asked for "a specific date".
- BM25's misses are real: literal keyword mismatch, plus a stray "or" match on short off-topic newsletters.
Security review 3 checks
- OAuth scope read-only, confirmed by test. Cannot modify, send, or delete.
- At-rest storage unencrypted in ChromaDB. An accepted tradeoff for a local single-user tool, stated plainly.
- Output-injection gap, open: snippets reach the Markdown renderer unescaped. Documented with a recommended fix.
Key design decisions
Whole-email embedding, no chunking. The earlier version chunked emails into overlapping windows. For an inbox of mostly short messages that hurt more than it helped, splitting a coherent email across vectors and pulling rankings toward fragments. Embedding each email as one document keeps the unit of retrieval equal to the unit a person actually wants back.
Drop replies, keep the thread opener. Reply chains quote the original under a different sender, which duplicates content across vectors and biases similarity toward repeated quoted text. Indexing only originals keeps one clean vector per thread.
Message-ID as the ChromaDB key. Keying on the Gmail message ID makes ingestion idempotent: re-run on an updated inbox and only genuinely new messages are added.
A BM25 baseline, on purpose. “Semantic search is better” is only worth anything against a baseline. Benchmarking against BM25 on a ground-truth corpus turns the claim into a number, and shows the specific vocabulary-gap cases where the embedding model wins and the bag-of-words matcher can’t.
Try it
What this became
This project established the core pattern that the Agentic Process Discovery system later extended. APD kept the same ingestion approach, BGE embeddings, and ChromaDB storage, then added WhatsApp as a second source, multi-stage query expansion, and an LLM synthesis step that turns retrieved messages into structured process narratives rather than just returning results.
Technologies:
Python ChromaDB BGE-base-en-v1.5 sentence-transformers Gmail API BeautifulSoup Gradio rank-bm25
Concepts / Algorithms:
RAG Vector embeddings Semantic search Whole-document embedding BM25 baseline Retrieval evaluation Email parsing