Groundline: A RAG Service That Counts the Money It Saves

Over the past few days I built a pet project called Groundline, and I want to share why I made it this way and which problems it solves.

Basic document search is no longer hard to build, there are thousands of tutorials. But most tutorial and demo projects stop at the stage of «great, it answers somehow». In a real business the main question is different: how much does it cost to run.

Every call to a language model costs real money and seconds of waiting. If users keep asking the same thing in different words, and the system runs the full vector search and generates an answer from scratch each time, the budget goes into repeating work that was already done.

So I decided to build a service that not only answers questions over documents, but also shows the savings plainly, in honest numbers.

You upload documents in PDF, TXT or Markdown and ask questions. You get a clear answer with precise references to the source: the file, the exact fragment and the page. The model is strictly forbidden to invent anything, it works only with what was actually found in the text.

When a question arrives, the system does not run straight to the model. It honestly walks the whole chain.

1. Semantic cache. We check whether we have already answered something similar. If so, the answer comes back instantly without spending a single token.
2. Rewriting. If the cache missed, the model cleans the question up and expands abbreviations into a proper search query.
3. Hybrid search. Vector search by meaning and full-text search by exact words run in parallel. The first one is great at synonyms, the second never loses specific part numbers, terms and codes.
4. Reranking. A dedicated model scores the retrieved chunks and reorders them not by formal similarity but by how well a fragment actually answers the question.
5. Self-check. The model judges whether the context is sufficient. If it is not, another search runs with a note about exactly what is missing, up to two extra attempts.
6. Generation. The answer is streamed to the user word by word.
7. Storing and caching. The answer goes into the history and into the cache for future similar questions.

The similarity threshold for the cache was not picked out of thin air. Every request logs how close it was to the nearest stored question, even when the cache did not fire. Those metrics let me tune the threshold so that it catches real rephrasings without dumping different questions into one bucket.

On my test set with repeated questions the cache successfully covers 60 to 65 percent of requests. On genuinely unique user scenarios the number will of course be lower, but the economic benefit is still tangible.

And the saving is visible while the service works:

• which pipeline step is running right now and how many milliseconds or tokens it consumed;
• a live chart of model spend against the money the cache saved.

1. CPU contention. The local embedding and reranking models competed for the processor with background indexing of new files. At peak, an ordinary request took 83 seconds instead of a few hundred milliseconds. The fix was strict serialization: background indexing and question handling now wait their turn for the models.
2. Leaking database connections. If a user closed the tab in the middle of generation, the connection got stuck in the pool. On free hosting with a microscopic connection limit that quickly took the service down. Fixed by protecting the final database write from cancellation.

• Backend: FastAPI, LangGraph, Python.
• Database: Postgres with the pgvector extension for vector search, SQLAlchemy and Alembic for migrations.
• Security: data isolation between users rests not only on the code but on Row-Level Security in the database itself. A forgotten filter in a query will not leak somebody else's files.
• Models and monitoring: Groq for fast answers, LangFuse for tracing every step, the ragas library for quality evaluation, including precision, context recall and the absence of hallucinations.

Those same seven steps are easier to see once than to read about. An interactive diagram carries a query through the whole chain and shows what changes at each stage:

Sign-in is a one-time code sent by email, no passwords. Upload your own document or take the test corpus, ask a question, then rephrase it. You will see the difference in speed and cost immediately.

I would be glad to get constructive feedback and discussion, especially from people who have already shipped similar services to production and know where else this kind of architecture likes to hide its rakes.