Story: build a RAG app that knows when to say no

Most "chat with your documents" demos fail the same way: ask something the documents do not say, and the model invents a confident answer. This build is about the fix — an agent whose answers come from retrieved passages, and whose instructions make "the documents don't say" a first-class answer.

This is the narrative version of the RAG chatbot tutorial. That page is the complete copy-paste walkthrough; this one tells the same build as a story — what each service is doing and why, and the decisions that make the difference between a demo and something you can defend.

The shape of the thing

Six services touch this build, and each one does exactly one job:

Service	Its one job here
VectorDB	Store the passages as points; find the nearest ones to a question
Agents	Run the chat endpoint and the reasoning loop
Tools	Let the agent search the index mid-conversation
Sessions	Keep the conversation coherent across turns
Secrets	Hold the embedding API key, injected, never echoed
Gateway	Publish the finished app with an API key on the door

The three decisions that matter

1. One embedding model, everywhere. VectorDB stores and searches vectors — it does not create them. You embed passages when loading and embed questions when asking, and the same model must do both. Switch models between the two and the numbers stop lining up: the search still runs, it just returns nonsense confidently. This is the failure people actually hit, so say it out loud in your runbook.

2. The payload carries the answer. Each point in the index holds the vector and the original text plus its source filename as payload. The agent never needs a second lookup: what comes back from the search is the quotable passage and the citation, ready to hand to the model. Embedding model output goes in the vector; everything a human would want to read goes in the payload.

3. The refusal is instructed. The agent's prompt says, in effect: answer only from the retrieved passages; if they do not cover the question, say so and name what is missing. Without that instruction, the model's training fills the gap with a plausible answer — the exact behavior you built RAG to avoid. With it, "the handbook doesn't cover parental leave" is a correct answer, and your users learn to trust the boundary.

The build, in six moves

1. Store the embedding API key as a project secret. Write-only, injected into the agent as an environment variable, every reveal audited.
2. Create the index — one VectorDB call. Unlike dedicated vector services elsewhere, an index is queryable the moment it exists: there is no second "deploy the index to an endpoint" step and no idle endpoint billing by the node-hour.
3. Embed and upsert the passages. A short Python loop: read each file, split on paragraphs, call the embeddings endpoint, upsert vector + {text, source} points. The tutorial's version is standard-library-only.
4. Write the agent. Its tool is a single function, search_handbook(question), that embeds the question and asks the index for the three nearest points. The agent code reaches the platform-side VectorDB address, which is not the same address your laptop uses — ask your administrator for the internal one.
5. Deploy and interrogate. Ask a question the documents answer, then one they do not. The second answer is the test: you are looking for a polite refusal that names the gap, not a creative paragraph.
6. Publish it through the Gateway with --auth apikey, and hand keys to the people who should have it. The endpoint fails closed: if protection cannot be applied, nothing serves.

Where it breaks, honestly

• Laptop-side address vs platform-side address. The VectorDB base URL you use to load documents is not the address the deployed agent should use. Mix them up and the agent's tool calls time out. The tutorial calls this out at the exact step it bites.
• Embedding endpoint configuration. Use an embedding endpoint (such as Crusoe Managed Inference or your configured OpenAI-compatible endpoint) to convert documents into vectors before loading.
• Retrieval quality is the product. Three chunks per document is a demo. A real corpus needs a splitting strategy, and it usually needs search_handbook to return more passages with a rerank in between. The shape of the build stays; the numbers grow.

When the documents change without you

This build loads the corpus with a script. That is right while you own every document and can re-run it. The moment other people edit the corpus, the loading step becomes the thing nobody remembers to run - and the answers go stale silently, which is worse than an error.

A support agent that re-indexes itself is the same idea with the loading replaced by an event chain: a file lands in a bucket and is searchable a minute later. It also moves retrieval into an MCP server and adds the rerank this page recommends and does not show.

What to read next

• RAG chatbot tutorial — the full copy-paste walkthrough of this same build.
• VectorDB: collections and points — the data model you just used.
• Search with payload filters — narrow retrieval by metadata once your corpus grows.
• Use VectorDB with agents — the built-in memory bank's version of this pattern.
• Research agent with memory — the next step up: retrieval plus long-term memory.
• A support agent that re-indexes itself — the production shape of this same build.