VectorDB overview
This page explains what a vector database is, how VectorDB works on the Crusoe Agent Platform, and when you should reach for it. No prior machine-learning knowledge needed.
What vector search is, in plain words
Normal search matches words. If you search a database of support tickets for "refund", you find tickets that contain the word "refund" — and miss the one that says "I want my money back".
Vector search matches meaning. It works in two steps:
- Turn text (or images, or audio) into a list of numbers. An AI model called an embedding model reads your content and outputs a fixed-length list of numbers — for example, 1,536 of them. That list is called an embedding or a vector. The model is trained so that content with similar meaning gets numbers that are close together. "I want my money back" and "please refund me" end up as nearly identical vectors, even though they share no words.
- Find the nearest neighbors. When someone searches, you embed their query the same way, then ask the database: "which stored vectors are closest to this one?" Closest vectors = most similar meaning.
A vector database is a database built to store millions of these vectors and answer that "which are closest?" question fast. VectorDB is the Crusoe Agent Platform's managed vector database.
VectorDB stores and searches embeddings — it does not generate them. You run your content through an embedding model (any provider works), then write the resulting vectors into VectorDB. Every vector in one index must come from the same model, so the numbers are comparable.
The 30-second mental model
- You create an index: a named container for vectors that all have the same width (the same number of dimensions) and are compared with the same distance rule.
- You write points into it. A point is one vector, plus an optional JSON payload (metadata like
{"tag": "alpha"}), plus an optional id. - You query with a vector and get back the top-k nearest points, with similarity scores, optionally filtered by payload.
Under the hood, VectorDB runs on a shared, platform-operated Qdrant engine. You never talk to Qdrant directly or pick internal storage names — the platform derives an internal collection name from your project, so two projects can both have an index called documents without colliding. Every index is scoped to your project and invisible to other projects.
When to use it
Use VectorDB when the question is "what is similar to this?":
- Semantic search — search docs, tickets, or products by meaning, not keywords.
- RAG (retrieval-augmented generation) — fetch the most relevant snippets from your own data and hand them to a language model so its answers are grounded in facts.
- Agent long-term memory — the platform's agent memory bank is built on the same engine.
- Recommendations and deduplication — "users who liked this", "have we seen this before?".
Do not use it as a general-purpose database. It has no joins, no transactions, and payload filtering is a search refinement, not a query language. For key-value data, use MemoryStore.
How it compares
AWS and Azure fold vector search into a general search engine. GCP has a dedicated product but recently rewrote its data model. VectorDB is a dedicated vector database and nothing else.
| Crusoe VectorDB | Vertex AI Vector Search (GCP) | OpenSearch vector engine (AWS) | Azure AI Search vectors | |
|---|---|---|---|---|
| What it is | Dedicated vector DB (Qdrant-backed) | Dedicated vector product (ScaNN-based) | Vector fields inside a search/analytics engine | Vector index type inside a search service |
| Ready to query | Immediately after the index reports ready — no separate "deploy to endpoint" step | Requires deploying an index to an endpoint (1.0 model) or the newer Collections model | After index mapping setup | After index + vectorizer/profile setup |
| Result count control | One parameter: top_k | neighbor_count | Two parameters (k and size) that must agree | k per vector query |
| Capacity planning | None — shared platform capacity | Node-hours per deployed replica, billed while idle | OCUs / instance sizing | Search Units = replicas × partitions |
| What they have that we don't (yet) | — | Global regions, SLAs, hybrid search, autoscaling at huge scale | Full-text + hybrid search, aggregations, mature IAM | Hybrid + semantic ranking, multi-language SDKs |
Honest limits: VectorDB is in alpha. There are no snapshots or backups — deleting an index destroys its data permanently. Capacity is shared across the platform with no per-project quota, so one enormous index can slow neighbors. One dense vector per point (no named or sparse vectors). If you need those things today, the big clouds have them; we tell you that instead of hiding it.
What's in this section
- Quickstart — create an index, insert points, and search in about five minutes.
- Indexes and points — the full data model: dimensions, distance metrics, payloads, ids, and what can and cannot change.
- Search — query patterns: top-k, filters, scores, browsing, and deleting points.
- Use with agents — how the agent memory bank relates to VectorDB.
- API reference — every endpoint, field, and error.