Embeddings

An embedding is a list of numbers that stands for a piece of text. Two texts that mean similar things get similar numbers, which is what makes "find me documents about X" work when the documents never use the word X.

This page covers generating them. Storing and searching them is VectorDB.

The model

Model id	Output	Task
qwen-embedding	4096 numbers per input	Dense text embedding

4096 is the dimension, and it is not a setting — it is a property of the model. An index built for a different width will reject these vectors, so the number has to match in both places. Read it from the platform rather than typing it:

platformctl platform limits -o json

Do not hardcode the dimension

The console once carried its own copy of this number, it was wrong, and every index created from that page was unusable. If you need the width in code, read EMBED_MODEL and ask the platform; do not paste 4096 into a config file.

Generate one

platformctl

platformctl inference embed "a sailboat on the water"

curl

curl -s "$CAI_API/v1/embeddings" \
  -H "Authorization: Bearer $CAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen-embedding", "input": "a sailboat on the water"}'

You should see:

{
  "object": "list",
  "data": [{"object": "embedding", "index": 0, "embedding": [0.0123, -0.0456, "..."]}],
  "model": "qwen-embedding",
  "usage": {"prompt_tokens": 6, "total_tokens": 6}
}

Console

Embedding is an API operation; the console does not have a text box for it. What the console does show is the embedding width your project's indexes must use, under VectorDB → Create index.

Generate many at once

input takes an array. One request with 64 chunks is dramatically cheaper in round trips than 64 requests, and the results come back in the order you sent them:

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["EMBED_BASE_URL"],   # injected; do not write a URL
    api_key=os.environ["MODEL_API_KEY"],
)

chunks = ["...", "...", "..."]              # your document, already split
resp = client.embeddings.create(model=os.environ["EMBED_MODEL"], input=chunks)

vectors = [d.embedding for d in resp.data]  # same order as `chunks`

Order is the contract, not index

data[i] corresponds to input[i]. Each item also carries index, which says the same thing — use whichever you like, but do not assume the server sorted anything.

Batch size is a trade: bigger batches mean fewer round trips and a larger request body. If a batch is refused for size, halve it and retry — the operation is stateless, so a retry costs nothing but time.

Chunking before you embed

An embedding of a whole 40-page document is an average of forty pages, and averages match nothing well. Split first.

A reasonable default that needs no tuning to be useful:

• Split on structure — headings, then paragraphs — rather than every N characters. A chunk that ends mid-sentence embeds badly.
• Aim for a few hundred words per chunk.
• Overlap slightly (a sentence or two) so a fact that straddles a boundary is in both chunks.
• Keep the source with the chunk — file name, heading, URL. You will want to cite it, and adding it later means re-embedding everything.

This is exactly the job an object-store-triggered function is for: a document lands in a bucket, the function splits it, embeds the chunks, and writes them to an index. Nothing has to be running when the file arrives.

Store them

The vector alone is not useful — you need to get back to the text. Write each vector to a VectorDB point with a payload:

points = [
    {
        "id": f"{doc_id}-{i}",
        "vector": vec,
        "payload": {"text": chunk, "source": doc_path, "heading": heading},
    }
    for i, (chunk, vec) in enumerate(zip(chunks, vectors))
]

A stable id derived from the document and chunk number means re-indexing the same document overwrites rather than duplicating — so a re-run is safe, and a document that changed does not leave its old chunks behind.

Use them from an agent

An agent's memory bank embeds automatically: everything committed to memory is indexed with qwen-embedding, and search_memory retrieves against it. You do not call this API for that.

You call it yourself when you are building your own corpus — product docs, tickets, a wiki — and exposing it to the agent as a tool or an MCP server.

Where to call it from

Caller	Base URL to use
An agent, function or MCP server on the platform	$EMBED_BASE_URL (injected)
Your laptop, or CI	$CAI_API — /v1/embeddings routes to Inference
An OpenAI SDK client	Either of the above as base_url

The public hostname does not route from inside a project. A workload that hardcodes one hangs rather than erroring, which is the worst way for this to fail — so read EMBED_BASE_URL.

Next steps

• Reranking — reorder the results this retrieves.
• VectorDB quickstart — create an index and write points.
• Search — query it, and combine with a filter.
• Object-store triggers — index a document the moment it is uploaded.