Skip to main content

Chat models

Every agent needs a model to think with. This page explains where that model comes from, how to change it, and what happens when you bring your own.

The short version​

The platform injects a working chat model into every agent it deploys. You do not have to choose one, sign up for anything, or put a key in your code:

from crusoe_adk.foundry import foundry_model
from google.adk.agents import Agent

# Uses whatever chat model the platform wired up.
root_agent = Agent(model=foundry_model(), name="research_buddy", instruction="...")

That is the whole setup. The rest of this page is for when you want something different.

Where the model comes from​

Chat models are served by Crusoe Foundry, an OpenAI-compatible inference service. "OpenAI-compatible" means it speaks the same HTTP shape as the OpenAI API — POST /v1/chat/completions, the same request and response JSON — so any client written for OpenAI works against it by changing one URL.

At deploy time the platform sets three environment variables on your agent:

VariableWhat it isYours to change?
CHAT_MODELThe model id, e.g. zai/GLM-5.2Yes
MODEL_BASE_URLThe Foundry endpoint your workload should callYes
MODEL_API_KEYThe credential for that endpointInjected from a secret
Read the environment; do not write the URL

MODEL_BASE_URL is the address that routes from inside your project. The public hostnames in this documentation do not: a request to one from inside a workload hangs rather than failing, which is much harder to diagnose than an error. Always read the injected variable.

foundry_model() resolves all three for you, in this order:

  • model — the argument you passed, else $CHAT_MODEL, else the platform default
  • base URL — the argument, else $MODEL_BASE_URL, else the public Foundry URL
  • API key — the argument, else $MODEL_API_KEY

See what your agent is running on​

platformctl agents env get research-buddy

You should see the resolved values:

KEY              VALUE
CHAT_MODEL       zai/GLM-5.2
EMBED_MODEL      qwen-embedding
MODEL_BASE_URL   <the private inference address the platform injects>/v1
EMBED_BASE_URL   <the private inference address the platform injects>/v1

MODEL_API_KEY is not listed here. Values of secrets are never returned, by this command or the API behind it — platformctl agents secrets get research-buddy answers whether it is set, and nothing answers what it is set to.

Pin a different model​

Three ways, from least to most binding.

1. In the agent's environment​

The model becomes a deploy-time setting, changeable without touching code.

platformctl agents env set research-buddy CHAT_MODEL=nvidia/NVIDIA-Nemotron-3-Super-120B-A12B

2. In code​

The model becomes part of what the agent is, and travels with the source.

root_agent = Agent(
    model=foundry_model("nvidia/NVIDIA-Nemotron-3-Super-120B-A12B"),
    name="research_buddy",
    instruction="...",
)

Code wins over environment: foundry_model() only falls back to $CHAT_MODEL when you passed no argument.

3. Per-call settings​

foundry_model() passes any extra keyword straight through to the underlying client, so sampling settings live next to the model:

model = foundry_model(temperature=0, max_tokens=1024)

Set temperature=0 when you want the same question to give the same answer — for a test, or for a step whose output another step parses.

Bring your own model​

The platform does not lock you to Foundry. Declare a native model and it is used as-is:

# Vertex / Gemini
root_agent = Agent(model="gemini-2.5-flash", name="research_buddy", instruction="...")
# Any LiteLLM-supported provider
from google.adk.models.lite_llm import LiteLlm

root_agent = Agent(model=LiteLlm(model="openai/gpt-4o"), name="research_buddy", instruction="...")
Your provider needs your credential

The platform injects Foundry's key and only Foundry's. A third-party provider needs its own, which belongs in Secrets Manager — never in the source you deploy:

platformctl secrets set research-buddy OPENAI_API_KEY=sk-...

The value arrives as an environment variable in the agent, and never appears in the console, the API, or a build log.

Call a chat model directly​

Agents are the common case, but nothing stops a function or a service from calling the model itself — a classifier in a Pub/Sub consumer, say, or a summariser in an object-store trigger.

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["MODEL_BASE_URL"],
    api_key=os.environ["MODEL_API_KEY"],
)

reply = client.chat.completions.create(
    model=os.environ.get("CHAT_MODEL", "zai/GLM-5.2"),
    messages=[{"role": "user", "content": "Summarise this in one sentence: ..."}],
)
print(reply.choices[0].message.content)
Only agents and MCP servers get the model environment

A plain serverless service or function is deployed without MODEL_BASE_URL, CHAT_MODEL or MODEL_API_KEY — the container contract deliberately carries no platform credentials. To call a model from one, put the endpoint and key in the workload's own secrets.

What this does not do​

  • It does not host your own weights. Foundry serves a catalog; deploying a fine-tune of your own is not part of this surface.
  • It does not cache responses. Two identical prompts are two model calls and two charges.
  • It does not retry for you. A model error surfaces to your code as an error from the client library, so you decide whether a retry is safe.

Next steps​