Autoscaling and scale to zero

This page explains how the platform decides how many copies of your service to run, what happens when that number is zero, and which three knobs you control. We follow one example service, checkout-api, through the whole page.

The three knobs

Every service has a scaling block with three settings:

{"scaling": {"min_scale": 0, "max_scale": 10, "container_concurrency": 80}}

• min_scale (default 0) — the floor: the fewest instances kept running, even with no traffic. 0 means the service is allowed to scale to zero.
• max_scale (default 10) — the ceiling: the platform will never run more instances than this, no matter the load.
• container_concurrency (default 80) — how many requests one instance handles at the same time before the platform adds another instance.

An instance is one running copy of your container. The math is simple: if 200 requests arrive at once and each instance handles 80, the platform runs 3 instances (200 ÷ 80, rounded up) — as long as 3 is between your floor and ceiling.

In the console, these appear in the deploy dialog's Advanced options as Minimum instances, Maximum instances, and Concurrent requests per instance.

max_scale: 0 does not mean unlimited

Today the platform treats max_scale: 0 (or a negative value) as the default of 10 when it configures the autoscaler. Do not set 0 expecting unbounded scaling — you will get a ceiling of 10. If you need a higher ceiling, set the number you actually want.

container_concurrency: 0 is different: it means "use the serverless default," which is effectively unbounded requests per instance. Negative values for any of the three are rejected with a 400, as is min_scale greater than max_scale.

The life of an idle service

checkout-api is deployed with the defaults (0 / 10 / 80). Here is its day:

1. Traffic arrives. One instance starts and serves requests. Load rises to ~160 concurrent requests; a second instance is added automatically.
2. Traffic stops. After roughly 90 seconds of no requests (the platform's stability window plus a grace period), instances drain to zero. The service now costs nothing in compute.
3. A request arrives at 3 a.m. This is a cold start: the request waits while the platform pulls up a fresh instance — expect a few seconds — then it is served normally. Requests right after it are fast, because the instance is now warm.

You can watch this live. The metrics endpoint (also shown on the console's Overview tab) reports the current instance count:

curl -sS -H "Authorization: Bearer $CAI_TOKEN" \
  "$CAI_SERVERLESS_API/v1/projects/$PROJECT/services/checkout-api/metrics" \
  | jq '.instances'

You should see:

{"current": 0, "desired": 0}

Honest note: metrics are a live readout only. There is no metrics database behind them yet, so historical charts — request rate over time, latency percentiles — are not available in the alpha, and the endpoint's note field says so plainly.

Asking for logs while scaled to zero is not an error. You get a 200 with an explanation:

no running pods: this service is scaled to zero. Send it a request and the logs will appear here.

Testing scale-to-zero

If you script around scale-to-zero (for example in CI), poll for the instance count to reach zero with a generous timeout of about 3 minutes, rather than sleeping a fixed 90 seconds. The drain time is approximate, not exact.

Keeping one instance warm

If a multi-second cold start is unacceptable — say checkout-api sits in a user-facing checkout flow — set the floor to one:

curl -sS -X PATCH "$CAI_SERVERLESS_API/v1/projects/$PROJECT/services/checkout-api" \
  -H "Authorization: Bearer $CAI_TOKEN" -H 'Content-Type: application/json' \
  -d '{"scaling": {"min_scale": 1, "max_scale": 10, "container_concurrency": 80}}'

The trade is explicit: min_scale: 1 means no cold starts, and also means one instance running (and drawing on your project's CPU/memory quota) around the clock. That is the whole deal — there is no separate "provisioned concurrency" product or pricing tier to learn.

A PATCH merges the scaling block

Notice the PATCH above sends all three scaling fields even though only min_scale changed. That is a good habit — it states your full intent, and it is what the console does — but it is not required. A PATCH scaling block is merged onto the service's current scaling, so a sub-field you omit keeps its current value; it does not fall back to the platform default.

Request timeouts

Each request gets up to timeout_seconds to complete — default 300, settable from 1 to 3600. The platform injects the value into your container as the environment variable CRUSOE_REQUEST_TIMEOUT_SECONDS, so your code can read its own deadline instead of hard-coding one.

Summary

Setting	Default	Range / rule	What it does
min_scale	0	≥ 0; must be ≤ max_scale	Floor. 0 allows scale to zero; 1 keeps a warm instance and eliminates cold starts
max_scale	10	≥ 0; 0 currently behaves as 10	Ceiling on instances under load
container_concurrency	80	≥ 0; 0 = platform default (effectively unbounded)	Simultaneous requests per instance before another is added
timeout_seconds	300	1–3600	Per-request deadline; mirrored into CRUSOE_REQUEST_TIMEOUT_SECONDS
Idle-to-zero time	~90 s	—	How long a service with min_scale: 0 sits idle before draining to zero
Cold start	a few seconds	—	Delay on the first request after idle

Next: publish your service to the internet, or see troubleshooting if scaling is not behaving as expected.