Topics and subscriptions

This page explains the whole Pub/Sub model: what a topic and a subscription really are, how your project's storage budget works, how long messages live, and which settings are locked after creation. One example runs through the page: an orders topic with a workers subscription.

Topics

A topic is a named channel you publish messages to. Creating one is an admin operation (REST or console; the CLI cannot create topics yet).

Field	Required	Default	Meaning
name	yes	—	Lowercase letters, digits, and dashes; must start with a letter, must not end with a dash; 63 characters max. Immutable — it is part of the broker address.
display_name	no	empty	A human label. Freely editable.
max_bytes	no	16Mi	The most unacknowledged backlog this topic may hold, as a Kubernetes quantity (16Mi, 1Gi). This amount is claimed from your project budget immediately.
max_age	no	none	Oldest a retained message may get, as a duration (24h, 30m). Empty means no age limit.
discard	no	old	What happens when the topic is full: old drops the oldest messages to make room; new refuses new publishes with a 409.

A newly created topic answers with "state": {"phase": "Pending", "ready": false}. The resource exists immediately; the broker side converges within a few seconds. Publishing before it is ready returns 409: this topic is not ready yet: <reason>.

Every topic also reports a broker address like persistent://p-<short>/main/orders. You only need it for direct Pulsar clients — the HTTP API never asks for it.

The storage budget is spent by claim

Your project has a Pub/Sub storage budget — 1 GiB by default. Here is the sharp edge:

Empty topics still spend budget

A topic reserves its full max_bytes the instant it exists. Four empty 16 MiB topics have spent 64 MiB of budget while every byte counter reads zero. When a create or a max_bytes raise doesn't fit, you get a 409 like:

this project's Pub/Sub storage budget is 1.0GiB, of which <N> is already claimed by <n> topic(s); a topic of <M> does not fit. Delete a topic, lower an existing topic's max_bytes, or ask the platform operator to raise the project's quota.

Consuming messages does not free budget. Only deleting a topic or lowering its max_bytes does.

Check where you stand at any time:

curl -s "$API/v1/projects/$CAI_PROJECT/pubsub/quota" \
  -H "Authorization: Bearer $CAI_TOKEN"

You should see:

{"limit_bytes":1073741824,"allocated_bytes":67108864,"backlog_bytes":0,"storage_bytes":0,
 "available_bytes":1006632960,
 "human":"64.0MiB of 1.0GiB claimed by 4 topic(s); 0B stored, 0B unacknowledged",
 "topics":4,"provisioned":true}

The console's Pub/Sub page shows the same numbers as a two-segment meter: bytes claimed versus bytes actually stored. Other per-project defaults: 100 topics, and 100 producers / 100 consumers per topic.

Message retention

A message is retained only while some subscription still owes an acknowledgement for it. This has three practical consequences:

1. Create subscriptions before publishing. Publishing to a topic with no subscriptions succeeds and returns a message ID, but the message is reclaimed on the broker's own schedule. The console even warns you before you publish to a subscription-less topic.
2. Once every subscription has acknowledged a message, it is deleted. There is no replay of acknowledged history.
3. A new subscription with start_from: all may still see recent stranded messages — treat that as a grace period, not a guarantee.

max_age adds a second limit: messages older than it are dropped even if unacknowledged.

Subscriptions

A subscription is a durable reader of one topic. It remembers its position, counts delivery attempts per message, and survives your consumers restarting. Each subscription on a topic receives its own copy of every message.

Field	Required	Default	Meaning
name	yes	—	Same naming rules as topics.
type	no	shared	Ordering model — see below. Immutable.
ack_deadline_seconds	no	30	How long you may hold a pulled message before it is redelivered. Range 1–600. For push, this is also the per-request timeout.
max_deliver	no	5	Attempts before a message is dead-lettered (push only — see the gotcha below).
start_from	no	all	all starts from the earliest retained message; new starts from now. Applied at creation only. Immutable.
max_ack_pending	no	1000	Cap on messages delivered but not yet acknowledged.
deliver.mode	no	pull	pull (your code fetches) or push (the platform POSTs to your service). Details in Publish and consume.
dead_letter	no	none	{"topic": "...", "after_attempts": 5} — where repeatedly failing push messages are parked. The topic must already exist in your project and must differ from the subscription's own topic.

Subscription types: the ordering choice

The type is the queue-versus-stream decision, fixed at creation:

Type	Plain words	Ordering	Pull via API?
shared	A queue. Competing consumers each take messages.	None	Yes
key-shared	A queue ordered per key. All messages with the same key go to the same reader, in order.	Per key	Yes
exclusive	A stream with exactly one reader.	Total	No — push or direct client only
failover	Like exclusive, but a standby reader takes over if the active one dies.	Total	No — push or direct client only

Ordered types (exclusive, failover) admit one reader at a time, so they cannot be served by the load-balanced HTTP API. Creating one with deliver.mode: pull is refused with:

a "exclusive" subscription admits one reader at a time, so it cannot be read through
this API. Use deliver.mode "push", read it with a direct client, or choose type
"shared" or "key-shared".

max_deliver only counts for push

max_deliver is enforced by the push worker only. On a pull subscription, a message you never acknowledge keeps coming back forever — use the delivery_attempt field in the pull response to decide when to give up.

What you can change later

Updates use PATCH with pointer semantics: fields you omit keep their value.

Resource	Editable in place	Locked at creation
Topic	display_name, max_bytes, max_age (empty string clears it), discard	name (it is the broker address)
Subscription	display_name, ack_deadline_seconds, max_deliver, max_ack_pending, push URL and content mode, dead_letter ({"topic":""} clears it)	topic, type, start_from

To change a locked field, delete and recreate. Raising a topic's max_bytes re-checks the budget and can 409 the same way a create can.

Deleting a topic deletes all of its subscriptions along with every retained message. The API answers 202 {"name":"orders","status":"deleting"} and finishes asynchronously.

Tenancy: your project is the boundary

Each project maps to its own isolated Pulsar tenant (p-<short>), with all topics in one namespace (main). Members can read everything and publish, pull, and acknowledge; admins can additionally create, edit, and delete topics and subscriptions and fetch direct broker credentials. Resources in other projects answer 404 — the API will not confirm they exist.

Summary checklist

• Name topics and subscriptions with lowercase letters, digits, and dashes (63 chars max, start with a letter).
• Budget is claimed by max_bytes at creation — size topics honestly, delete what you don't use.
• Create subscriptions before publishing; retention exists only for unacknowledged messages.
• Pick type carefully: it is immutable, and ordered types can't be pulled through the API.
• Set a dead_letter topic on any push subscription whose messages you cannot afford to lose.
• Remember max_deliver does nothing on pull subscriptions — watch delivery_attempt yourself.

Next: Publish and consume for the message-path details, or the API reference for every field and error.