Skip to main content

Pub/Sub overview

This page explains what Pub/Sub is in plain words, when to reach for it, and how it maps to the messaging services you may already know from AWS, GCP, or Azure.

What it is​

Pub/Sub is the platform's managed messaging service. It lets one part of your system send a message without knowing — or waiting for — whoever will read it.

The model has three pieces:

  • A topic is a named channel you send messages to.
  • A subscription is a durable reader attached to a topic. "Durable" means it remembers its position: if your consumer is offline for an hour, the messages wait for it.
  • Publish and consume are the two verbs. You publish messages to a topic; each subscription on that topic receives its own copy.

Everything works over plain HTTP with a normal bearer token. There is no client library to install, no long-lived connection to hold open, and no message-broker expertise required.

Consuming works two ways:

  • Pull: your code asks the subscription for waiting messages, processes them, and acknowledges each one when it's done.
  • Push: the platform POSTs each message to an HTTP service running inside your project and retries automatically until your service accepts it (or a dead-letter limit is reached).

When to use it​

Use Pub/Sub when you want to:

  • Decouple services, so a producer can keep working even when a consumer is slow or down.
  • Fan out one event to several independent consumers (each subscription gets its own copy).
  • Buffer bursts of work into a queue that workers drain at their own pace.
  • Trigger a function or other HTTP service in your project whenever something happens, via push delivery.

If instead you need a multi-step process with retries and state that survives crashes — "call the agent, then call it again with the result" — that is Durable Execution, not messaging.

The 30-second mental model​

A message is kept only while some subscription still owes an acknowledgement for it. Once every subscription has acknowledged it (or if no subscription exists to want it), it is gone. That is why you create subscriptions before you publish.

Isolation: every project is its own tenant​

Every project gets its own fully isolated messaging tenant with its own storage budget — 1 GiB by default. Nothing you publish is visible to, or affected by, any other project. Topics, subscriptions, quota, and credentials all live inside your project; the API refuses to even confirm whether another project's resources exist.

Under the hood the broker is Apache Pulsar, run and managed for you. You never touch Pulsar directly unless you ask for it: project admins can fetch a direct-access credential (a pulsar:// URL and token) for advanced clients. See Publish and consume for the honest caveats on that path.

Delivery guarantee​

Delivery is at-least-once: every message arrives, but a message can occasionally arrive twice. Write consumers so that handling the same message twice is harmless (this is called being idempotent). Ordering is opt-in per subscription — see Topics and subscriptions.

How it compares​

If you know...Ours is...Honest differences
GCP Pub/SubThe closest match: topics, subscriptions, pull and push, ack deadlines, dead-letter topics, even the :publish-style URLs.GCP offers exactly-once delivery, global multi-region replication, and message filtering. We are at-least-once, single-cluster, no filter language — filter in your consumer.
AWS SNS + SQS + EventBridgeOne primitive instead of three: a topic with multiple subscriptions gives you SNS-style fan-out; a topic with one shared subscription behaves like an SQS queue; push delivery covers the EventBridge-to-target pattern.AWS has SaaS event sources, content-based routing rules, FIFO queues with exactly-once processing, and published SLAs. We have none of those yet — and no decision guide needed, because there's only one service.
Azure Service Bus + Event GridService Bus topics/subscriptions map almost one-to-one; key-shared subscriptions approximate sessions (per-key ordering); Event Grid's HTTP push maps to our push delivery.Service Bus has transactions, scheduled messages, and larger enterprise features. We keep the surface small: HTTP in, HTTP or pull out.
Alpha honesty

This platform is in alpha. There is no public API hostname yet — automation reaches the Pub/Sub API through a port-forward or an admin-provided endpoint, and the web console is the public front door. The CLI can list, publish, and pull, but creating topics and subscriptions is done in the console or via the REST API. No SLAs, one region: your cluster.

Where to go next​