Skip to main content

Advanced: webhook fan-out, exactly once (Node.js)

Every webhook sender retries. They retry on a timeout, on a 500, and on a response that arrived after they gave up waiting — so a delivery you handled perfectly can still arrive again. A receiver that publishes on every call double-charges, double-emails, or double-anything.

The fix is not "be fast enough not to time out." It is to make the second delivery a no-op.

Source: examples/functions/webhook-fanout.

What you need​

  • A project and platformctl login.
  • A MemoryStore instance (the claim store) and a Pub/Sub topic.
  • About 25 minutes.

The mechanism, in three steps​

inbound delivery
      │
      ▼
 1. find the SENDER'S delivery id      retries carry the same one
      │
      ▼
 2. claim it: SET key NX EX 86400      atomic; null means someone won already
      │
      ├── null ──▶ 200 {"status":"duplicate"}      done, no publish
      │
      ▼ "OK"
 3. publish to Pub/Sub
      │
      ├── fails ──▶ RELEASE the claim, then throw
      │
      ▼
   202 {"status":"accepted"}

Three details in that diagram are the whole guide.

The id must come from the sender​

handler.js
const ID_HEADERS = [
  'x-github-delivery',
  'x-stripe-event-id',
  'x-shopify-webhook-id',
  'x-request-id',
  'idempotency-key',
];

An id we generate is different on every retry and proves nothing. Only the sender knows that attempt 2 is the same event as attempt 1, and it says so in a header.

The order matters: x-request-id is last because some proxies regenerate it per attempt. A sender-specific delivery id is the one that is actually stable.

No delivery id is a 400, not a 500

A sender with no such header cannot be deduplicated, and accepting it would mean silently giving up the guarantee the function exists to provide. A 4xx also stops the sender retrying — which is correct, because retrying will not add the header. The response names the headers it looked for, so the sender can add one.

The claim must be atomic​

handler.js
const won = await r.set(`webhook:${id}`, String(Date.now()), { NX: true, EX: CLAIM_TTL_SECONDS });

SET ... NX returns null when the key already existed — exactly "someone else got here first". A GET followed by a SET has a race between the two calls, and a webhook burst is precisely when two deliveries arrive at once. That is not a rare interleaving; it is the normal case under retry storms.

EX is not optional either. Without a TTL the claim store grows forever. 24 hours is a deliberate margin over any sender's retry window.

The key is namespaced under webhook: because MemoryStore is shared with whatever else the project put there.

Claim before publishing, and release on failure​

Publishing first and recording second leaves a window where a crash between them loses the record but keeps the message — and the retry publishes again.

But claiming first creates the opposite hazard, and it is worse:

handler.js
  try {
    await publish(...);
  } catch (err) {
    // Release the claim so the sender's retry can succeed.
    await r.del(`webhook:${id}`).catch(() => {});
    throw err;
  }
Without that del, one transient failure loses the event permanently

The claim is held, so every retry is treated as a duplicate and answered 200. The sender sees success, stops retrying, and the message was never published. A five-second Pub/Sub blip becomes silent, permanent data loss.

This is the single most important line in the function, and it is the one most easily left out — because everything looks fine without it until the day Pub/Sub hiccups.

A duplicate is a 200, not a 409​

Telling a sender "conflict" invites a retry of something already handled. Duplicate suppression is a success: the event is safely recorded, exactly once, which is what the sender wanted.

Dependencies​

Node.js functions declare theirs in a package.json beside handler.js; npm installs the production dependencies at build time.

package.json
{
  "name": "webhook-fanout",
  "main": "handler.js",
  "dependencies": { "redis": "^4.7.0" }
}

Deploy it​

platformctl memorystore create webhook-claims
platformctl pubsub topics create webhook-events

printf '%s' "$SA_KEY" | platformctl secrets put pipeline-key
printf '%s' "$MS_URL"  | platformctl secrets put memorystore-url

platformctl functions deploy ./examples/functions/webhook-fanout --name webhook-fanout

platformctl secrets bindings set webhook-fanout PIPELINE_KEY --secret pipeline-key
platformctl secrets bindings set webhook-fanout MEMORYSTORE_URL --secret memorystore-url
platformctl secrets bindings apply webhook-fanout

The apply is not optional: binding records intent, applying rolls the revision that carries it.

Then publish it and point your sender at the address:

platformctl gateway publish function/webhook-fanout --name hooks --auth apikey

Prove it works​

Send the same delivery id twice:

curl -sS -X POST "$HOOK_URL" -H 'x-github-delivery: test-1' -d '{"n":1}'
curl -sS -X POST "$HOOK_URL" -H 'x-github-delivery: test-1' -d '{"n":1}'
{"status":"accepted","delivery_id":"x-github-delivery:test-1"}
{"status":"duplicate","delivery_id":"x-github-delivery:test-1"}

One message on the topic, two deliveries handled.

The numbers to compare against​

WhatNumber
Warm request, claim won, published25-60 ms
Warm request, duplicate (no publish)5-15 ms
Cold start, first request after idle1-3 s

A duplicate is roughly four times cheaper than a first delivery, because it does one MemoryStore round trip and stops. That matters under a retry storm, which is when this function does its most useful work.

Cold start versus the sender's timeout

Most webhook senders time out at 5-10 seconds, so a 1-3 second cold start is survivable but not free — the sender may already have queued a retry, which your function then correctly suppresses. If your sender is stricter, keep one instance warm:

platformctl agents config set webhook-fanout --min-scale 1

That instance costs compute continuously. It is the right trade only when the sender's timeout is tight.

Traps, at the point you hit them​

TrapWhat you seeWhy
No claim release on publish failureEvents vanish after a brief outageEvery retry reads as a duplicate. The del in the catch is what prevents this.
GET then SET instead of SET NXOccasional double publishes under loadThe race is between the two calls, and a burst is when it happens.
No EX on the claimMemoryStore fills up over monthsClaims are never forgotten.
Unprefixed claim keyCollides with another workload's cachePrefix it, as webhook:.
409 for duplicatesThe sender retries foreverIt reads a 4xx as "not handled".
MEMORYSTORE_URL hardcodedConnect timeout after 30 sThe instance hostname is private and opaque; read it from the environment.
Secret bound, still undefinedCannot read properties of undefinedBinding is not applying. Run secrets bindings apply.

Teardown​

platformctl gateway endpoint delete hooks
platformctl delete webhook-fanout
platformctl pubsub topics delete webhook-events
platformctl memorystore delete webhook-claims

What it costs to leave running. The function scales to zero and costs no compute while idle — unless you set --min-scale 1, which costs continuously. The Pub/Sub topic reserves its max_bytes from the project budget the moment it exists, whether or not anything is published. The MemoryStore instance runs continuously and is the largest standing cost here; it does not scale to zero.

Where next​