Skip to main content

Known issues

The Crusoe Agent Platform is in private alpha. This page lists what is missing, shared, or rough — deliberately, in public, with the workaround where there is one. Nothing here is a surprise waiting for you.

If you hit something that is not on this list and not on the troubleshooting page, that is a real bug. Report it with the request_id from the error.

Identity and sessions​

Sign-in sessions last 12 hours and cannot be revoked​

What it means for you. A sign-in token is a signed, self-contained credential with a 12-hour life. Signing out clears it from your browser, and deleting an account blocks future sign-ins — but neither kills a token that has already been issued. If a token leaks, it stays valid until it expires.

What still works. Roles and grants are re-read from the database on every request, so demoting someone, removing a grant, or revoking an API key bites on their very next call. Only the raw session token itself outlives the change.

Workaround. For anything automated, use an API key instead of a session token — keys are revocable instantly. See service accounts and API keys.

There is no multi-factor authentication​

What it means for you. Sign-in is email plus password only. There is no second factor, no hardware key, and no SSO.

Workaround. Use a long, unique password from a password manager (the minimum is 12 characters). Give machines their own service accounts rather than sharing a human login, so you never have to hand a password to a script.

Reaching the platform​

There is no public API hostname​

What it means for you. The web console at https://console.codyhill.dev is the public front door. The management API is not published on the internet, so a script on your laptop cannot simply curl a well-known URL.

Workaround. Two paths:

# 1. An endpoint your administrator gives you
export CAI_API=https://api.example.internal

# 2. Or let the CLI tunnel in using your cluster credentials (needs kubectl + KUBECONFIG)
platformctl list

With neither, platformctl reports auto port-forward failed (pass --api or set CAI_API instead). The four service groups have their own variables: CAI_SERVERLESS_API, CAI_PUBSUB_API, CAI_MEMORYSTORE_API, CAI_VECTORDB_API. See API authentication.

Published workloads cannot use your own domain​

What it means for you. A published workload gets https://<name>-<project-short>.apps.codyhill.dev, and only that. The API accepts a custom hostname field and stores it, but the platform ignores it and always computes the hostname itself.

Workaround. Put your own reverse proxy or CDN in front of the platform hostname if you need a branded URL. See public endpoints and domains.

The control plane runs a single replica​

What it means for you. The management API runs one instance, so a restart or an upgrade is a brief window where deploys and management calls fail. Already-running agents, functions, and services are unaffected — they serve traffic independently of the control plane.

Workaround. Retry management calls; they are safe to repeat. Do not schedule a bulk deploy during a maintenance window.

Compute​

Everything scales to zero, so the first call after idle is slow​

What it means for you. An idle workload runs zero instances. The first request has to start one and load your code, inside the 60-second invoke timeout. Heavy imports can push that first call over the line; the retry succeeds against a warm instance.

Workaround. Set minimum instances to 1 on anything latency-sensitive (the Configure modal in the console, or PATCH /v1/agents/{name}/config). That trades capacity and quota for predictable latency. See autoscaling and scale to zero.

Serverless revisions draw down the project's Services quota​

What it means for you. Every revision of a serverless service permanently holds two Kubernetes Services, so each deploy of the same service spends the quota again. When the Services quota fills, new revisions stop becoming ready — and because the quota is project-wide, unrelated workloads in the same project stop becoming ready too.

Workaround. Watch the Services line under Quotas in the console, delete services you no longer need, and ask an admin to raise the quota before a large rollout. See quotas and audit.

max_scale: 0 on a serverless service does not mean unlimited​

What it means for you. The API accepts and echoes max_scale: 0, but the platform runs that service with a ceiling of 10 instances. Do not rely on 0 meaning unbounded.

Workaround. Set the ceiling you actually want, explicitly.

"Internal" is not the same as "authenticated"​

What it means for you. An unpublished service is off the internet, but it is not authenticated. Any workload that can reach the platform's internal gateway can call it.

Workaround. If a service must know who is calling, check a credential inside the service itself.

The console cannot upload large or binary source​

What it means for you. Browser deploys are capped at 32 MiB per archive, 1 MiB per file, and 100-byte file paths, and the "write" and "files" modes handle text only. The platform itself accepts 100 MiB.

Workaround. Use the CLI for anything bigger or binary:

platformctl deploy ./my-agent --name my-agent

Or pack a .tar.gz yourself and use the console's archive mode, which passes bytes through unchanged. See install the CLI.

Functions have no schedule, object-store, or Pub/Sub triggers​

What it means for you. A function is HTTP-triggered (and accepts CloudEvents). If you want a cron schedule or an object-storage event, a function alone will not do it.

Workaround. Deploy a serverless service instead — triggers attach there. See also HTTP and events.

Agent sessions and memory​

The session store and memory bank are shared infrastructure​

What it means for you. This is the most important limitation on the page. Your project has its own namespace, quota, and network policy — but the storage that backs agent sessions and the agent memory bank is one shared deployment across all projects, not a per-project instance with its own credential. Names are separated per project and agent, but the isolation is by naming, not by a separate credential or a separate server.

Note the distinction: the MemoryStore and VectorDB services you create yourself are per-project resources with their own passwords and their own endpoints. It is only the storage that agents use automatically, behind the scenes, that is still shared.

Workaround. Treat agent session transcripts and memorized content as alpha-grade confidential — good enough for internal work, not for regulated or customer-identifying data. If you need hard isolation today, create your own MemoryStore instance and VectorDB index in your project and write to those from your agent code. See use MemoryStore with agents and use VectorDB with agents.

The memory bank is shared by every caller of an agent​

What it means for you. Memory belongs to the agent, not to a person. Anything one caller memorizes can be retrieved by any later caller of that agent.

Workaround. This is why memorize always requires authentication even though invoking does not. Memorize deliberately, and never memorize content that one caller shouldn't be able to read back. See memory.

The agent data plane is open by default​

What it means for you. Anyone who can reach an agent's URL can invoke it, without a credential. Management (deploy, logs, secrets, delete) always requires authentication; invoking does not.

Workaround. Do not put sensitive capability behind an alpha agent's data plane. A platform-wide setting can require authentication to invoke — ask your administrator if you need it on. See invoke.

Data services​

There are no backups, snapshots, or restores​

What it means for you. Deleting a VectorDB index destroys every vector in it. Deleting a MemoryStore instance destroys its data volume. Neither is recoverable, and there is no snapshot to roll back to.

Workaround. Keep the source of truth somewhere you control and be able to re-ingest. Delete is admin-only for exactly this reason.

MemoryStore instances are single-node​

What it means for you. One server, no replica, no failover. Persistence is on by default and survives a restart with roughly a one-second loss window, but a lost node means downtime while it restarts, and traffic inside the platform is unencrypted.

Workaround. Use MemoryStore for cache, counters, queues, and session state — things you can rebuild. Choose the eviction policy at creation (allkeys-lru for caches; the default noeviction fails writes at the limit, which is what you want for session state and not what you want for a cache). See MemoryStore overview.

VectorDB index settings are mostly immutable​

What it means for you. dimensions, distance, shards, on-disk storage, and quantization are fixed at creation. Only replicas and payload placement can be edited afterward.

Workaround. Create a second index with the settings you want, re-ingest, then delete the old one. Pick your embedding model before you create the index. See collections and points.

External access means "reachable from your VPC", not "on the internet"​

What it means for you. The optional external endpoints for MemoryStore, VectorDB, and Pub/Sub terminate at a gateway firewalled to your Crusoe VPC. They are reachable from your own virtual machines, not from the open internet. They are also not equally finished:

ServiceExternal status
MemoryStoreThe rediss:// route works
Pub/SubConnection and topic lookup are proven; produce and consume still need the in-cluster address
VectorDBA per-project access token is coded but switched off by default, so no external per-project token is issued and the external URL is withheld

Workaround. Run the workload inside the platform (an agent, function, or serverless service) and use the internal address, which always works. Turn external exposure off when you no longer need it — the credentials involved do not expire on their own.

Shared capacity, no per-project storage quota on VectorDB​

What it means for you. VectorDB runs on shared capacity with no per-project size limit, so a very large index in one project can slow searches for everyone.

Workaround. Keep indexes to what you actually query, and delete experiments when you're done with them.

Messaging​

The Pub/Sub budget is spent by claim, not by use​

What it means for you. A topic reserves its whole max_bytes the moment it exists. Four empty 16 MiB topics have already spent 64 MiB of the project's 1 GiB budget while every byte counter reads zero, and the next create can fail with a quota error that looks wrong.

Workaround. Delete unused topics or lower an existing topic's max_bytes. Consuming messages never frees budget. See publish and consume.

Ordered subscriptions cannot be read over HTTP​

What it means for you. exclusive and failover subscriptions admit exactly one reader at a time, which a load-balanced HTTP API cannot be. Pulling one returns a 409 explaining this, and creating one for pull delivery is refused up front.

Workaround. Use push delivery for ordered types, or use a shared or key-shared subscription when you can tolerate less strict ordering.

Retry limits apply to push delivery only​

What it means for you. On a pull subscription, a message you never acknowledge redelivers forever — max_deliver is enforced by the push worker, not by the pull API. And a push message that exhausts its attempts with no dead-letter topic configured is dropped.

Workaround. On pull, use the delivery_attempt field to decide when to give up. On push, always configure a dead-letter topic for anything you cannot afford to lose. See topics and subscriptions.

Message counters lag; the publish response is the truth​

What it means for you. Topic and subscription counters refresh on a background cycle and can be up to about five minutes stale. They also reset when a broker restarts, so they are not a running total.

Workaround. Trust the message ids in the publish response for what was actually stored. Sample and difference the counters if you need rates.

The direct broker credential never expires​

What it means for you. The token that lets an outside client talk to the broker directly has no expiry, and revoking one project's token means rotating a key shared by everyone.

Workaround. Only fetch it when you truly need direct access, and treat it as a long-lived production credential.

Durable execution​

Durable execution is a sample integration, not a managed product surface​

What it means for you. There is no REST API, no CLI command, and no console page for workflows. The workflow engine runs inside the platform, but you drive it by deploying your own worker and using the engine's own tooling.

What that also means. The workflow engine's endpoint is reachable only from inside the platform, and it has no platform authentication of its own — network position is the boundary. So a worker has to run in-cluster; you cannot point a laptop at it. All projects share one workflow namespace, and each engine component runs a single replica.

Workaround. Follow the worked sample end to end in run a workflow; read durable execution overview for what is and isn't in scope.

Secrets​

Applied secrets are written onto the workload in plaintext​

What it means for you. Applying bindings reads each value and writes it into the agent's own storage, because agent instances cannot yet authenticate to the secret store on their own for environment delivery. The value is protected by the project boundary rather than by an end-to-end encrypted path.

Workaround. For the highest-value credentials, fetch them at call time from inside your code instead of injecting them as environment variables — that path uses a short-lived, read-only, path-scoped token and never leaves a copy behind. See use secrets in workloads.

Bind and apply are two steps; unbind is one​

What it means for you. Recording a binding changes nothing until you apply it. Removing one, by contrast, takes effect immediately — it revokes the value and rolls a new revision. The asymmetry is deliberate: granting access should be explicit; removing it should never wait.

Workaround. After binding, apply. After rotating a secret, apply again — latest-tracking bindings only pick up the new version on the next apply or deploy. See manage secrets.

The platform model key is shared by everyone​

What it means for you. The platform-wide model credential is shared across all accounts. Nobody can read it back, but anyone who can deploy an agent can deploy one that spends it. Rotating it reaches each agent on that agent's next cold start, not immediately.

Workaround. Give an agent its own key if its usage must be billed or limited separately:

platformctl secrets set my-agent MODEL_API_KEY=sk-...

See secrets and env.

Observability​

There are no historical metrics or charts​

What it means for you. The platform reports live values — instances running, revision readiness, sandbox pool state, MemoryStore memory and hit rate — and stores no time series. There is no request-rate graph, no latency percentile, and no way to ask what happened an hour ago.

Workaround. Log what you need from inside your own code and read it back with log history (14-day retention by default). See logs.

The audit log has no export and no date-range filter​

What it means for you. Every project member can read the project's append-only audit trail, newest first, a page at a time — including break-glass grants taken by platform staff. But there is no download, no date-range query, and no retention policy you can set. The console's filter box only searches the rows already loaded on the page.

Workaround. Page back through the log and copy what you need, or read the audit endpoint yourself and store the entries somewhere you control. See break-glass and audit.

Stored agent source keeps no history​

What it means for you. The browser editor stores only the latest version of each file. There is no version history and no undo, and every upload replaces the whole stored source tree.

Workaround. Keep your code in your own version control and treat the platform's copy as a deployment artifact. See files and editor.

Scope of the alpha​

No regions, no service-level agreement​

What it means for you. This is one deployment. There is no region to choose, no multi-region failover, and no published availability target or support commitment. Compare that honestly against a hyperscaler before you put revenue-carrying traffic on it — see how we compare to AWS and the service mapping.

Workaround. Use the platform for the things it is genuinely good at right now: building, deploying, and iterating on agents fast, with real managed primitives behind them. Keep a copy of anything you cannot recreate.

Some things exist only in the console, and some only in the CLI​

What it means for you. Coverage is uneven while the surface settles:

TaskConsoleCLI
Deploy / invoke / log / delete agents and functionsYesYes
Create or delete a serverless service, VectorDB index, MemoryStore instance, Pub/Sub topicYesNo
Publish and pull Pub/Sub messagesYesYes
Project secrets, bindings, revealYesNo (only direct per-agent values)
Organizations, users, projects, members, invitations, break-glass, audit, quotas, service accounts, API keys, MCP serversYesNo
Serverless service logsYesNo
Upload larger than 32 MiB, or binary filesNoYes

Workaround. Anything either tool is missing is available over the API. See the CLI overview and the API overview.