Production

Running ZBSearch Edge in production means protecting your API, offloading rebuilds from the Worker, and monitoring index health. This page covers the recommended production setup.

Cloudflare plan requirements

The IndexCoordinator Durable Object (WAL batching and rebuild locking) uses a SQLite-backed class, which is available on both plans - but without the Paid plan's CPU budget the API is not usable for search.

Authentication

Always set API keys before exposing the Worker publicly — a full-access write key, and a read-only key you can safely embed in public clients:

wrangler secret put WRITE_API_KEY --config deploy/cloudflare/wrangler.toml
wrangler secret put READ_API_KEY --config deploy/cloudflare/wrangler.toml

Clients must send:

Authorization: Bearer YOUR_SECRET_KEY

The read-only key covers searches and index/metadata reads only; it gets 401 on any write route. The legacy single API_KEY secret still works as a full-access key.

Public endpoints (no key required):

• GET /health
• GET /v1/info
• OPTIONS *

External rebuilds (recommended)

In-Worker rebuilds work for small indexes but share the Worker's CPU and memory limits. For production:

1. Set BUILDER_WEBHOOK_URL on the Worker
2. Run rebuilds via GitHub Actions, a cron job, or any CI that can call the rebuild CLI

When the webhook is configured, the Worker's cron does not rebuild inline - it only notifies your external runner.

wrangler secret put BUILDER_WEBHOOK_URL --config deploy/cloudflare/wrangler.toml

The Worker sends:

POST /your-webhook-url
Content-Type: application/json

{"indexId":"products","source":"scheduler"}

Your runner then executes:

source deploy/cloudflare/.env
pnpm --filter @zbsearch/edge-index-builder build
node packages/edge-index-builder/dist/cli.js rebuild products
# or
node packages/edge-index-builder/dist/cli.js rebuild --all

For initial or full catalog reloads, use import instead of per-document API writes - see Indexing & rebuilds.

GitHub Actions setup

The monorepo includes a reference workflow at deploy/github-actions/zbsearch-rebuild.yml.

1. Copy the workflow

Copy it to .github/workflows/zbsearch-rebuild.yml in your fork or deployment repo.

2. Add repository secrets

In GitHub -> Settings -> Secrets and variables -> Actions, add:

3. Trigger options

The reference workflow supports three triggers:

To connect the Worker webhook to GitHub dispatch, point BUILDER_WEBHOOK_URL at the GitHub API:

https://api.github.com/repos/OWNER/REPO/dispatches

With headers configured in your webhook proxy, or use a small relay (e.g. Cloudflare Worker, AWS Lambda) that translates the Worker's POST into a GitHub repository_dispatch with a personal access token.

4. Verify CI rebuild

After a workflow run, check index status:

curl "$WORKER_URL/v1/indexes/products/status" \
  -H "authorization: Bearer $API_KEY"

Confirm pendingOps is 0 and lastRebuildAt is recent.

Environment reference

Worker secrets (via wrangler secret put)

Worker vars (wrangler.toml)

Worker bindings (wrangler.toml)

Caching behavior

Search latency is dominated by which cache layer serves the snapshot:

Hot indexes on warm isolates skip both deserialization and network I/O entirely. Cold isolates in a warm colo pay one Cache API read plus deserialization; a new colo pays one R2 read. Expect the isolate cache to absorb the large majority of search traffic on any index queried more than occasionally, and the Cache API to cover most of the rest - R2 reads should be rare outside of fresh colos and just-rebuilt versions. Rebuilds change the version key, so post-rebuild traffic re-warms once per colo rather than serving stale data.

Rebuild CLI env

Build the CLI before first use (or after upgrading):

pnpm --filter @zbsearch/edge-index-builder build

Monitoring

Health checks

Point uptime monitoring at GET /health. No auth required.

Index health

Poll GET /v1/indexes/:id/status for:

• pendingOps - growing unbounded means rebuilds are failing or too infrequent
• status - building longer than your rebuild duration indicates a failed rebuild. With the coordinator binding, the abandoned lock becomes stealable after 15 minutes, so the next rebuild attempt recovers on its own.
• lastRebuildAt - stale timestamp means writes are not being merged

Cloudflare observability

wrangler.toml enables observability by default:

[observability]
enabled = true

Use the Cloudflare dashboard for Worker invocations, errors, and CPU time. Watch for CPU limit errors during in-Worker rebuilds - migrate to external rebuild if you see them.

Security checklist

• WRITE_API_KEY and READ_API_KEY set and enforced on all /v1/* routes; only the read key reaches public clients
• R2 API tokens scoped to the minimum bucket permissions
• R2 credentials stored as GitHub secrets, not in the repo
• Custom domain with Cloudflare SSL if exposing publicly
• Consider Cloudflare Access or WAF rules for additional protection
• External rebuild runner in a trusted environment (CI, not public HTTP)

Scaling guidance

Measured locally with the load-testing harness (~1.3 KB/doc snapshots, single-process wrangler dev - re-run against your deployment for authoritative numbers):

v1 stores full msgpack snapshots per shard. A group search that fans out in-process must hold every shard snapshot in one isolate (8× 16.5 MB = 132 MB > 128 MB - measured: 15-22 s latencies and OOM 503s at 100k docs). The ShardSearch Durable Object gives each shard a private isolate with a warm cache, which is the supported architecture past ~50k docs total. As a rule of thumb, at ~1.3 KB/document a shard of 25–50k documents keeps each snapshot comfortably under the default 64 MB SNAPSHOT_CACHE_MAX_BYTES, which also bounds the re-indexing cost of dirty searches (WAL overlay) per shard. Very large indexes will hit Worker memory limits during in-Worker rebuild - use the external builder and watch indexSizeBytes in /status.

One more rebuild caveat, learned the hard way in production testing: in-Worker rebuilds replay a shard's whole WAL backlog in a single invocation. Rolling rebuilds at the default 500-op threshold sail through 100k+ docs, but a large one-shot backlog (bulk API ingest faster than rebuilds drain, roughly 10k+ pending ops per shard) does not complete - the status wedges at building and retries. For bulk loads, use the external builder (zbsearch-edge-builder import) instead of the write API.

Cost considerations

You pay Cloudflare for:

• Workers - requests and CPU time (rebuilds in-Worker consume CPU ms)
• R2 - storage and Class A/B operations (WAL appends, snapshot reads)

Search-heavy workloads benefit from the per-isolate snapshot cache and Workers Cache reducing R2 reads and deserialization cost. Write-heavy workloads are batched into WAL segments (roughly 100 ops / 256 KB per R2 object when the coordinator is bound) and trigger periodic full snapshot rewrites.

Disaster recovery

R2 objects are the source of truth:

• indexes/*/meta.json - index registry state
• indexes/*/*/snapshot.msgpack - searchable data
• wal/* - unmerged writes (append-only)
• buffer/* - legacy unmerged writes (read during migration)

To recover: redeploy the Worker pointing at the same bucket. Unmerged WAL entries will be included in the next rebuild. Keep R2 versioning or lifecycle rules if you need point-in-time recovery.

Upgrading

When updating from a new zbsearch release:

git pull
pnpm install
pnpm --filter @zbsearch/edge-core build
wrangler deploy --config deploy/cloudflare/wrangler.toml

Rebuild indexes after engine upgrades if the snapshot format changes (check release notes). No automatic migration runs in v1.