Skip to content

feat: add webhooks via Cloudflare Queues and Cron Triggers#142

Open
ephraimduncan wants to merge 6 commits into
mainfrom
feat/webhooks-cloudflare
Open

feat: add webhooks via Cloudflare Queues and Cron Triggers#142
ephraimduncan wants to merge 6 commits into
mainfrom
feat/webhooks-cloudflare

Conversation

@ephraimduncan

Copy link
Copy Markdown
Contributor

Summary

Adds form webhooks — when a submission arrives, formbase POSTs it to a user-configured URL with retries — built entirely on Cloudflare-native primitives. This replaces the approach in #121 (BullMQ + Redis + a separately deployed Fly.io worker), which required an always-on external service.

  • Cloudflare Queues handle delivery, retry/backoff, and a dead-letter queue.
  • Cloudflare Cron Triggers run a safety-net sweep (re-drives stuck deliveries) and daily log cleanup.
  • No Redis, no BullMQ, no separate service.

How it works

POST /api/s/[id] / form.testWebhook
  → insert webhook_delivery_logs row (pending)
  → after() → WEBHOOK_QUEUE.send({ deliveryLogId, webhookUrl })   (deferred, never a floating promise)
        → worker.ts queue(): deliver via fetch + HMAC sign → mark success / retry w/ backoff
        → exhausted → DLQ → mark failed
  worker.ts scheduled(): */5 sweep stuck rows · 0 3 cleanup logs >90d

The custom OpenNext worker entry (apps/web/worker.ts) re-exports the generated fetch handler + cache DOs and adds queue()/scheduled(). It hydrates process.env from the bindings before dynamically importing the consumer/scheduled modules, so the workerd handlers can reach the validated env (which next dev's request context normally provides).

What's better than #121

  • HMAC signing (was unsigned): WebCrypto HMAC-SHA256 over timestamp.body, sent as X-Formbase-Signature / X-Formbase-Event / X-Formbase-Timestamp, with a per-form webhook_secret.
  • No floating promise: enqueue is deferred with after() (the pattern that caused the submission hang fixed in fix: prevent submission endpoint hang on Cloudflare Workers #141), not void promise.
  • SSRF-hardened URL validation: production requires public HTTPS and rejects localhost/private ranges.
  • Delivery visibility: a read-only "Recent deliveries" list + signing-secret field in form settings (the webhook_delivery_logs table in feat: add webhook queue system and worker #121 was never surfaced).
  • Idempotency: at-least-once delivery with skip-on-success; submission id is a stable idempotency key for receivers.
  • The signing secret is never exposed through the public getFormById procedure (explicit column allowlist).
  • The cron sweep atomically leases stuck rows (UPDATE … SET next_retry_at … RETURNING) so it can't double-deliver rows still in the queue's own retry window.

Changes

  • schema: webhook_delivery_logs table; forms.enable_webhook / webhook_url / webhook_secret; migration 0003_awesome_inhumans.
  • packages/api/lib/webhook.ts: pure, runtime-agnostic payload/log/query helpers.
  • apps/web/src/lib/webhooks/: producer, deliver (HMAC), consumer (backoff + DLQ), scheduled (sweep + cleanup).
  • tRPC: form.update webhook fields + server-generated secret, form.testWebhook, form.listDeliveries; setFormData now returns { id }; queue threaded through context.
  • config: wrangler.jsonc queues/DLQ/cron + main./worker.ts; cloudflare-env.d.ts gitignored; tsconfig checkJs:false.
  • Deleted the stale apps/worker/ and packages/queue/ leftovers from feat: add webhook queue system and worker #121.

Verification

  • @formbase/db, @formbase/utils, @formbase/api: typecheck clean.
  • apps/web: no typecheck/lint errors in any webhook file.
  • Reviewed by an adversarial multi-lens pass (workerd runtime, queue/retry/idempotency, HMAC, type boundaries, migration, SSRF).

Before deploying

  1. Create the queues: wrangler queues create formbase-webhooks and wrangler queues create formbase-webhooks-dlq.
  2. Apply the migration (packages/dbdb:migrate).
  3. Queue/cron handlers only run under wrangler dev / opennextjs-cloudflare preview (not next dev); validate end-to-end against a webhook.site URL.

Known follow-ups (low severity)

  • DLQ markFailed could be wrapped in try/finally before ack().
  • isValidWebhookUrl denylist has IPv6/numeric-host gaps (backstopped by global_fetch_strictly_public).
  • Test button reads the saved URL prop rather than the live field value.

Deliver submission.created webhooks using Cloudflare Queues (delivery,
retry/backoff, dead-letter queue) and Cron Triggers (safety-net sweep and
log cleanup) — no Redis, BullMQ, or separately deployed worker, replacing
the approach abandoned in #121.

- schema: webhook_delivery_logs table + forms.enable_webhook / webhook_url
  / webhook_secret (migration 0003)
- producer: submission route and form.testWebhook enqueue via WEBHOOK_QUEUE,
  deferred with after() (never a floating promise)
- consumer/cron: custom OpenNext worker entry (worker.ts) exporting queue()
  and scheduled(); process.env hydrated before dynamic imports so the
  workerd handlers can reach the validated env
- HMAC-SHA256 request signing via WebCrypto with X-Formbase-Signature,
  X-Formbase-Event and X-Formbase-Timestamp headers and a per-form secret
- UI: webhook settings, read-only signing secret, and recent deliveries list
- SSRF-hardened URL validation; signing secret is never exposed through the
  public getFormById procedure
- cron sweep atomically leases stuck rows to avoid duplicate delivery
@ephraimduncan ephraimduncan requested a review from aikins01 June 2, 2026 21:02
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 2, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
formbase-web 5222cad Commit Preview URL

Branch Preview URL
Jun 06 2026, 08:30 PM

@aikins01

aikins01 commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

sweet

Address low-severity review findings:
- consumer: wrap DLQ markFailed in try/finally so the message is always
  acked even if the status write throws (prevents a stranded DLQ batch)
- isValidWebhookUrl: close SSRF denylist gaps — full 127.0.0.0/8, 0.0.0.0,
  bracket-stripped IPv6 loopback/ULA/link-local, and integer/hex IP encodings
- form settings: gate the Test button on the live URL field + dirty state
  (and reset after save) so it can't test a stale, unsaved URL
Next 16 builds with Turbopack, which refuses to follow symlinks that
resolve outside the project root. Two things broke the build in this bun
monorepo: apps/web resolves dependencies from the hoisted root
node_modules (a level above apps/web), and bun's isolated linker also
symlinked some deps (@base-ui/react, @tailus/themer, tailwind plugins)
into its global cache outside the repo entirely.

- next.config.js: set outputFileTracingRoot + turbopack.root to the
  monorepo root so Turbopack can resolve the root node_modules
- bunfig.toml: use the hoisted install linker so every dependency lives
  inside the repo (no global-cache symlinks for Turbopack to reject)
- Use space-y-2 for the webhook signing-secret block to match sibling sections
- Drop stray pnpm from db:migrate so it runs tsx directly like the other db scripts

@aikins01 aikins01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for pushing this forward. i found a few blockers that need fixing before this can merge: CI is failing, the public form lookup exposes webhook targets, and the stuck-delivery sweep can overrun queue batch limits.

Comment thread packages/db/schema/forms.ts
Comment thread packages/api/routers/form.ts Outdated
Comment thread apps/web/src/lib/webhooks/scheduled.ts
Comment thread apps/web/src/app/api/s/[id]/route.ts Outdated
@ephraimduncan ephraimduncan requested a review from aikins01 June 6, 2026 20:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants