Skip to content

chore: update dependencies and streamline docs #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Bug-Finderr merged 1 commit into from
Jul 24, 2026
Merged

chore: update dependencies and streamline docs #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ jobs:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- uses: nubjs/setup-nub@47e5e7393d50f4e9544ddaf756aa277972afba2d # v0
with:
nub-version: 0.4.7
node-version: 24
nub-version: 0.5.0
node-version: 26
cache-dependency-path: nub.lock
- uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b
with:
Expand All @@ -25,8 +25,8 @@ jobs:
- run: |
uv venv --python 3.14
uv pip install -r test/requirements.txt
- run: nubx --no-install tsc --noEmit
- run: nubx --no-install biome check .
- run: nub exec tsc --noEmit
- run: nub exec biome check .
- run: nub audit --prod
- run: nubx --no-install wrangler deploy --dry-run
- run: nub exec wrangler deploy --dry-run
- run: nub run test
29 changes: 4 additions & 25 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,30 +6,14 @@ out
dist
*.tgz

# code coverage
coverage
*.lcov

# logs
logs
_.log
report.[0-9]_.[0-9]_.[0-9]_.[0-9]_.json

# dotenv environment variable files
.env
.env.development.local
.env.test.local
.env.production.local
.env.local

# caches
.eslintcache
.cache
*.tsbuildinfo
report.*.json

# local experiments / scratch
.tmp-*
HANDOFF.md
# secrets
.env*
!.env.example

# wrangler local state / build cache
.wrangler
Expand All @@ -44,11 +28,6 @@ HANDOFF.md
# Finder (MacOS) folder config
.DS_Store

CLAUDE.md

# claude local settings
.claude/settings.local.json

# python venv for the separate-runner compat tests + caches
.venv/
__pycache__/
110 changes: 44 additions & 66 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,121 +1,99 @@
# api-proxy

A single Cloudflare Worker that reverse-proxies the OpenAI, Anthropic, and Google Gemini APIs - over **HTTP and WebSocket** - behind **revocable proxy tokens**. You issue tokens from an admin dashboard and hand them out; each token is validated server-side and swapped for the real provider key before the request is forwarded. Consumers never see your real keys, and you can scope or revoke any token at any time.
A Cloudflare Worker that proxies OpenAI, Anthropic, and Google Gemini over HTTP and WebSocket. Clients use revocable proxy tokens; the Worker validates each token and replaces it with the provider key before forwarding.

## Use it

Works with the official **OpenAI**, **Anthropic**, and **Google GenAI** SDKs (Python and Node) - and, since the worker routes by auth header and forwards everything else verbatim, with anything that speaks those APIs: the Vercel AI SDK, LangChain, LiteLLM, OpenAI-compatible tools, or raw `curl`. A client changes only **two things**: the base URL and the API key (a proxy token).
Clients normally change only their base URL and API key. The compatibility suite covers the official SDKs and selected AI frameworks in `test/sdk-compat/`; other HTTP clients should work when they expose a base-URL override and use a supported credential slot, but are not automatically tested.

| Client | base URL | API key |
|---|---|---|
| OpenAI SDK (Python / Node) | `https://<worker>/v1` | proxy token |
| Anthropic SDK (Python / Node) | `https://<worker>` (no `/v1`) | proxy token |
| Google `@google/genai` (Node) | `httpOptions.baseUrl = https://<worker>` | proxy token |
| Gemini via the OpenAI SDK | `https://<worker>/v1beta/openai` | proxy token |
| Client | Base URL | API key |
| ----------------------------- | ---------------------------------------- | ----------- |
| OpenAI SDK | `https://<worker>/v1` | proxy token |
| Anthropic SDK | `https://<worker>` | proxy token |
| Google `@google/genai` | `httpOptions.baseUrl = https://<worker>` | proxy token |
| Gemini through the OpenAI SDK | `https://<worker>/v1beta/openai` | proxy token |

```python
# OpenAI SDK (Python); Node is identical
from openai import OpenAI

client = OpenAI(base_url="https://<worker>/v1", api_key="<proxy-token>")
client.chat.completions.create(
model="gpt-5.4", messages=[{"role": "user", "content": "Hello"}])
```

Or raw HTTP:

```bash
curl https://<worker>/v1/chat/completions \
-H "authorization: Bearer <proxy-token>" -H "content-type: application/json" \
-d '{"model":"gpt-5.4","messages":[{"role":"user","content":"Hello"}]}'
model="gpt-5.6", messages=[{"role": "user", "content": "Hello"}])
```

Browser apps work too - the worker answers the CORS preflight and reflects the request Origin (provider browser opt-ins still apply, e.g. Anthropic's `dangerouslyAllowBrowser`).
Browser requests are supported through CORS. Provider-specific browser opt-ins still apply, such as Anthropic's `dangerouslyAllowBrowser`.

### WebSocket / realtime
### WebSocket APIs

Realtime sockets proxy the same way - point the WebSocket at the worker and use a proxy token. The worker swaps the token for the real key on the upgrade handshake.
| API | URL | Proxy-token slot |
| ---------------- | ---------------------------------------------------- | --------------------------------------------------------------------------- |
| OpenAI Realtime | `wss://<worker>/v1/realtime?model=…` | Bearer header, or `openai-insecure-api-key.<token>` subprotocol in browsers |
| OpenAI Responses | `wss://<worker>/v1/responses` | Bearer header |
| Gemini Live | `wss://<worker>/ws/…BidiGenerateContent?key=<token>` | `?key=` query |

| WebSocket API | URL | token slot |
|---|---|---|
| OpenAI Realtime (server) | `wss://<worker>/v1/realtime?model=…` | `Authorization: Bearer <proxy-token>` |
| OpenAI Realtime (browser) | `wss://<worker>/v1/realtime?model=…` | `Sec-WebSocket-Protocol: realtime, openai-insecure-api-key.<proxy-token>` |
| OpenAI Responses (WebSocket mode) | `wss://<worker>/v1/responses` | `Authorization: Bearer <proxy-token>` |
| Gemini Live | `wss://<worker>/ws/…BidiGenerateContent?key=<proxy-token>` | `?key=` query |
A socket is authorized and rate-limited when it connects. Revocation affects the next connection, not an open stream.

A browser can't set the `Authorization` header on a WebSocket, so OpenAI places the key in the `openai-insecure-api-key.` subprotocol; the worker rewrites it to a Bearer header upstream. `x-api-key` upgrades route to Anthropic too. A long-lived socket is rate-limited and validated **once at connect**, so a revoke applies to the next connection, not an open stream.

## How it works

The proxy token rides in the SDK's normal auth slot. The worker validates its scope, strips the HTTP header/query auth slots, sets one provider key, and forwards the path, remaining query, body, and stream. WebSocket handshakes add OpenAI's browser subprotocol slot. See [docs/architecture.md](docs/architecture.md) for the routing table and design.
See [the architecture](docs/architecture.md) for routing, credential replacement, and consistency rules.

## Setup

Requires Node 24+ and Nub 0.4.7.
Requires Node 26+ and Nub 0.5.x.

```bash
nub install
nubx wrangler login
nubx wrangler kv namespace create api-proxy-tokens # paste the id into wrangler.toml
nub exec wrangler login
nub exec wrangler kv namespace create api-proxy-tokens
```

Set the secrets (only these four; never committed):
Put the namespace ID in `wrangler.toml`, then configure the secrets and deploy:

```bash
nubx wrangler secret put OPENAI_API_KEY
nubx wrangler secret put ANTHROPIC_API_KEY
nubx wrangler secret put GEMINI_API_KEY
nubx wrangler secret put ADMIN_SECRET # password for the admin dashboard
nubx wrangler deploy
nub exec wrangler secret put OPENAI_API_KEY
nub exec wrangler secret put ANTHROPIC_API_KEY
nub exec wrangler secret put GEMINI_API_KEY
nub exec wrangler secret put ADMIN_SECRET
nub exec wrangler deploy
```

Optional plain vars (NOT secrets) override the upstreams; they default to the real hosts and only need setting for testing: `OPENAI_UPSTREAM`, `ANTHROPIC_UPSTREAM`, `GEMINI_UPSTREAM`.

## Admin dashboard
`OPENAI_UPSTREAM`, `ANTHROPIC_UPSTREAM`, and `GEMINI_UPSTREAM` are optional plain variables used mainly to point tests at a mock upstream.

Visit `https://<worker>/admin`, sign in with `ADMIN_SECRET`, and create tokens with a label, provider scope, optional expiry, and either a custom value (minimum 12 characters) or a generated value. The plaintext is shown **once**. KV stores its SHA-256 hash plus metadata (`label`, `last4`, scope, status, creation/expiry), with `lastUsed` in a side key. The admin route checks for an existing custom-token hash, but KV's read-then-write is not transactional, so do not race identical creations.
## Admin and token controls

## Per-token controls
Visit `https://<worker>/admin` and sign in with `ADMIN_SECRET`.

- **Expiry** - optionally set or edit an expiry; past it the token is rejected and the dashboard shows it as `expired`.
- **Rate limit** - each token is capped at 100 requests / 60s (`429` + `Retry-After` over the limit). Tune `[[ratelimits]]` in `wrangler.toml`. It is a per-location, loose ceiling for abuse protection, not a strict quota.
- **Scope & revoke** - a token only reaches the providers you check; disable or delete to revoke (KV changes can take 60 seconds or more to reach other locations).
- Create a generated or custom token with a label, provider scope, and optional expiry. Plaintext is shown once.
- Edit expiry, disable a token, or delete it. KV changes may take 60 seconds or more to reach another Cloudflare location.
- Each token is checked against the default 100 requests per 60 seconds threshold, configured under `[[ratelimits]]` in `wrangler.toml`. This is loose per-location abuse control, not a strict quota.

## Security
Provider keys stay in Cloudflare secrets. KV stores token hashes and metadata, never usable token plaintext. Before forwarding, the Worker removes every recognized inbound credential slot and sets exactly one provider credential.

- Real provider keys are Cloudflare secrets, injected only into outbound requests - never in KV, never returned to callers.
- Token plaintext is not persisted; KV stores hashes, metadata, and separate last-used timestamps.
- The worker strips HTTP header/query auth slots and the WebSocket key subprotocol before setting the provider credential, so a proxy token is not forwarded upstream.
- Do not host the worker on a `*.openai.azure.com` / `*.cognitiveservices.azure.com` domain (the OpenAI SDK switches to Azure auth on those hostnames).
Do not host the Worker on an `*.openai.azure.com` or `*.cognitiveservices.azure.com` domain because OpenAI clients may switch to Azure authentication on those hostnames.

## Testing

```bash
nub run test:unit # tier 1: proxy logic in workerd (vitest-pool-workers), fast CI gate
nub run test:compat # tier 2: real client libs (official SDKs, Vercel AI SDK, LangChain, Genkit) + raw fetch + a real wss round-trip vs a mock upstream
nub run test:py # tier 2 (Python): LiteLLM, LlamaIndex, instructor, Pydantic AI through the worker (needs the venv below)
nub run test # all of the above
nub run test:unit
nub run test:compat
nub run test:py
nub run test
```

**Each client-named compatibility case doubles as a usage example.** Keep every client: shared auth-slot routing is only one invariant, while separate cases catch base-URL options, default endpoints, implicit headers, transport choices, and streaming behavior. Copy the wiring from the matching file, `fetch.ts`, or `websocket.ts`. The coverage rationale is [docs/learnings/compat-is-the-auth-slot-not-the-sdk.md](docs/learnings/compat-is-the-auth-slot-not-the-sdk.md); the harness is [docs/architecture.md](docs/architecture.md) §14.

Two gotchas: use Anthropic's normal API-key mode (its OAuth `authToken` mode sends `Bearer`, which would route to OpenAI), and the legacy `google-generativeai` Python SDK needs `transport="rest"` (it defaults to gRPC and won't traverse an HTTP proxy otherwise).

The Python runner uses a local venv. One-time setup with [uv](https://docs.astral.sh/uv/):
Python compatibility tests need a local environment:

```bash
uv venv
uv pip install -r test/requirements.txt
```

> **Gemini is untested with the actual API.** No test hits a live provider - all three run against a mock upstream. OpenAI and Anthropic are additionally verified live in deployment. A `GEMINI_API_KEY` secret is configured, but its validity and the Gemini route have not been verified against the real Google Generative Language API. Treat it as built-but-unproven until that live check passes.
The Python tier exits with an error when `.venv` is absent. All automated tests use fake credentials and mock upstreams; Gemini has not been verified against its live API. Client-specific setup and caveats are in [`test/sdk-compat/`](test/sdk-compat/) and the [compatibility learning](docs/learnings/compat-is-the-auth-slot-not-the-sdk.md).

## Cost

The Worker and SQLite Durable Object have separate allowances. Cloudflare's Free Durable Object tier lists **100,000 requests/day** and **13,000 GB-s/day** (pricing page updated 2026-06-19). Each OpenAI geo-block fallback consumes one DO request, and its execution contributes to active duration; each admin token edit or delete adds one `TokenWriter` DO request. See [Durable Objects pricing](https://developers.cloudflare.com/durable-objects/platform/pricing/); upstream API usage is billed by the provider.
The OpenAI geo-block fallback and every admin token create, edit, or delete use a Durable Object request in addition to the Worker request. See [Cloudflare Durable Objects pricing](https://developers.cloudflare.com/durable-objects/platform/pricing/). Provider API usage is billed separately.

## Contributing

Issues are welcome. External PRs are not accepted and will be auto-closed.
Issues are welcome. External pull requests are auto-closed.

## License

Expand Down
36 changes: 10 additions & 26 deletions _legacy/v1/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,35 +1,19 @@
# v1 - one worker per provider (archived)
# v1 - one worker per provider

The original proxy: **three separate Workers**, one per provider, each a thin pass-through
that swaps the hostname and injects the real key from its own secret.
The original design used three unauthenticated Workers, one for each provider. Each Worker rewrote the host and injected its own provider secret.

```
client ──▶ openai-proxy ──(Bearer OPENAI_API_KEY)──▶ api.openai.com
client ──▶ claude-proxy ──(x-api-key ANTHROPIC_KEY)─▶ api.anthropic.com
client ──▶ gemini-proxy ──(x-goog-api-key GEMINI)──▶ generativelanguage.googleapis.com
```

| File | Worker | Upstream | Key slot it sets |
|---|---|---|---|
| `openai.ts` / `wrangler.openai.toml` | `openai-proxy` | api.openai.com | `Authorization: Bearer` |
| `claude.ts` / `wrangler.claude.toml` | `claude-proxy` | api.anthropic.com | `x-api-key` |
| `gemini.ts` / `wrangler.gemini.toml` | `gemini-proxy` | generativelanguage.googleapis.com | `x-goog-api-key` (and strips `?key=`) |
| Worker | Upstream | Credential set |
|---|---|---|
| `openai-proxy` | api.openai.com | Bearer |
| `claude-proxy` | api.anthropic.com | `x-api-key` |
| `gemini-proxy` | generativelanguage.googleapis.com | `x-goog-api-key` |

## Why it was replaced

- **No auth.** Each worker injected the real upstream key for *any* caller. Anyone who knew
the URL spent the key. There were no shareable, revocable tokens.
- **Three deploys, three URLs.** Clients had to know which worker maps to which provider, and
each needed its own secret and deploy.
Anyone who knew a Worker URL could spend its shared provider key. There were no per-user scopes, revocation, or usage controls, and each provider needed a separate URL and deployment.

v2 (the active root worker) collapses all three into **one** worker that routes by auth header,
gates every request behind a hashed [proxy token](../../docs/learnings/proxy-token-security.md),
and adds the [OpenAI geo-403 egress fix](../../docs/learnings/openai-egress-geo-block.md). See
[provider routing by auth header](../../docs/learnings/provider-routing-by-auth-header.md) for how
one base URL serves all three.
The current root Worker adds hashed proxy tokens, one routing surface, and the OpenAI geo-block fallback. See the active [architecture](../../docs/architecture.md).

## Status

Reference only - **not deployed, not built, not tested.** Kept to document where the project
started. The `main` paths in these tomls point at files in this folder, so each could still be
deployed standalone (`wrangler deploy -c _legacy/v1/wrangler.openai.toml`) if ever needed.
Historical reference only. This directory is excluded from builds and tests and is not maintained as deployable code.
2 changes: 1 addition & 1 deletion biome.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"$schema": "https://biomejs.dev/schemas/2.5.3/schema.json",
"$schema": "https://biomejs.dev/schemas/2.5.5/schema.json",
"vcs": {
"enabled": true,
"clientKind": "git",
Expand Down
Loading
Loading