feat(server): namespace-scoped introspection filter — 87× per-instance memory (3/6)#1332
Open
yyyyaaa wants to merge 1 commit into
Open
feat(server): namespace-scoped introspection filter — 87× per-instance memory (3/6)#1332yyyyaaa wants to merge 1 commit into
yyyyaaa wants to merge 1 commit into
Conversation
…PECTION_FILTER) Every PostGraphile instance runs pg-introspection's full-catalog query and retains the entire introspection graph — the single biggest per-instance cost on production-shaped catalogs, and pure waste for an instance that serves 4 schemas out of 61k pg_class rows. Measured on the #1325 rig (48 tenant DBs, 61k pg_class rows in one database): - retained heap per instance: 1,305.6MB -> 14.7MB (87x) - cold build: 7.2s -> 0.42s - PG-side introspection spike: +1,283MiB -> +17MiB Mechanism (introspection-filter.ts): - The stock pg-introspection query gates namespaces with a fixed textual predicate that appears EXACTLY four times. We swap each occurrence, pre-parse, for `nspname = any (array[...]::text[])` over the keep set. - Keep set = served schemas ∪ public ∪ a bounded closure over cross-schema references (FK targets, attribute/procedure type owners), discovered with qualified catalog queries (MAX_CLOSURE_ROUNDS=5) and memoized per pool. - Fail-open on gate-count mismatch (a pg-introspection upgrade that changes the text runs UNFILTERED — correctness over savings, and the pinning test below fails loudly). Fail-closed on discovery errors. - pg-introspection@1.0.1 pinned as a devDependency: the test suite asserts the installed package still carries the exact gate text four times, so a dependency bump that drifts the text breaks CI here instead of silently disabling the filter in production. Wiring: opt-in via GRAPHILE_INTROSPECTION_FILTER=1|true. When on (and the instance has a concrete schema list), the pgService pool is wrapped with a thin interceptor that only touches the introspection statement; every other query passes through byte-identical. Flag off = pool selection identical to today. The metrics sampler gains swap/discovery counters. Extracted from #1325 (commit 7615a8c). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0122xqM2VkNbuAmZshK1YNSb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR 3/6 of the #1325 split. Scopes each instance's catalog introspection to the namespaces it actually serves, opt-in via
GRAPHILE_INTROSPECTION_FILTER=1. This turned out to be the memory wall — and it applies to dedicated (non-pooled) instances, so it delivers most of the per-instance win even if pooling (PRs 4–5) never merges.Measured on the #1325 rig (48 tenant DBs, 61k
pg_classrows in one database):Mechanism (
introspection-filter.ts)pg-introspectionquery gates namespaces with a fixed textual predicate that appears exactly four times. Pre-parse, each occurrence is swapped fornspname = any (array[…]::text[])over the keep set.public∪ a bounded closure over cross-schema references (FK targets, attribute/procedure type owners), discovered with qualified catalog queries (MAX_CLOSURE_ROUNDS=5) and memoized per pool.pg-introspection@1.0.1is pinned as a devDependency purely so the test suite can assert the installed package still carries the exact gate text.Wiring
~10 lines in
buildPreset: when the flag is on and the instance has a concrete schema list, the pgService pool gets a thin interceptor that touches only the introspection statement; every other query passes through byte-identical. Flag off ⇒ pool selection identical to today. Metrics sampler gains swap/discovery counters.Testing
graphql/server132/132 (14 suites) — includes the pinning test, gate-mismatch fail-open, literal escaping (quote doubling,pg_*/information_schemadrops), closure-loop memoization, and the dedicated-wrapper path. (The filter-through-rewriting-pool composition tests land with PR 4, where that seam exists.)Stack
Stacked on #1331 (
feat/scale-s2-cache-hardening); merge after it. Full stack map in #1330. Extracted from #1325 (commit 7615a8c; kept open as the validated reference).🤖 Generated with Claude Code
https://claude.ai/code/session_0122xqM2VkNbuAmZshK1YNSb