fix(triggers): rewrite user_ids = ANY(...) to @> so GIN indexes can be used#880
Merged
Merged
Conversation
… can be used
The on_user_challenge and on_challenge_disbursement triggers both do per-row
lookups against the 8 GB notification table to dedupe reward / cooldown
notifications, e.g.:
SELECT id FROM notification
WHERE type = 'reward_in_cooldown'
AND new.user_id = ANY(user_ids)
AND timestamp >= (new.completed_at - interval '1 hour')
PostgreSQL's GIN operator class supports @>, &&, <@ — but NOT `scalar = ANY(array)`.
So even with the full ix_notification GIN, AND with the partial GIN added by
0210 (#877), every trigger call fell back to a parallel sequential scan of
the entire 8 GB table:
Parallel Seq Scan on notification cost=0..963930.62
Rows Removed by Filter: 7,846,045 Execution: 13,640 ms
pg_stat_user_indexes confirmed idx_scan = 0 on the new partial GIN since
creation.
Rewriting the predicate to the canonical @> form lets the planner pick the
partial GIN — same EXPLAIN drops to a Bitmap Index Scan / 2 ms / 5 buffers.
Semantics are identical (both forms test array membership).
Three call sites updated:
- handle_user_challenges.sql (reward_in_cooldown dedupe)
- handle_challenge_disbursements.sql (challenge_reward dedupe, both
legacy and sol_reward_disbursement copies)
This is the actual fix the #877 partial GIN was supposed to enable — the
index was correct, but the trigger SQL couldn't reach it. After this deploys,
the IndexChallengesJob first-tick wedge clears (the per-upsert trigger cost
drops from ~13s to ~2ms) and challenge_disbursement throughput rises with it.
The wider codebase has ~50 other `= any(user_ids)` occurrences across other
trigger functions; cleaning those up is a separate sweep.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The actual fix for the
IndexChallengesJobwedge that #877 was supposed to enable. #877's partial GIN was correct — but the trigger SQL couldn't reach it.Root cause
The
on_user_challengeandon_challenge_disbursementtriggers both do per-row lookups against the 8 GB / ~23.5 M-rownotificationtable to dedupe reward / cooldown notifications, e.g.:PostgreSQL's GIN operator class supports
@>,&&,<@— but NOTscalar = ANY(array). Even with the fullix_notificationGIN, and the partial GIN added by0210(#877), every trigger call fell back to a parallel sequential scan of the entire 8 GB table.Confirmed in prod on cd94ede:
And
pg_stat_user_indexesshowedix_notification_cooldown_user_ids.idx_scan = 0since it was built — completely unused.Fix
Rewrite the predicate to the canonical
@>form. Semantics are identical (both test array membership), but only@>is GIN-eligible. Same EXPLAIN on the same row, with the same data:new.user_id = ANY(user_ids)(before)user_ids @> ARRAY[new.user_id](after)ix_notification_cooldown_user_idsThree call sites updated:
ddl/functions/handle_user_challenges.sqlreward_in_cooldowndedupe path ofhandle_on_user_challenge()— fires on everyis_complete=truewrite touser_challengesddl/functions/handle_challenge_disbursements.sql(×2)challenge_rewarddedupe in bothhandle_challenge_disbursement()(legacy table) andhandle_sol_reward_disbursement()(new indexer's table)Schema dump (
sql/01_schema.sql) and migration tracker checksums updated to match.Impact
IndexChallengesJobfirst-tick wedge clears once a fresh backend picks up the new function (so acore-indexerpod restart after this deploys is the last manual step, unless the deploy itself replaces the pod).cooldown_days > 0challenges (p,u, the Phase 2 ones, etc.) get the same speedup whenever they fire the trigger.Out of scope
The wider codebase has ~50 other
= any(user_ids)occurrences across other trigger functions (notification triggers added in #851, etc.). Same anti-pattern — same fix. Worth a separate sweep PR; I left it out here to keep this one minimal and reviewable.Test plan
go build ./...,go vet ./...clean (no Go changes; sanity check).EXPLAIN (ANALYZE, BUFFERS)that the@>form picksix_notification_cooldown_user_idsand completes in 2 ms (vs 13.6 s for the= ANYform).sql/03_migration_tracker.sqlupdated sopg_migrate.shre-applies them on deploy.sql/01_schema.sqlupdated in lockstep so a fresh test template reflects the new function bodies.🤖 Generated with Claude Code