tests/a11y/audit.test.tsx (the axe-core audit added in #79) intermittently fails the pre-push gate (and would in CI), while passing reliably locally in ~6s. It bounced a README-only push twice in one session before a retry went green.
Symptoms
- One or more
<surface>: no axe violations cases fail, often several at once (a cascade, not a single flake).
- Local
npx vitest run tests/a11y/audit.test.tsx → 10/10 pass, ~6s.
Root cause
Two things compound:
- Timeout under load.
axe.run on the large settings/ControlsPane DOM occasionally exceeds the per-test 30_000ms timeout when the docker gate is running pytest + vitest + coverage under resource contention (CI/gate is much slower than a dev box). The test already trims cost (scans only BUILDINGS_SECTION via axeMount, resultTypes: ['violations'], contrast/region rules off) — but it's still on the edge.
- axe is a singleton → timeouts cascade. The test's own comment notes it: a run that times out mid-flight leaves axe "running," so the next surface throws. So one slow surface fails multiple
its and bounces the whole push.
Directions (pick some)
- Break the cascade (highest value): ensure axe is reset between surfaces even after a timeout, e.g.
afterEach calling axe.teardown() / axe.reset(), or serialize surfaces so one wedge doesn't poison the rest. This alone turns a multi-failure bounce into at most one flaky case.
- Cut the wall-clock dependence: further shrink what's scanned per surface, or split the audit so each surface is its own isolated run.
- Reconsider gate placement: keep the cheap structural guards in the pre-push gate, but move the heavy
axe.run portion to CI-only (jsdom axe is slow and gate contention makes it flaky) — the structural guards catch most regressions fast.
- As a floor, bump the timeout, but that treats the symptom, not the singleton cascade.
Impact
Low correctness risk (it's a test, and it passes locally), but it costs real retries and erodes trust in the gate. Related: #79 (where the audit was added).
tests/a11y/audit.test.tsx(the axe-core audit added in #79) intermittently fails the pre-push gate (and would in CI), while passing reliably locally in ~6s. It bounced a README-only push twice in one session before a retry went green.Symptoms
<surface>: no axe violationscases fail, often several at once (a cascade, not a single flake).npx vitest run tests/a11y/audit.test.tsx→ 10/10 pass, ~6s.Root cause
Two things compound:
axe.runon the large settings/ControlsPaneDOM occasionally exceeds the per-test30_000mstimeout when the docker gate is running pytest + vitest + coverage under resource contention (CI/gate is much slower than a dev box). The test already trims cost (scans onlyBUILDINGS_SECTIONviaaxeMount,resultTypes: ['violations'], contrast/region rules off) — but it's still on the edge.its and bounces the whole push.Directions (pick some)
afterEachcallingaxe.teardown()/axe.reset(), or serialize surfaces so one wedge doesn't poison the rest. This alone turns a multi-failure bounce into at most one flaky case.axe.runportion to CI-only (jsdom axe is slow and gate contention makes it flaky) — the structural guards catch most regressions fast.Impact
Low correctness risk (it's a test, and it passes locally), but it costs real retries and erodes trust in the gate. Related: #79 (where the audit was added).