Skip to content

Improve parser correctness and performance - #37

Merged
ECorreia45 merged 7 commits into
masterfrom
agent/faster-correct-parser
Jul 26, 2026
Merged

Improve parser correctness and performance#37
ECorreia45 merged 7 commits into
masterfrom
agent/faster-correct-parser

Conversation

@ECorreia45

Copy link
Copy Markdown
Contributor

Summary

Replace the shared-regex parser with a reentrant single-pass scanner, improve the lightweight DOM implementation, add regression coverage for reproduced parser failures, and include a reproducible before/after benchmark.

Why

The previous parser passed its existing tests but failed several important cases:

  • trailing text after an unclosed element was appended to the root
  • mismatched closing tags corrupted nesting
  • > inside quoted attributes broke tokenization
  • markup-like content inside style became elements
  • namespace caching leaked across parents
  • nested parse() calls from callbacks corrupted the outer parse

Module-level regex state, dynamic regex creation, remaining-document slices for scripts, namespace caches keyed only by tag name, and repeated Set-to-array conversion also added avoidable overhead.

What changed

  • add a quote-aware, single-pass parser scanner
  • keep parser state local so parsing is reentrant
  • derive namespaces from the active parent
  • handle raw text and matching ancestor closures
  • use arrays in the lightweight DOM hot path
  • reuse the self-closing-tag regex
  • add six regression tests
  • add benchmark.compare.mjs for paired baseline/optimized measurements

Benchmark

Node 24, seven paired median samples, alternating before/after order with explicit GC between samples.

Workload Tokenizer Default Doc
Small HTML +31.7% +8.2%
Attributes +18.3% +4.5%
Medium HTML +36.9% +0.8%
Large HTML +32.8% +12.4%
Script-heavy +161.4% +18.5%
SVG-heavy +3.9% +4.3%
Malformed HTML +61.5% +5.9%
Geometric mean +43.3% +7.7%

The bundled parser increased from 11,212 bytes to 13,715 bytes (+22.3%). This is the main tradeoff to review.

Validation

  • 42/42 Jest tests pass
  • ESLint passes
  • Prettier passes
  • TypeScript --noEmit passes
  • git diff --check passes

@ECorreia45
ECorreia45 marked this pull request as ready for review July 26, 2026 22:01
@ECorreia45
ECorreia45 merged commit 5395df2 into master Jul 26, 2026
2 checks passed
@ECorreia45
ECorreia45 deleted the agent/faster-correct-parser branch July 26, 2026 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant