perf(lexer): identify literal delimiters with direct checks - #728
Conversation
There was a problem hiding this comment.
🟢 Approval recommended
The change is localized, preserves the existing call-site invariant for _parseLiteral, and is backed by targeted streaming split tests that exercise the new delimiter-detection logic.
Pull request overview
This PR optimizes literal delimiter detection in N3Lexer by replacing a regex-based opening-delimiter match with direct character checks, relying on the existing invariant that _parseLiteral is only called when the current character is a quote. It aims to reduce allocations on the hot path while keeping literal parsing behavior unchanged.
Changes:
- Replace regex matching in
_parseLiteralwith direct checks for single- vs triple-quoted delimiters. - Reuse fixed triple-quote delimiter strings (
"""/''') instead of allocating via substring/regex. - Add split-across-chunks tests covering both quote types, escaped closing quotes, multiline literals, and empty triple-quoted literals.
File summaries
| File | Description |
|---|---|
| src/N3Lexer.js | Reworks literal opening-delimiter identification to use direct character checks instead of a regex match. |
| test/N3Lexer-test.js | Adds parameterized streaming-split tests to ensure delimiter recognition remains correct across chunk boundaries. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
ccd21c0 to
ac5394a
Compare
|
🎉 This PR is included in version 2.7.9 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Identify a literal's opening delimiter with direct character checks instead of a regex match. Both callers of
_parseLiteralhave already identified a single or double quote, and the existing three-character lookahead guard remains. Triple delimiters reuse fixed strings, avoiding a new substring and regex match allocation.This is independent of #721's range/reuse fixes, #727's separator dispatch, and #726's direction markers. Escape handling, closing-delimiter searches, literal values, token lengths, and streaming behavior are unchanged.
All 6,902 tests pass with 100% statement, branch, function, and line coverage, plus ESLint and Node/browser IIFE/ESM builds. Added tests exercise both quote characters, escaped closing quotes, multiline strings, and empty triple-quoted strings at every two-chunk split. Existing malformed/unterminated literal tests remain unchanged.
Performance compares production Babel builds against main
4607e09on macOS arm64 / Node 25.1.0: seven paired rounds in rotating fresh-process order, 36 warmup and 36 measured iterations over 8,000-triple documents. Full token/quad digests agree. Values are median paired CPU changes with bootstrap 95% intervals; negative means less CPU.Dense-input controls are consistent with no change. These local synthetic results do not guarantee the same gains for every workload or runtime.
The current branch is rebased on main
e4e2148, retaining the merged #726 direction-marker optimization. The benchmark tables above describe the original comparison against4607e09; they have not been relabeled as measurements of the new base. The rebased source passes the full test suite at 100% coverage, lint, Node/browser builds, and all GitHub checks.