Skip to content

perf(lexer): identify literal delimiters with direct checks - #728

Merged
jeswr merged 1 commit into
rdfjs:mainfrom
jeswr:codex/perf-lexer-literal-delimiters
Sep 5, 2026
Merged

perf(lexer): identify literal delimiters with direct checks#728
jeswr merged 1 commit into
rdfjs:mainfrom
jeswr:codex/perf-lexer-literal-delimiters

Conversation

@jeswr

@jeswr jeswr commented Sep 5, 2026

Copy link
Copy Markdown
Member

Identify a literal's opening delimiter with direct character checks instead of a regex match. Both callers of _parseLiteral have already identified a single or double quote, and the existing three-character lookahead guard remains. Triple delimiters reuse fixed strings, avoiding a new substring and regex match allocation.

This is independent of #721's range/reuse fixes, #727's separator dispatch, and #726's direction markers. Escape handling, closing-delimiter searches, literal values, token lengths, and streaming behavior are unchanged.

All 6,902 tests pass with 100% statement, branch, function, and line coverage, plus ESLint and Node/browser IIFE/ESM builds. Added tests exercise both quote characters, escaped closing quotes, multiline strings, and empty triple-quoted strings at every two-chunk split. Existing malformed/unterminated literal tests remain unchanged.

Performance compares production Babel builds against main 4607e09 on macOS arm64 / Node 25.1.0: seven paired rounds in rotating fresh-process order, 36 warmup and 36 measured iterations over 8,000-triple documents. Full token/quad digests agree. Values are median paired CPU changes with bootstrap 95% intervals; negative means less CPU.

Workload CPU change 95% interval
lexer: dense -0.2% [-10.0%, +1.6%]
lexer: multiline -9.7% [-12.9%, -7.0%]
lexer: all-escaped -4.5% [-6.9%, -3.7%]
parser: dense -0.7% [-1.9%, +2.8%]
parser: multiline -8.4% [-9.8%, -5.7%]
parser: all-escaped -10.0% [-12.0%, -1.3%]

Dense-input controls are consistent with no change. These local synthetic results do not guarantee the same gains for every workload or runtime.

The current branch is rebased on main e4e2148, retaining the merged #726 direction-marker optimization. The benchmark tables above describe the original comparison against 4607e09; they have not been relabeled as measurements of the new base. The rebased source passes the full test suite at 100% coverage, lint, Node/browser builds, and all GitHub checks.

Copilot AI lite review requested due to automatic review settings September 5, 2026 21:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change is localized, preserves the existing call-site invariant for _parseLiteral, and is backed by targeted streaming split tests that exercise the new delimiter-detection logic.

Pull request overview

This PR optimizes literal delimiter detection in N3Lexer by replacing a regex-based opening-delimiter match with direct character checks, relying on the existing invariant that _parseLiteral is only called when the current character is a quote. It aims to reduce allocations on the hot path while keeping literal parsing behavior unchanged.

Changes:

  • Replace regex matching in _parseLiteral with direct checks for single- vs triple-quoted delimiters.
  • Reuse fixed triple-quote delimiter strings (""" / ''') instead of allocating via substring/regex.
  • Add split-across-chunks tests covering both quote types, escaped closing quotes, multiline literals, and empty triple-quoted literals.
File summaries
File Description
src/N3Lexer.js Reworks literal opening-delimiter identification to use direct character checks instead of a regex match.
test/N3Lexer-test.js Adds parameterized streaming-split tests to ensure delimiter recognition remains correct across chunk boundaries.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@jeswr
jeswr force-pushed the codex/perf-lexer-literal-delimiters branch from ccd21c0 to ac5394a Compare September 5, 2026 23:01
@jeswr
jeswr merged commit 824bdaf into rdfjs:main Sep 5, 2026
55 checks passed
@jeswr
jeswr deleted the codex/perf-lexer-literal-delimiters branch September 5, 2026 23:06
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 2.7.9 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants