Skip to content

[v3.1.0] List rendering robustness fixes (itemize / enumerate) - #422

Draft
OlgaRedozubova wants to merge 94 commits into
masterfrom
dev/olga/list-padding-block-item-marker
Draft

[v3.1.0] List rendering robustness fixes (itemize / enumerate)#422
OlgaRedozubova wants to merge 94 commits into
masterfrom
dev/olga/list-padding-block-item-marker

Conversation

@OlgaRedozubova

@OlgaRedozubova OlgaRedozubova commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Two groups of renderer fixes, shipped together as one 3.0.2 release:

  • Lists (1–4): four unrelated inputs made LaTeX itemize / enumerate lists render incorrectly; the fixes live in the list-env and footnote block rules.

  • Code blocks (5): code-text styles were absolute (px / rem), so a code block did not scale when a consumer sizes a rendered block by a single font-size; the styles are now relative.

  • Version bump: 3.0.1 → 3.0.2

  • Specs: pr-specs/2026-07-list-rendering-robustness.md, pr-specs/2026-07-code-block-font-scaling.md

  • Changelog: doc/changelog.md

  • Full test suite green.


1. Marker padding for block-content items

A top-level list gets its padding-inline-start from the widest custom \item[...] marker, but the width was only measured on the inline item path. Items whose content is a block environment (\begin{figure}, \begin{tabular}, a code fence) were skipped, so a list whose long-marker items all hold block content lost its padding.

MMD example

\begin{itemize}
\item[11.33] In Fig. 11-20, when pump YA delivers 5.00 cfs.
\begin{figure}
\includegraphics[alt={},max width=\textwidth]{https://cdn.mathpix.com/snip/images/QxUdugRlfaW7h_IxvYNbgRDXjd0egDGrEXiSjKH7FGU.original.fullsize.png}
\caption{Fig. 11-20}
\end{figure}
\item[11.34] In Fig. 11-21, which system has the greater capacity.
\begin{figure}
\includegraphics[alt={},max width=\textwidth]{https://cdn.mathpix.com/snip/images/QxUdugRlfaW7h_IxvYNbgRDXjd0egDGrEXiSjKH7FGU.original.fullsize.png}
\caption{Fig. 11-21}
\end{figure}
\end{itemize}
Before After
Screenshot 2026-07-28 at 18 32 50 Screenshot 2026-07-28 at 18 32 18

2. Marker width: fullwidth/CJK, math, and wrapped content

Marker width summed String.length over top-level text tokens only, so several markers were undercounted and the list got too small an indent (marker overlaps the content):

  • Fullwidth / CJK (\item[11.], U+FF0E) counted as narrow ASCII — now East-Asian Wide/Fullwidth chars count as 2.
  • Math (\item[$x^4+x^4$]) contributed 0 — now uses the token's rendered widthEx.
  • Wrapped content (\item[\textbf{…}]) contributed 0 — now measured through the wrapper's children.

ASCII text markers are unchanged.

MMD examples

\begin{itemize}
\item[I] Item 1.
\item[11.] Item 11..
\item[I] Item 1.
\end{itemize}
\begin{itemize}
\item[$x^4 + x^4$] a footnote note or item body
\end{itemize}
\begin{itemize}
\item[\textbf{x^4 + x^4}] a footnote note or item body
\end{itemize}
Case Before After
fullwidth 11. Screenshot 2026-07-28 at 18 09 56 Screenshot 2026-07-28 at 18 10 13
math $x^4+x^4$ Screenshot 2026-07-28 at 18 27 20 Screenshot 2026-07-28 at 18 27 11
bold \textbf{…} Screenshot 2026-07-28 at 18 29 21 Screenshot 2026-07-28 at 18 29 06

3. No env-state leak from an aborted list parse

The list block rule parses speculatively into a buffered state that shares env by prototype. On abort (unclosed list) or a silent probe it returned without restoring env.isBlock (and env.inheritedListType); the leaked isBlock = true then let the inline list fallback fire on the following content, so an unclosed itemize before a tabular rendered a broken partial list with empty <> item bodies. Those transient fields are now restored on both paths, so the unclosed list degrades to plain text — exactly as it does without a tabular.

MMD example

\begin{itemize}
\item[(d1)] Asad Ali Khan
\item[(d2)] Asad Ali Khan
\begin{tabular}{|l|l|}
cell
\end{tabular}
\begin{itemize}
Before After
Screenshot 2026-07-28 at 18 34 13 Screenshot 2026-07-28 at 18 34 27

4. Footnote block rules stop at a list start

The \footnote / \footnotetext block rules scan forward for their open tag, terminating at block boundaries so they don't swallow following blocks. The LaTeX list rule was not a terminator, so a \begin{itemize} between a paragraph and a later footnote (no blank line) did not stop the scan: the list was swallowed and rendered as literal text (a blank line masked the bug). The LaTeX list rule is now a terminator for both — \footnotetext via its full set, \footnote via a minimal fence + Lists (keeping its cheap scan).

This is also a performance fix: on repeated paragraph + list-with-footnotetext units without blank separators the missing terminator made the scan run across every list into the rest of the document — O(N²), seconds on large inputs; terminating at the list makes it linear (guarded by a scaling test).

MMD example

Wiener's life work, its enormous range notwithstanding, exhibits a coherence of thought from start to finish, reminiscent of a great work of art. To write a biography of a person of so towering a stature is an
\begin{itemize}
\item[] \footnotetext{
1 Fortunately, this correspondence is now being studied by Dr. Albert C. Lewis of the Bertrand Russell Editorial Project, McMaster University, Ontario, Canada, and may see the light of day within a few years.
}
\end{itemize}
Before After
Screenshot 2026-07-28 at 18 35 45 Screenshot 2026-07-28 at 18 35 55

5. Code-block styles scale with the em context

Code-text styles were pinned to absolute values, so when a consumer scales a rendered block by setting a single font-size on the container (e.g. image export), everything scaled except the code — px is fixed and rem resolves against the root, not the block's em. The four properties are now relative, calibrated so a 16px base is pixel-identical to before (only code padding moves 16px → 15px):

  • #setText pre { font-size: 0.9375em; } (was 85%)
  • #setText pre code { font-size: inherit; } (was 15px)
  • #setText pre code { line-height: 1.6; } (was 24px)
  • #setText pre code { padding: 1em; } (was 1rem)

Styles only — no change to lstlisting / fenced-code markup.

MMD example

\begin{lstlisting}
def area(r):
    return 3.14159 * r * r
\end{lstlisting}

Scale the rendered block by setting a large font-size on the container (as image export does).

Before After
Screenshot 2026-07-28 at 18 37 00 Screenshot 2026-07-28 at 18 37 13

Testing

List cases in tests/_data/_lists/_data.js and tests/_list-marker-padding.js: block-content markers (figure/fence), fullwidth 11., math and \textbf markers, an unclosed list + tabular, a list after a paragraph with a multiline \footnote{} / \footnotetext{}, and a markdown list not swallowed before a \footnote.

Guards: a scaling test (tests/_footnotes_latex.js) rejecting the O(N²) footnote scan; a silent-Lists env invariant (tests/_parse-isolation.js); selector-scoped code-style assertions (tests/_styles.js).

Full suite green.

Non-goals

  • The list padding heuristic itself (px-per-char, the > 3 threshold) is unchanged.
  • Malformed list input still degrades gracefully; the goal is plain text, not a partial list.
  • Nested lists keep the default indent even with long markers (pre-existing; separate ticket).
  • Code blocks: rendering at the default 16px base is unchanged apart from the 1px code padding; inline code, pre scroll/overflow, highlight colors, and table-cell padding are untouched.

These .d.ts files were emitted by an older tsc and only differ in tuple
formatting (e.g. [number, number][]); regenerate them with the pinned
4.9.5 compiler so committed declarations match the toolchain.
A top-level itemize/enumerate only got its inline-start padding when the
widest \item[...] marker was measured. That measurement lived only in the
inline item path, so items whose content is a block environment
(\begin{figure}, \begin{tabular}, code fence) were skipped. When every
long-marker item held block content, the list lost its padding entirely.

Extract the marker-width calc into a shared computeMarkerPadding helper and
apply it in the block item path too (setTokenListItemOpenBlock now returns
the created token). Add regression tests for fenced-code and figure items.
Marker padding summed String.length, so a fullwidth/CJK marker like
\item[11.] (U+FF0E, length 3) was undercounted versus its ASCII twin
\item[11.42] (length 5) and fell under the padding threshold, leaving the
list without indentation while a neighbouring ASCII list got it.

Count East-Asian Wide/Fullwidth characters as width 2 (iterating by code
point so surrogate pairs count once). ASCII markers are unaffected.
The Lists block rule parses speculatively into a buffered state, but that
state shares env by prototype, so parsing mutates the real env.isBlock (and
inheritedListType). On an unclosed list the rule returned false without
restoring them; a leaked isBlock=true then let the inline list fallback fire
on the following text, so an unclosed itemize before a tabular rendered a
broken partial list with empty <> item bodies instead of plain text.

Restore isBlock and inheritedListType when the speculative parse aborts, so
the unclosed list degrades to text exactly as it does without a tabular.
The \footnote/\footnotetext block rules scan forward for their open tag; their
terminator set included the core markdown list rule but not the LaTeX list rule,
so a \begin{itemize} between a paragraph and a later footnote (no blank line) was
swallowed and rendered as literal text. Add the LaTeX list rule to the footnote
terminator set and use it for both footnote block rules.

Bumps version to 3.0.2; adds the list-rendering-robustness spec and changelog
entry covering this branch's list fixes.
@OlgaRedozubova OlgaRedozubova self-assigned this Jul 28, 2026
Code-text styles were pinned to absolute values (`pre code` font-size 15px /
line-height 24px / padding 1rem, `pre` font-size 85%), so a code block did not
scale when a consumer sizes a rendered block via a single font-size on the
container (e.g. image export): everything else scaled but the code stayed ~15px.

Make them relative: `pre` font-size 0.9375em; `pre code` font-size inherit,
line-height 1.6, padding 1em. Calibrated so a 16px base is pixel-identical
except code padding (16px -> 15px). Styles only; no HTML change.

Regenerate style snapshots, add a regression gate that rejects absolute
font-size/line-height and rem padding for `pre code`; spec + changelog under 3.0.2.
…inators, perf guard

- Restore env.isBlock/inheritedListType on both the abort and silent paths of the
  Lists block rule, so a silent terminator probe never mutates shared state.
- \footnote uses a minimal fence+Lists terminator set (not the full set): fixes the
  list swallow with one cheap extra probe, avoiding a per-line cost regression.
- Add a first-char fast bail to the Lists rule (no substring allocation) since it now
  runs as a per-line terminator in paragraph/footnote scans.
- Measure markers by display width (East-Asian wide chars count as 2, BMP only).
- Tests: silent env invariant, marker width edge cases (math/emoji/nested), footnote
  recognition after block constructs, and a scaling guard that rejects the O(N^2)
  footnote scan across unterminated lists.
- Selector-scoped code-style gate; spec/changelog updated.
- Measure math markers by their rendered widthEx and wrapped markers (e.g. \textbf)
  by recursing into children, so such markers no longer get too small an indent
  (previously only top-level text tokens counted).
- Extract the char-based width primitives (isWideChar, displayWidth, tokenDisplayWidth)
  into common/display-width.ts — the char-based counterpart of getTextWidthByTokens;
  computeMarkerPadding now delegates to them.
- Round the padding px (marker width can be fractional once math width is included).
- Tests: wide-math and bold marker padding, markdown list not swallowed before a
  \footnote; explicit non-empty guard on the code-style assertions.
- Spec and changelog updated.
Empty `<>` item bodies from a leaked env.isBlock are fixed on both paths:
the speculative list parse restores its transient env fields on every exit
(abort/silent/commit/exception), and a tabular inside a list no longer carries
those flags into its envToInline snapshot (which core-inline replays onto the
shared env). Adds common/env-transient.ts as the single source of truth.

Marker padding is emitted in ex, not px, so it scales with the container
font-size like the marker's math SVG; this also fixes math markers being
clipped (exact widthEx + gap, both ex). The marker gap (.li_level padding-right)
and the default list indent move to ex/em. Marker width also trims edge
whitespace and falls back to source length when widthEx is absent.

Merges the two footnote terminator resolvers into resolveEnabledRuleFns.

Adds tests/_display-width.js, an env-leak adversarial matrix, exact ex-value
assertions, and updates fixtures/snapshot/specs/changelog.
Emit list marker padding-inline-start in em (was ex), converted from the ex
measurement via the default font metrics (EX_TO_EM = exDef/fonSizeDef). Custom
padding is emitted only when it exceeds the default (2.5em), so it never resolves
below it — no CSS max() or cross-unit comparison. The marker gap (0.625em) and
default indent (2.5em) move to em too; the attribute now carries its unit
("3.32em"). Math without a widthEx (non-SVG output) keeps the default indent
rather than a fabricated source-length estimate.

Text markers use 1.3 ex/char (reserve is per character cell, not per glyph);
this is ~5-14% tighter than 3.0.1 and wide all-caps markers can overlap —
documented in the spec Non-Goals and changelog.

Sync docs/comments with the code, add tests: EX_TO_EM vs fontMetrics, exact em
values, glyph-width limitation, MARKER_GAP_EM vs CSS.
A speculative (silent/aborted) list parse runs the body's \begin{figure} /
\begin{table}\caption and bumps the module-global caption counters, but its
tokens are discarded — so a \footnote after such a list, and the pre-existing
paragraph-terminator probe, shifted Figure N / Table N. Snapshot and restore the
counters when the parse is discarded: in the Lists block rule's finally and in
parseListEnvRawToTokens (inline path). A figure in a list is now Figure 1.

Extract the counters to the leaf module common/caption-counters.ts so the list
rule no longer imports the heavy begin-table module (removes an import-cycle
edge); the deep-import helpers Clear{Table,Figure}Numbers move there, renamed to
clear{Table,Figure}Numbers. Move the default font metrics (DEFAULT_FONT_SIZE_PX
/ DEFAULT_EX_PX) to consts as the single source for EX_TO_EM and FontMetrics.
Treat combining U+3099/U+309A as zero width in both isWideChar and displayWidth;
set committed before flush; drop the now-redundant CSS guard.

Bump 3.0.1 -> 3.1.0 (minor): the data-padding-inline-start attribute is now an
em value and caption numbering changes for existing docs (see changelog/spec).

Add regression tests: caption numbering (absolute, multi-float order, nested
list, \ref matches the caption number), attribute format contract, 3-char
threshold, combining marks in isWideChar and displayWidth.
Replace the flat per-character reserve with a per-glyph-class estimate in em
(narrow / normal / wide / extra-wide W@% / East-Asian full-width; combining
marks 0). Measured against real font metrics, the old flat 14px/char
over-reserved narrow/digit markers by 36-70% and under-reserved all-caps by
~14%; the class estimate holds a +11..+27% margin — a tighter, correct indent
for narrow/normal markers and a wider, safer one for all-caps. Math still uses
the exact widthEx; short markers fall to the 2.5em default; the reservation is
clamped at 20em so a pathological OCR marker can't blow out the content column.

Only text-like leaves (text / code_inline / text_special) are measured, so a
`code`-span marker now contributes its width while an html_inline marker's raw
markup is not counted. Remove the now-unused displayWidth; dedupe the combining
-mark predicate.

Add a font-metrics lock test (Arial fixture): the rendered indent is never
smaller than the marker's true glyph width + gap. Update fixtures, docs, and
the attribute-format / clamp / mixed-marker tests.
A list's marker padding was accumulated as the max over all markers at every
depth and written to the outermost list token, so a wide marker several levels
deep drove the top-level list's indent (e.g. a deep [3.1.1.1] gave the outer
list 4.31em while its own 1.–5. markers are narrow). Track a stack of open list
tokens and attribute padding to the current (innermost) list instead.

Nested lists now also emit and apply their own padding (drop the top-level-only
gate in finalizeListItems and the level>1 suppression in the list renderers), so
each level reserves for its own markers — a wide nested marker indents its own
list rather than overflowing the container. The outer list reflects only its own
markers (default when they're narrow).

Add regression tests (per-level attribution; deep marker not bubbled up) and
update changelog/spec.
Marker padding (on top of the per-nesting-level attribution): keep the
default 2.5em indent on every list and emit a custom padding-inline-start
only when a marker overflows the accumulated ancestor indent plus that
default, reserving just the shortfall (need - ancestorIndent) so nested
markers don't compound. Flat lists are unchanged (e.g. [11.33] -> 3.51em);
ordinary nested lists (numbering, bullets) stay at the default; only a
genuinely wide nested marker reserves extra, and less than its full width.

Fix \item detection: the item-command regexes matched too loosely -
LATEX_ITEM_COMMAND_INLINE_RE matched the bare word "item" (no backslash),
and LATEX_ITEM_COMMAND_RE / LATEX_LIST_BOUNDARY_INLINE_RE matched any
\item-prefixed command (\itemsep, \itemindent). A list-item body containing
such text split into a spurious item, and a multiline \footnote/\footnotetext
inside an item was broken mid-brace and rendered as literal text. All three
now require \item plus a command boundary (\item[...] or \item(?![a-zA-Z])).

Update changelog and the list-rendering pr-spec; add regression tests
(_lists.js: \itemsep/\itemindent; _footnotes_latex.js: "item" in a footnote
body; _list-marker-padding.js: per-level B2 attribution).
…ate padding style

Resolve per-list padding in one top-down pass (resolveListPadding) after the whole
environment is parsed, instead of emitting during finalize. finalizeListItems now only
records each list's widest marker width; the pass computes the indent once every list's
final width is known, so item order no longer skews a nested list (a wide parent item
after the sublist resolves the same as before it).

Clamp the total (ancestor + own), not just the added shortfall, to LIST_MAX_INDENT_EM,
so cumulative indent never exceeds the clamp on a pathological nested marker.

Validate data-padding-inline-start against /^\d+(\.\d+)?em$/ before inlining it as a
style, so only a bare em value can reach the rendered CSS.

Docs: the attribute is now emitted on nested lists (Migration); an empty parent item
followed by a wide sublist marker overlaps as LaTeX does (Non-Goals, verified in
Overleaf); image markers reserve by alt text; correct the code-block "16px" note for
PreviewStyle's 17px base. Add regression tests (order independence, cumulative clamp,
bold reserve lock).
Core-markdown <ul>/<ol> (no .itemize/.enumerate class) had no padding-inline-start,
so browsers used their 40px default, which doesn't scale when a consumer sizes
content by font-size. Add a scoped rule (#preview-content ul, #setText ul, ... ol)
setting padding-inline-start: 2.5em, matching the LaTeX-list default and scaling with
the container. Scoped only per 2026-03-mmd-css-scoping.md — a bare ul/ol would restyle
the host page. LaTeX lists are unaffected (their class rules and inline padding win by
specificity, same value); TOC is safe (separate #toc_container; the in-content
.table-of-contents is display:none).

Regenerate the listsStyles snapshot; update changelog and the list-rendering pr-spec.
…oped-only

Follow-up to the markdown-list padding rule (bc57340):
- The rule also covers the generated footnotes list (<ol class="footnotes-list">),
  which had no explicit padding — disclosed in changelog and the pr-spec/Migration.
- Migration: the wrapped ul/ol padding floor rises from the UA default (specificity 0)
  to (1,0,1), so a consumer's class-specificity rule no longer overrides the indent.
- Spec: the #toc_container-over-markdown precedence rests on bundle order (lists before
  toc), not specificity (both 1,0,1) — documented, with the specificity-boost option.
- Test: assert listsStyles has no bare ul/ol selector, so a future edit can't quietly
  regenerate the snapshot with one.
The markdown-list rule (#setText ul { padding-inline-start: 2.5em }) and
#toc_container ul { padding: 0 } are both (1,0,1), so which wins depended on bundle
order (lists before toc). TocStyle is only emitted when toc is enabled, so a consumer
that renders a visible #toc_container inside the wrapper without TocStyle got 2.5em on
the TOC list.

Add a guard in the always-emitted lists module — #preview-content #toc_container ul/ol,
#setText #toc_container ul/ol { padding-inline-start: 0 }, specificity (2,0,1) — so the
TOC list stays at 0 regardless of order or whether TocStyle is present. It matches any
ul/ol inside a wrapper-nested #toc_container (so it also outranks .itemize (1,1,0); a
theoretical LaTeX-in-TOC would be zeroed too — accepted over a :not(.itemize) carve-out).

Also address review follow-ups: correct the clamp wording (the reserved indent is
clamped to 20em, the per-level default step adds on top); widen the no-bare-ul/ol
tripwire to catch post-comma and combinator forms; regenerate the listsStyles snapshot;
update the pr-spec.
… test fixtures

Attribute marker padding through one shared registry (openTokens + allListTokens)
seeded in ListOpen and threaded into ListItems/processListChildToken, so all list
forms resolve identically — the block loop (own-line nested), the same-line form,
and the fully single-line / in-cell form (which the block Lists rule bails on and
ListOpen now handles, resolving on its single-line early-return). A wide nested
marker no longer inflates the outer list regardless of how the list is written,
and a single-line / in-cell list reserves for its marker too.

Review follow-ups: remove the dead `padding` plumbing (ListItemsResult /
ListInlineContext field, fallback branches, ListOpen local); make openTokens /
allListTokens required (no silent-no-op default); fill skipped depth before the
ancestor sum and clamp depth >= 0 in resolveListPadding (no RangeError / NaN);
fix the stale "for top-level lists" render comments; note the inline-opened
*_list_open prentLevel (0 -> depth) in Migration.

Convert the rendered-HTML tests to full-HTML fixtures (catch visual regression the
way the _data-driven tests do): marker padding, B2 nesting, empty-<>, \item
detection and footnote-terminator cases -> tests/_data/_lists/_data.js; caption
numbering -> _data/_captions/_data.js; "item"-word footnote body ->
_data/_footnotes_latex/_data-footnotetext.js. Keep the non-renderable tests
(display-width units, config-varying renders, env-state, the Arial glyph-width
invariant, resolveListPadding unit, the scaling test) as targeted assertions.
Changelog: cut spec-level internals (per-glyph constants, px measurements, the
3.0.1 over-reservation percentages, internal probe/env mechanism) from the 3.1.0
list-fix bullets, keeping what changed plus the migration — matching the concise
style of earlier entries.

pr-spec: rewrite the Testing section, which described tests that were moved into
full-HTML fixtures (the list-rendering, caption and footnote-body cases), to match
the actual layout; drop an intermediate example value from a Non-Goal.
Follow-up fixes on the 3.1.0 list work, addressing review findings.

Table-cell export:
- A list written entirely on one line inside a table cell unwraps its item
  body to bare inline tokens (not an `inline` wrapper), which the cell renderer
  only sent to HTML — so table-markdown/tsv/csv/smoothed dropped the body
  (`\item[x] d` exported as `x `). The leaf branch now collects the whole
  consecutive run through renderTableCellContent, matching the multi-line form.
- A markdown link in a `tabular` cell skipped two tokens too many, leaving its
  `</a>` unemitted and absorbing following siblings. It now walks to its own
  link_close by depth. Pre-existing; unrelated to lists.

Speculative-parse hardening:
- Caption env (caption/captionPos/captionIsLabelFormatEmpty/
  captionIsSingleLineCheck) written by nested float rules is now snapshotted and
  restored on a non-committing exit, alongside the caption counters; closed by
  audit (no reproduced defect). Shared snapshotEnvKeys/restoreEnvKeys helpers.
- openTokens pop on an inline `\end` is now by token identity, so the block and
  inline paths can't pop the same list twice.
- Silent Lists probe result is memoized per state (invalidated on state.src
  reassignment), so paragraph/footnote terminator scans don't re-run the whole
  speculative parse per line. Output is byte-identical with and without the memo.

Cleanups:
- resolveEnabledRuleFn resolves the single list terminator without a Set/array.
- resolveListPadding uses a local instead of the vestigial token.indentEm.
- Merge duplicate consts imports.

Tests: single-line list in a cell (table-markdown/tsv), link-in-cell, the
padding-style sanitization guard, and silent-probe memo isolation. Docs:
changelog + specs (clamp wording, margin range/font, migration notes).
Widen the speculative-parse env rollback beyond the caption keys. A list body
parsed speculatively (silent probe / aborted parse) runs nested float and
tabular rules that write more shared-env keys than caption alone: begin-table
also writes envType/align/alignEnvBlock/number/type, and begin-tabular writes
isInline/subTabular/tabulare. A silent probe leaked all of these.

- Rename LIST_SPECULATIVE_CAPTION_ENV_KEYS -> LIST_SPECULATIVE_ENV_KEYS and add
  the float/tabular keys, so the set can't drift from its write sites. Still
  restored only on a non-committing exit; a committed float keeps them, as it
  does without a list (pre-existing).

Tests:
- _parse-isolation: a silent Lists probe over a body holding a figure / an
  unclosed table leaves env untouched (full key+value snapshot) — fails without
  the widened set.
- _list-marker-padding: the default-indent overflow threshold (narrow glyphs
  straddle it).
- _lists fixtures: sibling sublists resolve independently in either order (only
  the wide one reserves).

Docs: spec fix 3 lists every key and its write site; changelog/migration note
the committed-float persistence and the concrete table-markdown/marker diffs.
Three behaviour-preserving optimizations for list-heavy parses, plus docs.

- textReserveEm: measure ASCII (the common case) through a precomputed
  glyph-class lookup table instead of running the class regexes per char. The
  table is built from the same regexes so it can't drift, and iteration moves to
  code units with an explicit surrogate skip (astral chars still count once).
  Verified equal to the old path across ASCII/combining/CJK/astral/surrogates.
- restoreEnvKeys: set previously-absent keys to `undefined` instead of
  `delete`-ing them; `delete` moved the shared parser `env` into dictionary mode
  for the rest of the parse. All readers test the value, so behaviour is
  unchanged — but a consumer inspecting its own `env` after a parse now sees the
  keys present with value `undefined` (documented in changelog/spec/migration).
- safeAssignToken: hoist its skip Set to a module constant (was rebuilt per
  flushed token). Kept the constant at the top of the file so the exported
  function keeps its JSDoc in the generated .d.ts.

Tests updated to assert env leakage at the value level. Docs note the perf
change and the consumer-visible `undefined`-vs-absent env difference.
Docs only; no code change.

- changelog: re-measure the parse-speed bullet and drop the earlier overclaim.
  The memo helps where a terminator scan re-probes the same line, so the win is
  on repeated paragraph + list + footnote units without blank separators
  (200 units 47 -> 24 ms, 400 units 152 -> 43 ms, now linear); a plain
  list-heavy document is roughly unchanged (4119 lines: 60 -> 58 ms). Keep the
  env-rollback migration note.
- spec: state the memo measurement as an on/off isolation (34.4 -> 20.4 ms on
  the re-probed shape; no effect on blank-separated lists), drop the
  unverifiable "33-shape matrix" phrase, and replace "separate ticket" with
  "out of scope here". Refresh the Testing section for the tests that landed
  (sibling sublists, the default-indent threshold, the padding-style guard,
  the figure/table env-isolation cases).
- textReserveEm: classify non-ASCII BMP letters by case (uppercase extra-wide,
  the rest wide) instead of counting them as narrow, so uppercase Cyrillic/
  Greek/accented-Latin markers are no longer clipped by the item text; widen the
  zero-width set to the real combining-mark blocks (a decomposed accent now
  costs only its base glyph); cache the class per code point. ASCII stays on the
  lookup table. Behaviour for ASCII is unchanged.
- ItemsListPush: split a line on a real \item (LATEX_ITEM_SPLIT_RE), so a mid-
  line \itemsep no longer starts a spurious item and drops the space before it.
- resolveListPadding: carry a prefix sum instead of re-reducing the ancestor
  indents per level (equivalent output, O(1) per token).
- snapshotEnvForInline: build the snapshot by copying wanted keys instead of
  spread-then-delete (delete dropped env into dictionary mode); carry symbol
  keys through.
- list-state: warn once per parse for the depth-desync diagnostics, so a silent
  probe can't flood a consumer's log.
- styles-lists: hold the in-content .table-of-contents list at padding 0 too.

Docs: changelog + spec cover the by-case width model (with the measured
under/over-reserve trade-offs), the item-split anchoring, and the perf notes.
Fixture comment corrected (astral emoji land in the wide class, not normal).
- textReserveEm/tokenMarkerWidth: measure code_inline and \texttt{…} markers at
  a flat 0.62em/char (monospace advance) instead of the proportional glyph
  classes, which underreserved narrow chars in a `<code>` face; cache only the
  non-ASCII case test (a Map, covering astral too) rather than a 64 KB table;
  a lone surrogate reserves 0.
- Lists probe memo: include the line's content offset (bMarks+tShift) and
  blkIndent in the key, so a blockquote — which shifts those for the same line
  numbers on one state — can't return a stale answer.
- processListChildToken: on the inline path, pop the shared registry only when
  the top is a list-open of the matching kind, mirroring the block path's
  identity check, so an unpaired close can't misattribute later markers.
- list-state: report each distinct list-depth diagnostic once per parse, capped
  at five, instead of one blanket message — a different depth is new info.

Docs: spec/changelog cover the monospace width model and the probe-key inputs.
The icon demo pages embed a copy of the stylesheet, and 2eb05a7 updated the
list and code-block rules in it. That was pointless: none of the eight pages
contains a <ul>, <ol> or <pre>, so every edited rule matches nothing there.
Restore all five files to their master state.
- resolveListPadding: round the reserve up (Math.ceil), never to nearest, so a
  marker's indent can't land a hundredth below what it needs.
- snapshotEnvForInline: drop undefined-valued LIST_SPECULATIVE_ENV_KEYS from the
  snapshot so replaying it can't clear a key that went live after the rollback
  (reproduced: env.align ended undefined instead of "center" when a list-with-
  tabular preceded \begin{center}). The begin-tabular trio (isInline/subTabular/
  tabulare) is exempt — the tsv/csv export needs their undefined to reach replay.
- tokenMarkerWidth: count monospace cells (code points, combining marks 0) for
  code_inline/\texttt markers instead of UTF-16 length; drop code_inline from
  TEXT_LIKE_TYPES since the monospace branch handles it.
- list-state: warn once per distinct case, no separate cap.
- README: note that lib/ and es5/ are committed build artifacts (build, don't
  compile, before committing) and how to check the tree.

Docs: spec/changelog updated — the env-replay fix (with the reproduced case),
the ceil'd fixture value, the by-case width model, and the release bundling.
- table-markdown getMdLink: build a link label by walking to the matching
  link_close instead of reading only the token after link_open, so a formatted
  label survives the export (`[**b** x](url)` was `[](url)`). Per inner token:
  text with `]` re-escaped, code_inline/smiles with their markers, else the
  getMdForChild marker or its content (keeps image alt / inline-math LaTeX).
- render-table-cell-content: also feed the consumed link tokens to the smoothed
  accumulator, so pptx no longer keeps an opening `<a>` with no text or close.
- Lists probe memo: key on the first line's end offset (eMarks) instead of
  blkIndent — the rule never reads blkIndent, and eMarks pins the content more
  tightly; the geometry still distinguishes a blockquote's shifted lines.
- render-tabular: only build the leaf run when an export (tsv/csv/md/smoothed)
  is requested; an HTML-only render was doing it for nothing.
- block-rule: fold the single list terminator back into resolveEnabledRuleFns
  (one resolver, not two).

Docs: changelog/spec cover the link-label composition (with the image/math/
smiles/escaping behaviour) and the probe-key inputs (blkIndent excluded, tested).
…walking the tail

- writtenAsText (verbatim or command argument) is the single "is this text?"
  predicate; hasCloserAhead and firstUsableCloser now skip an argument closer too,
  not only a code/math one, matching closesOurListWithin and the sibling count
- unclosedEnvsIn walks the tail as the parse loop does, so a closer in a code span
  no longer counts and a sibling that could never close is not opened
- mathOpenerOffsets sweeps the openers once; findVerbatimRanges skips a paragraph
  with no math instead of scanning to EOF per block (126 ms -> 1.2 ms on prose)
- resetEnvSnapshotPool keeps the pool when a snapshot is live (nested render);
  snapshotEnvAll releases its slot if the read throws
- table-markdown trims math in a link label, as the cell loop does; move
  splitInlineListEnv into list-source-model; drop dead non-global regex resets
- Code-block styles: a raw-HTML <pre> with no <code> now takes 15px (13.6 -> 15)
…require warn message

- restoreEnvKeysFromAll blanks a key the parse added but leaves absent a key
  neither side ever had, so Object.keys(env) matches 3.0.1 again — only the value
  differs where a parse leftover used to show
- The sibling-closable check counts closersLeftAfter (free closers, minus the
  openers that claim them), matching hasCloserAhead, not every structural closer
- resolveEnabledRuleFns does not cache while the ruler cache token is null
  (a toggle in flight), an entry that could never be hit
- warnDistinct takes message as a required parameter; encapsulate structuralCountIn
  and CLOSER_SUFFIX_KEY, export closersLeftAfter
- Docs: tsv/csv carry a link's href then the rest of the cell; new console.warn
  diagnostics; sub/sup/ins kept in a plain cell; the \\ fix holds for every command
…nv by what is open

- listHostFlags walks a container stack (open list vs open item) to decide which
  list opening straight inside a list gets a marker-less <li> and closes with one,
  replacing the prevToken/nextToken guesses that missed a list after a sibling
  close and broke on an unbalanced slice
- \end{itemize} over an open enumerate now closes with </ol>: setTokenCloseList
  and listCloseInline read the open type, not the command name, so no closing tag
  is left without its opener
- pairListTokens pairs item and list tokens in one pass (was O(n^2) for absorb);
  extractNextBraceContent and the cell marker-less item go through findEndMarker
- Extract the opaque-env handling into latex-list-opaque; snapshotEnvAll runs
  inside the try so a throwing env getter declines the rule, not the render
Reporting a problem must not be the thing that fails, and restoring `env`
must not outlive its own failure.

- warnDistinct checks `console.warn` per call. A headless runner leaving it
  unset threw from inside the handler that reports a failing rule, so the
  document lost the whole list instead of one warning.
- The `finally` in Lists wraps the whole restore and releases the pool slot
  in its own `finally`. A getter-only or frozen key made the write-back
  throw, skipping `releaseEnvSnapshot()`; since the per-render reset declines
  while a snapshot is live, that slot stayed claimed for the process. The
  failure is now reported as `env-restore-failed:` rather than propagated,
  which matches 3.0.1 (it never wrote back at all, so it never threw).
- A wrapper that closes on its own line hands its tail back to the opaque
  walk when the tail actually shrank. A second wrapper beside it reached the
  caller as text and kept the brace that opened it (`}y`); 3.0.1 dropped it.
- Drop the dead `snapshotDepth = 0` after the early return in
  resetEnvSnapshotPool: release never lets the depth go below zero.
- Cache the list host flags by length plus the two end token types, not
  length alone.

Tests: new tests/_list-source-model.js unit-tests the four readers of the
source model and the opaque walk directly (12 cases, including a property
test that the open-env count walks a tail the way the parse loop does);
_parse-isolation gains a missing-console case, a hostile-env case and three
registry checks (heading slug, label, footnote register once through a
speculative probe). Fixtures: two wrappers on one line, and a closer written
in math outside a wrapper pinned as a quirk.

Docs: the changelog no longer claims a math closer is text outside a wrapper,
names the two-wrapper fix, and warns tsv/csv readers and anyone catching an
exception in a block of their own. README documents how the diagnostic cap
behaves off the markdownToHTML path, that `[TexConvert]` is MathJax's own
channel, that a rule must be swapped through `ruler.at`, and that `env`
should stay small. /pr-specs no longer ships in the tarball.
CI failed with a 2000ms timeout on the fixture sweep. Two causes, both mine.

- The DOM-based sweep added last round duplicated `invalidChild`, which walks
  every child of every list element and not only the first — the "stops at the
  first child" limitation belongs to the regex check in the fuzz harness, not
  here. Verified by removing the host-flag fix: both sweeps failed on the same
  fixture, so one of them says nothing new. Removed, and the gate is still
  there: without the fix `holds across every list fixture` fails and names the
  shape.
- The three sweeps in that block each rendered all 218 fixtures, 1.3s of the
  2s budget before any assertion. They now share one render: the block runs in
  293ms against 729ms.
`$|x|$` exported `| |x| | 2 |`, a row a Markdown reader cuts into three
cells against a two-column header. The ascii branch now escapes like the
path beside it, and both read one chain through `asciiForMarkdown`, so they
cannot disagree on a token carrying only `ascii_md`. Reachable from `|x|`,
`\left|…\right|`, `\vert`, `\|` and a `|` inside `\text{}`; `\mid` is
U+2223 and unaffected. `tsv`/`csv` keep their own chains — a pipe is legal
in both. Over 338 documents that export table-markdown, one changes.

Review follow-ups, none of which move output: `\setcounter` on the inline
path falls back to 1 for a non-numeric argument as the block path does;
`wrapLooseRun` loses a parameter no caller passed; the itemize reset reads
`<= 0` like the enumerate branch beside it, where at zero it assigns zero.

Fixtures: five in `_data/_table-markdown` for the pipe, one of them `\mid`
holding the escape narrow; one in `_data-footnotetext` for a list before a
multi-line note. Five unit tests pin the `absorbSublistIntoWrapper` guards
that no document reaches — two are load-bearing, one of them what keeps the
walk terminating.

Spec: perf re-measured with its method and spread, the `\footnotetext`
terminator claim corrected to match the code, long bullets split and their
narration trimmed.
`findVerbatimRanges` clipped such a span to the blank line and kept it, so the
paragraph tail after an unpaired opener read as math. Everything asking whether
an `\end{itemize}` is text then answered wrong for that stretch: a marker in any
later paragraph disabled the wrapper guard for the text between them, the list
leaked `\begin{center}` and `\end{itemize}` as literal LaTeX, and the item after
the wrapper was lost. Nothing warned — the rule parsed, it just parsed differently.

The window rule now follows what pairs inside one inline token rather than a list
of markers: `$`, `$$`, `\[`, `\(` and both double forms. A math env keeps its
clipped span, its body being allowed to span paragraphs, and that clipping is
what still covers a closer written in the env's first paragraph. Verified for
every opener `RE_MATH_OPEN` admits: `\[` and `\(` render as text across a blank
line exactly as `$` does, so an unpaired one is not math.

One concept, one definition. The six list-structural token types were spelled out
in four places, one of them a copy this branch added under a second name; they now
come from `common/consts` and the structural set is built from the open and close
sets, so a seventh type cannot reach one reader and miss another. `structuralSuffix`
verifies that the array it counts is the one its key was cached for. Two literal
membership chains in the tabular renderer and two in the list tokens read the
shared sets instead. `isWideChar`'s zero-width test no longer runs twice per code
point. `render_item_inline` treats both list kinds alike.

The `20em` clamp no longer warns. It fired on valid input and, as the README had to
say, meant nothing a consumer could act on: a level whose ancestors already reserve
enough does not overlap. Removing it also removes the filter both list test files
carried to keep it out of CI.

Released as a minor by decision — recorded in the changelog with what strict semver
would have made major. Absolute timings dropped from the changelog: they move with
the shape of the input and the host, and the spec carries the method.

Fixtures pin both pairs that differ only by a trailing paragraph (`$` and `\[`), a
raw-HTML link label the export carries through, and the orphan `<li>` an `\item`
after `\end{itemize}` leaves; the fixture sweep now also rejects an `<li>` outside
any list. Fuzzing 12000 documents over the changed paths: no invariant violated,
where `master` violates on 42.
The readers that decide list structure now agree on what is not structure.
`maskNonStructure` blanks code spans and `\item[...]` markers before a walk looks,
keeping length and spaces so a match still applies to the line itself, and
`renewCommandSpanEnd` measures a `\renewcommand` so its body is skipped whole.
The body walk, `findFirstCompleteListEnv` and `unclosedEnvsIn` read the same
masked text; `splitInlineListEnv` no longer answers a question of its own.

What that reached. A closer beside a code span was read as text along with the
span and then closed a second list from inside item content, leaving a stray
`</ul>`; a list command in a marker cost the whole list, where it is now the
marker's text; `\renewcommand*`, `{\x}[1][d]{…}`, a bare `\name` and a space
before the star were unmeasured, so a closer sharing their line was swallowed
and the list stayed open. A span that cannot be measured at all — an unclosed
brace — goes to the walk instead and keeps its `<br>`: losing the closer is the
worse trade. `parseCommand` reads the starred form, skips the optional argument
and counts the brace that opens a bare-name body, so `\renewcommand\x{a{b}c}`
no longer prints `c}` to the reader.

A marker body is parsed with the block flag still set on `env`, so a list written
there opened a real one inside `<span class="li_level">`. All three marker parses
hold a counted flag while they run, and an `\item` with no list open stays text
rather than emitting an `<li>` that has nothing to sit in — which also fixes two
shapes that were pinned as quirks.

The backtick test asked whether a backtick stood on each side, not whether the
command sat in a span, so a command between two closed spans read as text. That
is what the masking replaces. Two costs went with it: the check parsed the line's
spans per match, and re-masking per match walked the line again — 360 transitions
with 720 spans on one line measured 199.5 ms and now measure 59.9, against 97.4
at the branch point.

Whether a list needs an `<li>` of its own is cached per token array and now per
render, so an entry paired before a structural edit is not reused. Re-rendering
one array is unsupported either way, `attrJoin` piling classes onto the same
tokens; the changelog says so.

Over 6000 generated documents in three option modes the branch leaves no
unbalanced tag, against 26 to 54 at the branch point and 54 on `master`, and
renders 487 documents better than `master` with none worse. What remains flagged
there is a list with no items — the empty `<ul>` every build produces for
`\begin{itemize}\end{itemize}`. 4694 tests, 286 list fixtures, fourteen quirks.
The inline rule closed what it was asked to and nothing tracked that the list had
already ended, so a second closer on a line emitted a bare `</ul>` — a tag that
reaches the surrounding page. It declines now when no list is open, as the item
rule already does. Both shapes that carried an unbalanced tag were pinned as
quirks; they render as their own text and move to `_data.js`, leaving no fixture
and no quirk with unbalanced markup.

One line pays for it: with a crossed closer, a dangling `\item` and a dangling
`\begin` together, the whole line stays literal where it used to build a list and
leave a stray `</ul>` and an unclosed `<ol>` beside it. The text is all still
there, unformatted. Four neighbouring shapes — a crossed closer inside a matched
pair among them — build their list as before, three of them where `master` gives
up.

`markerParseDepth` is cleared per render like the level stack, and `endMarkerParse`
clamps at zero. Nothing reaches a stuck flag today, every call sitting in a
`finally`, but a stuck one would mute the four inline list rules for the rest of
the process with no diagnostic. A buffered state takes its own copy of `types`,
which is popped in place: an open always comes first today, and that no longer has
to stay true.

Two paths asked for a warning and got a test instead. The offset anchor answers -1
to two readers that act on it differently, so its contract is pinned directly —
only our own callers can break the suffix invariant, and CI is where that belongs.
The loose-run wrapper stays silent: over every fixture it never sees a state
without a `Token` constructor, and the message would name a caller error the
document cannot act on. The `env` snapshot check keeps its comment corrected
instead of a generation counter, which would have cost the pool its point while
the handle it guards against cannot be reached.

The spec quoted counts from a fuzzer that is not in the repository, and three of
its numbers had gone stale — 222 fixtures against 288, six table-markdown ones
against twenty-nine, 4354 tests against 4701. The counts a reader cannot reproduce
are gone, from the changelog too; the rest are one command each.
Follow-up on the 3.1.0 list work, from the last review round.

Fixes:
- The tail that may open a sibling list is read masked, like the rest of the
  walk. Unmasked, an opener written in a code span counted as a sibling, so the
  walk went on and the closer after it emitted a `</ol>` with no opener —
  a tag-balance regression against `master`, which drops that tail instead.
  The same slice now feeds the walk's own step, so the two cannot drift.
- A run left at list level gets its `<li>` wherever it comes from
  (wrapListLevelRuns). Fix 17 wrapped a chunk before the first `\item` on the
  block path only, so a list opened flush inside an inline run
  (`\begin{enumerate}\begin{itemize}```) put its content straight in the `<ul>`.
  Read from the emitted tokens, not hooked to a rule: the item close belongs to
  whichever rule owns the list.
- srcValueCached takes an `isFresh` predicate and rewrites a stale slot in
  place, so the structural-suffix cache repairs itself instead of recomputing
  the whole walk on every later ask. Its warnDistinct line is gone with it.
- prentLevel falls back to 0 and the two `state.level` assignments send NaN to
  0, so a state built outside the rules cannot poison the depth.

Tests:
- tests/_lists-fuzz.js: seeded corpus over list shapes (xorshift32, separator
  per gap), asserting no unpartnered tag, no crossed tags, no direct child of a
  list that is not an `<li>`, no list markup in a marker. 500 documents per
  option mode by default (2s), LIST_FUZZ_DOCS=6000 for the full sweep (20s,
  400MB peak). Over 6000 documents `master` violates the invariants 567 times,
  the branch point 5, this build none. One open shape sits in KNOWN, pinned in
  the quirks file with what it renders; a list with no item is out of scope,
  since an empty environment renders `<ul></ul>` on `master` too.
- The env-growth timing test is split: the key-count invariant is its own `it`,
  and the timing one retries, takes medians of five and keeps a wide margin.
- Fixtures: three list shapes for the wrap and the masked tail, one quirk, two
  captions — the caption argument is cut at the first `}` on the env's own line,
  pinned as it stands (identical on 3.0.1; pairing it belongs with the caption
  renderer, not here).
- Unit tests for the cache repair and for a state without prentLevel.

Measured: 4718 tests; option matrix 0/64 invalid; no exception escapes on 13
hostile inputs; public API identical to `master`; corpus of 3329 documents
differs from the branch point in 7, all named above; speed within ±2.2% and
peak RSS flat, medians of nine, copy-vs-copy.
Two review rounds on the 3.1.0 list work. Both P0/P1 defects below are this
branch's own, measured against `master`.

Fixes:
- A list closed by an inline `\end` in its body no longer gets a second closing
  tag. `closingList` was read before `finalizeListItems`, which is where such an
  `\end` closes the list, so the handle went stale and the walk emitted a
  `</ul>` with no opener — a tag that reaches the page around the document.
  The emit is decided after the parse and from the level stack. Over 25000 fuzz
  documents this was the last violation left (#24057); `master` has 2425.
- Marker tokens are frozen, and one carrying `attrs` or `children` is copied per
  list. Sharing them cost two defects: a consumer's write reached every later
  list with the same marker, and `link_open` — which pushes `target`/`style` on
  every render — piled those onto the shared token, so the second `<a>` carried
  each attribute twice and the third three times. Cloning every token per read
  measured 12–29% slower, so the plain ones stay shared; the copy costs
  −0.4% to +1.1% with output identical over the corpus.
- `encodeURI` on a marker no longer takes the document down: a lone surrogate
  (`\item[\uD800]` under `forDocx`) threw `URIError` through the whole render,
  where the attribute is a docx-side convenience. That marker's attribute is
  empty now, the rest of the document renders.
- The inline scanner asks its four patterns at an index instead of of
  `src.slice(pos)`, which copied the document tail per character. A 60-item list
  followed by 320KB of text: 24.4ms → 6.2ms; 800 items: 12.1ms → 4.4ms.
- Marks that sit on the base letter — Hebrew, Arabic, Devanagari and Thai — plus
  a soft hyphen and a BOM reserved 0.90em each in the marker width estimate.
  They reserve nothing, so a vowelled marker measures as the bare one.
- `renewCommandSpanEnd` reads a control word's name by char code, as the rest of
  the file does.

Documentation, corrected against measurement:
- The `tsv`/`csv` note claimed a flat one-line list in a cell was unchanged. It
  changes whenever the cell holds content before the first `\item`: that chunk
  used to be dropped and now exports as its own line. Three forms plus a control
  are pinned in `_data/_tsv` and `_data/_csv`.
- `env-transient` claimed a per-render reset guards the section counters. It
  does not — a probe runs inside the render, and `1.` reads `3.`. Inherited from
  3.0.1, now a Non-Goal with the measurement.
- The inline-scanner Non-Goal described the slice that is gone; renumbered.

Tests: the fuzz default is 2000 documents (`npm run test:fuzz` sweeps 25000);
fixtures for the orphan closer, the surrogate marker, the export forms and the
`\caption` shapes that lose text; a test that no fixture reports the level stack
and the token registry disagreeing; two for the zero-width marks.

Measured: 4785 tests; fuzz 25000 × 3 option modes clean; corpus of 3312
documents unchanged except the shapes named here; option matrix 0/64 invalid;
no exception escapes on 13 hostile inputs; public API identical to `master`.
Answers to two review rounds. Nothing here changes rendered output: the corpus
of 3313 documents is byte-identical to the previous commit.

Tests:
- A test that died before releasing its env snapshot left the depth raised, and
  from then on the pool reset and both restores declined — so one failure read as
  four, in tests that do nothing wrong. Root hooks now drain the depth after every
  test of the run and report it once the run ends, naming the test that leaked.
  `envSnapshotDepth()` is exported for that. Verified by leaving a snapshot on
  purpose, from a file that sorts after the reporting one.
- The unmatched-brace growth test compared 2000 against 8000 braces, where both
  sides land at 1–6ms and one tick of the millisecond clock reads as the whole
  bound. It compares 8000 against 32000 now — 5ms and 17ms — so the ratio measures
  the work rather than the clock. An absolute floor was tried first and dropped:
  at these times it would have let a 4× regression through.
- A fixture pins the section numbers under `\footnotetext` above a list holding a
  `\section`: the counters are not rolled back with a speculative parse, and this
  branch made `Lists` one of that scan's terminators, so the numbers must not move
  again from either.

Performance:
- `renewCommandSpanEnd` builds the inline-code index once per call instead of once
  per `findEndMarker`, and only when there is an argument to pair. On 6400
  `\renewcommand`+`\item` pairs in one paragraph: 89ms → 61ms. On 6400 malformed
  ones, where the eager version cost 29ms against 21ms, it is 18ms.

Documentation:
- Fixture and test counts are gone from the spec: they rot with every fixture
  added, and the files are the source of truth. Measurements stay.
- Non-Goals now carry the decisions a reviewer would otherwise re-open: the silent
  -1 from `absoluteOffsetOf` (its branch is unreachable over the fixture corpus and
  6000 fuzz documents), the eight-source cache limit (measured, no cliff at
  document level), and the `prentLevel` typo. The perf method notes that its
  scripts are not in the repository.
- README drops the build-artifact advice it had added. What remains is the one
  non-obvious part: `npm run compile` prints tuple types differently from the
  build, so its output shows up as a diff in files no source change touched.

Measured: 4786 tests, three runs; the growth bounds green under load; fuzz at
25000 documents clean; option matrix 0/64 invalid; public API identical to
`master`; speed within noise.
@OlgaRedozubova
OlgaRedozubova marked this pull request as ready for review August 19, 2026 11:25
@OlgaRedozubova
OlgaRedozubova marked this pull request as draft August 19, 2026 11:26
… its body

Two rules replace guesses the previous commit made about a document's structure.
Over the corpus of 3321 documents four render differently, each pinned by a
fixture; the seeded fuzz is clean at 25000 documents in all three option modes.

Braces:
- A group is an argument only of a command this package parses. Markdown keeps
  braces in prose, unbalanced ones included, so counting every balanced pair let
  two braces written around a list hold its closer: `opens {` above and
  `closes }` below cost the list its second item and printed the tail as literal
  LaTeX. `\foo{ \end{itemize} }` kept a sibling list from opening at all.
- The names are one frozen list, LATEX_BRACE_ARG_COMMANDS, and the sweep is built
  from it so the two cannot drift. Nine of them — the underline family and
  `\author` — build their names by alternation after a backslash check, so a
  source sweep for `\\name` does not find them; left off the list they shielded
  nothing, where the same closer inside `\caption{…}` was text. A test now renders
  `\name{ \end{itemize} }` for every name on the list, with an unsupported name as
  the control, so neither direction can drift silently. `\alph|\Alph|\arabic|
  \roman|\Roman` stay off deliberately: their pattern admits only `enumi…enumiv`,
  so no closer can sit inside an argument that is parsed at all.
- The pairing moved to `common/argument-spans.ts` in two layers — `braceMatches`
  for where each balanced `{` closes, `commandArgumentSpans` for the groups a
  supported name takes — so the caption and footnote readers can reuse it.

Wrappers:
- A wrapper env owns its body: what stands inside goes to the wrapper's own rule,
  a list opened there included. Refusing on an opener inside cost the crossed
  shape both frames — the list broke in two and the wrapper printed as LaTeX.
- A wrapper holding the list's *only* closer is still declined. Handing it the
  body was measured to cost more: the closer goes with it, the list stays open,
  and an unclosed list is printed rather than rendered, so the first item falls to
  text as well. Both shapes are pinned, and the decision is in the spec.
- The order walk that decides whether our closer stands inside the wrapper had no
  fixture left once the opener condition went: a tally passes every other shape.
  One is added — closer first, two openers after it, nothing closing the list past
  the wrapper — where a tally loses the whole list.

Marker tokens:
- Every read of a cached marker returns a copy, always. Copying only the tokens a
  rule writes to left the frozen original reaching a consumer whose rule wrote to
  it, and strict mode threw a TypeError out of `md.render`. Not measurable over
  the corpus (2817/2902ms against 2840/2903 without the copies); on 400 lists of
  five items with a math marker it costs about 1ms of 22, against 31ms on master.

Tests:
- `canCloseAfter` gets its own cases, including the one a net count fails: an
  unclosed env below the sibling must not subtract a closer it reaches first.
- A fence in a table cell inside a leaf run is pinned in `_data/_tsv` and
  `_data/_csv` with a control: only content-free tokens may end a run, so adding
  `fence` to that set drops its export — verified by mutation, six tests fail.
- The fuzz balance check reads list tags per name and in order: counting `ul` and
  `ol` together let `<ul>…</ol>` pass as balanced.

Documentation:
- The changelog and the spec described the intermediate marker-token behaviour and
  quoted a 12–29% figure for copy-on-read that measurement does not reproduce.
  Both now carry only what was measured for the behaviour that ships.
…pened

Four defects where a rule reported success having lost part of its input, and one
where two readers of the same command disagreed. Over the corpus of 3334 documents
17 render differently, each with more structure than before: 19 lists become 28, 21
items become 30, 11 documents holding literal LaTeX become 4, no unbalanced tags
either way. A grid of 228 well-formed shapes goes from 112 violations to 40; the
seeded fuzz is clean at 25000 documents in three option modes.

Line ownership:
- The opaque pass took the prefix before a `\begin{center|figure|tabular|lstlisting}`
  into an item body, so a closer standing there never reached the walk: the strict
  bail dropped the parse and `\end{itemize} \begin{center}x\end{center}` cost a level
  with the rest of the list printed as LaTeX. It declines such a line now — but only
  while a list stays open past the prefix. Declining when the prefix closes the last
  one left the leftover to a walk that cannot emit it, which lost a `tabular` and an
  `lstlisting` that used to render; the live level count answers that.
- A leftover after the outermost closer is handed back to the block phase by offset
  instead of being dropped, so `\end{itemize} tail text` keeps `tail text` — the same
  `<div>` a standalone paragraph gives. Applied in the commit branch alone: the line
  arrays are shared with the real state by prototype, and written from the walk a
  silent probe moved the document's lines under a parse that never applied, taking
  valid nested lists apart. Only `bMarks` and `tShift` move — `sCount` names the
  container, and zeroed it read as a dedent, breaking a markdown list in two. A
  leftover holding list structure stays with the walk: read as a fresh document it
  would open as a top-level list, skipping the closer count that keeps
  `\caption{ \end{itemize} }` from opening a sibling.

Paragraph boundaries:
- `paragraphDiv` read an opener only at a line start, so `text \begin{itemize}` left
  the env reading closed and the paragraph broke on the next opener — half the env
  went to a block below, the rest printed as LaTeX, on master too. It asks the list
  model by offset now, and the list rule declines to end such a paragraph. Two guards
  keep the cost where it belongs: the question is asked only in a paragraph holding a
  mid-line opener, and only for an env the source can still close. Asked per line
  unconditionally, 1000 unclosed units went 61ms to 2761ms — a growth test caught it.
- The gate reads the line through `maskNonStructure`, so an opener in code or in an
  `\item[…]` marker does not fire it; the raw pattern runs first, so the masking only
  costs the rare line.

Backticks:
- `skipBackticks` answered "code to the end of the source" for a run with no partner,
  so the scanner read a list's own `\end` as code and built nothing — while the
  renderer printed those backticks as text, no `<code>` in the output at all. It skips
  the run itself now. Over 8000 documents built from backticks and list structure this
  changes 97, each from literal LaTeX to a rendered list.

One syntax, one reader:
- `\renewcommand{\labelitemi} [1]{Z}` took `[1]{Z` for the marker where the span
  reader measured the same string whole — space before that argument is legal LaTeX.
  Both forms of whitespace are pinned.

Tests:
- The shape grid gains a third invariant: the follower must reach the output. Without
  it a lost env keeps the level count and the no-literal check happy while
  disappearing, which is how the `tabular` regression above passed this very grid. The
  shapes where a follower is still lost are listed by name with their owners.
- Six expectations move, every one of them toward keeping content: a `}` left by an
  unclosed command, a whole code span, a `y$`, and a fence that now opens where it is
  written instead of a line later, where it swallowed the list's closer.

Documentation:
- The claim that a speculative parse makes two headings read `3.` and `4.` does not
  reproduce, here or on master: a `\section` in a list body stays text, so no counter
  sees it. Measured over seven shapes; the comment and the Non-Goal now state the
  missing guarantee rather than a defect.
- `commandArgumentSpans` returns inclusive spans while `isInsideRanges` is half-open;
  said out loud, since the closing brace's own offset reads as outside.
- A dead line about the caller adding `endPos` to `state.pos` is gone — no caller does.
A chunk before the first `\item` gets a marker-less `<li>` so the `<ul>` holds only
`<li>`. A chunk of nothing but `\setcounter` or `\renewcommand` was wrapped too, and
those apply to the list rather than print: the wrapper rendered to nothing, so the
HTML was unchanged and every fixture, the corpus of 3334 documents and both fuzz
runs stayed green — while a consumer walking the token stream for LaTeX emitted an
`\item` the source never had.

    \begin{enumerate}\setcounter{enumi}{35}     →  \item   \setcounter{enumi}{35}
    \item Test36                                   \item Test36

Reached from both paths: the command on the opener's line goes through the inline
one, on its own line through the block one. The run guard now reads such a token as
no content, and the token stream matches `master` on every shape.

The regression test asserts the stream, not the HTML — asserting the output is what
let this through. It pins both paths and keeps a control: a run that does render
still gets its item, or the `<ul>` would hold a child that is not an `<li>`.
Verified by mutation: with the old guard the test fails.
…de a closer

Answers to two review rounds. Four defects, each measured before and after; the
corpus of 3339 documents moves by one — the fixture added for the first of them.

Argument spans:
- A group in prose after a command's own argument was taken as another argument of
  it, so `\textbf{x} {opens \end{itemize} closes}` hid the closer written in that
  prose. That is the unsafe side — an uncovered transition counts as structure — and
  it disabled this branch's own sibling-list fix: with a `\label{L}` before the brace
  the sibling lost its item, without it the item rendered. A further group now counts
  only with no space before it, which keeps `\renewcommand{\x}{y}` and `{a}{b}`.
- Three places skipped a `[...]` option and each had its own version: the span reader
  paired through `findEndMarker`, the sweep used `indexOf(']')` with a line check, the
  rule applying the command used a bare `indexOf`. Over eight forms they disagreed on
  four — a `]` in a code span, `\]`, `[[m]]`, and a `]` one line down. `skipOptionalArg`
  answers for all three now, with the line rule as its parameter, and a property test
  pins the eight forms in both modes.

Dropped content:
- `ItemsAddToPrev` skipped a chunk that *contains* an end-of-list command where the
  comment promised one that *is* only those: `\begin{center} keep \end{itemize} me`
  was dropped whole, taking the wrapper's opener with it, and the `\end{center}` below
  came out as literal text. Anchored now (`ONLY_LIST_CLOSERS_RE`), so the line renders
  and `\end{itemize}\end{itemize}` is still dropped.
- The branch that looked like it kept a leftover after the outermost closer is gone:
  the items are flushed above it, so nothing it added could reach a token. Verified by
  mutation — with the body removed the suite passes unchanged. The drop is what the
  spec chooses, and the comment now says so.

Marker tokens:
- `Object.freeze` is shallow and so was the copy, so `meta` and `map` travelled by
  reference between a cached original and every list reading it. Both are frozen and
  copied now. Measured: no marker body produces either today, so this closes the
  guarantee the comment already claimed rather than a reachable defect — and the test
  written for it was dropped as vacuous rather than left to look like coverage.

Documentation:
- The changelog claimed both that text after the outermost closer is dropped and that
  it is not. The narrow statement is the true one: what is still dropped is a leftover
  that opens a list of its own. Both shapes are pinned — the one-level form as a quirk
  matching 3.0.1, the two-level form beside it in `_data.js`.
… the clock

Two nets, both answering a review: the shape grid covered the block path only, and
the growth bounds measured the runner.

The inline path:
- 64 shapes — a whole env inside one line, at a line start and mid-paragraph, two
  depths, four things after it, with and without text following. Same invariants as
  the block grid: as many lists out as openers in, no list command left as text, the
  follower and the trailing text present. 16 shapes still lose a `center` after the
  env, and the set is byte-identical to `master`: the align rule takes that line
  before this one is asked, as it does in the block grid. Listed by name, so any
  other shape fails. Verified by mutation — disabling the prefix guard fails 22.

Growth:
- The three bounds divided wall time at two sizes and needed `retries(2)` to stay
  green, which measures load rather than complexity. Each now divides the shape's
  growth by that of a plainly linear document measured in the same process, so the
  machine cancels: 1.0–1.4 measured here against 2.8–3.7 with the quadratic walk the
  first of them exists to catch. No absolute thresholds, no retries.
- Counting rule invocations was tried first and dropped: it sees only work that
  crosses a rule boundary, and reported the same ×8.0 with that walk as without it —
  the bound would have gone quiet on exactly the regression it was written for.

Not added: the same content assertion over the seeded corpus. That corpus carries
malformed constructs on purpose — an unclosed macro or `\caption`, an extra closer,
an `\item[...]` marker — and each legitimately consumes the text beside it. Six such
families surfaced, every one byte-identical to `master`, and excluding them takes a
predicate wider than the assertion. Content is asserted where inputs are well-formed
by construction: the two grids.
A perf regression this branch introduced, reported in review and reproduced.
`a2d7ac8b` replaced the argument sweep's `indexOf(']')` with the shared
`skipOptionalArg`, which pairs through `findEndMarker`. On an option that never
closes that reader walks to the end of the source accumulating content and
rebuilds the inline-code index — once per `[`, over the whole document, since
the sweep runs on the source. Measured on a list of unclosed `\caption[`:
126 ms at 1600 units against `master`'s 45, growth per doubling ×5.7 against
×3.8. The closed form was unaffected, as the report said (32 against 31).

In `sameLine` mode the reader now answers from the line before pairing: with no
`]` before the newline the option cannot close, which is the same answer at the
cost `master` paid. 49 ms and ×3.8 now — parity.

No answer changed, and not by argument: the old reader and the new were compared
at every position of 40000 generated forms — 478494 comparisons, no difference.
The 3339-document corpus is byte-identical to `093bba23`, the suite passes 4913,
and the 25000-document fuzz is clean in all three modes.

The `anyLine` reader was checked for the same shape rather than assumed safe: it
passes a cached code index and measures at parity (37 ms against 38).

Tests:
- a growth bound on this form, which the four existing ones did not cover — the
  metric reads ×1.8–2.2 with the fix and ×4.9–7.3 without, against a limit of 3.
- three fixtures for the three branches of the reader, each observable in HTML
  through whether a sibling list opens: option closed, no `]` at all, and a `]`
  one line down.

Also from review: the quirk comment blamed `ItemsAddToPrev` for dropping the
text beside an extra closer. That branch is gone — instrumented, the function is
not called on that input at all. The leftover holds list structure, so the
handoff declines it by decision, and nothing else emits it.
…of skipping

Four review rounds. The output does not move: the 3345-document corpus is
byte-identical to `01edf8eb`, the public API to `master`.

A failing rule is caught and renders literal LaTeX — valid output, so no HTML
comparison, grid count or fuzz invariant reports one, and a `TypeError` of ours
would have passed the whole suite. Rather than rethrow, which would break the
documented promise that `markdownToHTML` does not, the engine counts degraded
renders and a root hook fails the run on any test that had one. Measured first:
zero over 344 fixtures and 25000 fuzz documents, so asserting zero is strict.
Verified by injecting a `TypeError` — the fuzz file passes all three invariants
with one, and the hook names those very tests. It also found seven places that
degrade on purpose and had to say so; an eighth is now loud. `test:fuzz` loads
the file registering the hook, the deep sweep having run past the counter.

`listTailFrom` is an own property of each buffered state now. `Object.create`
had it inherited, and the commit branch rewrites the real `bMarks` from it, so
the invariant rested on the call order inside the branch's hardest function.

Tests that were weaker than they read:
- The one-line grid skipped every assertion for its eight LOST shapes. Asserted
  now, and that split them in two: written at a line start the *list* is lost
  whole (`BeginAlign` takes the line, traced), mid-paragraph the follower is.
  Both pinned as quirks.
- A `KNOWN` entry of the fuzz corpus that matches no document fails the run,
  instead of going quiet — asked at the full corpus only, since a shortened one
  misses entries harmlessly. A non-numeric `LIST_FUZZ_DOCS` read as `NaN` and
  passed all three tests in 2ms on an empty corpus; it falls back now.
- The growth bounds read whole milliseconds, so the small side was 1–2 and one
  GC pause tripled the ratio: red about once in three runs. Fractional
  milliseconds and five samples; measured 1.0–2.2 against limits of 2.2 and 3.

Claims corrected, each traced rather than reasoned about:
- `\end{itemize} \begin{center}c\end{center}` over one level was documented as
  fixed. It is not: the list rule declines that line (asked at ruler position 15
  against `paragraphDiv`'s 29), the paragraph takes the document, and a `center`
  has no rule to render it there — literal LaTeX where `master` dropped it, so
  the output changed. `tabular` and `lstlisting` have an inline rule and render.
  The wrapper is now pinned by a quirk; nothing pinned it before, the grid
  asserting only that the follower did not render.
- Both grids blamed the align rule for that line and claimed byte-identity with
  `master`. Neither holds: of the twenty LOST keys the eight `center`/`figure`
  ones differ from `master` and the twelve fence ones do not.

Also: the `[...]` reader's `sameLine` short-circuit is described where the
option reader is, the irreversible `bMarks` move where `sCount` is, and the
parameter shadowing the `fence` import is renamed.

The five declarations that move here are what `npm run build` writes. It is
`tsc && webpack`, and the webpack step runs `ts-loader` with no options, so it
reads the same `tsconfig.json` — `declaration: true` included — and emits into
`lib/` a second time. The batch compiler prints a tuple from its original syntax
node and `ts-loader` synthesizes it, so the formatting depends on which step
wrote last. Every commit so far held the `tsc` form, so the documented command
left those five dirty; they now hold what it writes, which makes the build
idempotent. Types are identical: 46 lines of tuple line breaks, no `.js` moved.
All four timed out on CI at mocha's 2000ms default. They render documents of up
to 32000 units six times a side, which is 0.4–1.2s here and past the default on
a shared runner. Mine to fix: the samples went from three to five in the commit
before this, which is what pushed them over.

The samples stay at five — measured over eight rounds, three of them spread the
ratio 0.96–1.56 against 1.06–1.19, and the bound sits at 2.2, so the tighter
spread is worth 250ms. What was wrong was the default timeout, not the work: the
bound is a ratio taken in-process, so a generous cap measures the same thing.
60000ms, as the timing tests in `_footnotes_latex.js` already use.

Checked the rest of the suite for the same trap rather than fixing only what CI
named: every other test over a second — the fuzz corpus, the footnote scans, the
unclosed `tabular` bound at 1945ms — already sets its own timeout. These four
were the only ones left on the default.
Four review rounds. Two P0s of ours, a pre-existing defect found on the way, and
a net for each — the 3345-document corpus moves only by the four fixtures added
for the first of them, in every output mode.

The leftover handed back to the block phase was located from the line's start,
but four branches above the walk hand it a line with a prefix already eaten — a
`\setcounter` before a block env, a wrapper closed on the line, a
`\renewcommand`, a bare `\setcounter`. Counted from the start the offset pointed
inside the eaten command, so the block phase re-read from there:
`\renewcommand{\labelitemi}{Z} \end{itemize} TAIL` printed `labelitemi}{Z} …`,
and a `tabular` there put its cell in twice. Two of the four shapes rendered
correctly on `master`. Which anchor to use was measured, not argued: over 12000
documents the line's end held on 4529 of 4529 initialisations and, once fixed, on
2313 of 2313 uses; the start held on 94.8% and 93.5%. It is the anchor
`absoluteOffsetOf` uses twenty lines away.

The marks are put back now, the shape `blockquote` uses for the same arrays. Two
attempts at that were wrong and both were caught here rather than in review: a
walk per leftover nests a frame each and overflowed the stack at 3000, losing the
whole document; bounding the walk to one line cut a fence opening in the leftover
short. The outermost leftover owns the walk and runs to the end of the range,
later ones record their marks for it to restore — 200000 leftovers in a 10.7MB
document keep every one, and the phase ends with the marks as it found them.

`commandArgumentSpans` rebuilt the inline-code index over the whole source for
every `[...]` option. The cost split says it was all of it: the pairing scans for
1000 options take 0.08ms against one index at 0.42ms, and the rebuild made 367ms.
End to end a list of `\caption[short]{full}` lines took 424ms against `master`'s
21 at 52KB, growing quadratically; the index is built once per source now — 23.4
against 21.3, no answer changed over 60000 generated inputs.

Found while measuring the last review item, and not ours: the diagbox inline rule
runs at every backslash and asked an unanchored pattern, so standing at one it
matched a `\diagbox` further along and consumed everything between —
`\alpha VISIBLE text \diagbox{a}{b}` lost `VISIBLE text` on 3.0.1 too. Anchored,
which also drops the scan of the rest of the source per backslash: 340ms against
1206 on a backslash-dense 183KB document.

Nets, since none of these had one:
- a grid over the shapes whose line lost a prefix, 104 cells, 52 of which broke;
- growth bounds on the closed option, on a backslash-heavy math tail closed and
  unclosed, and on a document of 4000 leftovers — each verified by mutation, the
  last one red the moment the nesting comes back;
- `absoluteOffsetOf` counts an anchor that would not confirm, and a root hook
  asserts zero over the run;
- the root hooks moved to `tests/hooks/root-hooks.js`, required through the
  `mocha` key, so they cover a run of one file — registered from a test file an
  injected `TypeError` passed the fuzz in silence;
- fixtures for all of the above, including the inline diagbox shapes.

Rejected with measurements rather than argued: a floor on the inline `openTokens`
pop (12333 of 13815 pops legitimately close a list opened outside the stream), a
checksum in the host-flag key (3200 queries over 9600 tokens would pay per query
what the cache saves), and a source sweep for command-list drift (names built by
alternation are invisible to it, which is how nine were missed).
…s an offset

The block rule claimed its whole line and renders to nothing, so anything
sharing the line was dropped — in a list body once a chunk with a block env
took the block path, and in any document before that. It applies the command
and leaves the line to the paragraph when a tail remains. Verbatim needs no
guard: no block rule runs inside a fence, `lstlisting` or `tabular`.

An unclosed macro no longer eats what follows it: `parseOneCommand` answers
the length of what is left, so `state.pos` ran past `posMax` and the
tokenizer dropped every later line. Bounded to its own line now, which also
fixes the older mid-paragraph form.

The name is read with a word boundary in both rules — disagreeing on
`\renewcommandfoo` cost the line its text, and half-eaten arguments printed
where an unsupported command should print whole.

`renewCommandSpanEnd` answers at an offset with an optional code-span index.
A slice per command rebuilt that index over the rest: 1215ms on 8000
commands beside one code span against 3, and the two inline readers cost
1079 and 1515ms on `master` against 5 and 18 here. Two growth bounds pin the
shapes, thresholds calibrated against the state they must catch.

`token.latex` keeps the shape each rule writes — making them agree gained the
LaTeX converter a blank line per command. A test pins both payloads.

Measured: corpus 1347 sources × 3 modes unmoved; 1728-shape grid 268 forms
changed, none losing text, 16 newly differing from `master`, all of them
`\renewcommand{\x}` with no body where the line below was swallowed; 576
wrapper cases untouched in verbatim; 1152 differential forms identical.
Measured once, the inline one read 0.6 idle and 1.23 under the deep fuzz in
the same process — a coin flip against its 1.2 threshold, which failed one
run in two of `LIST_FUZZ_DOCS=200`. Both now take the lowest of three and
sit at 0.8–1.05 under that load, against 3.5 and 4.1 with the slices they
guard. Verified: nine runs of both modes on Node 16 and two on Node 20, and
the regressed build still fails at ×3.47–3.56.
A rule that re-parses its own argument shares the `env` the list set, so the
block flag stays up in there and an `\end{itemize}` written in the argument
took the list's level from inside that nested parse. The real closer then
found no list to close: `\footnote{f \end{itemize} g}` left the `<ul>` open
and the heading and prose below it went inside, while `\text{…}` emitted a
`</ul>` with no opener on `3.0.1` too. `\caption`, `\label` and `\url` were
never affected — their own rules take the argument first, which is why the
argument model on the block side never had to answer for this.

The list marks which source it handed to the inline parser, and the four
inline list rules answer only for that one — the same shape `isParsingMarker`
already gives a marker body. A symbol, so `Object.keys(env)` and JSON keep
the set they had.

A balance precondition before the commit point was written first and removed:
with the cause fixed it declined nothing in 148134 checks over the suite and
11808 over the fuzz, and in one experiment it turned "unbalanced tags" into
"no list at all", which reads as a fix.

The net that class needs is the fuzz instead: six fragments whose argument
holds a closer, and an oracle over the token stream, where an unpaired open
shows before any tag does. With the guard disabled that corpus now fails on
three documents; it passed before. `KNOWN` empties — its one entry balances
on its own now, measured, so the exclusion goes away rather than going quiet.

Measured: corpus 1347 sources × 3 modes unmoved; 1728-shape grid unchanged at
16 divergences from `master`, all gains; 576 wrapper cases untouched in
verbatim; 1152 differential forms identical; suite 5045 on Node 16 and 20.
`nextMathSpan` resolved each env opener's closer from a copy of the tail and,
when that found none, asked `findEndMarkerPos` over the same tail again. A run
of unclosed openers paid both per opener, so `findVerbatimRanges` — which the
list readers reach through `verbatimRangesOf` — ran quadratically on input the
OCR output produces routinely: 28/106/425/1884 ms at 1000…8000 openers, ×4.0
per doubling. It is 1/1/2/4 ms now, ×1.6.

Both skips rest on one fact: a closer starts with `\end`, whatever the env
name, the spacing in its braces or a star. One `lastIndexOf` per call answers
"is there any past here", and `$`-like markers are excluded by the same test
rather than by a list of their own. No regex, so nothing can drift from
`endTag`.

Two other shapes were measured and dropped: resolving the closer over the
whole string instead of the slice is faster still but changed the answer on 6
of 4000 generated forms — the reader's state depends on where it starts; and a
hand-written sweep regex per env name is a second copy of `endTag`'s pattern.

Equivalence: 4000 generated forms of math, envs, fences, lstlisting and code
spans — identical to the previous build. Corpus 1347 sources × 3 modes
unmoved; 1728-shape grid, 576 wrapper cases and 1152 differential forms all
unchanged. A growth bound pins the axis the two neighbouring ones miss: they
scale the tail after one opener, this one scales the openers, and it fails at
×15.8 with either skip removed.
… changelog

`md.block.parse` hands its fourth argument on as it got it, and the list rule
both reads and writes there — so a falsy one threw out of `Object.keys` and,
caught by the rule's own guard, turned every list of the document into literal
LaTeX with one warning to explain it. `3.0.1` threw a TypeError out of
`md.block.parse` instead. The rule normalises `env` at entry as
`init_math_cache` already does, the inline guards read it with `?.` so a rule
reached without the block rule declines rather than throws, and
`snapshotEnvAll` answers with an empty snapshot, keeping its slot paired.

`parseOneCommand` reads to the end of `src`, which is past `posMax` inside a
link label or an option, so `state.pos` could land beyond the window — the
tokenizer then ends its loop, which is how the rest of a document went missing
before the line bound. Clamped unconditionally now.

Both pinned, and both tests fail with their fix removed.

Two review items came out the other way, and the code now says so. Taking the
tail decision before applying the macro looked cleaner and cost `master`
parity: a later list reads the marker a declined line set, so the order is
load-bearing. And `silent` never reaches this rule — only the '' chain has it,
measured at 0 silent calls of 229282 over the suite — so a guard for it would
be code that cannot run. `srcValueCached` needed nothing either: `recall` runs
before the `isFresh` branch and has already refreshed the slot's age and the
hot slot. `setTokenCloseList` loses a `@returns` for a value it never returned.

The 3.1.0 changelog entry is 8859 characters, from 30418. Nine breaking entries
instead of sixteen — the rest were fixes needing no action, and are grouped
with them. One claim is corrected: a marker's tokens are copied per list, so
lists no longer leak into each other, but within one list each `<li>` still
re-renders the same tokens and a link marker still collects `target`/`style`
once, twice, three times down the list — measured identical to `3.0.1`.

Corpus 1347 sources × 3 modes unmoved, 1728-shape grid, 576 wrapper cases and
1152 differential forms unchanged; suite 5048 on Node 16 and 20, fuzz 25000.
…osed with numbers

Both readers of a `\diagbox` argument built the inline-code index over the whole
string, twice per command, so a line of them followed by a code span was
`n^1.85`: 342 ms at 3200 commands, 31 ms now, and 394 ms of that was
`getSubDiagbox` alone. Inherited — `master` measures 334 ms on the same input.
The index is built once per string in the block path and once per source in the
inline one, through the same `srcValueCached` bucket the renewcommand rule
already uses, and the arguments are read at absolute offsets in `src` rather
than in a fresh slice. Without a backtick in the string the old code was already
linear, `getInlineCodeListFromString` returning at once, which is why the axis
had to be built with a code span past the commands to show anything.

That rewrite cost the tail once on the way: with absolute offsets the rule's end
position is absolute too, and `state.pos += endIndex` doubled it, so everything
after a `\diagbox` went missing. A grid of 910 diagbox forms — five commands,
thirteen argument shapes, seven contexts, both output modes — caught it at 60
divergences and reads 0 with `state.pos = endIndex`. Three fixtures and four
fuzz fragments pin the offsets; a growth bound divides the axis with a code span
by the same one without, ×1.15 here against ×6.17 before.

A bare `\name` is skipped only at the command name now. Between arguments there
is no such form in LaTeX, and skipping one there took `\textbf{x}\unknown{q}`
for two arguments of `\textbf` — so a closer written in a command this package
does not parse was masked when glued to one it does, and not when a space stood
between. A grid of 180 forms now reads glued and spaced alike; two fixtures.

`RENEWCOMMAND_STICKY_RE` was left hot by every render. Harmless today, its one
exec site assigning `lastIndex` first, but the invariant is what makes it so:
a test now asserts every `g`/`y` constant of the module comes back at zero, and
it found this one.

Three review items came out as no change, each with a number. `srcValueCached`
refreshes the slot's age on the recompute path — `recall` runs before the
freshness check and has already re-inserted the entry and set the hot slot; a
test pins it and fails when the two are reordered. The five `src.slice(pos)` at
the list inline rule entries feed anchored patterns, so the match fails at
offset 0 and the tail is never read: 14 ns per call against 10 ns for a sticky
exec, flat from a 1KB tail to 1MB. And the reported ×1.9 on symmetric math does
not reproduce — 620/617 ms on the branch against `master`'s 677/686 at 4000 of
the reported unit, the branch faster on both forms.

Smaller: `hasOwnProperty` rather than `in` for the list's inline-source symbol,
so an inherited one cannot make an own enumerable property; the growth bounds
take the median of three ratios instead of the lowest, which biased down; the
`\diagbox`, empty-`\item` and leftover-as-block behaviours reach the changelog;
the README says where the Markdown exports escape and what a consumer's `env`
comes back as; and `SetItemizeLevelTokens` carries the reason its call sits
where it does.

Suite 5080 on Node 16 and 20, fuzz 25000 with four new fragments, corpus 1401
sources × 3 modes unmoved, 910 diagbox forms and 180 glued/spaced forms at zero.
…acts

The list spec repeated two shapes once per fix: "X was tried and reverted
because Y", twenty-six times, and "measured over N units: a/b/c against d/e/f
on master", each with a few hundred characters of prose around a single number.
Both are tables now — Alternatives measured and rejected, and Measurements —
and the fix list keeps the rule and the shape that identifies it.

The measurement table compares two things instead of three. It had a "Before"
column meaning this branch with one change reverted, which reads as a third
baseline beside `3.0.1` and says nothing a reader can act on; it is master
against the branch now, twelve rows. The numbers that only compare our own two
implementations moved to the rejected-alternatives table, where "instead of /
it cost" is the shape they already had, and the single figures that support a
boundary went back into that boundary's own line.

Two errors of mine in that table are fixed: the cell-export row had the sides
swapped (`tsv`+`csv` cost ×9 here and ×8.5 on `3.0.1`, not the other way), and
the `\renewcommand` grid diff count sat among the measurements as though it
were one.

Nothing was dropped: every fix, boundary, rejected alternative and number was
checked by name against the previous revision after each pass, and the four
that fell out during rewriting — the prototype-chain reads, the option-reader
comparisons, the marker-copy corpus timing and the corpus size — are back.

Also renamed for what they hold: Desired Behavior to Fixes, Non-Goals to
Boundaries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant