Problem
punctuationCleanup remains a hardcoded normalization policy island. The YAML pipeline names the step, but the actual pause-punctuation set, lexical apostrophe protection, placeholder protocol, and strip policy are still embedded in TypeScript regex code.
Evidence
public/rules/frontends/qlatt-english/normalization-pipeline.yaml has punctuation_cleanup as type: builtin, handler: punctuationCleanup.
src/g2p/text-normalize.ts implements punctuationCleanup with a private placeholder ("\x00") and hardcoded punctuation regexes for , . ? ! ; :.
- The same punctuation marks are already declarative transcription data in
public/rules/frontends/qlatt-english/frontend.yaml under transcription.punctuation_tokens, so there is drift risk between normalization and transcription.
Acceptance criteria
- The pause-generating punctuation inventory is loaded from declarative config, not duplicated as a regex literal in
punctuationCleanup.
- Lexical apostrophe preservation and strip policy are represented as explicit config or a generic named normalizer whose behavior is table-driven.
- Add tests proving a config change to punctuation tokens affects normalization/tokenization consistently.
- Remove the hardcoded
[,.?!;:] policy from src/g2p/text-normalize.ts.
npm.cmd run typecheck:core passes.
Problem
punctuationCleanupremains a hardcoded normalization policy island. The YAML pipeline names the step, but the actual pause-punctuation set, lexical apostrophe protection, placeholder protocol, and strip policy are still embedded in TypeScript regex code.Evidence
public/rules/frontends/qlatt-english/normalization-pipeline.yamlhaspunctuation_cleanupastype: builtin,handler: punctuationCleanup.src/g2p/text-normalize.tsimplementspunctuationCleanupwith a private placeholder ("\x00") and hardcoded punctuation regexes for, . ? ! ; :.public/rules/frontends/qlatt-english/frontend.yamlundertranscription.punctuation_tokens, so there is drift risk between normalization and transcription.Acceptance criteria
punctuationCleanup.[,.?!;:]policy fromsrc/g2p/text-normalize.ts.npm.cmd run typecheck:corepasses.