fix: iterate normalize_confusables to a fixed point (idempotent + complete)

[raeq]

The bug

normalize_confusables isn't idempotent on some confusable-base + combining-mark inputs. The normalize_confusables_idempotent proptest finds it only non-deterministically — it took an unlucky CI seed ("¥\u{340}") on the 0.11.1 release PR (#522) to surface it. It's a latent bug on main.

Root cause — a two-way interaction

Confusable folding and canonical composition feed each other, so a single pass is never guaranteed to be a fixed point:

¥ + ◌̀   → compose → ¥+U+0300 → fold ¥→Y → Y+U+0300 → compose → Ỳ   (fold exposes a composition)
Ҫ + ◌̧   → fold Ҫ→C → C+◌̧ → compose → Ç → fold Ç→C → C               (composition exposes a NEW fold)

The second case also breaks the completeness invariant: a one-shot recompose stops at Ç, which is itself a confusable (is_confusable stays true).

Fix

Re-run the fold/compose pass until the output stops changing. Idempotent and complete by construction — the loop can only exit once nothing folds. Converges fast (every fold moves toward the target script, composition only shrinks length → no cycles; observed max 3 internal passes, capped at MAX_PASSES=8 + debug_assert). The no-op fast path is preserved; the pipeline-internal normalize_confusables_into is untouched.

Why the testing changed — this is the point

A random \PC* proptest can only stumble onto the 2-code-point adjacency that triggers this. That's why 1000-case runs missed it for so long, and why an earlier one-shot fix that passed the proptest, a 100k-case stress run, and the full suite still left 61 residual failures — found only by enumerating the whole class:

• New Tier-3 exhaustive gate exhaustive_fold_compose_idempotent_and_complete: every confusable source code point × every combining mark (~9M pairs, ~6s release), asserting idempotency and completeness. 0 counterexamples.
• Wired ignored lib-level exhaustive tests into tier3.yml (--lib -- --ignored) — also picks up a previously-orphaned presets fast-path sweep that the gate never ran.
• Replaced fold_never_drops_chars: iterating to a fixed point can legitimately shorten output (Ҫ+◌̧ → C), so its char-count proxy is false; its real guarantee (no empty table value) is now a deterministic table test.
• Explicit regression test (both directions) + pinned proptest seed.

Verification

Full Rust suite, clippy (core + ext), perf_lint, pytest (3529), the ~9M-pair exhaustive gate (0 counterexamples), and a 50k-case debug proptest — all green.

Unblocks 0.11.1 (#522), which should rebase onto this.

Assisted-by: Claude Code:claude-opus-4-8

[Copilot]

Pull request overview

Fixes a latent non-idempotence bug in normalize_confusables when a confusables fold changes the base character in a way that enables a new canonical composition with an already-present combining mark (e.g. "¥\u{0340}"), ensuring normalize_confusables reaches a fixed point in one pass and stays aligned with is_confusable’s composed-form behavior.

Changes:

• Re-compose the post-fold output in the compose-at-lookup path to make normalize_confusables idempotent when folding exposes additional canonical composition opportunities.
• Add a deterministic regression unit test for the minimal failing input and pin the proptest regression seed.
• Refine the fold_never_drops_chars property test to separate “fold doesn’t delete content” from “post-fold recomposition is content-preserving”.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
src/confusables.rs	Recompose folded output to guarantee idempotence; adds regression unit test and adjusts related proptests.
proptest-regressions/confusables.txt	Pins the newly discovered minimal failing case to prevent regressions.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[raeq]

Resolved the Copilot thread as obsolete: it flagged compose_str(&out).into_owned() from the earlier one-shot-recompose version of this PR. That approach was replaced by the fixed-point loop (force-pushed), which reverts that line — the flagged code no longer exists.