Skip to content

ai rewrite: retry once when the model drops a protected-region placeholder #90

Description

@lsimons

Summary

mdd ai rewrite refuses a rewrite when a protected-region placeholder does not
come back from the model, and dumps the rejected output to
<file>.rewrite.fail. That is the right default — a missing placeholder means a
table, fence or export header would be silently dropped. But the failure is
often transient, and a whole page currently has to be re-run by hand.

Observed on a real corpus:

  • A page failed with __MDD_PROTECTED_0__ missing on one run and succeeded on
    the very next run with byte-identical input. The failure is per-request
    randomness, not a property of the page.
  • The failure concentrated on pages where the placeholder is a large share of
    the input — a bare token on its own line reads to the model as noise worth
    tidying away.

Two changes already reduce it: the constraints prompt now has an explicit
placeholder-preservation section, and chunking means fewer tokens per request
and so fewer chances to drop one (#89). Neither eliminates it.

Proposal

Retry once, with the failure named, before giving up.

  • On a stitch failure, re-issue the same request with a correction appended to
    the user message: which placeholder tokens are missing, and that every token
    must be reproduced verbatim on its own line.
  • Exactly one retry. If the retry also fails, refuse as today and dump the
    retry's output.
  • Log the retry at warning level so a run's output shows it happened; a silent
    retry hides a systematic prompt problem behind a success.
  • The corrected request has a different user message, so it lands on its own
    cache key naturally. Confirm the failed attempt is not what gets cached.
  • Count retries in the run summary, so a corpus-wide rate is visible rather
    than having to be reconstructed from logs.

Out of scope

Retrying a truncated completion. Truncation is a budget problem, not a
compliance problem, and re-issuing the same request would hit the same cap.

Acceptance criteria

  • A dropped placeholder is recovered without operator intervention when the
    retry succeeds.
  • A persistent failure still refuses, still dumps, and does not loop.
  • The retry rate is visible in the run summary.
  • Tests: retry succeeds; retry fails and refuses; the retry prompt names every
    missing token; no more than one retry per chunk.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestneeds-triageMaintainer needs to evaluate this issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions