Ask the model again when its reply holds its own markup, then remove it

docs/plan.md

@@ -12,6 +12,7 @@
   1. `feat/support-check` stays for a later try.
   2. Tuning resumes only on the owner's word, from what staging shows.
   3. Staging, 2026-10-26: after one search, "Add grocery delivery to the PRD." got a reply whose whole text was a JSON object with a `question` field, shown as is, nothing saved; one run, an observation (`docs/records/staging-2026-10-26-grocery-json.md`, local). The owner chose the display fix only: such a reply now shows the sentence inside (`unwrappingJsonReplies`, `src/server/AGENTS.md`), on `main` since 42fcca6; `verify` passed locally (600 unit tests); unverified on staging until such a reply comes again. The plain wording is measured only on their word.
+  4. 2026-11-02, seen by the owner on "Draft the first section.": replies whose text held the model's own markup, three times, none in the 141 records before. The owner chose the display fix and one automatic retry, on `fix/hide-model-markup` (`retryingModelMarkup`, `src/server/AGENTS.md`); unverified on the real model until such a reply comes again, and the two warnings in the logs show how often it happens and how often the retry helps. No cause is isolated; temperature (Cloudflare's default, 0.6) is unmeasured, and measured only on their word.
 - **Lean notes** (the owner, 2026-10-22; this lane only): this section, the measurement's parts of `src/server/AGENTS.md` and the lane's decisions hold only what a later session needs to test, fix or decide: the current state, the lessons that still apply, the next steps. Replace what's outdated instead of adding to it; one line of evidence at most, with the details in `docs/records/` (local).
 - **Each batch:** `--plan` first, then real runs only, never a probe; a refused request costs nothing, and the planned batch is sent again on the owner's word or after 00:00 UTC. The agent decides the batches within the day (`docs/decisions.md`, 2026-10-21). Plan updates go on a documents-only branch from `main`.
 - **Before each batch after a merge of the build lane:** read that merge's diff of `src/server` and `src/lib`. If it changed what the model receives (the prompt, the tools' descriptions and schemas, the generation settings, how messages and earlier answers reach the model, the search) or the transport (`src/server/ai/cloudflare.ts`, which slice 8a may change), stop and tell the owner before sending anything. The 9d comparisons list such merges as code differences; read the file list before trusting that warning.

src/server/AGENTS.md

@@ -73,7 +73,7 @@ What shapes the model (2026-10-22):
 | Searches and saves per answer | `chat.ts`: `MAX_SEARCHES`, `MAX_SAVE_TRIES`, `MAX_STEPS` | three searches, each result saying how many are left; two tries to save |
 | How earlier answers look to the model | `chat.ts`: `forEarlierAnswers` | searches cut to passage ids and interviewees; saved sections whole; reasoning left out |
 | Its reasoning between tool calls in one answer | the AI SDK, `@ai-sdk/openai-compatible`, `ai/cloudflare.ts` | passed back as `reasoning`; whether Cloudflare reads it shows in the records, see below |
-| The transport | `ai/cloudflare.ts` | whole replies; empty text instead of `null`; reasoning sent as `reasoning` and tool results named, as Cloudflare's provider does (since 2026-10-22); no retry at the quota, retry waits of at most 5 seconds (slice 8a); a reply of one repeated character refused (since 2026-10-22); a reply that is only a JSON object of one text field becomes that text (since 2026-10-26) |
+| The transport | `ai/cloudflare.ts` | whole replies; empty text instead of `null`; reasoning sent as `reasoning` and tool results named, as Cloudflare's provider does (since 2026-10-22); no retry at the quota, retry waits of at most 5 seconds (slice 8a); a reply of one repeated character refused (since 2026-10-22); a reply that is only a JSON object of one text field becomes that text (since 2026-10-26); a reply's text holding the model's markup is asked for once more, then removed (since 2026-11-02) |
 | The search | `search.ts` and the embedder | fake word overlap here, bge-large-en-v1.5 on staging; the five closest (the reranker is off since 2026-10-22, *The reranker*) |
 | The app's own messages to the model | the check-in (`src/features/AGENTS.md`) and `checkReport` | fixed texts |
 | The visitor's wording | the visitor | free |
@@ -145,6 +145,7 @@ The tool that measures the model's behaviour (`docs/plan.md`, slices 9a to 9c; d
 - **The quota and other 429s** (slice 8a): `refusingQuota` wraps `fetch` for chat and embeddings alike, under whatever `fetch` the settings pass (the measurement's meter and recorder see every reply first). A 429 with code 3036 or 4006 (`QUOTA_CODES`; 4006 is what `/ai/v1` sent on 2026-10-20) throws `QuotaExceededError` from `fetch`, which `handleFetchError` in `@ai-sdk/provider-utils` passes through and the SDK doesn't retry: nothing works before 00:00 UTC. Any other 429 (the per-minute limit, code 3040 out of capacity) goes on to the SDK's two retries, but a `retry-after-ms` or `retry-after` asking for more than `MAX_RETRY_WAIT_MS` (5 seconds) is dropped first, so the SDK's own 2 and 4 seconds apply instead of up to 60 each, inside the route's 60 seconds (`getRetryDelayInMs` in `ai/dist/index.js`, read 2026-10-20). The request bodies are untouched. The quota codes are also in `measurement/budget.ts` (`QUOTA_CODES`); keep the two lists the same. Replayed replies in `tests/unit/cloudflare.test.ts`; the 4006 one is Cloudflare's own of 2026-10-20.
 - **Degenerate replies** (`refusingDegenerateReplies`, 2026-10-22): Cloudflare has answered with a run of "!" up to the output limit, with no reasoning and no tool call (2026-10-21 and 2026-10-22). A reply with no tool call, finish reason `length` and text at least 95% one character throws `DegenerateReplyError`, so the visitor gets the generic error and "Try again", never the text; the AI SDK doesn't retry it, and there's no retry of our own. The `wrapGenerate` middleware comes after `simulateStreamingMiddleware` in the array, since the last one is wrapped directly around the model and so sees each whole reply before it becomes a stream (the installed `LanguageModelMiddleware` type in `node_modules/ai/dist/index.d.ts` and `wrapLanguageModel` in `dist/index.js`; https://ai-sdk.dev/docs/reference/ai-sdk-core/wrap-language-model, read 2026-10-22 through Context7). The recorded replies are replayed in `tests/unit/cloudflare.test.ts`; the measurement counts such an answer "not run".
 - **Replies written as JSON** (`unwrappingJsonReplies`, 2026-10-26): on staging, gpt-oss-120b's whole reply after a search was `{"question":"…"}`, shown to the visitor as is. A reply with no tool call whose whole text is a JSON object of one non-empty text field, bare or in a code fence, becomes that text; anything else, a save written as text included, passes as it came. A `wrapGenerate` middleware next to `refusingDegenerateReplies`, for the same reason; the reply is replayed in `tests/unit/cloudflare.test.ts`. It changes what earlier answers look like to the model too, since the browser sends the unwrapped text back.
+- **Replies holding the model's markup** (`retryingModelMarkup`, 2026-11-02): the owner saw gpt-oss-120b write its control tokens (https://github.com/openai/harmony/blob/main/docs/format.md, "Special Tokens") as the reply's text, three times on "Draft the first section.": a search after a save, its reasoning and made-up searches, and a search in an answer's first call, so nothing ran or saved. None of the 141 records before it held such a text. The same request goes once more, at most once per model call, so a second call's cost and time only when it happens; the reply carries both calls' tokens, and the meter counts both HTTP requests while a record's model call shows only the second reply. If the second reply holds markup too, its whole text goes, since the words before the first token can be reasoning; a tool call in the same reply stays. The chat then treats it as a reply without words (*Failures*, above). Two warnings without the text show in the logs how often it happens and how often the retry helps. No cause is isolated, and whether the retry usually works is unmeasured. The replies are replayed in `tests/unit/cloudflare.test.ts`.
 - **`includeUsage: true`** makes streamed replies report tokens; kept for when streaming returns.
 - **No token count for embeddings:** on staging (2026-10-17) Cloudflare's `/embeddings` reply had no `usage`, and the AI SDK then reports `NaN` tokens. `cloudflare-embedder.ts` logs "Cloudflare reported no token count" instead.
 - **Embedding batches:** Cloudflare's bge models take at most 100 texts a call; the package would send 2,048, so `cloudflare-embedder.ts` splits them.

src/server/ai/cloudflare.ts

@@ -296,6 +296,81 @@ const unwrappingJsonReplies: LanguageModelMiddleware = {
   },
 };
 
+/**
+ * gpt-oss's own control tokens
+ * (https://github.com/openai/harmony/blob/main/docs/format.md, "Special Tokens").
+ */
+const MODEL_MARKUP =
+  /<[|](?:start|end|message|channel|constrain|return|call)[|]>/;
+
+/** Whether a reply's text holds the model's markup. */
+function holdsMarkup(content: readonly { type: string; text?: string }[]) {
+  return content.some(
+    (part) => part.type === "text" && MODEL_MARKUP.test(part.text ?? ""),
+  );
+}
+
+/** Two calls' count of one kind of token; undefined if neither reported it. */
+function addedTokens(first: number | undefined, second: number | undefined) {
+  if (first === undefined && second === undefined) return undefined;
+  return (first ?? 0) + (second ?? 0);
+}
+
+/**
+ * gpt-oss can write its own markup as the reply's text: a search after a save,
+ * its reasoning and made-up searches, or a search in an answer's first call,
+ * so nothing runs (seen by the owner, 2026-11-02; replayed in
+ * `tests/unit/cloudflare.test.ts`). The same request goes once more, and the
+ * reply carries both calls' tokens. If the second reply holds markup too, its
+ * whole text goes, since what comes before the first token can be reasoning.
+ * The chat then treats it as a reply without words: a saved section stands
+ * alone, and otherwise the visitor gets the usual error and "Try again".
+ */
+const retryingModelMarkup: LanguageModelMiddleware = {
+  specificationVersion: "v4",
+  async wrapGenerate({ doGenerate }) {
+    const first = await doGenerate();
+    if (!holdsMarkup(first.content)) return first;
+    console.warn(
+      "The chat model wrote its own markup into a reply; asking it again.",
+    );
+    const second = await doGenerate();
+    const { inputTokens: input, outputTokens: output } = first.usage;
+    const usage = {
+      ...second.usage,
+      inputTokens: {
+        total: addedTokens(input.total, second.usage.inputTokens.total),
+        noCache: addedTokens(input.noCache, second.usage.inputTokens.noCache),
+        cacheRead: addedTokens(
+          input.cacheRead,
+          second.usage.inputTokens.cacheRead,
+        ),
+        cacheWrite: addedTokens(
+          input.cacheWrite,
+          second.usage.inputTokens.cacheWrite,
+        ),
+      },
+      outputTokens: {
+        total: addedTokens(output.total, second.usage.outputTokens.total),
+        text: addedTokens(output.text, second.usage.outputTokens.text),
+        reasoning: addedTokens(
+          output.reasoning,
+          second.usage.outputTokens.reasoning,
+        ),
+      },
+    };
+    if (!holdsMarkup(second.content)) return { ...second, usage };
+    console.warn(
+      "The chat model wrote its own markup again; its text was removed.",
+    );
+    return {
+      ...second,
+      usage,
+      content: second.content.filter((part) => part.type !== "text"),
+    };
+  },
+};
+
 /**
  * The chat model, asked for whole replies. Streamed, Cloudflare sends
  * gpt-oss's tool calls as reasoning text without the tool's name and ends
@@ -314,6 +389,7 @@ export function createCloudflareChatModel(
     middleware: [
       simulateStreamingMiddleware(),
       unwrappingJsonReplies,
+      retryingModelMarkup,
       refusingDegenerateReplies,
     ],
   });