coordinator: clarify CheckStatus error handling precedence

[3AceShowHand]

What did you do?

While reviewing the table route conflict fix, we found that coordinator/changefeed.Backoff.CheckStatus has two different precedence rules for maintainer-reported errors:

• fast-fail / unretryable errors are handled before checkpoint progress;
• retryable errors are only handled after checkpoint progress does not advance.

A deterministic code-level case is:

1. Backoff.checkpointTs is 1.
2. Maintainer reports a new MaintainerStatus with CheckpointTs = 2.
3. The same status also contains a retryable status.Err.

Current code path: CheckStatus observes checkpoint progress first and returns StateNormal, so the retryable error is not surfaced in that call.

The current PR only fixes the narrower table-route fast-fail path by checking unretryable errors before checkpoint progress. The broader retryable-error precedence is intentionally left for a separate change because it affects backoff/recovery semantics.

What did you expect to see?

We should clarify and likely enforce one consistent invariant for CheckStatus:

• if a maintainer status reports Err, should the error always be handled before considering checkpoint progress?
• should m.checkpointTs advance when the same status also reports an error?
• when should a later no-error status be allowed to recover the changefeed from warning to normal?

If the intended invariant is “reported error wins”, then CheckStatus should handle status.Err first and let HandleError classify fast-fail vs retryable errors. The function comment should also be updated so it does not imply retryable errors are ignored whenever checkpoint advances.

What did you see instead?

Today the behavior is split:

• fast-fail errors can fail the changefeed immediately;
• retryable errors can be bypassed by checkpoint progress in the same status.

Changing this is not just a one-line bug fix. It affects:

• when a changefeed enters warning;
• when isRestarting is set;
• whether m.checkpointTs advances on an errored status;
• when a later no-error status recovers the changefeed.

Suggested follow-up tests:

• retryable error + advanced checkpoint;
• fast-fail error + advanced checkpoint;
• recovery from warning only after a later no-error status advances checkpoint.

Related code:

• coordinator/changefeed/backoff.go
• coordinator/changefeed/backoff_test.go

Versions of the cluster

This is a code-path issue found during PR review, not an environment-specific runtime report.

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

N/A

Upstream TiKV version (execute tikv-server --version):

N/A

TiCDC version (execute cdc version):

current development branch

[asddongmen]

Reproduced this exact fast-fail precedence bug on release-8.5 while investigating PR #5761.

CI evidence:

• Failing job: pull-cdc-mysql-integration-heavy #2744, case table_route_conflict_detection, subcase route-conflict-multi-rename.
• At 2026-07-27 11:07:30.553 +08:00, the maintainer detected and cached CDC:ErrTableRouteConflict.
• The maintainer checkpoint had advanced from 467958928354574344 to 467958928826171410 shortly before the error heartbeat.
• Coordinator emitted no changefeed maintainer report an error log for this subcase; all 20 API polls continued to return state=normal, error=null.
• Maintainer.GetMaintainerStatus() drains runningErrors, so after Backoff.CheckStatus returns early on checkpoint progress, the one-shot fast-fail error is no longer present in later heartbeats. routing.Admin.failed also prevents the same error callback from being reported again.

The same flaky failure reproduced on unrelated PR #5757 in job #2740, while the same head passed later jobs #2741/#2742, confirming this is timing-dependent and not introduced by #5761.

The fix from #5424 is absent from the current #5761 head (618b69346745f5fb04d7c89a1a392d44c91746e5). Its release-8.5 cherry-pick is now tracked by #5771.

[3AceShowHand]

close this by #5424