ticdc: document Debezium source.start_ts option - #21896
Conversation
ekexium
added
area/ticdc
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
ti-chi-bot
Bot
added
the
size/M
📝 WalkthroughWalkthrough文档新增 Debezium JSON DML 消息中 ChangesDebezium 事务开始 TSO
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: 🔵 Low · up to The documentation adds the optional Debezium transaction TSO field, but the Open API reference still needs to clarify where it applies, how URI precedence works, and that Avro rejects the option. The PR is mergeable with explicit owner follow-up on this bounded documentation contract gap. Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (4 skipped: 4 unsupported.) ✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
该 PR 为 TiCDC 的 Debezium JSON 输出补充文档：新增/补全 source.start_ts（源事务开始时的原始 PD TSO）相关配置、OpenAPI 字段说明，以及消息字段表格条目，帮助下游消费者实现更精确的事务分组或 exactly-once 处理。
Changes:
- 在 Debezium 协议文档中新增“包含事务开始 TSO（v8.5.9 引入）”小节，说明开启方式、影响范围与精度注意事项。
- 在 Kafka sink URI 参数、changefeed 配置项、OpenAPI v2 参数说明中补充
include_start_ts/include-start-ts及对应 URI 参数。 - 在 Debezium DML payload 字段表中补充
payload.source.start_ts行说明。
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| ticdc/ticdc-sink-to-kafka.md | 为 Kafka sink URI 参数表新增 debezium-include-start-ts 说明与链接。 |
| ticdc/ticdc-open-api-v2.md | 为 OpenAPI v2 的 sink.debezium 参数表新增 include_start_ts 字段说明。 |
| ticdc/ticdc-debezium.md | 新增“包含事务开始 TSO”小节，并在 DML 字段表中新增 payload.source.start_ts 说明。 |
| ticdc/ticdc-changefeed-config.md | 为 changefeed 配置 sink.debezium 增加 include-start-ts 配置项说明与引用链接。 |
Suppressed comments (2)
ticdc/ticdc-debezium.md:59
- 为避免读者将
uint64误认为普通文本，建议将类型名用反引号标注，并将 “IEEE-754” 按常见写法写为 “IEEE 754”。
> `start_ts` 是原始的 uint64 PD TSO，不是毫秒时间戳。消费者必须将其作为 64 位整数或十进制字符串处理。不要将其解析为 JavaScript `Number` 或 IEEE-754 `float64`，后者无法精确表示 18 位 TSO。
ticdc/ticdc-debezium.md:614
- 表格行里同样建议将类型名
uint64用反引号标注，便于读者识别为数据类型，并与上文注意事项的写法保持一致。
| `payload.source.start_ts` | 数值 | 源事务的开始 TSO。仅在启用 `debezium-include-start-ts` 或 `[sink.debezium] include-start-ts` 时出现。原始 uint64 PD TSO，不是毫秒时间戳。 |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| ``` | ||
| kafka://127.0.0.1:9092/topic-name?protocol=debezium&debezium-include-start-ts=true | ||
| ``` |
| - DML 消息的 value 会在 `commit_ts` 旁增加整型字段 `source.start_ts`，JSON schema 将该字段声明为 `int64`。 | ||
| - DDL 事件、WATERMARK 事件、key 消息和 Debezium Avro 不受影响。将该选项与 Debezium Avro 协议一起使用会被拒绝。 | ||
| - 如需回滚，关闭该选项即可。关闭期间产出的消息与此前格式保持兼容。 |
| | `compression` | 设置发送消息时使用的压缩算法（可选值为 `none`、`lz4`、`gzip`、`snappy` 和 `zstd`，默认值为 `none`）。注意 Snappy 压缩文件必须遵循[官方 Snappy 格式](https://github.com/google/snappy)。不支持其他非官方压缩格式。| | ||
| | `auto-create-topic` | 当传入的 `topic-name` 在 Kafka 集群不存在时，TiCDC 是否要自动创建该 topic（可选，默认值 `true`）。 | | ||
| | `enable-tidb-extension` | 可选，默认值是 `false`。当输出协议为 `canal-json` 时，如果该值为 `true`，TiCDC 会发送 [WATERMARK 事件](/ticdc/ticdc-canal-json.md#watermark-event)，并在 Kafka 消息中添加 TiDB 扩展字段。从 6.1.0 开始，该参数也可以和输出协议 `avro` 一起使用。如果该值为 `true`，TiCDC 会在 Kafka 消息中添加[三个 TiDB 扩展字段](/ticdc/ticdc-avro-protocol.md#tidb-扩展字段)。| | ||
| | `debezium-include-start-ts` | 可选，从 v8.5.9 开始引入，默认值是 `false`。仅当 `protocol` 为 `debezium` 时生效。如果该值为 `true`，TiCDC 会在 Debezium JSON DML 消息中添加 `source.start_ts`（源事务的原始 PD TSO）。显式指定的 URI 参数优先于配置文件中的 `[sink.debezium] include-start-ts`。该选项与 Debezium Avro 一起使用会被拒绝。详情请参考 [TiCDC Debezium Protocol](/ticdc/ticdc-debezium.md#包含事务开始-tso)。 | |
| - 控制 Debezium JSON DML 消息是否包含 `source.start_ts`（源事务的原始 PD TSO）。 | ||
| - 默认值：`false` | ||
| - 该参数只有当 sink 类型为 MQ 且输出协议为 Debezium JSON 时才生效。与 Debezium Avro 一起设置会被拒绝。 | ||
| - 你也可以设置等价的 URI 参数 `debezium-include-start-ts`。显式指定的 URI 参数优先于该配置项，包括使用 `false` 覆盖 `true`。 |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 799e85ae-e1f8-4ad7-b327-ef5b30376447
📒 Files selected for processing (4)
ticdc/ticdc-changefeed-config.mdticdc/ticdc-debezium.mdticdc/ticdc-open-api-v2.mdticdc/ticdc-sink-to-kafka.md
Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review.
| | 参数名 | 说明 | | ||
| |:-------------------|:-------------------------------------------------------------------| | ||
| | `output_old_value` | `BOOLEAN` 类型，是否输出行数据更改前的值。默认值为 `true`。关闭后，Update 事件不会输出 "before" 字段的数据。 | | ||
| | `include_start_ts` | `BOOLEAN` 类型。从 v8.5.9 开始引入。控制 Debezium JSON DML 消息是否包含 `source.start_ts`（源事务的原始 PD TSO）。默认值为 `false`。 | |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
补齐 include_start_ts 的完整配置契约。
当前条目没有说明该配置仅对 MQ 的 Debezium JSON 生效，也没有说明 Debezium Avro 的拒绝行为和 debezium-include-start-ts URI 参数优先级。API 用户同时提交 Open API 配置和 sink-uri 时，无法确定最终值。
可直接提交的替换
-| `include_start_ts` | `BOOLEAN` 类型。从 v8.5.9 开始引入。控制 Debezium JSON DML 消息是否包含 `source.start_ts`（源事务的原始 PD TSO）。默认值为 `false`。 |
+| `include_start_ts` | `BOOLEAN` 类型。从 v8.5.9 开始引入。仅当下游为 MQ 且输出协议为 Debezium JSON 时生效。控制 Debezium JSON DML 消息是否包含 `source.start_ts`（源事务的原始 PD TSO）。默认值为 `false`。显式指定的 URI 参数 `debezium-include-start-ts` 优先于该配置项，包括使用 `false` 覆盖 `true`。设置为 `true` 时，与 Debezium Avro 协议一起使用会被拒绝。 |As per path instructions：该 Markdown 问题可通过连续行的精确替换安全修复。
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| | `include_start_ts` | `BOOLEAN` 类型。从 v8.5.9 开始引入。控制 Debezium JSON DML 消息是否包含 `source.start_ts`（源事务的原始 PD TSO）。默认值为 `false`。 | | |
| | `include_start_ts` | `BOOLEAN` 类型。从 v8.5.9 开始引入。仅当下游为 MQ 且输出协议为 Debezium JSON 时生效。控制 Debezium JSON DML 消息是否包含 `source.start_ts`（源事务的原始 PD TSO）。默认值为 `false`。显式指定的 URI 参数 `debezium-include-start-ts` 优先于该配置项，包括使用 `false` 覆盖 `true`。设置为 `true` 时，与 Debezium Avro 协议一起使用会被拒绝。 | |
Source: Path instructions
What is changed, added or deleted? (Required)
Document the optional TiCDC Debezium JSON
source.start_tsfield (new in v8.5.9). Downstream consumers can use the original PD TSO of the source transaction for exact-once or transaction grouping. The option is disabled by default.debezium-include-start-ts, TOML[sink.debezium] include-start-ts, URI precedence, DML-only scope, Avro rejection, rollback, and uint64 TSO precision (do not parse as JavaScriptNumber/ float64).include_start_tsfield, and the DMLpayload.source.start_tstable row.Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions (in Chinese).
What is the related PR or file link(s)?
AI agent involvement
Do your changes match any of the following descriptions?
Summary by CodeRabbit
新功能
source.start_ts）。文档