feat(TAP-8714): mysql support auto adjust stream read size - #671

Draft

11000100111010101100111 wants to merge 356 commits into

TAP-8714-auto-set-batch-size-mastr-devfrom

TAP-8714-auto-set-batch-size-mysql-dev

This pull request is big! We're only showing the most recent 250 commits

Commits on Jan 6, 2026

feat: mysql innodb cluster
jarad0628
committed
5bde218

Commits on Jan 7, 2026

fix: TAP-9317 mysql add column with default function
mnianqi
committed
005c62e

Commits on Jan 8, 2026

fix: postgres and mongo io exception retry
jarad0628
committed
0d44773

Commits on Jan 9, 2026

fix: TAP-9316 修正 MongoDB 心跳事件的时间设置，及 v4+ 的更新操作无法更新子文档问题
 HarsenLin
committed
3a1a400

Commits on Jan 14, 2026

Commits on Jan 15, 2026

fix: TAP-9317 mysql add column with default function
mnianqi
committed
0344f50
Merge pull request #699 from tapdata/TAP-9317
jarad0628
authored
dc5a9ee

Commits on Jan 16, 2026

Commits on Jan 17, 2026

feat: mysql-connector remove debezium
jarad0628
committed
9a33e98

Commits on Jan 19, 2026

feat: mysql-connector remove debezium
jarad0628
committed
210747c
Merge remote-tracking branch 'origin/develop' into develop
jarad0628
committed
d35de09
revert: RegisterMain
jarad0628
committed
85aa825
Merge branch 'main' into develop
jarad0628
committed
6c44d50
fix: Fix paimon writing issue
ply0011
committed
1ad9b71
Merge remote-tracking branch 'origin/develop' into develop
jarad0628
committed
be7a86b
Merge pull request #704 from tapdata/develop
jarad0628
authored
c244bb6

Commits on Jan 20, 2026

fix: Fix paimon writing issue
ply0011
committed
e1a37cb
fix: mysql highPerformance default false
jarad0628
committed
46ede15
Merge remote-tracking branch 'origin/develop' into develop
jarad0628
committed
521afb0
fix: Fix paimon writing issue
ply0011
committed
b9d81be
Merge branch 'develop' of https://github.com/tapdata/tapdata-connectors into develop
ply0011
committed
67f2e59
feat: Paimon supports updating primary key values
ply0011
committed
1489387

Commits on Jan 21, 2026

Commits on Jan 22, 2026

fix: pgto money bug
jarad0628
committed
a3889fe
Merge branch 'develop'
jarad0628
committed
dd0cdf5
fix: MysqlReaderV2 all data type
jarad0628
committed
79c1181
fix: postgres query time(6)
jarad0628
committed
40b426b

Commits on Jan 23, 2026

fix: MysqlReaderV2 ddl collect
jarad0628
committed
dbe8084
fix: pgto money bug
jarad0628
committed
0054019
fix: Fix paimon IllegalThreadStateException
ply0011
committed
0f16275

Commits on Jan 26, 2026

Commits on Jan 27, 2026

fix: starrocksTest streamLoad url
jarad0628
committed
2f39dab
fix: MysqlReaderV2 ddl bug
jarad0628
committed
860181a

Commits on Jan 30, 2026

Commits on Feb 2, 2026

fix: pgto money bug
jarad0628
committed
ff01049

Commits on Feb 3, 2026

Commits on Feb 4, 2026

feat: paimon as source
jarad0628
committed
9967392

Commits on Feb 5, 2026

fix: TAP-9678 StarRocks cache data is incomplete
mnianqi
committed
3aaf105

Commits on Feb 6, 2026

fix: starrocks label with dot problem
jarad0628
committed
46c5851

Commits on Feb 10, 2026

fix: paimon reader
jarad0628
committed
8767968
feat:When the engine encounters an extremely large event, a warning is required.
weiliang110100
committed
c3b708a

Commits on Feb 12, 2026

fix: paimon reader all data type
jarad0628
committed
ae837e8

Commits on Feb 13, 2026

fix: write into paimon error
ply0011
committed
592ac82

Commits on Feb 23, 2026

Commits on Feb 25, 2026

Commits on Feb 26, 2026

Commits on Feb 28, 2026

fix: execute and call procedure in JS
jarad0628
committed
94324b1

Commits on Mar 1, 2026

fix: execute and call procedure in JS
jarad0628
committed
a466bb1

Commits on Mar 3, 2026

fix: add paimon connection string
jarad0628
committed
f73453b

Commits on Mar 4, 2026

Commits on Mar 5, 2026

fix: TAP-10060 custom connector add global map to cache token
mnianqi
committed
563c297
Merge pull request #717 from tapdata/fix-develop-TAP-10095
jarad0628
authored
d45ab7c
Merge pull request #718 from tapdata/fix-main-TAP-10095
jarad0628
authored
b069028
Merge pull request #719 from tapdata/TAP-10060
jarad0628
authored
f10096b
Merge pull request #720 from tapdata/main
jarad0628
authored
97a5209
Merge pull request #721 from tapdata/develop
jarad0628
authored
17fce56

Commits on Mar 6, 2026

fix: TAP-10148 custom connector throw exception when write records failed
mnianqi
committed
42702ee

Commits on Mar 9, 2026

Commits on Mar 10, 2026

Commits on Mar 11, 2026

feat: kafka enhance support kerberos
jarad0628
committed
26a7807
Merge pull request #727 from tapdata/TAP-10169
jarad0628
authored
49c433d
feat: json support paimon for each table partition key
jarad0628
committed
d6992ac

Commits on Mar 12, 2026

feat: json support paimon for each table partition key and data transfer
jarad0628
committed
91a7a80
fix: TAP-10169 mariadb read binlog position failed
mnianqi
committed
b8dd332
Merge pull request #729 from tapdata/TAP-10169
jarad0628
authored
3e0d610
feat: redis support tableNodeConfig
jarad0628
committed
916c654
Merge pull request #731 from tapdata/main
jarad0628
authored
02e9bc8
feat: TAP-9677 starrocks supports HA, fix for cache data is incomplete
mnianqi
committed
8ac41e4
feat: paimon support hash key when many primary keys
jarad0628
committed
4e1ec60
fix: registry avro json ignore primary key
jarad0628
committed
4afed32

Commits on Mar 13, 2026

Merge pull request #732 from tapdata/TAP-9677
jarad0628
authored
8320767
feat: add tableNodeConfig for doris
jarad0628
committed
dc53c39
fix: mongo spec.json timestamp and regexpression
jarad0628
committed
691638d

Commits on Mar 16, 2026

Merge pull request #734 from tapdata/develop
jarad0628
authored
8edefa5

Commits on Mar 17, 2026

fix: kafka avro default value not Serializable
jarad0628
committed
5284118

Commits on Mar 19, 2026

Commits on Mar 21, 2026

feat: upgrade 2.0.7 support local file connector to push heartbeat
jarad0628
committed
d0f0028

Commits on Mar 23, 2026

hot-fix: api 2.0.6 support push control event for starrocks
jarad0628
committed
de4ddf3
hot-fix: api 2.0.6 support push control event for starrocks
jarad0628
committed
19f7ca1
hot-fix: api 2.0.6 support push control event for starrocks
jarad0628
committed
9b3172f
Merge pull request #738 from tapdata/fix-main-processControl
jarad0628
authored
63f65b1
hot-fix: api 2.0.6 support push control event for starrocks
jarad0628
committed
ace9526
hot-fix: starrocks lost data
jarad0628
committed
dc23496
hot-fix: starrocks lost data
jarad0628
committed
670fd30
hot-fix: starrocks lost data
jarad0628
committed
633263d
hot-fix: starrocks lost data
jarad0628
committed
de7e291
Merge pull request #739 from tapdata/fix-main-processControl
jarad0628
authored
e3dfd45

Commits on Mar 24, 2026

feat:TAP-9662 Build a logical view based on MDM via MongoDB aggregation.
weiliang110100
committed
5f265f5
fix: postgresTest create slot add timeout
jarad0628
committed
413bc88
fix: old kafka record null cause NPE
jarad0628
committed
39981a7
fix: old kafka record null cause NPE
jarad0628
committed
e78236e
Merge pull request #742 from tapdata/fix-main-processControl
jarad0628
authored
c8ea217
fix: postgres can use specific slot to cdc again
jarad0628
committed
d289a9e

Commits on Mar 25, 2026

fix: postgresTest create slot add timeout
jarad0628
committed
b289972
fix: TAP-7203 support mongodb ssl with password-protected certificates
mnianqi
committed
22d1bff
Merge pull request #743 from tapdata/TAP-7203-v2.0.6
jarad0628
authored
8726a3f

Commits on Mar 26, 2026

fix: postgres money null bug
jarad0628
committed
c7463b2
feat: add connector snowflake
jarad0628
committed
b6309cf
fix: postgres money null bug
jarad0628
committed
82673fc
fix: starrocks update partial columns bug
jarad0628
committed
261be08
fix: starrocks update partial columns bug
jarad0628
committed
ab8de5d
Merge pull request #744 from tapdata/fix-main-processControl
jarad0628
authored
65bd5c1
Merge pull request #740 from tapdata/feat-TAP-9662-develop
jarad0628
authored
4f0a892
Merge branch 'main' into develop
jarad0628
committed
714c822
Merge remote-tracking branch 'origin/develop' into develop
jarad0628
committed
ccc22a7

Commits on Mar 27, 2026

fix: Upgrade GitHub workflow config from 'actions/checkout@v3' and 'actions/checkout@v2' to 'actions/checkout@v4'.
HarsenLin
committed
bc303ea
fix: Fix the Sync-Code-to-Office error.
HarsenLin
committed
ebe4402
fix: Fix reusable workflow error.
HarsenLin
committed
bd343a9
fix: Fix reusable workflow error.
HarsenLin
committed
58d563b
fix: Fix reusable workflow error.
HarsenLin
committed
e788e8a
fix: Fix reusable workflow error:
HarsenLin
committed
9a07ac9
fix: Fix reusable workflow error:
HarsenLin
committed
c17fda2
fix: Fix reusable workflow error
HarsenLin
committed
359495e
fix: Fix reusable workflow error
HarsenLin
committed
61a76e6
chore(workflow): Update code synchronization workflow configuration
HarsenLin
committed
c30dd6c
fix: TAP-10570 Multi-update operations are not allowed when updating the shard key field
mnianqi
committed
1e29478

Commits on Mar 28, 2026

feat: add connector snowflake
jarad0628
committed
98c8b75

Commits on Mar 31, 2026

Commits on Apr 1, 2026

feat: recode kafka test
jarad0628
committed
576bcbd
fix: starrocks httpclient bug
jarad0628
committed
97fcfbd
Merge pull request #746 from tapdata/harsen/fix-develop-upgradeGithubActionsCheckoutv4
jarad0628
authored
ac10c42
Merge pull request #747 from tapdata/TAP-10570
jarad0628
authored
c2985da
fix: TapMoney convert to decimal(10,4)
jarad0628
committed
a537eab
feat: recode kafka test
jarad0628
committed
c5b9518
Merge remote-tracking branch 'origin/develop' into develop
jarad0628
committed
4e1a533
fix: remove writeFormat from doris perTable
jarad0628
committed
06d4486

Commits on Apr 2, 2026

Merge pull request #748 from tapdata/develop
jarad0628
authored
02d3fc6
fix: starrocks default flush time 30s
jarad0628
committed
a6d8923

Commits on Apr 5, 2026

Commits on Apr 6, 2026

Commits on Apr 7, 2026

feat: PaimonService perf testcase support init table mock；
lebron-hj
committed
8104e7b
feat: PaimonService perf testcase support S3
lebron-hj
committed
2fcfb35
feat: PaimonService perf support compact log output
lebron-hj
committed
b5375d7

Commits on Apr 8, 2026

feat: PaimonService perf support Paimon s3
lebron-hj
committed
3aa4141
fix: kafka error code with detail
jarad0628
committed
def50a7
Merge pull request #749 from tapdata/develop-paimon-perf
jarad0628
authored
32421e8
Merge remote-tracking branch 'origin/develop' into develop
jarad0628
committed
2e6dc4a
fix: starrocks remove httpclient cache
jarad0628
committed
92f827e
fix: paimon add afterInitial api
jarad0628
committed
862a869
fix: paimon add disk overflow write
jarad0628
committed
09c3e3e
fix: paimon add afterInitial api
jarad0628
committed
8fb2f04
fix: paimon add disk overflow write
jarad0628
committed
6f45311
fix: walminer same invalid update event
jarad0628
committed
af256a7
fix: paimon support bucketCount -2
jarad0628
committed
84da8a1

Commits on Apr 9, 2026

fix: paimon support bucketCount -2
jarad0628
committed
42c0a90
fix: paimon support cache File for each table
jarad0628
committed
4daa4b8
fix: paimon support time and timestamp with local time zone
jarad0628
committed
e5f8ef0

Commits on Apr 10, 2026

Commits on Apr 12, 2026

Merge pull request #751 from tapdata/develop
jarad0628
authored
0cbd036
feat: snowflake all data type
jarad0628
committed
98975a3

Commits on Apr 13, 2026

Commits on Apr 14, 2026

Commits on Apr 15, 2026

fix: dameng support ddl with note
jarad0628
committed
442a446
Merge remote-tracking branch 'origin/develop' into develop
jarad0628
committed
e7db73f

Commits on Apr 18, 2026

feat: support specific partition write
jarad0628
committed
e8e211d
feat: kafka support splitUpdatePk
jarad0628
committed
2e2868e

Commits on Apr 19, 2026

feat: kafka support transaction
jarad0628
committed
04979ae

Commits on Apr 21, 2026

fix: gen single test perf project
lebron-hj
committed
efc3493
fix: gen single test perf project
lebron-hj
committed
ea45513
feat:TAP-11093 Aggregation node optimization
weiliang110100
committed
ed327a7

Commits on Apr 22, 2026

Commits on Apr 23, 2026

fix: delete postpone bucket support
lebron-hj
committed
2dd90dc
Merge pull request #753 from tapdata/develop-paimon-perf
jarad0628
authored
57c80e5

Commits on Apr 24, 2026

Commits on Apr 25, 2026

fix: auto-inject MongoDB HA timeout options when creating MongoClient
ply0011
committed
72b35a3

Commits on Apr 27, 2026

Commits on Apr 30, 2026

Merge pull request #762 from tapdata/main
jarad0628
authored
9cc9730

Commits on May 7, 2026

fix:TAP-11586 MongoDB mining tasks frequently encounter read timeout errors.
weiliang110100
committed
572d302

Commits on May 11, 2026