Skip to content

dumpling-overview: add Parquet export documentation #23933

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
D3Hunter wants to merge 1 commit into
base: master
Choose a base branch
from
Open

dumpling-overview: add Parquet export documentation #23933

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 27 additions & 5 deletions dumpling-overview.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

# Use Dumpling to Export Data

This document introduces the data export tool - [Dumpling](https://github.com/pingcap/tidb/tree/master/dumpling). Dumpling exports data stored in TiDB/MySQL as SQL or CSV data files and can be used to make a logical full backup or export. Dumpling also supports exporting data to Amazon S3.
This document introduces the data export tool - [Dumpling](https://github.com/pingcap/tidb/tree/master/dumpling). Dumpling exports data stored in TiDB/MySQL as SQL, CSV, or Parquet data files and can be used to make a logical full backup or export. Dumpling also supports exporting data to Amazon S3.

<CustomContent platform="tidb">

Expand Down Expand Up @@ -50,7 +50,7 @@

Dumpling has the following advantages:

- Support exporting data in multiple formats, including SQL and CSV.
- Support exporting data in multiple formats, including SQL, CSV, and Parquet.
- Support the [table-filter](https://github.com/pingcap/tidb-tools/blob/master/pkg/table-filter/README.md) feature, which makes it easier to filter data.
- Support exporting data to Amazon S3 cloud storage.
- More optimizations are made for TiDB:
Expand Down Expand Up @@ -151,13 +151,32 @@
>
> *Strings* and *keywords* are not distinguished by Dumpling. If the imported data is the Boolean type, the value of `true` is converted to `1` and the value of `false` is converted to `0`.

### Export to Parquet files

Starting from v9.0.0, you can export data to Parquet files by adding the `--filetype parquet` argument.

{{< copyable "shell-regular" >}}

```shell
tiup dumpling -u root -P 4000 -h 127.0.0.1 -o /tmp/test --filetype parquet -t 8 -F 256MiB
```

In the command above:

- Dumpling exports table data as Parquet files, and still exports the table schema as SQL files. You can use the exported files as the data source of [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md).
- You can use the `--parquet-compress` option to specify the compression algorithm for Parquet files. It supports `no-compression`, `snappy`, `gzip`, and `zstd`. The default value is `snappy`. When compression is enabled, the algorithm is added to the file name. For example, with the `snappy` compression, the exported data file is named as `test.t1.000000000.snappy.parquet`.

@coderabbitai coderabbitai Bot Sep 22, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

🔎 Supported by static analysis

🏁 Script executed:

rg -n -C 4 'snappy\.parquet|parquet\.snappy|parquet-compress' --glob '*.go' --glob '*.md'

Repository: pingcap/docs

Length of output: 7314

🏁 Script executed:

#!/bin/bash
set -e
printf '%s\n' '--- tracked relevant files ---'
git ls-files | rg '(^|/)(dumpling|tidb-lightning).*|dumpling-overview\.md'
printf '%s\n' '--- Lightning filename sections ---'
sed -n '300,380p' tidb-lightning/tidb-lightning-data-source.md
printf '%s\n' '--- filename-pattern references ---'
rg -n -C 3 'filename|file name|parquet|compress|schema\}|\{.*csv|sql.*parquet' tidb-lightning/tidb-lightning-data-source.md dumpling-overview.md

Repository: pingcap/docs

Length of output: 35353

🌐 Web query:

official GitHub Dumpling parquet compression filename snappy.parquet tidb-lightning parser

💡 Result:

<source_evidence>

<title>dumpling: support dump data in parquet format</title> GitHub pull request 68345 in pingcap/tidb (link omitted to avoid creating a cross-reference) ### What problem does this PR solve? Issue Number: ref `#68069` Problem Summary: NOTE: this is partial CP from https://github.com/tidbcloud/tidb-cse/pull/1937, most logic are kept as is, we can refactor later Dumpling currently supports SQL and CSV output. The linked issue requests support for exporting table data in Parquet format. ### What changed and how does it work? - Adds Parquet as a Dumpling file type and wires parquet-specific compression, page size, and row group size flags. - Converts SQL column metadata into `pkg/dumpformat/parquetfile.ColumnInfo` and writes raw row bytes through the Parquet writer. - Updates output naming so Parquet dumps include the selected Parquet compression suffix when compression is enabled. ### Check List Tests - [x] Unit test - [ ] Integration test - [x] Manual test (add detailed scripts or steps below) - [ ] No need to test > - [ ] I checked and no code files have been changed. > Manual test steps: - ... 7.0.0.1 ... 4000`; `SELECT tidb ... ()` reported ` ... .0.0-beta.2.pre-169 ... -ge96b62123 ... `. - Checked out and built on this branch: ... gh pr checkout 68345` and `make ... _dumpling`. - Recreated the same Parquet table ... defined in `dumpling/tests/e2e/run.sh`. - Source table: `e2e_parquet ... src.t_types`; destination table ... `. - ... 2e_parquet_ ... 0.79` bytes/row ( ... 10.39 MiB`, targeting ~ ... `./bin/dumpling -u root -h 127.0.0.1 -P 4000 -B e2e_parquet_src -T e2e_parquet_src.t_types --filetype parquet --parquet-compress ... /tmp/dum ... -F 256MiB` - Generated files under `/tmp/dumpling_parquet_68 ... 5_nocompress_20260514_151509 ... `/tmp/ ... pling_parquet_68345_nocompress_20260514_151509 ... e2e_parquet_src ... 00000000 ... 2026 ... 514_151509/e2e_parquet_src.t_types.000000001.parquet` (`248M`), `/tmp/dumpling_parquet_68345_nocompress_20 ... 60514_151509 ... _src.t_types.000000002.parquet` (`63M`). - Imported with ... `IMPORT ... 2e_parquet_dst. ... tmp/dumpling_parquet_68345_ ... 60514_151509 ... e2e_parquet_ ... .t_types.*.parquet&`#39`; ... parquet&`#39`;;` - Validation results: source row count = `524288`, destination row count = `524288`, bidirectional `EXCEPT` diff rows = `0`; source signature = `524288:1125219626002674`, destination signature = `524288:1125219626002674`. Side effects - [ ] Performance regression: Consumes more CPU - [ ] Performance regression: Consumes more Memory - [ ] Breaking backward compatibility Documentation - [x] Affects user behaviors - [ ] Contains syntax changes - [ ] Contains variable changes - [ ] Contains experimental features - [ ] Changes MySQL compatibility ### Release note Please refer to Release Notes Language Style Guide to write a quality release note. ```release-note Dumpling now supports exporting table data in Parquet format. ``` ## Summary by CodeRabbit * **New Features** * Parquet export support with CLI flags for compression, page size, and row-group memory tuning * Parquet-aware output writing and filenames that reflect chosen compression * **Changes** * Default Parquet compression changed to Snappy * Export metadata exposes column info and raw-byte accessors * File-format validation and handling updated for Parquet-specific options * Prometheus metrics ... for Parquet write flows * **Tests ... ions and sizing settings ... Parquet config parsing ... Review ... Stack ... > |**Parquet Configuration and Parsing** `dumpling/export/config.go`, `dumpling/export/config_test.go`|New `Config` fields `ParquetCompressType`, `ParquetPageSize`, `ParquetRowGroupSize` with defaults; CLI flags `--parquet-compress`, `--parquet-page-size`, `--parquet-row-group-size`; parsing helpers including `ParseParquetCompressType` and size parsing; file-format validation disallows `--compress` with Parquet; tests for defaults and overrides.| ... > |**Parquet Library Default Updates** `pkg/dumpformat/parquetfile/writer.go`, `pkg/dumpformat/parquetfile/writer_behavior_test.go`|Default compression changed to Snappy; `defaultRowGroupMe…[truncated] <title>TiDB Lightning Data Sources | TiDB Docs</title> https://docs.pingcap.com/tidb/stable/tidb-lightning-data-source/ | Type | Pattern | ... | Data file | If the data file contains data for a whole table, the file is imported into a table named `${db_name}.${table_name}` | `${db_name}.${table_name}.${csv|sql|parquet}` | | Data file | If the data for a table is split into multiple data files, each data file must be suffixed with a number in its filename | `${db_name}.${table_name}.001.${csv|sql|parquet}` | | Compressed file | If the file contains a compression suffix, such as `gzip`, `snappy`, or `zstd`, TiDB Lightning will decompress the file before importing it. Note that the Snappy compressed file must be in the official Snappy format. Other variants of Snappy compression are not supported. | `${db_name}.${table_name}.${csv|sql|parquet}.{compress}` | ... ## Parquet ... TiDB Lightning currently only supports Parquet files generated by Amazon Aurora, Apache Hive, and Snowflake. To identify the file structure in S3, use the following configuration to match all data files: ... [[mydumper.files]] # The expression needed for parsing Amazon Aurora parquet files pattern = &`#39`;(?i)^(?:[^/]*/)*([a-z0-9\-_]+).([a-z0-9\-_]+)/(?:[^/]*/)*(?:[a-z0-9\-_.]+\.(parquet))$&`#39`; schema = &`#39`;$1&`#39`; table = &`#39`;$2&`#39`; type = &`#39`;$3&`#39`; ... TiDB Lightning currently supports compressed files exported by Dumpling or compressed files that follow the naming rules. Currently, TiDB Lightning supports the following compression algorithms: `gzip`, `snappy`, and `zstd`. When the file name follows the naming rules, TiDB Lightning automatically identifies the compression algorithm and imports the file after streaming decompression, without additional configuration. ... . - TiDB Lightning does not support `parquet` files compressed through another compression tool, such as `db.table.parquet.snappy`. If you want to compress `parquet` files, you can configure the compression format for the `parquet` file writer. ... - TiDB Lightning v6.4.0 and later versions only support the following compressed data files: `gzip`, `snappy`, and `zstd`. Other types of files cause errors. If an unsupported compressed file exists in the directory where the source data file is stored, this will cause the task to report an error. You can move those unsupported files out of the import data directory to avoid such errors. - The Snappy compressed file must be in the official Snappy format. Other variants of Snappy compression are not supported. ... issue, you can use `[[myd ... .files]]` to match data files in your customized expression. ... Take the Aurora snapshot exported to S3 as an example. The complete path of the Parquet file is `S3://some-bucket/some-subdir/some-database/some-database.some-table/part-00000-c5a881bb-58ff-4ee6-1111-b41ecff340a3-c000.gz.parquet`. ... Based on the preceding Parquet file path, you can write a regular expression like `(?i)^(?:[^/]*/)*([a-z0-9\-_]+).([a-z0-9\-_]+)/(?:[^/]*/)*(?:[a-z0-9\-_.]+\.(parquet))$` to match the files. In the match group, `index=1` is `some-database`, `index=2` is `some-table`, and `index=3` is `parquet`. ... You can write the configuration file according to the regular expression and the corresponding index so that TiDB Lightning can recognize the data files that do not follow the default naming convention. For example: ... [[mydumper.files]] # The expression needed for parsing the Amazon Aurora parquet file pattern = &`#39`;(?i)^(?:[^/]*/)*([a-z0-9\-_]+).([a-z0-9\-_]+)/(?:[^/]*/)*(?:[a-z0-9\-_.]+\.(parquet))$&`#39`; schema = &`#39`;$1&`#39`; table = &`#39`;$2&`#39`; type = &`#39`;$3&`#39`; ... - schema: The name of the target database. The value can be: ... - The group index obtained by using a regular expression, such as `$1`. - The name of the database that you want to import, such as `db1`. All matched files are imported into `db1`. - table: The name of the target table. The value can be: ... - The group index obtained by using a regular expression, such as `$2`. - The name of the table that you want to import, such as `table1`. All ma…[truncated] <title>TiDB Lightning Data Sources - dev | TiDB Docs</title> https://docs.pingcap.com/tidb/dev/tidb-lightning-data-source/ TiDB Lightning supports importing data from multiple data sources to TiDB clusters, including CSV, SQL, and Parquet files. ... | Data file | If the data file contains data for a whole table, the file is imported into a table named`${db_name}.${table_name}` | `${db_name}.${table_name}.${csv|sql|parquet ... | Data file | If the data for a table is split into multiple data files, each data file must be suffixed with a number in its filename | `${db_name}.${table_name}.001.${csv|sql|parquet}` | ... | Compressed file | If the file contains a compression suffix, such as`gzip`,`snappy`, or`zstd`, TiDB Lightning will decompress the file before importing it. Note that the Snappy compressed file must be in the official Snappy format. Other variants of Snappy compression are not supported. | `${db_name}.${table_name}.${csv|sql|parquet}.{compress}` | ... file`pattern` is`&`#39`;^({schema_regrex})\.({table_regrex})\. ... file_serial_regrex})\.(csv|parquet ... `&`#39`;$4 ... ## Parquet ... TiDB Lightning currently only supports Parquet files generated by Amazon Aurora, Apache Hive, and Snowflake. To identify the file structure in S3, use the following configuration to match all data files: ... ``` [[mydumper.files]] # The expression needed for parsing Amazon Aurora parquet files pattern = &`#39`;(?i)^(?:[^/]*/)*([a-z0-9\-_]+).([a-z0-9\-_]+)/(?:[^/]*/)*(?:[a-z0-9\-_.]+\.(parquet))$&`#39`; ... schema = &`#39`;$1&`#39`; table = &`#39`;$2&`#39`; type = &`#39`;$3&`#39`; ... TiDB Lightning currently supports compressed files exported by Dumpling or compressed files that follow the naming rules. Currently, TiDB Lightning supports the following compression algorithms:`gzip`,`snappy`, and`zstd`. When the file name follows the naming rules, TiDB Lightning automatically identifies the compression algorithm and imports the file after streaming decompression, without additional configuration. ... - TiDB Lightning does not support`parquet` files compressed through another compression tool, such as`db.table.parquet.snappy`. If you want to compress`parquet` files, you can configure the compression format for the`parquet` file writer. ... - TiDB Lightning v6.4.0 and later versions only support the following compressed data files:`gzip`,`snappy`, and`zstd`. Other types of files cause errors. If an unsupported compressed file exists in the directory where the source data file is stored, this will cause the task to report an error. You can move those unsupported files out of the import data directory to avoid such errors. ... - The Snappy compressed file must be in the official Snappy format. Other variants of Snappy compression are not supported. ... To resolve this issue, you can use`[[mydumper.files]]` to match data files in your customized expression. ... Take the Aurora snapshot exported to S3 as an example. The complete path of the Parquet file is`S3://some-bucket/some-subdir/some-database/some-database.some-table/part-00000-c5a881bb-58ff-4ee6-1111-b41ecff340a3-c000.gz.parquet`. ... Based on the preceding Parquet file path, you can write a regular expression like`(?i)^(?:[^/]*/)*([a-z0-9\-_]+).([a-z0-9\-_]+)/(?:[^/]*/)*(?:[a-z0-9\-_.]+\.(parquet))$` to match the files. In the match group,`index=1` is`some-database`,`index=2` is`some-table`, and`index=3` is`parquet`. ... You can write the configuration file according to the regular expression and the corresponding index so that TiDB Lightning can recognize the data files that do not follow the default naming convention. For example: ... ``` [[mydumper.files]] # The expression needed for parsing the Amazon Aurora parquet file pattern = &`#39`;(?i)^(?:[^/]*/)*([a-z0-9\-_]+).([a-z0-9\-_]+)/(?:[^/]*/)*(?:[a-z0-9\-_.]+\.(parquet))$&`#39`; schema = &`#39`;$1&`#39`; table = &`#39`;$2&`#39`; type = &`#39`;$3&`#39`; ``` ... The value can be: ... - - The group index obtained by using a regular expression, such as`$1`. ... such as`db1`. All matched files are imported into`db1`. ... the target table ... - type: The file type. Supports`sql`,…[truncated] <title>dumpling/export/config.go</title> https://github.com/pingcap/tidb/blob/master/dumpling/export/config.go "no- ... = "no ... "csv-null-value" ... flagSQL = "sql" ... CaseSensitive = "case-sensitive ... Database = " ... -empty-database" ... TidbMemQuotaQuery = "tidb-mem- ... " flagCA = "ca" flagCert ... Key = "key" flagCsvSeparator = "csv-separator" flagCsvDelimiter = "csv-delimiter" flagCsvLineTerminator = "csv-line ... terminator" flagOutputFilenameTemplate = "output-filename-template" flagCompleteInsert = "complete-insert" flagParams = "params" flagReadTimeout = "read-timeout" flagTransactionalConsistency = "transactional-consistency" flagCompress = "compress" flagCsvOutputDialect = "csv-output-dialect" flagPDAddr = "pd" flagClusterSSLCA = "cluster-ssl-ca" flagClusterSSLCert = "cluster-ssl-cert" flagClusterSSLKey = "cluster-ssl-key" flagPartitions = "partitions" flagParquetCompress = "parquet-compress" flagParquetPageSize = "parquet-page-size" flagParquetRowGroupSize = "parquet-row-group-size" // FlagHelp represents the help flag FlagHelp = "help" ) ... // ClusterSSLCA/ClusterSSLCert/ClusterSSLKey override Security.* when connecting // to PD endpoints for GC control. ClusterSSLCA string ClusterSSLCert string ClusterSSLKey string // ParquetCompressType is the parquet row-group compression type. ParquetCompressType compressedio.CompressType // ParquetPageSize is the parquet data page size in bytes. ParquetPageSize int64 ... // ParquetRowGroupSize is the parquet row-group flush threshold by accounted // in ... memory bytes. ParquetRowGroupSize int64 ... StatementSize, OutputDirPath: ".", Server ... : ServerInfoUnknown, Sort ... Pk: true, Tables: nil, Snapshot: "", Consistency: ConsistencyTypeAuto, NoViews: true, NoSequences: true, Rows: UnspecifiedSize ... Where: "", EscapeBackslash ... true, FileType ... "", NoHeader: false, NoSchemas ... false, NoData: false ... Csv ... Value: "\\N", SQL: "", TableFilter: allFilter, DumpEmptyDatabase: true, CsvDelimiter: "\"", CsvSeparator: ",", CsvLineTerminator: "\r\n", SessionParams: make(map[string]any), OutputFileTemplate: DefaultOutputFileTemplate, PosAfterConnect: false, CollationCompatible: LooseCollationCompatible, CsvOutputDialect: CSVDialectDefault, ... Tables: false, PromFactory: promutil.NewDefaultFactory(), PromRegistry: promutil.NewDefaultRegistry(), TransactionalConsistency: true, PDAddr: "", ClusterSSLCA: "", ClusterSSLCert: "", ClusterSSLKey: "", ParquetCompressType: parquetfile.DefaultCompressionType, ParquetPageSize: units.MiB, ParquetRowGroupSize: parquetfile.DefaultRowGroupMemoryLimitBytes, } ... default &`#39`;\\r\\n&`#39`;") ... .String( ... FilenameTemplate, "", "The output filename template (without file extension). When used with --rows/-r or --filesize/-F in split mode, include {{.Index}} (for example: &`#39`;{{.DB}}.{{.Table}}.{{.Index}}&`#39`;) to avoid overwriting chunk files") ... complete INSERT statements that include ... names") ... nil, ` ... used while dumping ... client=latin1,character_set_connection=latin1"`) flags.Bool(FlagHelp, false, "Print ... message and quit") flags.Duration(flagReadTimeout, 15*time.Minute, "I/O read timeout for db connection.") _ = flags.MarkHidden(flagReadTimeout) flags.Bool(flagTransactionalConsistency, true, "Only support transactional consistency") _ = flags.MarkHidden(flagTransactionalConsistency) flags.StringP(flagCompress, "c", "", "Compress output file type, support &`#39`;gzip&`#39`;, &`#39`;snappy&`#39`;, &`#39`;zstd&`#39`;, &`#39`;no-compression&`#39`; now") flags.String(flagCsvOutputDialect, "", "The dialect of output CSV file, suppo…[truncated] <title>Dumpling Overview | TiDB Docs</title> https://docs.pingcap.com/tidb/stable/dumpling-overview/ ### Compress the exported data files ... You can use the `--compress ` option to compress the CSV and SQL data and table structure files exported by Dumpling. This parameter supports the following compression algorithms: `gzip`, `snappy`, and `zstd`. The compression is disabled by default. ... - This option only compresses individual data and table structure files. It cannot compress the entire folder and generate a single compressed package. - This option can save disk space, but it also slows down the export speed and increases CPU consumption. Use this option with caution in scenarios where the export speed is critical. - For TiDB Lightning v6.5.0 and later versions, you can use compressed files exported by Dumpling as the data source without additional configuration. ... The Snappy compressed file must be in the official Snappy format. Other variants of Snappy compression are not supported. ... - The `--compress ` option specifies the compression format of the dump. It supports the following compression algorithms: `gzip`, `snappy`, and `zstd`. This option can speed up dumping of data if storage is the bottleneck or if storage capacity is a concern. The drawback is an increase in CPU usage. Each file is compressed individually. ... | `-c` or `--compress` | Compresses the CSV and SQL data and table structure files exported by Dumpling. It supports the following compression algorithms: `gzip`, `snappy`, and `zstd`. | "" |

Citations:

🌐 Web query:

site:github.com/pingcap/tidb "snappy.parquet" "parquet-compress"

💡 Result:

<source_evidence>

<title>dumpling/export/config.go</title> https://github.com/pingcap/tidb/blob/master/dumpling/export/config.go ences = "no ... = "order-by-primary-key" flagStatusAddr = "status-addr" flagRows = "rows" flagWhere = "where" flagEscapeBackslash = "escape-backslash" ... Filetype = "filetype" flagNo ... = "no-header" flagNoSchemas = "no-schemas" flagNoData = "no-data" flagCsvNullValue = "csv-null-value" flagSQL = "sql" flagFilter = "filter" flagCaseSensitive = "case-sensitive" flagDumpEmptyDatabase = "dump-empty-database" flagTidbMemQuotaQuery = "tidb-mem-quota-query" flagCA = "ca" flagCert = "cert" flagKey = "key" flagCsvSeparator = "csv-separator" flagCsvDelimiter = "csv-delimiter" flagCsvLineTerminator = "csv-line-terminator" flagOutputFilenameTemplate = "output-filename-template" flagCompleteInsert = "complete-insert" flagParams = "params" flagReadTimeout = "read-timeout" flagTransactionalConsistency = "transactional-consistency" flagCompress = "compress" flagCsvOutputDialect = "csv-output-dialect" flagPDAddr = "pd" flagClusterSSLCA = "cluster-ssl-ca" flagClusterSSLCert = "cluster-ssl-cert" flagClusterSSLKey = "cluster-ssl-key" flagPartitions = "partitions" flagParquetCompress = "parquet-compress" flagParquetPageSize = "parquet-page-size" flagParquetRowGroupSize = "parquet-row-group-size" // FlagHelp represents the help flag FlagHelp = "help" ) ... // ClusterSSLCA/ClusterSSLCert/ClusterSSLKey override Security.* when ... // to PD endpoints for GC control. ... SSLCA string ClusterSSLCert string ClusterSSLKey string // ParquetCompressType is the parquet row-group compression type. ParquetCompressType compressedio.CompressType // ParquetPageSize is the parquet data page size in bytes. ParquetPageSize int64 // ParquetRowGroupSize is the parquet row-group flush threshold by accounted // in-memory bytes. ParquetRowGroupSize int64 ... Size, ... : ".", ... Unknown, ... Pk: true ... : "", Consistency: ConsistencyTypeAuto ... Views: true ... Sequences: true ... Database: true, ... "\"", ... Separator: ",", ... [string]any ... Connect: false ... Collation ... : LooseCollationCompatible ... Csv ... ect: CSVD ... PromFactory: promutil ... PromRegistry: ... .NewDefault ... TransactionalConsistency ... true, PDAddr: "", ClusterSSLCA: "", Cluster ... LCert: "", Cluster ... "", ParquetCompressType: parquetfile.DefaultCompressionType, Parquet ... Size: units.MiB, ParquetRowGroupSize: parquetfile.DefaultRowGroupMemoryLimitBytes, ... .String( ... &`#39`;\\r\\ ... .String( ... output filename template (without file extension ... When used with ... rows/-r or --filesize/-F in split mode, ... {{.Index}} (for example: &`#39`;{{.DB}}.{{. ... to avoid overwriting chunk ... client=latin1,character_set_connection=latin1"`) flags.Bool(FlagHelp, false, "Print help message and quit") flags.Duration(flagReadTimeout, 15*time.Minute, "I/O read timeout for db connection.") _ = flags.MarkHidden(flagReadTimeout) flags.Bool(flagTransactionalConsistency, true, "Only support transactional consistency") _ = flags.MarkHidden(flagTransactionalConsistency) flags.StringP(flagCompress, "c", "", "Compress output file type, support &`#39`;gzip&`#39`;, &`#39`;snappy&`#39`;, &`#39`;zstd&`#39`;, &`#39`;no-compression&`#39`; now") flags.String(flagCsvOutputDialect, "", "The dialect of output CSV file, support &`#39`;snowflake&`#39`;, &`#39`;redshift&`#39`;, &`#39`;bigquery&`#39`; now") flags.StringSlice(flagPartitions, nil, "The table partitions to dump. Every listed partition must exist on all selected base tables; incompatible with --sql. TiDB >= v5.0.0 only"…[truncated] <title>dumpling: support dump data in parquet format</title> GitHub pull request 68345 in pingcap/tidb (link omitted to avoid creating a cross-reference) --- ### What problem does this PR solve? Issue Number: ref `#68069` Problem Summary: NOTE: this is partial CP from https://github.com/tidbcloud/tidb-cse/pull/1937, most logic are kept as is, we can refactor later Dumpling currently supports SQL and CSV output. The linked issue requests support for exporting table data in Parquet format. ### What changed and how does it work? - Adds Parquet as a Dumpling file type and wires parquet-specific compression, page size, and row group size flags. - Converts SQL column metadata into `pkg/dumpformat/parquetfile.ColumnInfo` and writes raw row bytes through the Parquet writer. - Updates output naming so Parquet dumps include the selected Parquet compression suffix when compression is enabled. ### Check List Tests - [x] Unit test - [ ] Integration test - [x] Manual test (add detailed scripts or steps below) - [ ] No need to test > - [ ] I checked and no code files have been changed. > Manual ... - Local cluster: TiDB on ` ... 27.0.0.1:4000`; `SELECT tidb_version()` reported `v9.0.0-beta.2.pre-1699-ge96b621239`. - Checked out and built on this branch: `gh pr checkout 68345` and `make build_dumpling`. - Recreated the same Parquet table schema defined in `dumpling/tests/e2e/run.sh`. - Source table: `e2e_parquet_src.t_types`; destination table: `e2e ... `. - Generated `524288` rows in `e2e_parquet_src.t_types` with average payload `1020.79` bytes/row (about `510.39 MiB`, targeting ~`512 MiB`). - Dumped Parquet using: `./bin/dumpling -u root -h 127.0.0.1 -P 4000 -B e2e_parquet_src -T e2e_parquet_src.t_types --filetype parquet --parquet-compress no-compression -o /tmp/dumpling_parquet_68345_nocompress_20260514_151509 -F 256MiB` - Generated files under `/tmp/dumpling_parquet_68345_nocompress_20260514_151509`: `/tmp/dumpling_parquet_68345_nocompress_20260514_151509/e2e_parquet_src.t_types.000000000.parquet` (`248M`), `/tmp/dumpling_parquet_68345_nocompress_20260514_151509/e2e_parquet_src.t_types.000000001.parquet` (`248M`), `/tmp/dumpling_parquet_68345_nocompress_20260514_151509/e2e_parquet_src.t_types.000000002.parquet` (`63M`). - Imported with: `IMPORT INTO e2e_parquet_dst.t_types FROM &`#39`;/tmp/dumpling_parquet_68345_nocompress_20260514_151509/e2e_parquet_src.t_types.*.parquet&`#39`; FORMAT &`#39`;parquet&`#39`;;` - Validation results: source row count = `524288`, destination row count = `524288`, bidirectional `EXCEPT` diff rows = `0`; source signature = `524288:1125219626002674`, destination signature = `524288:1125219626002674`. Side effects - [ ] Performance regression: Consumes more CPU - [ ] Performance regression: Consumes more Memory - [ ] Breaking backward compatibility Documentation - [x] Affects user behaviors - [ ] Contains syntax changes - [ ] Contains variable changes - [ ] Contains experimental features - [ ] Changes MySQL compatibility ### Release note Please refer to Release Notes Language Style Guide to write a quality release note. ```release-note Dumpling now supports exporting table data in Parquet format. ``` ## Summary by CodeRabbit * **New Features** * Parquet export support with CLI flags for compression, page size, and row-group memory tuning * Parquet-aware output writing and filenames that reflect chosen compression * **Changes** * Default Parquet compression changed to Snappy * Export metadata exposes column info and raw-byte accessors * File-format validation and handling updated for Parquet-specific options * Prometheus metrics adjusted for Parquet write flows * **Tests** * E2E Parquet export/import verification across compressions and sizing settings * Unit tests for Parquet config parsing and writer behavior * **Chores** * CI: upload dumpling test logs on failures Review Change Stack ## ... > |**Parquet Configuration and Parsing** `dumpling/export/config.go`, `dumpling/export/config_test.go`|New `Config` fields `ParquetCompressType`, `ParquetPageSize`, `ParquetRowGroupSize` with defaults; CLI flags `--parquet-compress`, `--parquet-page-size`, `--par…[truncated] <title>lightning: Compress Reader/Writer supports reading/writing Snappy/Zstd type compressed files · Pull Request `#38603` · pingcap/tidb</title> GitHub pull request 38603 in pingcap/tidb (link omitted to avoid creating a cross-reference) ## lightning: Compress Reader/Writer supports reading/writing Snappy/Zstd type compressed files ... Compress Reader supports reading Snappy/Zstd type compressed files Compress Writer supports writing Snappy/Zstd type compressed files ... snappy part LGTM. Looking forward to lzo/zstd. ... **lyzx2001** pushed commit `f5ef657`: compress reader/writer support reading/writing zstd type compressed files · Oct 28, 2022 at 3:56am ... > zstd is supported now. ... **lyzx2001** changed the title from "lightning: Compress Reader/Writer supports reading/writing Snappy type compressed files" to "lightning: Compress Reader/Writer supports reading/writing Snappy/Zstd type compressed files" · Oct 28, 2022 at 3:58am ... **lyzx2001** pushed commit `215831a`: compress reader/writer support reading/writing snappy/zstd type compressed files · Oct 28, 2022 at 10:46am <title>lightning: support import compressed csv/sql files to save cost</title> GitHub issue 38514 in pingcap/tidb (link omitted to avoid creating a cross-reference) # lightning: support import compressed csv/sql files to save cost - State: closed - Author: lichunzhu - Created: 2022-10-18T06:05:37Z - Updated: 2023-01-05T12:48:12Z - Repository: pingcap/tidb - Number: `#38514` ## Labels - type/feature-request - component/lightning - component/dumpling --- ## Feature Request **Is your feature request related to a problem? Please describe:** > For checkpoint I&`#39`;m thinking of skip the leading bytes until reader&`#39`;s pos exceeds chunk.Offset. > > It does work. However, can we look into whether we can seek directly on the compressed file? One idea is to save the state of compress reader to checkpoint, so we continue reading from last pos after lightning restarts. **lichunzhu** commented on 2022-11-03T06:18:45Z: > > > For checkpoint I&`#39`;m thinking of skip the leading bytes until reader&`#39`;s pos exceeds chunk.Offset. > > > > It does work. However, can we look into whether we can seek directly on the compressed file? One idea is to save the state of compress reader to checkpoint, so we continue reading from last pos after lightning restarts. > > I&`#39`;m afraid that not every compress reader supports reading start from an offset. I will do a survey to check whether it&`#39`;s okay. - Referenced by PR `#38910`: dumpling: compress supports snappy/zstd - Referenced by PR `#38946`: lightning: support checkpoint read for compress files - Referenced in commit 481f5ab - Referenced in commit 78d1905 - Referenced by PR `#39153`: lightning: support compress for lightning, add compress unit and integration tests - Referenced by PR `#39219`: lightning: refine progress for compress files import - Referenced by PR `#12083`: dumpling, lightning: add document for compressed files support - Referenced in commit 5b9d96b - Referenced by PR `#12100`: add v6.5.0 release notes - Referenced in commit 10b3bc7 - Referenced by PR `#39680`: *: support sample for compressed files for adjustment - lichunzhu added label "component/dumpling" - Referenced in commit 0c18082 - lichunzhu closed - Referenced in commit 5ad0c09 <title>lightning, importinto: validate Parquet format before parsing metadata</title> GitHub pull request 66949 in pingcap/tidb (link omitted to avoid creating a cross-reference) ### What problem does this PR solve? Issue Number: close `#66948` Problem Summary: When users misconfigure the data format (e.g. using the Parquet parser on CSV/SQL files), the Arrow library produces confusing "invalid offset" errors from footer parsing. Users cannot tell from the error message that they specified the wrong format. **Additionally, when calculating the failure rate, errors in statistics may also occur. For example, we shouldn&`#39`;t treat this kind of scenarios as failure.** ### What changed and how does it work? - Add `parseParquetMetaData()` in `parquet_wrapper.go` that reads and validates the Parquet footer ourselves, producing three categories of sentinel errors: - `ErrNotParquet`: file is not Parquet (wrong magic, too small) - `ErrParquetCorrupt`: file has Parquet magic but broken metadata - `ErrParquetEncrypted`: encrypted Parquet (`PARE` magic) is not supported - Raw I/O errors are returned unwrapped - Pass parsed metadata to Arrow via `file.WithMetadata()` to skip its internal `parseMetaData()`, eliminating double-parse overhead and extra I/O. - Use `SourceFileMeta.FileSize` (always initialized from file iteration) instead of `SeekEnd` to avoid an extra round-trip on cloud storage. - Simplify `NewParquetParser` and `SampleStatisticsFromParquet` signatures to accept `SourceFileMeta` instead of scattered `path`/`meta`/`fileSize` params. ### Check List Tests - [x] Unit test - [ ] Integration test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test > - [ ] I checked and no code files have been changed. > Side effects - [ ] Performance regression: Consumes more CPU - [ ] Performance regression: Consumes more Memory - [ ] Breaking backward compatibility Documentation - [ ] Affects user behaviors - [ ] Contains syntax changes - [ ] Contains variable changes - [ ] Contains experimental features - [ ] Changes MySQL compatibility ### Release note Please refer to Release Notes Language Style Guide to write a quality release note. ```release-note Improve error messages when the Parquet parser is used on non-Parquet files (e.g. CSV or SQL), replacing confusing "invalid offset" errors with clear diagnostics ... format mismatch. ... ## Summary by CodeRabbit ... > > > > > 📝 Walkthrough > > ## Walkthrough > > Callers now pass a single file metadata struct (SourceFileMeta / FileMeta) into Parquet helpers; a new parseParquetMetaData routine validates/parses Parquet footers and returns explicit Parquet errors before creating the Arrow Parquet reader via WithMetadata. > > ## Changes > > |Cohort / File(s)|Summary| > |---|---| > |**Parquet core & wrapper** `pkg/lightning/mydump/parquet_parser.go`, `pkg/lightning/mydump/parquet_wrapper.go`|Added `parseParquetMetaData` to read/validate Parquet footer and new explicit errors (`ErrNotParquet`, `ErrParquetCorrupt`, `ErrParquetEncrypted`). Changed `NewParquetParser` and `SampleStatisticsFromParquet` to accept `SourceFileMeta`/`FileMeta`; parser now uses parsed metadata with `WithMetadata`.| ... > |**Call sites / importer flow** `lightning/pkg/importer/chunk_process.go`, `lightning/pkg/importer/get_pre_info.go`, `pkg/executor/importer/import.go`, `pkg/lightning/mydump/loader.go`|Updated callers to construct/pass a single metadata struct (`SourceFileMeta`/`FileMeta`) and adjusted argument order; added explicit error checks after parser creation where applicable.| ... > |**Tests & test helpers** `pkg/lightning/mydump/parquet_parser_test.go`, `pkg/lightning/mydump/loader_test.go`, `tests/realtikvtest/importintotest/parquet_test.go`|Tests updated to stat files to obtain `FileSize`, construct `SourceFileMeta`, adapt helpers to new `NewParquetParser` signature, added metadata-parsing unit test and an IMPORT INTO invalid-parquet test covering corrupt/encrypted cases.| ... > |**Error codes & mapping** `pkg/errno/errcode.go`, `pkg/errno/errname.go`, `pkg/util/dbterr…[truncated]

Citations:

  • 1: https://github.com/pingcap/tidb/blob/master/dumpling/export/config.go
  • 2: GitHub pull request 68345 in pingcap/tidb (link omitted to avoid creating a cross-reference)
  • 3: GitHub pull request 38603 in pingcap/tidb (link omitted to avoid creating a cross-reference)
  • 4: GitHub issue 38514 in pingcap/tidb (link omitted to avoid creating a cross-reference)
  • 5: GitHub pull request 66949 in pingcap/tidb (link omitted to avoid creating a cross-reference)
  • 6: GitHub issue 67849 in pingcap/tidb (link omitted to avoid creating a cross-reference)

🏁 Script executed:

python3 - <<'PY'
import json
import urllib.request

url = "https://api.github.com/repos/pingcap/tidb/pulls/68345/files?per_page=100"
request = urllib.request.Request(url, headers={"Accept": "application/vnd.github+json", "User-Agent": "review-inspection"})
with urllib.request.urlopen(request, timeout=20) as response:
    files = json.load(response)

for item in files:
    patch = item.get("patch", "")
    text = (item.get("filename", "") + "\n" + patch).lower()
    if any(term in text for term in ("parquet", "filename", "extension", "compress")):
        print(f"--- {item['filename']} ---")
        print(patch)
PY

Repository: pingcap/docs

Length of output: 2436

🏁 Script executed:

set -e
curl -kfsSL -H 'Accept: application/vnd.github+json' -H 'User-Agent: review-inspection' \
  'https://api.github.com/repos/pingcap/tidb/pulls/68345/files?per_page=100' |
python3 -c '
import json, sys
for item in json.load(sys.stdin):
    patch = item.get("patch", "")
    text = (item.get("filename", "") + "\n" + patch).lower()
    if any(term in text for term in ("parquet", "filename", "extension", "compress")):
        print("--- " + item["filename"] + " ---")
        print(patch)
'

Repository: pingcap/docs

Length of output: 33228

Clarify the native Parquet filename exception in the TiDB Lightning documentation.

Dumpling emits test.t1.000000000.snappy.parquet, not test.t1.000000000.parquet.snappy. TiDB Lightning has a dedicated route for native Parquet compression and matches the snappy.parquet form before the generic compressed-file route. Keep the Dumpling examples unchanged.

The generic pattern in tidb-lightning/tidb-lightning-data-source.md:27 conflicts with this behavior and with its note at line 368. Limit that pattern to CSV and SQL files, or document the native Parquet exception explicitly.

- You can use the `--parquet-page-size` option to set the data page size of Parquet files. The default value is `1MiB`.
- You can use the `--parquet-row-group-size` option to set the row group memory limit of Parquet files. The default value is `120MiB`. Dumpling flushes buffered rows when the accounted in-memory usage of a row group reaches this limit.
- The `--compress` option is not supported for Parquet files. To compress Parquet files, use the `--parquet-compress` option instead.

### Compress the exported data files

You can use the `--compress <format>` option to compress the CSV and SQL data and table structure files exported by Dumpling. This parameter supports the following compression algorithms: `gzip`, `snappy`, and `zstd`. The compression is disabled by default.

- This option only compresses individual data and table structure files. It cannot compress the entire folder and generate a single compressed package.
- This option can save disk space, but it also slows down the export speed and increases CPU consumption. Use this option with caution in scenarios where the export speed is critical.
- For TiDB Lightning v6.5.0 and later versions, you can use compressed files exported by Dumpling as the data source without additional configuration.
- This option does not apply to Parquet files. To compress Parquet files, use the [`--parquet-compress`](#option-list-of-dumpling) option instead.

> **Note:**
>
Expand Down Expand Up @@ -207,7 +226,7 @@
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
```

- `{schema}.{table}.{0001}.{sql|csv}`: The date source file
- `{schema}.{table}.{0001}.{sql|csv|parquet}`: The data source file. For compressed Parquet files, the file name also contains the compression algorithm, for example, `test.t1.000000000.snappy.parquet`.

{{< copyable "shell-regular" >}}

Expand Down Expand Up @@ -400,7 +419,7 @@
| `-m` or `--no-schemas` | Do not export the schema with only the data exported |
| `-s` or `--statement-size` | Control the size of the `INSERT` statements; the unit is bytes |
| `-F` or `--filesize` | The file size of the divided tables. The unit must be specified such as `128B`, `64KiB`, `32MiB`, and `1.5GiB`. |
| `--filetype` | Exported file type (csv/sql) | "sql" |
| `--filetype` | Exported file type (csv/sql/parquet) | "sql" |
| `-o` or `--output` | Specify the absolute local file path or [external storage URI](/external-storage-uri.md) for exporting the data. | "./export-${time}" |
| `-S` or `--sql` | Export data according to the specified SQL statement. This command does not support concurrent export. |
| `--consistency` | flush: use FTWRL before the dump <br/> snapshot: dump the TiDB data of a specific snapshot of a TSO <br/> lock: execute `lock tables read` on all tables to be dumped <br/> none: dump without adding locks, which cannot guarantee consistency <br/> auto: use --consistency flush for MySQL; use --consistency snapshot for TiDB | "auto" |
Expand All @@ -423,7 +442,10 @@
| `--status-addr` | Dumpling's service address, including the address for Prometheus to pull metrics and pprof debugging | ":8281" |
| `--tidb-mem-quota-query` | The memory limit of exporting SQL statements by a single line of Dumpling command, and the unit is byte. For v4.0.10 or later versions, if you do not set this parameter, TiDB uses the value of the `mem-quota-query` configuration item as the memory limit value by default. For versions earlier than v4.0.10, the parameter value defaults to 32 GB. | 34359738368 |
| `--params` | Specifies the session variable for the connection of the database to be exported. The required format is `"character_set_client=latin1,character_set_connection=latin1"` |
| `-c` or `--compress` | Compresses the CSV and SQL data and table structure files exported by Dumpling. It supports the following compression algorithms: `gzip`, `snappy`, and `zstd`. | "" |
| `-c` or `--compress` | Compresses the CSV and SQL data and table structure files exported by Dumpling. It supports the following compression algorithms: `gzip`, `snappy`, and `zstd`. This option does not apply to Parquet files. | "" |
| `--parquet-compress` | The compression algorithm for Parquet files. It supports `no-compression`, `snappy`, `gzip`, and `zstd`. | "snappy" |
| `--parquet-page-size` | The data page size of Parquet files. The unit must be specified such as `128B`, `64KiB`, `32MiB`, and `1.5GiB`. | 1MiB |

Check failure on line 447 in dumpling-overview.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[PingCAP.Units] Put a nonbreaking space between the number and the unit in '1MiB'.

Raw Output:
{"message":"[PingCAP.Units] Put a nonbreaking space between the number and the unit in '1MiB'.","location":{"path":"dumpling-overview.md","range":{"start":{"line":447,"column":141},"end":{"line":447,"column":145}}},"severity":"ERROR","code":{"value":"PingCAP.Units","url":"https://developers.google.com/style/units-of-measure"}}
| `--parquet-row-group-size` | The row group memory limit of Parquet files, which is the flush threshold by accounted in-memory bytes. The unit must be specified such as `128B`, `64KiB`, `32MiB`, and `1.5GiB`. | 120MiB |

Check failure on line 448 in dumpling-overview.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[PingCAP.Units] Put a nonbreaking space between the number and the unit in '120MiB'.

Raw Output:
{"message":"[PingCAP.Units] Put a nonbreaking space between the number and the unit in '120MiB'.","location":{"path":"dumpling-overview.md","range":{"start":{"line":448,"column":213},"end":{"line":448,"column":219}}},"severity":"ERROR","code":{"value":"PingCAP.Units","url":"https://developers.google.com/style/units-of-measure"}}
Comment on lines +447 to +448

@coderabbitai coderabbitai Bot Sep 22, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

🔎 Supported by static analysis

🏁 Script executed:

rg -n --hidden -S 'PingCAP\.Units|Vale|vale|non.?breaking|Units' . --glob '!vendor/**' --glob '!node_modules/**'
sed -n '430,455p' dumpling-overview.md

Repository: pingcap/docs

Length of output: 41016

🤖 get_repo_knowledge executed:

get_repo_knowledge pingcap/docs /tmp/coderabbit-repo-knowledge/pingcap-docs-9c73f70c/conventions /tmp/coderabbit-repo-knowledge/pingcap-docs-9c73f70c/learnings

Length of output: 4452

🏁 Script executed:

#!/bin/bash
set -u
printf '%s\n' '--- relevant tracked files ---'
git ls-files | rg '(^|/)(\.vale\.ini|vale|Vale|markdownlint|dumpling-overview\.md|ci\.yaml)$|styles/PingCAP/Units\.yml|scripts/' | head -120
printf '%s\n' '--- .vale.ini ---'
if [ -f .vale.ini ]; then cat -n .vale.ini; else printf '%s\n' 'missing'; fi
printf '%s\n' '--- Units rule ---'
if [ -f styles/PingCAP/Units.yml ]; then cat -n styles/PingCAP/Units.yml; else printf '%s\n' 'missing'; fi
printf '%s\n' '--- Vale references in scripts/workflow ---'
rg -n -S 'vale|Vale|PingCAP\.Units|stylesPath|BasedOnStyles|formats|glob|exclude' scripts .github .vale.ini styles 2>/dev/null | head -200
printf '%s\n' '--- target lines and nearby Markdown unit examples ---'
cat -n dumpling-overview.md | sed -n '420,455p'
printf '%s\n' '--- relevant Markdown guidance/config ---'
rg -n -S 'Markdown|Vale|nonbreaking|non-breaking|unit|nbsp' README.md CONTRIBUTING.md .github resources scripts 2>/dev/null | head -200

Repository: pingcap/docs

Length of output: 39267

🏁 Script executed:

#!/bin/bash
set -u
printf '%s\n' '--- CI Vale job ---'
cat -n .github/workflows/ci.yaml | sed -n '55,78p'
printf '%s\n' '--- changed-file Vale plumbing ---'
rg -n -S 'vale-action|vale |Vale|extract-changed-markdown|changed.*md|\.md' .github/workflows .github/scripts scripts 2>/dev/null | head -120
printf '%s\n' '--- repository nonbreaking-space examples ---'
rg -n -S '&nbsp;|&`#160`;|U\+00A0' --glob '*.md' --glob '*.mdx' . | head -80
printf '%s\n' '--- all target literals in the file ---'
rg -n -F -e '1MiB' -e '120MiB' dumpling-overview.md

Repository: pingcap/docs

Length of output: 15224

🏁 Script executed:

#!/bin/bash
set -u
printf '%s\n' '--- nearby Parquet section ---'
cat -n dumpling-overview.md | sed -n '155,175p'
printf '%s\n' '--- targeted diff for dumpling-overview.md ---'
git diff --unified=2 -- dumpling-overview.md | rg -n -C 3 'parquet|MiB|@@' | head -180

Repository: pingcap/docs

Length of output: 2022

Add nonbreaking spaces to the Parquet size defaults.

PingCAP.Units applies to this Markdown table and flags 1MiB and 120MiB. Use &nbsp; between each number and unit.

Committable replacement 
-| `--parquet-page-size` | The data page size of Parquet files. The unit must be specified such as `128B`, `64KiB`, `32MiB`, and `1.5GiB`. | 1MiB |
-| `--parquet-row-group-size` | The row group memory limit of Parquet files, which is the flush threshold by accounted in-memory bytes. The unit must be specified such as `128B`, `64KiB`, `32MiB`, and `1.5GiB`. | 120MiB |
+| `--parquet-page-size` | The data page size of Parquet files. The unit must be specified such as `128B`, `64KiB`, `32MiB`, and `1.5GiB`. | 1&nbsp;MiB |
+| `--parquet-row-group-size` | The row group memory limit of Parquet files, which is the flush threshold by accounted in-memory bytes. The unit must be specified such as `128B`, `64KiB`, `32MiB`, and `1.5GiB`. | 120&nbsp;MiB |
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
| `--parquet-page-size` | The data page size of Parquet files. The unit must be specified such as `128B`, `64KiB`, `32MiB`, and `1.5GiB`. | 1MiB |
| `--parquet-row-group-size` | The row group memory limit of Parquet files, which is the flush threshold by accounted in-memory bytes. The unit must be specified such as `128B`, `64KiB`, `32MiB`, and `1.5GiB`. | 120MiB |
| `--parquet-page-size` | The data page size of Parquet files. The unit must be specified such as `128B`, `64KiB`, `32MiB`, and `1.5GiB`. | 1&nbsp;MiB |
| `--parquet-row-group-size` | The row group memory limit of Parquet files, which is the flush threshold by accounted in-memory bytes. The unit must be specified such as `128B`, `64KiB`, `32MiB`, and `1.5GiB`. | 120&nbsp;MiB |
🧰 Tools
🪛 GitHub Check: vale

[failure] 448-448:
[vale] reported by reviewdog 🐶
[PingCAP.Units] Put a nonbreaking space between the number and the unit in '120MiB'.

Raw Output:
{"message":"[PingCAP.Units] Put a nonbreaking space between the number and the unit in '120MiB'.","location":{"path":"dumpling-overview.md","range":{"start":{"line":448,"column":213},"end":{"line":448,"column":219}}},"severity":"ERROR","code":{"value":"PingCAP.Units","url":"https://developers.google.com/style/units-of-measure"}}

[failure] 447-447:
[vale] reported by reviewdog 🐶
[PingCAP.Units] Put a nonbreaking space between the number and the unit in '1MiB'.

Raw Output:
{"message":"[PingCAP.Units] Put a nonbreaking space between the number and the unit in '1MiB'.","location":{"path":"dumpling-overview.md","range":{"start":{"line":447,"column":141},"end":{"line":447,"column":145}}},"severity":"ERROR","code":{"value":"PingCAP.Units","url":"https://developers.google.com/style/units-of-measure"}}


## Output filename template

Expand Down
Loading