Skip to content

Commit e75b4a0

Browse files
zhangwl9张文领
andauthored
[AMORO-4186] Add support for Parquet row-group merging in IcebergRewriteExecutor (#4188)
* Add support for Parquet row-group merging in IcebergRewriteExecutor * Enhance Parquet row-group merge support for Iceberg V2 tables and add version checks * Add cleanup for orphaned merged output files in Parquet row-group merge * Refactor ParquetFileMergeRunner to simplify ParquetWriter initialization * Implement Parquet schema context preparation for row-group merging in IcebergRewriteExecutor * Add minimum average row-group size threshold for Parquet row-group merge * fixup * fixup style Removed unnecessary line breaks in the class documentation. * fixup style --------- Co-authored-by: 张文领 <zhangwl9@chinatelecom.cn>
1 parent 4679309 commit e75b4a0

7 files changed

Lines changed: 1004 additions & 26 deletions

File tree

amoro-format-iceberg/src/main/java/org/apache/amoro/optimizing/AbstractRewriteFilesExecutor.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,7 @@ private List<DeleteFile> equalityToPosition() throws Exception {
145145
return posDeleteWriter.result().deleteFiles();
146146
}
147147

148-
private List<DataFile> rewriterDataFiles() throws Exception {
148+
protected List<DataFile> rewriterDataFiles() throws Exception {
149149
List<DataFile> result = Lists.newArrayList();
150150
TaskWriter<Record> writer = dataWriter();
151151

0 commit comments

Comments
 (0)