Write delete manifests with content='deletes' file metadata#1097
Merged
Conversation
WriteToFile hardcoded the Avro file-metadata 'content' key to "data" for every
manifest, including delete manifests. The Iceberg spec requires this key to
reflect what the manifest holds ("data" or "deletes"); spec-strict readers
cross-check it against the manifest list entry's content type and reject a
delete manifest mislabeled as "data". Derive it from the manifest's content.
Adds a test asserting a v3 DELETE's delete manifest carries content='deletes'
(and not the mislabeled 'data') in its Avro file metadata.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Raghvendra Singh <raghav@cashify.in>
Tishj
added a commit
that referenced
this pull request
Jun 22, 2026
…-content-v1.5-variegata Backport #1097 to v1.5-variegata: write delete manifests with content='deletes'
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
WriteToFileinsrc/core/metadata/manifest/iceberg_manifest.cpphardcodes the Avrofile-metadata
contentkey to"data"for every manifest it writes, including deletemanifests:
Per the Iceberg spec, a manifest's
contentmetadata must reflect what it holds —"data"for a data manifest,
"deletes"for a delete manifest. So a v3DELETE(which writes adeletion-vector delete manifest) produces an Avro file whose
contentmetadata saysdata, contradicting the manifest list entry, which correctly records it as a deletemanifest.
Readers that take a manifest's content from the manifest list entry tolerate this (so it
does not affect query results in duckdb-iceberg itself), but spec-strict readers cross-check
the manifest file's own
contentmetadata against the list entry and reject the mismatch.For example, iceberg-go's
ReadManifesterrors with:Fix
Derive the value from the manifest's content type (the
IcebergManifestFilethe functionalready receives):
Testing
test/sql/local/catalog_test_config_setup/catalog_agnostic/delete/test_delete_manifest_content_type.test:after a v3
DELETE, resolves the delete manifest viaiceberg_metadataand asserts itsAvro file metadata carries
content='deletes'(and not the mislabeledcontent='data').Found alongside #1095 — both are v3 deletion-vector write-correctness fixes.