Write geometry stats#56
Closed
Tmonster wants to merge 66 commits into
Closed
Conversation
…ion with a build flag
…uires manually created equality delete file
…predicates prune files
[BugFix] Equality delete columns must always be referenced
Iceberg schema fields carry an optional `doc` attribute ("a comment
describing the field" per the spec). It was parsed but then discarded,
so `duckdb_columns().comment` was always NULL.
- IcebergColumnDefinition::ParseType now stores the parsed `doc` (it was
received as a parameter but never assigned), and Copy() preserves it.
- GetColumnDefinition() maps a non-empty `doc` to the DuckDB column
comment via ColumnDefinition::SetComment().
- IcebergTableSet::Scan returns the already-resolved table entry (which
now carries the comments) instead of the lazy placeholder, so the
comment surfaces in catalog listings once a table has been resolved
(e.g. via DESCRIBE or a scan). Untouched tables stay lazy, preserving
the behaviour from duckdb#515.
Fixes duckdb#1025
The upper bound for a string longer than the bound length was built by truncating to 16 bytes and incrementing the last byte. That splits multi-byte UTF-8 characters and usually yields invalid UTF-8, so writing e.g. full-width Japanese or Turkish titles failed the whole insert with "Could not write upper bounds for string column". Truncate on a code point boundary and increment the last code point instead (skip the surrogate range; if a code point is already the maximum, drop it and carry). Adds a regression test.
- Spark data generator `column_doc_comment` creates a table whose columns carry COMMENT clauses (mapped to Iceberg field `doc`). - SQL test attaches the catalog, resolves the table, and asserts that `duckdb_columns().comment` returns the documentation (NULL where a field has no `doc`). Verified locally against the apache/iceberg-rest-fixture catalog.
…mment Surface Iceberg field `doc` as DuckDB column comment
…d-non-ascii Compute string upper bounds on code point boundaries
Pin versions in requirements.txt
…ported in Iceberg
…berg into write_geometry_stats
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.