[WIP] Delta v0.25.0 update #328
Open
benfleis wants to merge 3 commits into
Open
Conversation
Cargo feature flags on the delta_kernel_ffi crate are unaffected by the default-engine extraction into its own crate. The FFI struct ABI moved more than the changelog suggests for a C++ consumer: - visit_literal_null gained type_tag/precision/scale params (v0.22.0) - visit_transform_expr/visit_field_transform replaced by visit_struct_patch_expr/visit_field_patch with a reshaped signature (three field lists instead of one, keep_input/optional replacing is_replace) (v0.25.0); re-encoded our internal transform-op ParsedExpression shape and FindPartitionValues to match - new required EngineExpressionVisitor::visit_array and EngineSchemaVisitor::visit_void struct fields (v0.25.0) - get_write_context renamed to get_unpartitioned_write_context and became fallible (v0.22.0) - commit() now returns a Handle<ExclusiveCommittedTransaction> instead of a bare version number (v0.23.0) See workspace/tasks/delta-v0.25--summary.md for the full investigation, what was ruled out, and open testing concerns (notably: no local test coverage for the new struct-patch "append" path, gated behind GENERATED_DATA_AVAILABLE fixture data this sandbox can't produce).
…encoding delta_list_files(..., transform_expression=1) prints our internal delta_kernel_transform_expression debug representation verbatim. The v0.25.0 struct-patch rewrite changed that representation's shape (is_replace -> keep_input/optional/kind), so the fixture's expected strings needed updating to match. Not a behavior change -- verified via the full generated-data test suite (24 files) plus the full local suite (22 files): all pass except two pre-existing failures traced to a parquet-writer boolean-column stats limitation in the regenerated fixture data, confirmed via parquet_metadata() and unrelated to the kernel bump. Also confirmed (via temporary instrumentation, since reverted) that none of the 46 available test files exercise the new struct-patch "append" path -- see workspace/tasks/delta-v0.25--summary.md for the full accounting and the one remaining open item.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI tests first, description to come