Skip to content

fix: return error tuple on dtype/operator mismatch in Query.where predicates - #27

Merged
stuartjohnpage merged 5 commits into
mainfrom
sp/apply-correct-predicate
Sep 3, 2026
Merged

stuartjohnpage merged 5 commits into
mainfrom
sp/apply-correct-predicate

Conversation

@stuartjohnpage

@stuartjohnpage stuartjohnpage commented Sep 3, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • fix: return clean error tuple on dtype/operator mismatch in Results.filter/2 #12 guarded Results.filter/2 against ordering operators (>, <, >=, <=) on non-orderable columns, but the same unguarded dispatch remained in Client.apply_filter/4
  • Client.apply_filter/4 now performs the same dtype check and returns {:error, "operator >= not supported on :string column 'revision'"}.
  • Query.process_files/3 previously hard-matched {:ok, df} from parse_parquet_files/3; it now passes an error through.

Drive-bys

  • cleaned up some aliases
  • orderable_dtype?/1 abstracted out

Test plan

  • mix test — 8 doctests + 110 tests pass
  • New regression test: Client.parse_parquet_files/3 with revision >= 1 on a string column returns {:error, msg} where msg =~ "operator >=", msg =~ "revision", msg =~ "string"
  • New positive test: id >= 2 on a numeric column through the same path still filters rows correctly
  • Existing Results.filter/2 dtype tests continue to pass against the shared orderable_dtype?/1

@stuartjohnpage stuartjohnpage self-assigned this Sep 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The functional changes are cohesive, error propagation is handled, and new tests cover both the regression and a positive path, with only a minor docstring inconsistency noted.

Pull request overview

This PR aligns client-side Parquet row filtering with the existing Results.filter/2 behavior by guarding ordering operators (>, <, >=, <=) against non-orderable Explorer dtypes, returning a descriptive {:error, message} instead of raising, and propagating that error up through query execution.

Changes:

  • Add dtype/operator validation to Client.apply_filter/4 and return an error tuple for invalid ordering comparisons.
  • Propagate Client.parse_parquet_files/3 failures through Query.process_files/3 instead of hard-matching {:ok, df}.
  • Extract orderable_dtype?/1 into PredicateParser and add regression/positive tests for Parquet row filtering.
File summaries
File Description
test/delta_query/client_test.exs Adds adapter-backed tests covering valid numeric ordering filters and invalid string ordering filters.
lib/delta_query/results.ex Refactors aliases and switches to shared PredicateParser.orderable_dtype?/1 for ordered operator validation.
lib/delta_query/query.ex Ensures process_files/3 passes through Client.parse_parquet_files/3 errors via with.
lib/delta_query/predicate_parser.ex Introduces public orderable_dtype?/1 helper with docs/spec for shared dtype checks.
lib/delta_query/client.ex Returns `{:ok, df}
Review details
  • Files reviewed: 5/5 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread lib/delta_query/client.ex

@emmy-bishop emmy-bishop left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lookin good!

@stuartjohnpage
stuartjohnpage merged commit 5d341b0 into main Sep 3, 2026
1 check passed
@stuartjohnpage
stuartjohnpage deleted the sp/apply-correct-predicate branch September 3, 2026 19:43
@stuartjohnpage stuartjohnpage mentioned this pull request Sep 3, 2026
1 task done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants