fix: hash-based encoding for object-dtype join/groupby keys (v2.2.5) - #32
Conversation
np.unique's internal argsort raises TypeError on object arrays mixing Python types (e.g., str and int) because cross-type `<` is unsupported. The pure-Python encoding paths in _encode_columns and _encode_columns_paired now use a dict-based codebook for object dtype, matching the C accelerator's hash-equality semantics. Fixes joins/groupbys that previously failed when both sides had _dtypes='object' with heterogeneous Python values. Also fixes a pre-existing mypy return-type error in to_pandas via cast(), and documents the three version-bump files in CLAUDE.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Fixes TypeError raised when left_join / inner_join / group_by operate on object-dtype key columns containing mixed Python types (e.g., str and int). The pure-Python encoding fallback used np.unique(..., return_inverse=True), which internally argsorts and fails on cross-type <. A new dict-based hash codebook (GroupSet._hash_encode_object) is added with semantics matching the C accelerator (_c_encode_strings), and used for object arrays in both single-column (_encode_columns) and paired (_encode_columns_paired) paths.
Changes:
- Add
GroupSet._hash_encode_object()and route object-dtype arrays through it in_encode_columnsand_encode_columns_paired. - Wrap
Tafra.to_pandas()return incast(DataFrame, ...)to silence a pre-existing mypyreturn-valueerror against the lightweightDataFrameProtocol. - Bump version to 2.2.5 across
pyproject.toml,recipe/meta.yaml,docs/changelog.md; document the three bump locations inCLAUDE.md.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tafra/group.py | New hash-based object encoder; used in single-column and paired encoding paths below the C threshold |
| tafra/base.py | cast(DataFrame, ...) around pd.DataFrame(...) return in to_pandas |
| test/test_tafra.py | Three regression tests for mixed-type object-dtype joins and group_by |
| docs/changelog.md | 2.2.5 release notes |
| pyproject.toml | Version bump 2.2.4 → 2.2.5 |
| recipe/meta.yaml | Conda recipe version bump 2.2.4 → 2.2.5 |
| CLAUDE.md | Documents the three version-bump locations |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Summary
left_join/inner_join/group_byonobject-dtype key columns containing heterogeneous Python types (e.g.,strandintacross sides, or mixed within one side) no longer raiseTypeError: '<' not supported between instances of ...._encode_columnsand_encode_columns_pairedcallsnp.unique(..., return_inverse=True), which internallyargsorts the values. Object arrays with mixed Python types fail cross-type<comparison. The C accelerator (_c_encode_strings) handles this correctly via hash equality but only triggers at >= 50k rows;_validate_dtypeslets both sides through because_dtypes='object'on both sides is an equality match.GroupSet._hash_encode_object— a dict-based codebook used for object-dtype arrays in the pure-Python path. Semantics match_c_encode_strings(hash equality, not ordering).return-valueerror inTafra.to_pandasviacast(DataFrame, ...)— the protocol is intentionally a "fake class to satisfy typing of apandas.DataFramewithout a dependency" and can't structurally match pandas's actual signatures.pyproject.toml,recipe/meta.yaml,docs/changelog.md) inCLAUDE.md.Test plan
TestStringDtype:test_join_on_object_dtype_mixed_python_types_across_sidestest_join_on_object_dtype_mixed_within_sidetest_group_by_object_dtype_mixed_python_typesruff check tafra— cleanmypy tafra— clean (was 1 pre-existing error, now resolved)Generated with Claude Code