Fix Double -> Long conversion after Parquet migration - #122
Fix Double -> Long conversion after Parquet migration#122kevindetry-milaboratories wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a helper function stripDecimalOnIntColumns to strip trailing decimals from integer columns and axes, preventing conversion failures after Parquet migration. The helper is applied across several tree-export functions. Feedback highlights a potential issue where negative integers represented as floats (e.g., -1.0) would fail to match the current regular expression pattern, and suggests updating the pattern to support negative numbers.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
0aaabe8 to
072f138
Compare
Greptile Summary
This PR fixes a type-conversion regression introduced by the Parquet migration, where integer columns (
Long/Int) arrive float-formatted (e.g.,"192.0") — either because Pandas promotes NA-bearing integer columns to float or because MiXCR emits certain counts as doubles. A module-levelstripDecimalregex step and astripDecimalOnIntColumnshelper are added; the helper is called in all four export functions (shmTree,shmTreeNodes,shmTreeNodesWithClones,shmTreeNodesUniqueIsotype) before theirpfconvParamsis returned.stripDecimal— new module-level constant: a single-elementpreProcessarray containing aregexpReplacethat strips a trailing.digitssuffix (e.g.,"192.0"→"192"), making the value safe for the subsequent strict-cast toLong/Int.stripDecimalOnIntColumns— new helper function that iterates all axes (spec.type) and columns (spec.valueType) and appendsstripDecimalto anyLong/Intentry'spreProcess, preserving any pre-existing steps.stripDecimalOnIntColumns(axes, columns)immediately before theirreturnstatement, ensuring everyLong/Intfield across all table shapes is covered without a fragile hand-picked column list.Confidence Score: 4/5
Safe to merge; the fix is well-scoped and all four export functions are updated consistently.
The logic change is small and correct — the regex handles the described float-formatted-integer cases and is a no-op for already-clean integers. All four export functions are covered. The only note is a changeset version bump (minor vs patch) that has no runtime impact but affects release semantics.
.changeset/soft-ghosts-shake.md — version type may want a second look before the release is cut.
Important Files Changed
minor— conventionally apatchfor a bug fix, but may be intentional given the data-behaviour change introduced by the Parquet migration.stripDecimalregex step andstripDecimalOnIntColumnshelper; correctly wired into all four export functions (shmTree, shmTreeNodes, shmTreeNodesWithClones, shmTreeNodesUniqueIsotype) before their return statements.Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD A["Export function called\n(shmTree / shmTreeNodes /\nshmTreeNodesWithClones /\nshmTreeNodesUniqueIsotype)"] --> B["Build axes[] and columns[]\nwith Long/Int specs"] B --> C["stripDecimalOnIntColumns(axes, columns)"] C --> D{"For each axis:\nspec.type == Long or Int?"} D -- Yes --> E["Append stripDecimal\nregexpReplace to axis.preProcess\ne.g. '192.0' → '192'"] D -- No --> F["Leave axis unchanged"] E --> G{"For each column:\nspec.valueType == Long or Int?"} F --> G G -- Yes --> H["Append stripDecimal\nregexpReplace to col.preProcess\ne.g. '99.0' → '99'"] G -- No --> I["Leave column unchanged"] H --> J["Return pfconvParams\nwith patched axes + columns"] I --> J J --> K["xsv.importFileMap reads Parquet\nString → stripDecimal → strict-cast Long/Int"]%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% flowchart TD A["Export function called\n(shmTree / shmTreeNodes /\nshmTreeNodesWithClones /\nshmTreeNodesUniqueIsotype)"] --> B["Build axes[] and columns[]\nwith Long/Int specs"] B --> C["stripDecimalOnIntColumns(axes, columns)"] C --> D{"For each axis:\nspec.type == Long or Int?"} D -- Yes --> E["Append stripDecimal\nregexpReplace to axis.preProcess\ne.g. '192.0' → '192'"] D -- No --> F["Leave axis unchanged"] E --> G{"For each column:\nspec.valueType == Long or Int?"} F --> G G -- Yes --> H["Append stripDecimal\nregexpReplace to col.preProcess\ne.g. '99.0' → '99'"] G -- No --> I["Leave column unchanged"] H --> J["Return pfconvParams\nwith patched axes + columns"] I --> J J --> K["xsv.importFileMap reads Parquet\nString → stripDecimal → strict-cast Long/Int"]Prompt To Fix All With AI
Reviews (1): Last reviewed commit: "Fix Double -> Long conversion after Parq..." | Re-trigger Greptile