Skip to content

fix: transform/convert dead ends and JSON round-trip (#537) - #549

Closed
Shashankss1205 wants to merge 1 commit into
mainfrom
fix/537-transform-deadends
Closed

fix: transform/convert dead ends and JSON round-trip (#537)#549
Shashankss1205 wants to merge 1 commit into
mainfrom
fix/537-transform-deadends

Conversation

@Shashankss1205

Copy link
Copy Markdown
Collaborator

Fixes #537. Four transform/convert dead ends:

BUG-21 — np.ndarray handles are a dead end

Converting a handle to an index-less mtype (np.ndarray, numpy3D, ...) produced a handle with no time index that broke inspect_data/split_data/format and fabricated a cutoff. Those targets are now rejected up front with a message pointing to pd.Series/pd.DataFrame.

NB-09 — Series→Panel convert dumped a raw mtype error

convert(Series handle -> pd-multiindex) returned sktime's multi-paragraph "No valid mtype could be identified ..." dump. Caught and replaced with a clean domain message: "Cannot convert a Series-scitype handle to 'pd-multiindex'. That target expects a different scitype (Panel)."

NB-10 — JSON was save-only

  • Added a JSON reader to the file adapter (.jsonjson, pd.read_json records orient); the load error hint no longer points into a dead end.
  • save_data json now writes the index as a "time" column (records orient dropped it), so a saved JSON round-trips through load_data_source(time_column="time").

BUG-19 — duplicate timestamps hard-rejected before auto-format

Duplicate timestamps were a validation error, so no handle was returned and the documented remedy (transform_data(action='format', remove_duplicates=True)) was unreachable. Downgraded to a warning, so the handle loads and auto-format de-duplicates it (keeping the first of each).

Testing

  • New tests/test_transform_deadends.py (5 tests): ndarray target rejected; Series→Panel clean error (no mtype dump); valid convert works; duplicate-timestamp load succeeds and is de-duplicated; JSON save→load round-trip.
  • Full suite: 285 passed.

🤖 Generated with Claude Code

- BUG-21: converting a handle to an index-less mtype (np.ndarray, numpy3D, ...)
  produced a handle that broke inspect/split/format and fabricated a cutoff.
  Reject those targets with a message pointing to pd.Series/pd.DataFrame.
- NB-09: a scitype-incompatible convert (Series handle -> pd-multiindex) dumped
  sktime's multi-paragraph mtype-inference error. Catch it and return a clean
  "that target expects a different scitype (Panel)" message.
- NB-10: JSON was save-only. Add a JSON reader to the file adapter (.json ->
  json), and make save_data write the index as a "time" column so JSON files
  round-trip through load_data_source(time_column="time").
- BUG-19: duplicate timestamps were a hard load error, making the documented
  remedy (auto-format remove_duplicates) unreachable. Downgrade to a warning so
  the handle loads and auto-format de-duplicates it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Shashankss1205

Copy link
Copy Markdown
Collaborator Author

Closing: this PR's content is already on main as commit caaa83b (identical patch-id; it was landed directly rather than through the PR merge button). Nothing further to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] transform/convert dead ends: np.ndarray, Series→Panel, JSON save-only, duplicate-timestamp ordering

1 participant