[BUG] `format_data_handle` silently reindexes data to daily frequency for unrecognized time intervals

[Om-Dhondge]

[BUG] format_data_handle silently reindexes data to daily frequency for unrecognized time intervals

Problem

When format_data_handle cannot infer a time series frequency using pd.infer_freq, it falls
through a manual if-elif chain that only handles six specific intervals: 1 day, 1 hour, 1 minute,
1 second, 7 days, and 28-31 days. Any other interval (bi-weekly, quarterly with irregular spacing,
annual with leap years, 15-minute, etc.) hits the final else branch and silently sets
freq = "D".

The series is then reindexed to a full daily date range. The inserted rows are filled with real
values via forward/backward fill. The call returns success: True with no warning.

Steps to Reproduce

import sys
sys.path.insert(0, "src")
import pandas as pd
from sktime_mcp.runtime.executor import get_executor

executor = get_executor()

# Bi-weekly survey data with slight real-world irregularity
# (month-end shifts, one collection came a day early)
dates = [
    pd.Timestamp("2023-01-02"),
    pd.Timestamp("2023-01-16"),  # 14 days
    pd.Timestamp("2023-01-30"),  # 14 days
    pd.Timestamp("2023-02-13"),  # 14 days
    pd.Timestamp("2023-02-28"),  # 15 days - month-end shift
    pd.Timestamp("2023-03-13"),  # 13 days
    pd.Timestamp("2023-03-27"),  # 14 days
    pd.Timestamp("2023-04-10"),  # 14 days
]
values = [10.0, 12.0, 11.0, 14.0, 13.0, 15.0, 12.0, 16.0]
df = pd.DataFrame({"date": dates, "value": values})

load_result = executor.load_data_source({
    "type": "pandas",
    "data": df,
    "time_column": "date",
    "target_column": "value",
})
handle = load_result["data_handle"]

fmt_result = executor.format_data_handle(handle)
print("Input rows: ", len(df))
print("Output rows:", fmt_result["metadata"]["rows"])
print("Frequency:  ", fmt_result["metadata"]["frequency"])
print("Success:    ", fmt_result["success"])

Output:

Input rows:  8
Output rows: 99
Frequency:   <Day>
Success:     True

Expected vs Actual

	Expected	Actual
Output rows	8 (unchanged)	99 (expanded to daily)
Frequency	None or bi-weekly, with a warning	<Day>, silently
success	True, but with a warning in the response	True, no indication anything changed
Inserted values	N/A	91 rows forward/backward filled from real observations

What's Happening

In executor.py around line 750, when pd.infer_freq returns None, the code attempts manual
inference by looking at the most common time difference between observations. The chain only covers
six exact intervals:

if most_common_diff == pd.Timedelta(days=1):
    freq = "D"
elif most_common_diff == pd.Timedelta(hours=1):
    freq = "h"
elif most_common_diff == pd.Timedelta(minutes=1):
    freq = "min"
elif most_common_diff == pd.Timedelta(seconds=1):
    freq = "s"
elif most_common_diff == pd.Timedelta(days=7):
    freq = "W"
elif most_common_diff.days >= 28 and most_common_diff.days <= 31:
    freq = "MS"
else:
    freq = "D"  # ← silent fallback for anything else

Bi-weekly data (most common diff = 14 days) does not match any branch, so freq becomes "D".
The series is then passed to pd.date_range(..., freq="D"), which generates a daily index spanning
the full observation window. The original 8 data points are reindexed across 99 daily slots, and
fill_missing backfills the gaps with real values. The caller gets back a structurally corrupted
dataset with no indication that anything went wrong.

The same failure applies to any interval not in the list: quarterly with irregular spacing,
15-minute, 30-minute, 2-hourly, 4-hourly, annual with leap year variation, and others.

Impact

This is silent data corruption. A model trained on the output of format_data_handle for a
bi-weekly series is being trained on 91 fabricated daily observations, not 8 real ones. The
success: True response gives no signal that the data was altered, so there is no opportunity
to catch it downstream.

Suggested Fix

Replace the catch-all else with an explicit skip. If the interval is unrecognized, leave the
frequency unset and report it in the response rather than guessing:

else:
    freq = None  # unrecognized interval, do not reindex

And surface it to the caller:

if freq is None and len(time_diffs) > 0:
    changes_made["frequency_warning"] = (
        f"Could not determine frequency from most common interval "
        f"({most_common_diff}). Reindexing skipped."
    )

This preserves the original data, gives the caller the information they need to decide what to do,
and avoids silently fabricating observations.

Environment

• sktime-mcp version: 0.1.0
• Python: >=3.10

Code changes are minimal, hence creating a PR for the fix right away for integration. Happy to iterate on the approach if the maintainers prefer a different handling strategy

[Shashankss1205]

Still reproduces on main exactly as described (8 bi-weekly rows become 99 daily rows with success true), and because auto-format runs on load the damage is done in load_data_source already. PR #553 fixes the 'D' fallback and is being merged now; the broader off-anchor reindex corruption for recognised frequencies (e.g. mid-month dates -> MS wipes all values to NaN) is item 1 of the consolidated follow-up issue.