Skip to content

fix: classify intraday outages from declared exchange bar grids - #100

Merged
stefan-jansen merged 24 commits into
mainfrom
fix/intraday-schedule-gaps
Oct 17, 2026
Merged

stefan-jansen merged 24 commits into
mainfrom
fix/intraday-schedule-gaps

Conversation

@stefan-jansen

@stefan-jansen stefan-jansen commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Closes #99

Outcome

Replace the fixed US-clock intraday gap approximation with authoritative exchange bar positions. Explicit calendar overrides shared feed metadata, then NYSE; explicit None disables detection. Continuous24/7 remains an expected schedule. Require complete version-2 frequency/timestamp/anchor/label/partial-bar declarations and timezone for naive instants. Off-grid, contradictory, event-count and incomplete declarations fail rather than infer a grid from prices.

The grid restarts at each tradable interval after breaks and declared interruptions. Left/right labels and included/excluded shorter closing bars follow shared Specs701bde6. Count expected positions instead of expanding every missing timestamp, including a whole absent session. UTC output bounds, expected-bar counts and elapsed duration remain distinct. No filling or input mutation; classify each asset separately. Omitted frequency inherits feed metadata, otherwise daily; explicit conflicting frequency fails.

Intentional compatibility change: unqualified intraday calls must supply a complete feed declaration or explicit None. The old approximation cannot silently claim complete data. Daily label behavior remains separate; legacy weekly/monthly and range-planner limitations are not newly claimed as schedule grids.

Executed evidence

  • Base1f979aa8ee216c62dc9092b7cc2b68209e0304c8: real SyntheticProvider July7 minute bars13:30/13:32UTC map to valid NYSE session, but the old detector returns no gap despite missing13:31. Retained public-workflow log. Regression-only8902caf5b4814c0d2bf71033174e332c7af4951c has20 failures, comprising strict rejection of the inaccurate old API plus declared-grid capability failures; these are not20 historical numerical bugs.
  • Earlier59238a0c92441371d9836f9811aa89481faa2de9:135 focused gap/session cases pass in8.74s after non-rewriting integration of merged PR94. The grid reuses the same tradable-interval helper as native assignment/completion. ResourceWarning as error, all native caps1; these checks completed before the strict replacement hold. Current integrated-head results follow below.
  • Oldaf776ee failed seven legacy tests that omitted required bar-grid declarations, confirmed from actual multi-platform logs and locally reproduced7fail/17pass. Test-only migration2e239ce uses real SyntheticProvider rows and exact counts/rejection assertions;89 focused tests pass. No production relaxation, skip, xfail or blind workflow rerun. Full stacked-base qualification later passed, but does not replace current main-target gates.
  • Real provider cases cover DST/winter, London, holidays/shortened sessions, all-asset absent session, HK lunch/close side, CME overnight, continuous outages, explicitNone, naive/ambiguous local clocks, registered custom breaks/halts and metadata validation.
  • Review regressions additionally expose nanosecond truncation, declared frequency incorrectly falling back to daily, ignored scheduled halts and an intermediate partial-close indexing error. Each has retained failing evidence; fixed lane passes. Zero-length CME holiday breaks are a real provider schedule case and remain valid.
  • Ruff lint/format, ty, normal hooks, diff checks and strict MkDocs passed. All applicable full hosted CI/resource/platform/package/coverage/docs gates remain required after the prerequisite and retarget; local focused evidence is not full qualification.

Dependencies and review

Current head a84453f integrates main 721adee without rewriting history, after the session, ingestion, futures, crypto-weekend and daily-timestamp repairs merged. The normal merge preserves shared selection and daily label/instant semantics while documenting the complete intraday grid requirement. No asset-class inference was restored. The benchmark hold was explicitly released at 19:13 UTC before local integration and verification. Current-head hosted CI remains required; prior stacked-head results do not substitute for it.

On the integrated head, the full local suite passed 3,984 tests in 361.23s with 86.25% coverage. A combined focused lane passed 318 tests in 50.60s with ResourceWarning as error. Four additional public DataManager workflows use real SyntheticProvider acquisition and Flat/Hive storage: main721adee fails two inherited-grid cases and passes two explicit calendar=None controls; the fix passes all four. Only the clock is patched. Full-source Ruff check/format, ty, strict MkDocs, normal commit hooks and pre-commit --all-files passed. Both wheel and sdist installed in isolation and passed dependency, import, CLI and README quick-start checks. These development artifacts are not a release.

AI-assisted implementation/tests/prose under maintainer direction. Owner reviewed actual diff, declared boundary sides, scheduled outages independent of observed data, arithmetic bar counts, aliases, atomic errors, memory scaling and compatibility. No independent reviewer claimed. Daily timestamp and futures repairs are merged separately. Validation, general intraday acquisition-range planning, schedule-aware filling and CME crypto cutover remain separate accepted work, not capabilities claimed by this PR. Published0.2.1 unchanged; no release/deploy.

Stefan Jansen added 21 commits October 17, 2026 10:17
Copilot AI balanced review requested due to automatic review settings October 17, 2026 15:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Base automatically changed from fix/session-calendar-contract to main October 17, 2026 17:16
Stefan Jansen added 2 commits October 17, 2026 13:18
…-gaps

# Conflicts:
#	docs/user-guide/sessions.md
#	src/ml4t/data/sessions/assigner.py
#	src/ml4t/data/sessions/completer.py
#	tests/test_session_calendar_contract.py
@stefan-jansen
stefan-jansen merged commit 74f4e48 into main Oct 17, 2026
27 checks passed
@stefan-jansen
stefan-jansen deleted the fix/intraday-schedule-gaps branch October 17, 2026 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: classify intraday outages against declared exchange bar grids

2 participants