Skip to content

Make baseline evaluation configurable - #10

Merged
amarakramali merged 1 commit into
mainfrom
agent/configure-baseline-evaluation
Jul 27, 2026
Merged

Make baseline evaluation configurable#10
amarakramali merged 1 commit into
mainfrom
agent/configure-baseline-evaluation

Conversation

@amarakramali

Copy link
Copy Markdown
Owner

Summary

  • make the baseline evaluator import-safe and configurable through CLI arguments
  • validate timestamps, hourly continuity, and required numeric feature columns
  • support custom metrics and plot destinations while creating parent directories
  • document baseline evaluation in the synthetic no-download quickstart
  • cover metrics, invalid inputs, and generated command artifacts with tests

Why

The synthetic quickstart could generate feature data, but baseline evaluation still executed on import and used hard-coded production paths. That prevented users from evaluating the bundled sample workflow or safely reusing the evaluation functions in tests and other code.

Impact

Existing default paths and the 30-day evaluation window remain available. Users can now evaluate any compatible feature CSV, choose evaluation and plot windows, and place artifacts wherever needed.

Validation

  • python -m pytest -q — 56 passed
  • uvx --from ruff==0.16.0 ruff check src tests streamlit_app.py — passed
  • python -m compileall -q src tests streamlit_app.py — passed
  • synthetic 40-day data → features → 30-day baseline evaluation smoke test — metrics CSV and 101 KB PNG generated

@amarakramali
amarakramali merged commit 671d698 into main Jul 27, 2026
2 checks passed
@amarakramali
amarakramali deleted the agent/configure-baseline-evaluation branch July 27, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant