Skip to content

Panel history tabs, calculation dry run and opt-in calculation log - #793

Open
frankyhun wants to merge 12 commits into
altmenorg:devfrom
frankyhun:master
Open

Panel history tabs, calculation dry run and opt-in calculation log#793
frankyhun wants to merge 12 commits into
altmenorg:devfrom
frankyhun:master

Conversation

@frankyhun

@frankyhun frankyhun commented Jul 29, 2026

Copy link
Copy Markdown

This PR brings the work done in the frankyhun/HAsmartirrigation fork (releases v2026.7.1 → v2026.7.4) back to dev. It is a mix of panel features, a calculation "preview" mode, an opt-in audit log for calculations, and two small fixes, plus tests and documentation.

Panel

Irrigation history tab (view-history.ts, new)
A new tab that lists what was actually watered. Runs are recorded when a valve run finishes (valve_runner.py) and when observed watering is registered (observed_watering.py), and persisted in the store. A new websocket command smart_irrigation/irrigation_history returns the raw records (start time in UTC, duration in seconds, water in litres) so the panel can group them by local calendar day and convert to the user's unit system — bucketing on the backend would use the server's idea of "a day".

Weather service retrieval history (view-weatherservice.ts)
The Weather service tab now shows the last retrievals, newest first, via a new smart_irrigation/weatherservice_history websocket command. Only sensor groups that actually take at least one value from the weather service are considered, and within a record only the weather-service-sourced values are returned — showing a sensor or static value on that tab would be misleading.

Durations as hh:mm:ss with water volume (view-info.ts, view-zones.ts, helpers.ts)
Run times were shown as a raw number of seconds. They are now formatted as hh:mm:ss on the Info tab and the Zones tab, and the corresponding water volume is shown next to them.

Panel module URL is versioned (panel.py)
After an update, browsers kept serving the cached smart-irrigation.js, so a new bundle only appeared after a hard refresh. The panel module URL now carries the integration version as a query parameter.

Hungarian label fix
hu.json: corrected the label for the sensor update interval.

Dry run for the calculate services

calculate_zone and calculate_all_zones accept a new dry_run: true option. A dry run computes the result and returns it without touching any irrigation data: the bucket, the zone, the collected weather data and the internal "last calculated" marker are all left exactly as they were.

The motivation: a normal manual calculation consumes the weather data collected so far and moves the "last calculated" marker, so the scheduled run later that day only sees a partial window. ET is computed as a full-day rate from that window and then scaled by its length, and because min/max temperature come from the samples inside the window, a partial window never contains the full daily temperature swing — the scheduled run then produces a lower daily total. A dry run avoids this.

One thing a dry run does still do: with PyETO and forecast_days > 0 it fetches forecast data, because the preview has to be computed from the same inputs to be meaningful. This is documented, together with the API-quota implication.

Since nothing is stored, the outcome is only available as the service response — result.zones carries delta, bucket, duration, current_drainage and et_deficiency per zone; all five keys are always present and are null when the module does not produce that value.

This also fixed a pre-existing bug found on the way: scheduler.py called the calculation helpers with the wrong arguments (test_scheduler_calculate_action.py covers it).

Opt-in calculation log

New calc_log.py. Two days with near-identical weather can produce very different watering volumes, and after the fact there is no way to see why — the diagnostics file shows the current state, not how it was reached.

When Calculation log is switched on in the general settings (off by default), each zone calculation appends one JSON object to config/smart_irrigation/calc_log.jsonl, in metric units, holding the whole chain:

  • identification — local and UTC timestamp, zone, sensor group, calculation module, integration version;
  • inputs — the interval used, and per field the aggregated value, the aggregation method applied, how many records went into it, their min/max, the source (sensor / weather service / static) and whether the value was carried over;
  • module intermediates — for PyETO the latitude, elevation, coastal flag, day of year, et_rad, cs_rad, sol_rad (and whether it was measured or estimated from temperature), net_in_sol_rad, avp, net_out_lw_rad, net_rad and eto per day, plus the deltas and their mean; Passthrough and Static record their (fewer) inputs the same way;
  • outputs — ET deficiency, interval multiplier, precipitation, delta, bucket before/after, max bucket, drainage rate and drainage, precipitation rate, resulting duration and volume.

Dry runs are logged too — they are exactly when one asks "why this number?" — but every record carries a dry_run flag so a dry run is never mistaken for a real calculation.

The file is capped at 2 MB and rotated (one backup kept), so it can be left on for a season. The most recent records are also included in the diagnostics download, with coordinates rounded and entity ids removed, so they can be attached to an issue in one step.

Tests and docs

New: tests/test_calc_log.py, tests/test_calculate_dry_run.py, tests/test_scheduler_calculate_action.py; test_diagnostics.py and test_panel.py extended.

Docs: a "Calculation log" section in configuration-general.md, a "Dry run" section in usage-services.md, and a "The calculated duration looks wrong" entry in usage-troubleshooting.md.

Translations updated for en, fr and hu.

Notes for the reviewer

  • The series is rebased on current dev (10241e7) and merges cleanly. It contains only the 12 feature/fix commits — the fork's own release: v2026.7.2v2026.7.4 commits and its CLAUDE.md / README changes are deliberately left out, so no version bump is included: manifest.json, const.py and frontend/src/const.ts stay at v2026.7.1 and the version is yours to bump when you release.
  • frontend/dist/smart-irrigation.js was rebuilt from source on top of this branch (npm run build), so the committed bundle matches the sources at v2026.7.1 rather than carrying the fork's build.
  • Verified locally against this branch: ruff check custom_components/smart_irrigation/ and black --check clean, and both pytest jobs from .github/workflows/pytest.yml green (217 passed in tests/, 40 in the test_helpers / test_performance / test_diagnostics selection).
  • Upstream's recent fixes are preserved — the Riemann-sum-in-days fix (Riemann sum aggregation multiplies by dt in seconds instead of days, causing massive ET0/bucket blow-up #784) and the Passthrough precipitation fix (Wrong Calculation of Irrigation time using Passthrough #790) are untouched by the calculation changes here, and their tests still pass.

frankyhun and others added 11 commits July 29, 2026 07:22
…nfo tab

The Info tab printed every duration as a raw second count, which is hard to
read as soon as a run is longer than a couple of minutes, and it never showed
how much water those durations actually represent.

- Add `formatDuration()` (hh:mm:ss, hours uncapped) and `waterVolume()`
  (duration / 60 * throughput) to helpers, plus a `ZONE_WATER_VOLUME` unit
  case so the volume is labelled L in metric and gal in imperial.
- Use hh:mm:ss for the per-zone duration, the next irrigation duration and
  the total irrigation duration.
- Show the implied water volume next to each of those durations. The totals
  sum over the enabled (automatic/manual) zones, i.e. exactly the set the
  backend uses for total_irrigation_duration, so the two stay consistent.
- Add the `water` / `total-water` labels to en.json and hu.json; the other
  languages fall back to English until translated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 937fc43)
The weather service tab only showed the current configuration, so there was
no way to tell from the panel when the service was last queried or what it
actually returned.

Add a `smart_irrigation/weatherservice_history` websocket command that walks
the sensor groups, keeps only those that take at least one value from the
weather service, and returns the most recent retrievals (newest first). Inside
a record only the weather-service-sourced values are returned: a sensor or
static value would be misleading on this tab.

The tab renders them as a table below the settings: retrieval time, the sensor
group (only when more than one takes weather data) and one column per weather
value that actually carries data, with its unit in the header. Values are shown
in metric, which is how they are stored internally. The table scrolls sideways
so a wide set of values does not break the panel on a phone.

Field labels reuse the existing, already translated mapping item keys; the new
strings are added for en, hu and fr, other languages fall back to English.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 20f161c)
The panel could show what watering was planned, but nothing about what had
actually happened: only a per-zone "last irrigation" stamp and a cumulative
water total, with no way to see individual runs or how usage moved over time.

Record one entry per credited run in the store (start time, zone id and name,
duration, water used) and expose them over a new
`smart_irrigation/irrigation_history` websocket command. The hook sits in
`_apply_volume_credit`, the single crediting path for both direct valve control
and observed watering, so every run that reaches the bucket is recorded exactly
once. Recording failures are caught and logged: history is a display
convenience and must never break crediting. Entries are pruned to 90 days and
capped at 1000, and ride along with backup/restore since they live in the
storage file.

The zone name is stored next to the id so a run keeps its label after the zone
is renamed or deleted. Records are returned raw (UTC, seconds, litres) and the
panel groups them by local calendar day, so days break where the user's clock
says they do rather than where the server's does.

The new History tab lists the runs as a table (start, zone, duration, water),
then charts total water per day over the last 30 days, then one chart per zone
that watered in that window. The charts are inline SVG: the panel ships no
charting library and 30 fixed buckets do not need one. Volumes are converted to
gallons for imperial users, matching the rest of the panel.

Direct valve control divides the zone multiplier back out when crediting the
bucket, which is right for the bucket but understates the tap: the valve was
open for the whole elapsed time. The history and the water-used total now count
that gross delivered volume, which is what the water-used sensor already
documents itself as ("cumulative water delivered to this zone").

Strings are added for en, hu and fr; other languages fall back to English.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 151e0e0)
The zone card header showed the calculated irrigation time as a raw
second count, which is hard to read once a run is longer than a couple
of minutes. Format it as hh:mm:ss and append the water volume the run
implies, reusing the formatDuration/waterVolume helpers and the unit
resolution already used on the info tab, so the sub-line now reads
"hh:mm:ss (12.3 L)" (gallons on an imperial setup).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 2ec56d8)
"ennyente" is not a Hungarian word. Reword the auto-update-interval
label to "Érzékelőadatok frissítésének gyakorisága", which reads
naturally and describes the field as a frequency.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit c239cc3)
…ndle

The panel bundle is registered at a fixed path, so a browser that already
holds it cached has no way to tell one release from the next.
cache_headers=False stops Home Assistant sending long-lived cache headers,
but it does not reach the frontend's service worker, so a desktop browser
could keep serving the previous panel after an update while the companion
app -- whose WebView cache has a different lifecycle -- showed the new one.

Append the integration version to the module URL. The URL then changes on
every release and the fetch misses the cache on its own, with no manual
cache clearing. The static path registration is untouched: routing matches
on path, so the query string does not affect it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 2d2a124)
A manual calculation during the day consumes the collected weather data and
advances the mapping's last-calculation marker. The scheduled run later that
day then only sees a partial window, and because evapotranspiration is computed
as a full-day rate from that window's statistics before being scaled by the
window length, the split is not additive: minimum and maximum temperature are
taken from the samples inside the window, so a partial window never contains
the full diurnal swing. That lowers both the vapour pressure deficit and, when
no solar radiation sensor is configured, the Hargreaves radiation estimate,
which is proportional to the square root of the temperature range. The day's
total irrigation volume ends up noticeably lower than if the scheduled run had
seen the whole day.

`delete_weather_data: false` does not fix this on its own: the last-calculation
marker is written unconditionally during aggregation, and retaining the data
while re-baselining the `delta` aggregates double-counts precipitation.

Add `dry_run` to `calculate_zone` and `calculate_all_zones`. A dry run computes
the result and returns it without writing anything: the bucket, the zone and
the collected weather data are all left untouched, and the last-calculation
marker is not advanced. Both services now declare `SupportsResponse.OPTIONAL`
so the outcome is reachable via `response_variable`, since a dry run stores
nothing to read back.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 2fe77d5)
Assert that a dry run writes nothing (last-calculation marker, zone/bucket,
collected weather data) while still aggregating and returning the result, and
that a normal run keeps its existing write behaviour.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 622a8fb)
…require

A recurring schedule with action "calculate" never calculated anything:
`_async_calculate_all()` was called without `delete_weather_data` and
`async_calculate_zone(zone_id)` without `weatherdata`, so both raised a
TypeError that the surrounding try/except only logged.

The per-zone path now gathers the data the helper expects, mirroring the
ATTR_CALCULATE branch of `async_update_zone_config`: fetch the zone and its
mapping, aggregate the sensor data, and fetch forecast data when the module is
PyETO with forecasting enabled. Zones without sensor data (or with forecasting
but no weather service) are skipped with the same error as the manual path.

Sensor data is cleared once all the requested zones have been calculated, since
zones can share a mapping and clearing it mid-loop would starve the rest.

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 74de51c)
…run in calculate_all

Two regressions in the dry-run change:

- The `ATTR_CALCULATE` branch returned the calculation result directly, which
  short-circuited the tail of `async_update_zone_config`. That skipped
  `register_start_event()` and `async_setup_observed_watering()` for *every*
  real calculation, not just dry runs, so a freshly calculated duration never
  reached the schedule. Reachable from the `calculate_zone` service, the
  per-zone Calculate button and the panel. The result is now held in a local
  and returned after the bookkeeping; a dry run still returns early, since it
  wrote nothing to re-register.

- The `ATTR_CALCULATE_ALL` branch hardcoded `delete_weather_data=True` and
  dropped `dry_run`, even though `ATTR_DRY_RUN` was added to the zone view
  schema. A caller posting `{calculate_all: true, dry_run: true}` got a
  committed calculation plus a wipe of all collected sensor data. It now
  forwards `dry_run` and returns early for a dry run.

Tests cover both paths and the unchanged real-run behaviour; the three new
regression tests fail against the previous commit.

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 4044449)
- Consolidate the "a dry run must not consume the collected data" rule. It was
  enforced in three places; each path now has exactly one guard, sitting next to
  the deletion it protects: `async_calculate_zone` for the per-zone path, and the
  bulk-clear condition in `_async_calculate_all` for the all-zones path. The
  caller-side `not dry_run and data.get(...)` in `async_update_zone_config` is
  gone, so `delete_weather_data` is now forwarded as the caller asked.

- Replace `if dry_run: return results` / `return results` in
  `_async_calculate_all` with a single `if not dry_run` around the start-event
  registration and one return.

- `_summarize_calculations` now emits every documented key, as null when the
  module did not produce it, so a template can index the response without
  guarding. The docs describe this and the zone-skipping behaviour.

- Document that a dry run still fetches forecast data for PyETO zones with
  `forecast_days > 0`, and drop the "writes nothing" overclaim. Skipping the
  fetch would make the preview diverge from the real calculation, so the call is
  intentional; the docs now say so and warn about the API quota.

- Drop the `name`/`description` from the `dry_run` entries in services.yaml.
  Home Assistant renders the translations, and the sibling `delete_weather_data`
  already relies on en.json alone.

Tests cover the bulk-clear guard in both modes plus an explicit
`delete_weather_data=False`, since removing the outer guard made the inner one
load-bearing. Verified by mutation: weakening it fails the new test.

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 383d697)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant