Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions documentation/api/change_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ v3.0-32 | July XX, 2026
- Extended ``GET /api/v3_0/jobs/<uuid>`` with a ``result`` field containing ``unresolved`` and ``resolved`` arrays, each keyed by asset ID. For scheduling jobs, this surfaces soft state-of-charge constraint analysis: ``soc-minima`` and ``soc-maxima`` violations (with a ``violation`` magnitude) or satisfied constraints (with a ``margin`` headroom). Both arrays are empty when no SoC constraints were defined.
- Added a ``group`` field to the storage flex-model, accepted by the `/assets/(id)/schedules/trigger <../api/v3_0.html#post--api-v3_0-assets-id-schedules-trigger>`_ (POST) endpoint, referencing a power sensor representing a group of devices (e.g. a shared inverter or feeder). The group's ``power-capacity`` is enforced as a hard constraint on the group's aggregate power, while its ``consumption-capacity``/``production-capacity`` are enforced as soft constraints with default breach prices; the group's scheduled aggregate power is saved to the group sensor.
- The ``group`` field also accepts a ``{"asset": <id>}`` reference (besides ``{"sensor": <id>}``), pointing at an asset whose own (DB-stored) flex-model defines the group's constraints. Such a group defines no power sensor of its own; its aggregate schedule is instead saved via its ``consumption``/``production`` output sensor references, following the same conventions as any other asset-only flex-model entry. This lets the entire flex-model for a device tree (including groups) live in the DB, with ``flex-model`` omitted or empty on the trigger request.
- Fixed: `/assets/(id)/schedules/trigger <../api/v3_0.html#post--api-v3_0-assets-id-schedules-trigger>`_ (POST) now also accepts the legacy ``force_new_job_creation`` field name (in addition to ``force-new-job-creation``), matching the sensor-level trigger endpoint. Previously, only the sensor-level endpoint accepted both spellings.
- Several query parameters now additionally accept a canonical spelling (hyphenated where applicable), while the legacy underscored spelling keeps working: ``per_page``/``per-page``, ``sort_by``/``sort-by``, ``sort_dir``/``sort-dir`` (all paginated list endpoints), ``account_id``/``account`` and ``asset_id``/``asset`` (``GET /sensors``, ``GET /assets``, ``GET /users``). New clients should prefer the canonical form.

v3.0-31 | 2026-06-01
""""""""""""""""""""
Expand Down
10 changes: 9 additions & 1 deletion flexmeasures/api/common/schemas/assets.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,13 @@ def _serialize(self, values: list[str], attr, obj, **kwargs) -> str:


class AssetAPIQuerySchema(PaginationSchema):
legacy_field_aliases = {
**PaginationSchema.legacy_field_aliases,
"account_id": "account",
}

sort_by = fields.Str(
data_key="sort-by",
required=False,
validate=validate.OneOf(["id", "name", "owner"]),
metadata=dict(
Expand All @@ -55,7 +61,7 @@ class AssetAPIQuerySchema(PaginationSchema):
),
)
account = AccountIdField(
data_key="account_id",
data_key="account",
required=False,
metadata=dict(
description="Select assets from a given account (requires read access to that account). Per default, the user's own account is used.",
Expand Down Expand Up @@ -118,10 +124,12 @@ class PublicAssetAPISchema(Schema):

class AssetPaginationSchema(PaginationSchema):
sort_by = fields.Str(
data_key="sort-by",
required=False,
validate=validate.OneOf(["id", "name", "resolution"]),
)
sort_dir = fields.Str(
data_key="sort-dir",
required=False,
validate=validate.OneOf(["asc", "desc"]),
)
16 changes: 14 additions & 2 deletions flexmeasures/api/common/schemas/generic_schemas.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,22 @@
from marshmallow import Schema, fields, validate
from flexmeasures.api.common.schemas.search import SearchFilterField
from flexmeasures.api.common.schemas.utils import SupportsLegacyFieldAliases


class PaginationSchema(Schema):
class PaginationSchema(SupportsLegacyFieldAliases, Schema):
legacy_field_aliases = {
"per_page": "per-page",
"sort_by": "sort-by",
"sort_dir": "sort-dir",
}

# note: the absence of this parameter would signal to the API to not paginate (so there is no default set here)
page = fields.Int(required=False, validate=validate.Range(min=1))
per_page = fields.Int(
required=False, validate=validate.Range(min=1), load_default=10
data_key="per-page",
required=False,
validate=validate.Range(min=1),
load_default=10,
)
filter = SearchFilterField(
required=False,
Expand All @@ -15,12 +25,14 @@ class PaginationSchema(Schema):
),
)
sort_by = fields.Str(
data_key="sort-by",
required=False,
metadata=dict(
description="Sort results by this field.",
),
)
sort_dir = fields.Str(
data_key="sort-dir",
required=False,
validate=validate.OneOf(["asc", "desc"]),
metadata=dict(
Expand Down
1 change: 1 addition & 0 deletions flexmeasures/api/common/schemas/users.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ def _serialize(self, value: User, attr, obj, **kwargs) -> int:

class AccountAPIQuerySchema(PaginationSchema):
sort_by = fields.Str(
data_key="sort-by",
required=False,
validate=validate.OneOf(["id", "name", "assets", "users"]),
)
1 change: 1 addition & 0 deletions flexmeasures/api/common/schemas/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
VariableQuantityField,
VariableQuantityOpenAPISchema,
)
from flexmeasures.data.schemas.utils import SupportsLegacyFieldAliases # noqa: F401


def make_openapi_compatible( # noqa: C901
Expand Down
20 changes: 19 additions & 1 deletion flexmeasures/api/v3_0/assets.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@
SensorsToShowSchema,
)
from flexmeasures.data.schemas.scheduling import AssetTriggerSchema
from flexmeasures.data.schemas.utils import SupportsLegacyFieldAliases
from flexmeasures.data.services.scheduling import (
create_sequential_scheduling_job,
create_simultaneous_scheduling_job,
Expand Down Expand Up @@ -99,6 +100,22 @@ def sensor_term_filter(term: str):
return or_(*filters)


class AssetTriggerSchemaV3(SupportsLegacyFieldAliases, AssetTriggerSchema):
"""v3_0-only wrapper around the shared `AssetTriggerSchema`.

`AssetTriggerSchema` itself stays canonical (no legacy field-name
aliasing) since it's also used outside the versioned API, e.g. by the
CLI. This subclass adds v3_0's backward compatibility for the legacy
`force_new_job_creation` field name, so it can be deleted in one place,
along with the rest of `flexmeasures/api/v3_0/`, once this API version is
sunset -- without needing to touch the shared domain schema.
"""

legacy_field_aliases = {
"force_new_job_creation": "force-new-job-creation",
}


class AssetTriggerOpenAPISchema(AssetTriggerSchema):

def __init__(self, *args, **kwargs):
Expand Down Expand Up @@ -153,6 +170,7 @@ class AssetChartDataKwargsSchema(Schema):

class AssetAuditLogPaginationSchema(PaginationSchema):
sort_by = fields.Str(
data_key="sort-by",
required=False,
validate=validate.OneOf(["event_datetime"]),
)
Expand Down Expand Up @@ -1427,7 +1445,7 @@ def update_keep_legends_below_graphs(self, **kwargs):
}, 200

@route("/<id>/schedules/trigger", methods=["POST"])
@use_args(AssetTriggerSchema(), location="args_and_json", as_kwargs=True)
@use_args(AssetTriggerSchemaV3(), location="args_and_json", as_kwargs=True)
# Simplification of checking for create-children access on each of the flexible sensors,
# which assumes each of the flexible sensors belongs to the given asset.
@permission_required_for_context("create-children", ctx_arg_name="asset")
Expand Down
38 changes: 23 additions & 15 deletions flexmeasures/api/v3_0/sensors.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
from flask_classful import FlaskView, route
from flask_json import as_json
from flask_security import auth_required, current_user
from marshmallow import fields, pre_load, Schema, ValidationError, validates_schema
from marshmallow import fields, Schema, ValidationError, validates_schema
import marshmallow.validate as validate
from rq.job import Job, JobStatus, NoSuchJobError
from webargs.flaskparser import use_args, use_kwargs
Expand All @@ -26,7 +26,10 @@
unprocessable_entity,
fallback_schedule_redirect,
)
from flexmeasures.api.common.schemas.utils import make_openapi_compatible
from flexmeasures.api.common.schemas.utils import (
make_openapi_compatible,
SupportsLegacyFieldAliases,
)
from flexmeasures.api.common.schemas.sensor_data import ( # noqa F401
SensorDataDescriptionSchema,
GetSensorDataSchema,
Expand Down Expand Up @@ -220,14 +223,23 @@ def validate_time_window(self, data, **kwargs):
)


class SensorKwargsSchema(Schema):
account = AccountIdField(data_key="account_id", required=False)
asset = AssetIdField(data_key="asset_id", required=False)
class SensorKwargsSchema(SupportsLegacyFieldAliases, Schema):
legacy_field_aliases = {
"account_id": "account",
"asset_id": "asset",
"per_page": "per-page",
}

account = AccountIdField(data_key="account", required=False)
asset = AssetIdField(data_key="asset", required=False)
include_consultancy_clients = fields.Boolean(required=False, load_default=False)
include_public_assets = fields.Boolean(required=False, load_default=False)
page = fields.Int(required=False, validate=validate.Range(min=1))
per_page = fields.Int(
required=False, validate=validate.Range(min=1), load_default=10
data_key="per-page",
required=False,
validate=validate.Range(min=1),
load_default=10,
)
filter = SearchFilterField(
required=False,
Expand All @@ -238,7 +250,11 @@ class SensorKwargsSchema(Schema):
unit = UnitField(required=False)


class TriggerScheduleKwargsSchema(Schema):
class TriggerScheduleKwargsSchema(SupportsLegacyFieldAliases, Schema):
legacy_field_aliases = {
"force_new_job_creation": "force-new-job-creation",
}

start_of_schedule = AwareDateTimeField(
data_key="start",
format="iso",
Expand Down Expand Up @@ -295,14 +311,6 @@ class TriggerScheduleKwargsSchema(Schema):
),
)

@pre_load
def support_legacy_field_name(self, data, **kwargs):
"""Accept old snake_case input for backwards compatibility."""
if "force_new_job_creation" in data and "force-new-job-creation" not in data:
data["force-new-job-creation"] = data.pop("force_new_job_creation")

return data


class SensorAPI(FlaskView):
route_base = "/sensors"
Expand Down
80 changes: 80 additions & 0 deletions flexmeasures/api/v3_0/tests/test_asset_trigger_schema_v3.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
"""Tests for AssetTriggerSchemaV3, the v3_0-only wrapper that adds legacy
field-name backward compatibility on top of the shared, canonical
AssetTriggerSchema (see flexmeasures/api/v3_0/assets.py for why this split
exists: sunsetting v3_0 should be able to delete this compatibility layer in
one place, without touching the domain schema also used by the CLI).
"""

from marshmallow.validate import ValidationError
import pytest
from werkzeug.datastructures import MultiDict

from flexmeasures.api.v3_0.assets import AssetTriggerSchemaV3


def test_asset_trigger_schema_v3_accepts_legacy_force_new_job_creation_field():
"""Regression test for a gap that made flexmeasures-client#218 necessary: the
sensor-level scheduling trigger schema already accepted both
`force_new_job_creation` (legacy) and `force-new-job-creation` (canonical),
but the asset-level trigger schema only accepted the canonical spelling and
rejected the legacy one (as an unknown field, yielding a 422).
"""
schema = AssetTriggerSchemaV3()
normalized = schema._apply_legacy_field_aliases({"force_new_job_creation": True})
assert normalized == {"force-new-job-creation": True}


def test_asset_trigger_schema_v3_load_accepts_legacy_force_new_job_creation_field(
db, app
):
"""Same regression as above, but exercised through the real `schema.load(...)`
deserialization path (not by calling the `@pre_load` helper directly), so
this fails if the hook ever stops being registered/applied by Marshmallow
(e.g. decorator removed, or an MRO change means the hook is no longer
picked up).

Uses a nonexistent asset id, so `load()` is expected to still raise -- but
only about the asset, never about `force_new_job_creation` being an
unrecognized field. If the legacy alias stopped working, Marshmallow's
default `unknown` handling would additionally report `force_new_job_creation`
as an unknown field.
"""
schema = AssetTriggerSchemaV3()
with pytest.raises(ValidationError) as e_info:
schema.load(
{
"id": 2**31 - 1, # some asset id that doesn't exist
"start": "2026-01-15T10:00:00+01:00",
"force_new_job_creation": True, # legacy spelling
}
)
messages = e_info.value.messages
assert "force_new_job_creation" not in messages, (
"the legacy field name should have been aliased to "
"`force-new-job-creation` before validation, not rejected as an "
f"unknown field; got: {messages}"
)
assert (
"id" in messages
), f"expected the (nonexistent) asset id to fail; got: {messages}"


def test_asset_trigger_schema_v3_preserves_multidict_when_aliasing():
"""Regression test: aliasing a legacy field must not destroy MultiDict
semantics (e.g. `getlist`) for other, untouched keys -- this is relied on
by `AssetTriggerSchema.normalize_flex_context_format` to detect a
multi-commodity `flex-context` list sent as repeated keys.
"""
schema = AssetTriggerSchemaV3()
data = MultiDict(
[
("start", "2026-01-15T10:00+01:00"),
("force_new_job_creation", True),
("flex-context", "electricity"),
("flex-context", "heat"),
]
)
normalized = schema._apply_legacy_field_aliases(data)
assert normalized.getlist("flex-context") == ["electricity", "heat"]
assert normalized["force-new-job-creation"] is True
assert "force_new_job_creation" not in normalized
9 changes: 8 additions & 1 deletion flexmeasures/api/v3_0/users.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ class AuthRequestSchema(Schema):

class UserAuditlogSchema(PaginationSchema):
sort_by = fields.Str(
data_key="sort-by",
required=False,
validate=validate.OneOf(
["username", "email", "event_datetime", "lastLogin", "lastSeen"]
Expand All @@ -61,11 +62,17 @@ class UserAuditlogSchema(PaginationSchema):


class UserAPIQuerySchema(PaginationSchema):
legacy_field_aliases = {
**PaginationSchema.legacy_field_aliases,
"account_id": "account",
}

sort_by = fields.Str(
data_key="sort-by",
required=False,
validate=validate.OneOf(["username", "email", "lastLogin", "lastSeen"]),
)
account = AccountIdField(data_key="account_id", required=False)
account = AccountIdField(data_key="account", required=False)
include_inactive = fields.Bool(load_default=False)


Expand Down
8 changes: 8 additions & 0 deletions flexmeasures/data/schemas/scheduling/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -1513,6 +1513,14 @@ class AssetTriggerSchema(Schema):
},
]
}

This schema is also used outside of the (versioned) API, e.g. by the CLI,
so it stays canonical (no legacy field-name aliasing here). API-version-
specific backward compatibility, such as accepting the legacy
`force_new_job_creation` field name, is layered on top in
`flexmeasures/api/v3_0/assets.py` (`AssetTriggerSchemaV3`), so it can be
deleted in one place once v3_0 is sunset, without touching this shared
domain schema.
"""

asset = GenericAssetIdField(
Expand Down
8 changes: 8 additions & 0 deletions flexmeasures/data/schemas/tests/test_scheduling.py

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a question about moving on.
When at some day API v3 is sunset, then these older remnants of old field names will still linger in schemas. Should we plan on cleaning them out, as well?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, and it's a real risk, not just a hypothetical — thanks for flagging it.

Short answer: schemas shouldn't be versioned wholesale, but version-specific compatibility shims need to live in the versioned API layer, not in shared domain schemas.

Most of flexmeasures/data/schemas/ isn't API-only — it's reused directly by the CLI and the scheduler internals (FlexContextSchema, StorageFlexModelSchema, AssetTriggerSchema, etc.). Duplicating that whole layer per API version would mean either heavy code duplication, or forcing non-API callers like the CLI to somehow pick an "API version" too, which doesn't make sense.

So the right split is:

  • Domain schemas (flexmeasures/data/schemas/*) stay canonical and version-agnostic: always just the current, correct field names, no compatibility cruft.
  • Version-specific wire-format concerns — legacy field aliases on the way in, dual field names on the way out — belong in flexmeasures/api/v3_0/ (or a thin adapter it owns).

That placement is what makes sunset cleanup free: when v3_0 is retired, we already delete/replace the whole api/v3_0/ package (same mechanism as the existing v1/v2_0 sunset blueprints in flexmeasures/api/sunset/__init__.py). If every v3-only compatibility shim lives inside api/v3_0/, deleting that package deletes all the cruft in one motion — nothing left to hunt down in the domain layer years later.

Concretely, on this PR: TriggerScheduleKwargsSchema (sensor-level trigger) was already scoped correctly — its alias lives in flexmeasures/api/v3_0/sensors.py. But I'd initially put AssetTriggerSchema's alias directly in the shared domain module, which was exactly the kind of permanent lingering you're describing. Fixed now: added AssetTriggerSchemaV3, a thin v3_0-only subclass in flexmeasures/api/v3_0/assets.py that adds the legacy force_new_job_creation alias, while AssetTriggerSchema itself stays canonical (and CLI-safe). Tests moved to flexmeasures/api/v3_0/tests/test_asset_trigger_schema_v3.py accordingly.

Original file line number Diff line number Diff line change
Expand Up @@ -1455,3 +1455,11 @@ def test_asset_trigger_schema_rejects_malformed_flex_context(app):
with pytest.raises(ValidationError) as e_info:
schema.normalize_flex_context_format({"flex-context": "not-a-dict-or-list"})
assert "flex-context" in str(e_info.value)


# Note: AssetTriggerSchema itself no longer aliases legacy field names (e.g.
# force_new_job_creation) -- that's v3_0-specific backward compatibility,
# layered on top by AssetTriggerSchemaV3 in flexmeasures/api/v3_0/assets.py,
# and tested there (flexmeasures/api/v3_0/tests/test_asset_trigger_schema_v3.py).
# This schema stays canonical since it's also used outside the API, e.g. by
# the CLI.
Loading
Loading