Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
bf18c4d
use the 202 status consistently in triggering endpoints (forecasting,…
nhoening Jun 5, 2026
70913c8
let get_schedule endpoint use status 202 as well, if jobs are not fin…
nhoening Jun 5, 2026
3db2ed7
Merge branch 'main' into feat/api/use-202-status-consistenly-in-trigg…
nhoening Jun 18, 2026
d99d3db
Merge branch 'main' into feat/api/use-202-status-consistenly-in-trigg…
nhoening Jul 11, 2026
4de328d
simplify deprecation, was mostly unused
nhoening Jul 11, 2026
2b921dc
Merge branch 'main' into feat/api/use-202-status-consistenly-in-trigg…
nhoening Jul 12, 2026
f0364e6
chsnge 200 to 202 in some more tests
nhoening Jul 12, 2026
fe17212
job status emdpoint also gets 202 status (and 422 for failures); use …
nhoening Jul 14, 2026
946f996
Merge branch 'main' into feat/api/use-202-status-consistenly-in-trigg…
nhoening Jul 14, 2026
e758489
Put info about deprecated fields into the header; also react to some …
nhoening Jul 17, 2026
f5cb24d
Merge branch 'feat/api/use-202-status-consistenly-in-triggering-and-f…
nhoening Jul 17, 2026
d833490
Merge branch 'main' into feat/api/use-202-status-consistenly-in-trigg…
nhoening Jul 17, 2026
5a00aee
clarify how to signal deprecated fields in API endpoints, introducing…
nhoening Jul 20, 2026
3eba45a
Merge branch 'feat/api/use-202-status-consistenly-in-triggering-and-f…
nhoening Jul 20, 2026
20b54ba
Merge branch 'main' into feat/api/use-202-status-consistenly-in-trigg…
nhoening Jul 20, 2026
7269549
some new tests still needed to assert 202 instead of 200
nhoening Jul 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion documentation/api/change_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,20 @@ v3.0-32 | July XX, 2026
""""""""""""""""""""""""

- Extended ``GET /api/v3_0/jobs/<uuid>`` with a ``result`` field containing ``unresolved`` and ``resolved`` arrays, each keyed by asset ID. For scheduling jobs, this surfaces soft state-of-charge constraint analysis: ``soc-minima`` and ``soc-maxima`` violations (with a ``violation`` magnitude) or satisfied constraints (with a ``margin`` headroom). Both arrays are empty when no SoC constraints were defined.
- **Field canonicalization** for background job tracking:
* The ``job`` field is now the canonical way to identify background jobs returned by `/sensors/<id>/schedules/trigger`, `/assets/<id>/schedules/trigger`, and `/sensors/<id>/forecasts/trigger` endpoints. If applicable, the triggered response now also returns a ``results-url`` pointing to the sensor-specific results endpoint, alongside the generic ``job-url``.
* Legacy ``schedule`` field (in scheduling endpoints) and ``forecast`` field (in forecasting endpoints) are deprecated but remain in responses for backward compatibility. Responses containing these fields include a dated ``Deprecation`` header, a ``Link`` header pointing to migration guidance, and a ``FlexMeasures-Deprecated-Response-Fields`` header naming the deprecated fields.
- ``GET /api/v3_0/jobs/<uuid>`` now returns ``202 Accepted`` while a job is queued or running, ``422 Unprocessable Entity`` for failed jobs, and ``200 OK`` for finished jobs. See :ref:`api_background_jobs` for the response format and polling flow.
- ``GET /api/v3_0/jobs/<uuid>`` now returns kebab-case metadata fields such as ``func-name`` and ``enqueued-at``. Legacy snake_case metadata fields such as ``func_name`` and ``enqueued_at`` remain available and are signalled with a dated ``Deprecation`` header, a ``Link`` header, and a ``FlexMeasures-Deprecated-Response-Fields`` header.
- Added a ``group`` field to the storage flex-model, accepted by the `/assets/(id)/schedules/trigger <../api/v3_0.html#post--api-v3_0-assets-id-schedules-trigger>`_ (POST) endpoint, referencing a power sensor representing a group of devices (e.g. a shared inverter or feeder). The group's ``power-capacity`` is enforced as a hard constraint on the group's aggregate power, while its ``consumption-capacity``/``production-capacity`` are enforced as soft constraints with default breach prices; the group's scheduled aggregate power is saved to the group sensor.
- The ``group`` field also accepts a ``{"asset": <id>}`` reference (besides ``{"sensor": <id>}``), pointing at an asset whose own (DB-stored) flex-model defines the group's constraints. Such a group defines no power sensor of its own; its aggregate schedule is instead saved via its ``consumption``/``production`` output sensor references, following the same conventions as any other asset-only flex-model entry. This lets the entire flex-model for a device tree (including groups) live in the DB, with ``flex-model`` omitted or empty on the trigger request.

v3.0-31 | 2026-06-01
""""""""""""""""""""
- Support filtering assets and sensors by ID prefix in the ``filter`` query parameter of ``GET /assets``, ``GET /assets/<id>/sensors`` and ``GET /sensors``.

- Support filtering assets and sensors by ID prefix in the ``filter`` query parameter of ``GET /assets``, ``GET /assets/<id>/sensors`` and ``GET /sensors``.
v3.0-31 | 2026-06-01
""""""""""""""""""""
- Added a unified job status endpoint ``GET /api/v3_0/jobs/<uuid>`` that returns the current execution status and a human-readable result message for any background job (scheduling, forecasting, etc.) identified by its UUID.
- Switched from ``force_new_job_creation`` to ``force-new-job-creation`` (maintaining backwards compatibility) and added the field to `/assets/(id)/schedules/trigger` (POST) endpoint, too.
- Support both snake_case and kebab-case fields in `/sensors/<id>/data` (GET), while only documenting the kebab-case ones.
Expand Down
143 changes: 140 additions & 3 deletions documentation/api/introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,93 @@ Here is a client-side code example in Python for handling 303 redirects (this me
print(f"Failed to fetch fallback schedule: {response.status_code} {response.text}")
return response

.. _api_background_jobs:

Background job monitoring
--------------------------

Several API endpoints queue background jobs for asynchronous processing (scheduling, forecasting, data ingestion) and return a ``202 Accepted`` response.
These responses include a ``job`` field (the canonical identifier) that clients can use to monitor job progress and retrieve results.
They also include both ``job-url`` for generic status monitoring and (if applicable) ``results-url`` for the sensor-specific results endpoint.

**Example 202 Accepted response from a scheduling endpoint:**

.. code-block:: json

{
"status": "ACCEPTED",
"job": "364bfd06-c1fa-430b-8d25-8f5a547651fb",
"job-url": "/api/v3_0/jobs/364bfd06-c1fa-430b-8d25-8f5a547651fb",
"results-url": "/api/v3_0/sensors/3/schedules/364bfd06-c1fa-430b-8d25-8f5a547651fb",
"message": "Request has been accepted for processing."
}

**Monitoring job status:**

Clients should use the ``job.id`` to query the unified job status endpoint:

.. code-block:: bash

GET /api/v3_0/jobs/<job-id>

This returns the current execution status and a human-readable result message. For example:

.. code-block:: python

import requests
import time

def wait_for_job(job_id, job_url, timeout=300, poll_interval=5):
"""Wait for a background job to complete and return the result.

Parameters
----------
job_id : str
The UUID of the background job, we use it for logging here..
job_url : str
The URL to query for job status (e.g., "/api/v3_0/jobs/<uuid>").
timeout : int
Maximum seconds to wait for job completion.
poll_interval : int
Seconds between status checks.
"""
start_time = time.time()
while time.time() - start_time < timeout:
response = requests.get(job_url)
if response.status_code not in (200, 202, 422):
raise RuntimeError(
f"Failed to query job status: {response.status_code} {response.text}"
)

job_data = response.json()
status = job_data.get("status")

if response.status_code == 202:
print(f"Job {job_id} is still {status.lower()}...")
time.sleep(poll_interval)
elif status == "FINISHED":
return job_data.get("result")
else: # Failed, error, etc.
raise RuntimeError(f"Job failed with status {status}: {job_data.get('message')}")

raise TimeoutError(f"Job {job_id} did not complete within {timeout} seconds")

.. note::

For **schedules**, after the job completes successfully, use the job ID (same value as the legacy ``schedule`` field) to retrieve the actual schedule or follow the returned ``results-url``:

.. code-block:: bash

GET /api/v3_0/sensors/<sensor_id>/schedules/<job-id>

For **forecasts**, after the job completes successfully, use the job ID to retrieve the forecast or follow the returned ``results-url``:

.. code-block:: bash

GET /api/v3_0/sensors/<sensor_id>/forecasts/<job-id>

Both of these endpoints will also return `202 Accepted` if the job is still being computed, so clients can continue to poll them directly if they prefer.
Comment thread
nhoening marked this conversation as resolved.

.. _api_deprecation:

Deprecation and sunset
Expand All @@ -166,12 +253,62 @@ For more information on our multi-stage deprecation approach and available optio

.. _api_deprecation_clients:

Clients
^^^^^^^
Deprecated response fields
^^^^^^^^^^^^^^^^^^^^^^^^^^^

In addition to deprecating entire endpoints, we sometimes deprecate individual fields in API responses while maintaining backward compatibility by including both the legacy and canonical fields.
Comment thread
nhoening marked this conversation as resolved.
When this happens, responses include a ``Deprecation`` response header with the deprecation date, a ``Link`` header pointing to migration guidance, and a ``FlexMeasures-Deprecated-Response-Fields`` header identifying the deprecated fields in that response.

For example, when scheduling endpoints switched from ``schedule`` to ``job`` as the canonical field identifier for background jobs, a response that still contains the legacy ``schedule`` field also carries deprecation headers:

.. code-block:: http

HTTP/1.1 202 Accepted
Deprecation: Wed, 01 Jul 2026 00:00:00 GMT
Link: <https://flexmeasures.readthedocs.io/latest/api/v3_0.html#post--api-v3_0-sensors-id-schedules-trigger>; rel="deprecation"; type="text/html"
FlexMeasures-Deprecated-Response-Fields: schedule
Content-Type: application/json

{
"status": "ACCEPTED",
"job": "364bfd06-c1fa-430b-8d25-8f5a547651fb",
"schedule": "364bfd06-c1fa-430b-8d25-8f5a547651fb",
"job-url": "/api/v3_0/jobs/364bfd06-c1fa-430b-8d25-8f5a547651fb",
"results-url": "/api/v3_0/sensors/3/schedules/364bfd06-c1fa-430b-8d25-8f5a547651fb"
}

In this example, clients should treat ``job`` as the canonical field and ``schedule`` as a backward-compatible alias.
The ``FlexMeasures-Deprecated-Response-Fields`` header tells clients which response fields are deprecated, the ``Deprecation`` header tells clients when they were deprecated, and the ``Link`` header points to migration guidance for this endpoint.

Professional API users should monitor API responses for the ``"Deprecation"`` and ``"Sunset"`` response headers [see `draft-ietf-httpapi-deprecation-header-02 <https://datatracker.ietf.org/doc/draft-ietf-httpapi-deprecation-header/>`_ and `RFC 8594 <https://www.rfc-editor.org/rfc/rfc8594>`_, respectively], so system administrators can be warned when using API endpoints that are flagged for deprecation and/or are likely to become unresponsive in the future.
The ``Deprecation`` header may describe either the endpoint itself or specific response fields.
Clients can tell the difference by checking for ``FlexMeasures-Deprecated-Response-Fields``: if it is present, only the named response fields are deprecated; if it is absent, the deprecation applies to the endpoint or API version as a whole.

For deprecated response fields, clients should:

- Monitor the ``Deprecation`` response header to detect deprecated API behavior.
- Use the ``FlexMeasures-Deprecated-Response-Fields`` header to identify which response fields are deprecated.
- Follow the ``Link`` response header to find migration guidance.
- Migrate to use the canonical field names documented in the API schema.
- Plan upgrades based on the deprecation guidance to avoid breakage when deprecated fields are eventually removed in a future API version.

Client code should therefore inspect both headers and body fields, for example:

.. code-block:: python

response = requests.post(trigger_url, json=payload, headers=headers)
response.raise_for_status()

deprecated_fields = response.headers.get("FlexMeasures-Deprecated-Response-Fields")
if deprecated_fields:
print(f"Deprecated response fields detected: {deprecated_fields}")
print(f"See {response.headers.get('Link')}")

data = response.json()
job_id = data["job"]

The deprecation header field shows an `IMF-fixdate <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.1.1>`_ indicating when the API endpoint was deprecated.
For endpoint deprecations, the deprecation header field shows an `IMF-fixdate <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.1.1>`_ indicating when the API endpoint was deprecated.
For deprecated response fields, the deprecation header field also shows an IMF-fixdate, while the presence of the ``FlexMeasures-Deprecated-Response-Fields`` header narrows the deprecation to the named response fields.
The sunset header field shows an `IMF-fixdate <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.1.1>`_ indicating when the API endpoint is likely to become unresponsive.

More information about a deprecation, sunset, and possibly recommended replacements, can be found under the ``"Link"`` response header. Relevant relations are:
Expand Down
3 changes: 3 additions & 0 deletions documentation/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ FlexMeasures Changelog
v1.0.0 | July XX, 2026
============================

.. warning:: As of this release we standardize asynchronous job responses to use the ``job`` field and return HTTP ``202 Accepted`` while a background job is queued or running. See :ref:`api_background_jobs` for the response format and polling flow. Legacy response fields such as ``schedule`` and ``forecast`` will be deprecated; clients should migrate to ``job`` (see the Infrastructure / Support section below for migration details).

.. warning:: Upgrading to this version requires running ``flexmeasures db upgrade`` (you can create a backup first with ``flexmeasures db-ops dump``).

New features
Expand Down Expand Up @@ -36,6 +38,7 @@ Infrastructure / Support
* ``flexmeasures db upgrade`` now runs ``VACUUM ANALYZE`` after upgrading by default, so Postgres has fresh planner statistics right after a migration; opt out with ``--no-vacuum`` [see `PR #2333 <https://www.github.com/FlexMeasures/flexmeasures/pull/2333>`_]
* Upgraded dependencies [see `PR #1485 <https://www.github.com/FlexMeasures/flexmeasures/pull/1485>`_, `PR #2215 <https://www.github.com/FlexMeasures/flexmeasures/pull/2215>`_ and `PR #2243 <https://www.github.com/FlexMeasures/flexmeasures/pull/2243>`_]
* Prepare the ``device_scheduler`` to deal with commitments per device group [see `PR #1934 <https://www.github.com/FlexMeasures/flexmeasures/pull/1934>`_]
* Standardize job-trigger API responses to return ``202 Accepted`` and a canonical ``job`` field; legacy response fields such as ``schedule`` and ``forecast`` are preserved for backward-compatibility but marked deprecated with migration guidance in :ref:`api_background_jobs` [see `PR #2224 <https://github.com/FlexMeasures/flexmeasures/pull/2224>`_].
* Speed up scheduling on longer horizons using a recursive stock model, making the solve time linear with the horizon instead of quadratic (about 10x faster solves at the 2-day default horizon, 23x at 3 days) [see `PR #2282 <https://www.github.com/FlexMeasures/flexmeasures/pull/2282>`_]
* Support storing encrypted connection secrets on organisations and assets, including utility functions, encryption key configuration, CLI commands to set and delete secrets, and UI tables that show stored secret names and optional expiration times without exposing their values [see `PR #2236 <https://www.github.com/FlexMeasures/flexmeasures/pull/2236>`_]
* Documentation section on the modelling choice for recording measurements, forecasts and schedules under one or multiple sensors [see `PR #2217 <https://www.github.com/FlexMeasures/flexmeasures/pull/2217>`_]
Expand Down
4 changes: 4 additions & 0 deletions documentation/tut/toy-example-expanded.rst
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,10 @@ This will have an effect on the available headroom for the battery, given the ``
}
}

.. note::
The API returns a **202 Accepted** response containing a ``job`` field.
Use this UUID to monitor the job status via :ref:`api_background_jobs`.

.. tab:: FlexMeasures Client

Using the `FlexMeasures Client <https://pypi.org/project/flexmeasures-client/>`_:
Expand Down
14 changes: 14 additions & 0 deletions documentation/tut/toy-example-from-scratch.rst
Comment thread
Flix6x marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,20 @@ There is more information being used by the scheduler, such as the battery's cap
}

.. note:: You can try this right in Swagger UI, too! You should find it at `http://localhost:5000/api/v3_0/docs <http://localhost:5000/api/v3_0/docs>`_ after starting FlexMeasures locally.

The API returns a **202 Accepted** response with a ``job`` field to track the scheduling job:

.. code-block:: json

{
"status": "ACCEPTED",
"job": "364bfd06-c1fa-430b-8d25-8f5a547651fb",
"schedule": "364bfd06-c1fa-430b-8d25-8f5a547651fb",
"job-url": "/api/v3_0/jobs/364bfd06-c1fa-430b-8d25-8f5a547651fb",
"results-url": "/api/v3_0/sensors/2/schedules/364bfd06-c1fa-430b-8d25-8f5a547651fb"
}

**Important:** Use the ``job`` field to reference the scheduling job. You can also follow the returned ``results-url`` to fetch the schedule results once they are ready. For more details, see :ref:`api_background_jobs`.

.. tab:: FlexMeasures Client

Expand Down
64 changes: 55 additions & 9 deletions flexmeasures/api/common/responses.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,34 @@
ResponseTuple = tuple[dict, int] | tuple[dict, int, dict]


DEPRECATED_RESPONSE_FIELDS_HEADER = "FlexMeasures-Deprecated-Response-Fields"


def deprecated_response_fields_headers(
fields: Sequence[str],
deprecation_link: str,
deprecation_date: str,
) -> dict[str, str]:
"""Headers used when a response contains deprecated fields."""
return {
"Deprecation": deprecation_date,
"Link": f'<{deprecation_link}>; rel="deprecation"; type="text/html"',
DEPRECATED_RESPONSE_FIELDS_HEADER: ", ".join(fields),
}


def is_response_tuple(value) -> bool:
"""Check if an object qualifies as a ResponseTuple"""
if not isinstance(value, tuple):
return False
if not len(value) == 2:
if len(value) not in (2, 3):
return False
if not isinstance(value[0], dict):
return False
if not isinstance(value[1], int):
return False
if len(value) == 3 and not isinstance(value[2], dict):
return False
return True


Expand Down Expand Up @@ -383,16 +401,44 @@ def request_processed(message: str) -> ResponseTuple:
def request_accepted_for_processing(
job_id: str,
message: str = "Request has been accepted for processing.",
legacy_key: str | None = None,
job_results_url: str | None = None,
deprecation_link: str | None = None,
deprecation_date: str | None = None,
) -> ResponseTuple:
return (
dict(
status="ACCEPTED",
message=message,
job_monitor_url=url_for("JobAPI:get_job_status", uuid=job_id),
job_id=job_id,
),
202,
"""
Standard 202 response when a background job is accepted.

Optional backwards-compatibility: if `legacy_key` is provided the response
will include the job id under that legacy key (e.g. `schedule` or
`forecast`) and response headers marking the field deprecation.

Optional `job_results_url` may be supplied to provide a direct link to the
sensor-specific job results endpoint, e.g. `/api/v3_0/sensors/<id>/schedules/<uuid>`.
"""
resp: dict[str, object] = dict(
status="ACCEPTED",
message=message,
job=job_id,
)
resp["job-url"] = url_for("JobAPI:get_job_status", uuid=job_id)
if job_results_url:
resp["results-url"] = job_results_url

if legacy_key:
# keep legacy key for backwards compatibility
resp[legacy_key] = job_id
assert deprecation_link is not None
assert deprecation_date is not None
return (
resp,
202,
deprecated_response_fields_headers(
[legacy_key], deprecation_link, deprecation_date
),
)

return resp, 202


def request_too_large(message: str) -> ResponseTuple:
Expand Down
Loading
Loading