Skip to content

Add GOPACS integration documentation #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MartinaeyNL merged 3 commits into from
Mar 10, 2026
Merged

Add GOPACS integration documentation #14

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
073f2c9
Add GOPACS integration documentation
Miggets7 Mar 4, 2026
9ea4291
Updated as result of feedback for better readability and use English …
Miggets7 Mar 9, 2026
74bd57d
Clarify that Include GOPACS attribute only works during asset creation
Miggets7 Mar 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 124 additions & 0 deletions ems/src/main/java/org/openremote/extension/ems/manager/gopacs/GOPACS.md
View file
Open in desktop

@MartinaeyNL MartinaeyNL Mar 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to understand most of the documentation without much prior knowledge of GOPACS.
Although it might be worth taking a second look yourself @Miggets7 to make sure everything is described
correctly, and can be understood by people like me. 👍

Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# GOPACS Integration

## What is GOPACS?

[GOPACS](https://www.gopacs.eu/) (Grid Operators Platform for Congestion Solutions) is a platform operated by Dutch grid operators (DSOs and TSO) to resolve grid congestion through flexibility trading. When the electricity grid is at risk of overloading, GOPACS sends flexibility requests to market participants (aggregators) who can adjust their energy consumption or production to relieve congestion.

The communication between GOPACS and market participants uses the **UFTP** (Universal Flexibility Trading Protocol), part of the [USEF](https://www.usef.energy/) framework, implemented via the [Shapeshifter](https://github.com/shapeshifter/shapeshifter-library-java) library.

For detailed documentation, see: [GOPACS documents and manuals](https://www.gopacs.eu/en/documents-and-manuals/)

## Getting Started

To participate in GOPACS flex trading through OpenRemote, you need:

1. **A GOPACS account** — Register as a Trading Company at [gopacs.eu](https://www.gopacs.eu/)
2. **OAuth2 client credentials** (`client_id` and `client_secret`) — See [OAuth2 Client Credentials for API Clients](https://www.gopacs.eu/wp-content/uploads/2025/12/GOPACS-OAuth2-Client-credentials-for-API-Clients-03-12-2025.pdf)
3. **A signing key pair** — An Ed25519 private key file for signing UFTP messages. The corresponding public key must be registered with GOPACS
4. **A contracted EAN** — The EAN (European Article Number) identifying your grid connection point, as agreed with your DSO

### Configuration

The following environment variables must be set on the OpenRemote manager:

| Variable | Required | Description |
|---|---|---|
| `GOPACS_PRIVATE_KEY_FILE` | Yes | File path to the Ed25519 private key for signing UFTP messages |
| `GOPACS_CLIENT_ID` | Yes | OAuth2 client ID from GOPACS |
| `GOPACS_CLIENT_SECRET` | Yes | OAuth2 client secret from GOPACS |
| `GOPACS_PARTICIPANT_URL` | No | Address book base URL (default: `https://clc-message-broker.gopacs-services.eu`) |
| `GOPACS_OAUTH2_URL` | No | OAuth2 token endpoint (default: `https://auth.gopacs-services.eu/realms/gopacs/protocol/openid-connect/token`) |
| `GOPACS_RESPONSE_DELAY_SECONDS` | No | Delay before auto-responding to messages (default: `10`) |
| `GOPACS_FLEX_OFFER_DELAY_SECONDS` | No | Delay before sending a flex offer (default: `30`) |

### Asset Setup

In OpenRemote, create an **EMS GOPACS Asset** as a child of an **EMS Energy Optimisation Asset** and set the `contractedEAN` attribute to your grid connection's EAN.

Alternatively, when creating a new **EMS Energy Optimisation Asset**, you can enable the "Include GOPACS" attribute to have the GOPACS child asset created automatically. Note that this only works during initial asset creation — if the **EMS Energy Optimisation Asset** already exists, you need to manually create the **EMS GOPACS Asset** as a child.

## Developer Guide

### Components

```
gopacs/
GOPACSHandler.java Core orchestrator — handles all UFTP message processing,
signing, OAuth2 auth, and scheduling
GOPACSServerResource.java JAX-RS interface for the inbound endpoint (POST /gopacs/message)
GOPACSServerResourceImpl.java Delegates incoming XML to GOPACSHandler::processRawMessage
GOPACSAuthResource.java RESTEasy client proxy for OAuth2 token requests
GOPACSAddressBookResource.java RESTEasy client proxy for DSO participant lookup
FlexRequestISPTypeHelper.java Converts ISP numbers to timestamps (with DST handling)
OAuth2TokenResponse.java DTO for OAuth2 token responses
```

Related files outside this package:
- `agent/EmsGOPACSAsset.java` — JPA entity defining the GOPACS asset type (contracted EAN, power attributes)
- `manager/EmsOptimisationService.java` — Manages `GOPACSHandler` lifecycle (creates/destroys handlers when assets are added/removed)
- `manager/EmsOptimisationSetupService.java` — Setup class that optionally creates GOPACS assets

### Data Flow

OpenRemote acts as an **AGR (Aggregator)** in the UFTP protocol. The message exchange with the DSO (Distribution System Operator) follows this flow:

```
DSO (Grid Operator) OpenRemote (AGR)
| |
|──── 1. FlexRequest ───────────────────────>| DSO requests flexibility for a congestion point
| | (contains ISPs with max/min power limits)
| |
|<─── 2. FlexRequestResponse ────────────────| Auto-response after configurable delay
| |
|<─── 3. FlexOffer ──────────────────────────| Sent after flex offer delay
| | (mirrors request, price EUR 0.00)
| |
|──── 4. FlexOfferResponse ─────────────────>| DSO accepts or rejects the offer

@MartinaeyNL MartinaeyNL Mar 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume it should be "We accept or reject the offer from the DSO" ?
Because, from my understanding, Step 3 sends an offer, and we can either reject/accept it by "responding" in Step 4.
Then in Step 5 we can "order" based on the offer that we agreed upon earlier.

And if I misunderstand this, then the docs are not clear enough 😂

@Miggets7 Miggets7 Mar 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We send the offer, as AGR to the DSO. They accept/reject it by sending a FlexOfferResponse

@MartinaeyNL MartinaeyNL Mar 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We send the offer, as AGR to the DSO.

I have no clue what an AGR is. 😂

They accept/reject it by sending a FlexOfferResponse

Wait, then the direction of the arrows in the diagram don't make sense right?
(if they send a Response, instead of us responding)

@Miggets7 Miggets7 Mar 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it says it here:

OpenRemote acts as an AGR (Aggregator) in the UFTP protocol.

It starts with a FlexRequest from the DSO. We respond with a FlexRequestResponse
Then we send a FlexOffer to the DSO. They respond with a FlexOfferResponse
Then they send a FlexOrder and we respond back with a FlexOrderResponse

So I think the lines are correct. Maybe read it once again? #

@MartinaeyNL MartinaeyNL Mar 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OOOHHH I've indeed read it the wrong way around.
I assumed that the left column was OpenRemote, and the right one the DSO.
Maybe because I tend to think from the perspective of the OpenRemote integration.

I think I understand it now.

| |
|──── 5. FlexOrder ─────────────────────────>| DSO orders the accepted flexibility
| | (updates predicted data points on asset)
| |
|<─── 6. FlexOrderResponse ──────────────────| Auto-response confirming the order
| |
```

**How flex orders feed into optimisation:**

1. `FlexOrder` power values are written as predicted data points on the `EmsGOPACSAsset` attributes (`powerLimitMaximumProfileFlexOrder`, `powerLimitMinimumProfileFlexOrder`)
2. `EmsOptimisationService.updatePowerLimitProfileTotalForecasts()` merges these GOPACS constraints with manual power limits from the parent `EmsEnergyOptimisationAsset`
3. The combined limits are used by the optimisation methods to constrain energy scheduling

### Inbound Endpoint

The handler deploys a JAX-RS web application at `/gopacs`. Incoming signed UFTP XML messages are posted to:

```
POST /gopacs/message
Content-Type: application/xml
```

Processing steps:
1. Deserialize signed XML envelope
2. Verify cryptographic signature using the sender's public key (from address book)
3. Deserialize UFTP payload
4. Process business logic (update asset attributes, schedule data points)
5. After a delay, send the auto-response (ensures the HTTP response is returned first)

### Authentication

- **Inbound messages**: Verified using the DSO's public key, fetched from the GOPACS address book (`GET /v2/participants/DSO?contractedEan=<EAN>`) and cached in memory
- **Outbound messages**: Signed with the private key from `GOPACS_PRIVATE_KEY_FILE`, delivered with an OAuth2 Bearer token obtained via client credentials flow from the GOPACS Keycloak instance

### ISP Handling

ISPs (Imbalance Settlement Periods) are 15-minute intervals. `FlexRequestISPTypeHelper` converts ISP numbers to timestamps and includes special handling for European DST transitions (CET/CEST) on the last Sundays of March and October.

## Testing

GOPACS provides a dedicated testing environment. See [Testing UFTP API Flex Messages](https://www.gopacs.eu/wp-content/uploads/2025/12/GOPACS-Testing-receiving-and-sending-flex-messages-by-UFTP-testing-functionality-04-12-2025.pdf) for their guide on sending and receiving flex messages via the UFTP testing functionality.

For additional context on the protocol and contract types, see [Flex Trading with CSC and ATR (UFTP Messages)](https://www.gopacs.eu/wp-content/uploads/2026/02/GOPACS-Flex-trading-with-Capacity-Limiting-Contracts-using-UFTP-messages-11-02-2026.pdf).

### Company Setup for Testing

To configure your Trading Company for testing Capacity Steering Contracts, follow: [Company Settings for CSC Participation](https://www.gopacs.eu/wp-content/uploads/2025/06/GOPACS-Company-settings-for-participating-in-CSC-Capacity-Steering-Contracts.pdf)
Loading