From 3aa76c5881defcb8aaffae3228a71ce14aa7be4f Mon Sep 17 00:00:00 2001 From: Ari Sulistiono Date: Sat, 24 Oct 2026 18:20:26 +0700 Subject: [PATCH 1/2] docs(public): consolidate user-first ARStack onboarding, community paths and M3.H product presentation --- .github/ISSUE_TEMPLATE/bug_report.yml | 62 +-- .github/ISSUE_TEMPLATE/interoperability.yml | 38 ++ CONTRIBUTING.md | 33 +- README.md | 76 ++-- SUPPORT.md | 19 + apps/arstack_studio/README.md | 160 ++------ docs/COMMUNITY_TESTING.md | 31 ++ docs/DEVELOPER_GUIDE.md | 368 +----------------- docs/FEATURE_MATRIX.md | 10 +- docs/GETTING_STARTED.md | 72 ++++ ...OCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md | 91 +++-- index.html | 8 +- scripts/check_product_naming.py | 24 +- 13 files changed, 377 insertions(+), 615 deletions(-) create mode 100644 .github/ISSUE_TEMPLATE/interoperability.yml create mode 100644 SUPPORT.md create mode 100644 docs/COMMUNITY_TESTING.md create mode 100644 docs/GETTING_STARTED.md diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index c59ca6485..6e87653b5 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -1,46 +1,52 @@ name: Bug report -description: Report a reproducible ARStack application, protocol or device issue +description: Report a problem with ARStack Studio, board connection, SCL import or an engineering tool title: "Bug: " +labels: [] body: - type: markdown attributes: - value: "Please avoid posting private station configurations, credentials, or customer packet captures. For security vulnerabilities, use the private reporting instructions in SECURITY.md." - - type: dropdown - id: area - attributes: - label: Component - options: - - ARStack Studio (Windows or Linux) - - ESP32-P4 SV / PTP - - IED Browser / Workbench - - IED Simulator - - Build / installation - - Other - validations: - required: true + value: | + Thanks for testing ARStack! Short reports are welcome. Please remove confidential details from screenshots and configuration files. - type: textarea - id: repro + id: problem attributes: - label: How to reproduce - description: Steps and input settings sufficient to reproduce in a lab - placeholder: "1. Open Studio... 2. Connect... 3. Observe..." + label: What happened? + description: Tell us what you tried and what went wrong. + placeholder: "I opened a .scd file, but the selected SV stream was not shown." validations: required: true - type: textarea - id: observed + id: expected attributes: - label: Expected and actual behavior - placeholder: "Expected: ... Actual: ..." + label: What did you expect? + placeholder: "I expected to see the SV ID and multicast address." validations: - required: true + required: false - type: input id: version attributes: - label: Version and environment - description: Release tag, Windows/Linux version and ESP32-P4 firmware Build ID, if relevant - placeholder: "M3.H; Windows 11; firmware a922e9bdba4e57a4" + label: ARStack version + description: See Help / About or the downloaded release. + placeholder: "M3.H" + validations: + required: false + - type: dropdown + id: area + attributes: + label: Where did it happen? + options: + - ARStack Studio + - ESP32-P4 firmware / device connection + - SCL or engineering file import + - SV / PTP traffic + - Build / installation + - Other + validations: + required: false - type: textarea id: evidence attributes: - label: Sanitized evidence (optional) - description: Logs, screenshots or short anonymized captures; never secrets or customer-identifying files + label: Screenshot, logs or sample file (optional) + description: You can drag a screenshot or sanitized sample here. + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/interoperability.yml b/.github/ISSUE_TEMPLATE/interoperability.yml new file mode 100644 index 000000000..8b4bed906 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/interoperability.yml @@ -0,0 +1,38 @@ +name: Community interoperability experiment +description: Share what you observed in an isolated IEC 61850 Process Bus test +title: "Community test: " +labels: [] +body: + - type: markdown + attributes: + value: | + Thanks for sharing an experiment! This form is intentionally short. Results can be successful, partial or unexpected. Never upload private substation configurations. + - type: textarea + id: setup + attributes: + label: Your lab setup + description: Which ARStack release, firmware, analyzer or IEC 61850 subscriber did you use? + placeholder: "M3.H Studio + ESP32-P4 → lab switch → Wireshark / subscriber" + validations: + required: true + - type: textarea + id: configuration + attributes: + label: SV and PTP configuration + placeholder: "4I+4V / 4 kHz, APPID, svID, PTP source mode and relevant settings" + validations: + required: false + - type: textarea + id: observation + attributes: + label: What did you observe? + description: What worked? What was unexpected? + validations: + required: true + - type: textarea + id: evidence + attributes: + label: Optional capture or screenshot + description: Attach sanitized PCAP, logs or images only if you have permission to share them. + validations: + required: false diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 439e3ced1..b50b33e90 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,27 +1,30 @@ # Contributing to ARStack61850 -ARStack61850 is an open-source IEC 61850 engineering toolkit. Improvements to interoperability, usability, timing evidence, tests and documentation are welcome. +Thanks for helping make IEC 61850 experimentation accessible to more engineers. + +You can contribute without writing C++: improve a tutorial, report a reproducible issue, translate an explanation, contribute an example, or share results from an isolated lab setup. ## Start here -- **Using the software?** Follow the [installation guide](docs/INSTALL.md) and select the [latest stable release](https://github.com/masarray/arstack61850/releases/latest). -- **Found a problem?** Open a [GitHub issue](https://github.com/masarray/arstack61850/issues/new/choose) with the steps to reproduce it, the expected result, and the actual result. The exact Studio version, firmware Build ID, OS and relevant IEC 61850 profile help when available. -- **Want to improve the code?** Read [Developer Guide](docs/DEVELOPER_GUIDE.md) for the existing build and test workflow. Check for an existing issue before implementing a large new feature. +- New to ARStack? Follow [Getting Started](docs/GETTING_STARTED.md). +- Found a problem? Use the [bug report form](https://github.com/masarray/arstack61850/issues/new?template=bug_report.yml). +- Tested another IEC 61850 setup? Share a [community interoperability report](docs/COMMUNITY_TESTING.md). +- Want to change source code? Open an issue describing the improvement before undertaking a large change. -## Submitting a pull request +## Pull requests -1. Branch from current `main`; do not reuse an old divergent release-candidate branch. -2. Keep each PR focused on one change. Describe the engineering problem, proposed behavior and affected modules. -3. Include a reproducible test or captured evidence for the behavior you changed. Run relevant C++/Qt/embedded checks and link the exact GitHub Actions results. -4. Preserve defined protocol behavior and fail closed on unknown SCL/MMS identity or unsupported SV/PTP synchronization claims. -5. State clearly which checks passed and what still requires physical equipment, a real peer or longer endurance testing. +1. Fork the repository and create a branch for your change. +2. Keep each PR focused, with a short description of why it helps users. +3. For code changes, run the relevant build and tests and describe what you actually tested. +4. Attach a screenshot for user-interface changes when possible. +5. Keep public copy **vendor-neutral**, friendly to first-time users, and accurate about supported workflows. -The current public M3.H Lab Simulator has a pinned firmware Build ID and immutable release artifacts. Future source changes belong to a new reviewed milestone; never overwrite old published binaries. +Please do not submit proprietary firmware, confidential substation configurations or credentials. -## Vendor-neutral evidence +## Product and engineering documentation -Use standards-based descriptions rather than manufacturer-specific workarounds. Sanitise captures and SCL files before publishing: remove site names, station IPs, customer identifiers, credentials, private certificates and proprietary details. Prefer small synthetic regression fixtures that reproduce the same observable protocol behavior. +User-facing guides should put the action first: what to click, what to expect, and how to troubleshoot. Put architecture notes, internal test records and historical decisions in [developer documentation](docs/DEVELOPER_GUIDE.md). -Security vulnerabilities should be reported privately using [SECURITY.md](SECURITY.md), not through a public exploit issue. +Active SV/PTP injection belongs in an isolated laboratory network. See the [release notes](https://github.com/masarray/arstack61850/releases/latest) for the current firmware/application pair. -Contributions are subject to the repository's [GPL-3.0 license](LICENSE) and applicable third-party notices. +By contributing, you agree that your contributions are provided under the repository's [GPL-3.0 license](LICENSE). diff --git a/README.md b/README.md index b2da02838..160e74f3a 100644 --- a/README.md +++ b/README.md @@ -1,61 +1,59 @@ # ARStack61850 -An open-source **IEC 61850 engineering lab** for exploring Sampled Values, PTP timing and process-bus behavior. Configure waveforms in a native desktop app, then generate real Ethernet traffic with an ESP32-P4 board. +**Open-source IEC 61850 Process Bus engineering lab** -[![Latest release](https://img.shields.io/github/v/release/masarray/arstack61850?label=Latest%20release)](https://github.com/masarray/arstack61850/releases/latest) -[![C++ CI](https://github.com/masarray/arstack61850/actions/workflows/cpp-ci.yml/badge.svg)](https://github.com/masarray/arstack61850/actions/workflows/cpp-ci.yml) -[![License: GPL-3.0](https://img.shields.io/badge/License-GPL--3.0-blue)](LICENSE) +Generate real Sampled Values (SV) traffic, explore PTP synchronization, and experiment with digital-substation signals using **ARStack Studio** and an **ESP32-P4 Ethernet board**. -[**Download latest release**](https://github.com/masarray/arstack61850/releases/latest) · [Product website](https://masarray.github.io/arstack61850/) · [Installation guide](docs/INSTALL.md) · [Report an issue](https://github.com/masarray/arstack61850/issues) +[**Download the latest stable release**](https://github.com/masarray/arstack61850/releases/latest) · [**Getting started**](docs/GETTING_STARTED.md) · [**Product website**](https://masarray.github.io/arstack61850/) · [**Get help**](SUPPORT.md) -## What can you do with ARStack? +## What you can do -| Capability | Your workflow | +| Explore | With ARStack | | --- | --- | -| 4I + 4V Sampled Values | Set current, voltage, frequency and phase scenarios in ARStack Studio, then publish SV from an ESP32-P4 Ethernet board | -| PTP laboratory timing | Explore a simulated grandmaster and observe Announce, Sync and Follow_Up frames | -| Capture and inspect | Review traffic, counters, waveform/phasor behavior and timing evidence | -| IEC 61850 engineering files | Explore SCL-based configurations and reference profiles in the desktop tooling | -| Open protocol foundations | Build on C++20 MMS, GOOSE, Sampled Values, SCL and COMTRADE components | +| **Generate Sampled Values** | Publish 4 current + 4 voltage channels over real Ethernet at 4,000 SV frames/s. | +| **Experiment with PTP** | Run an internal laboratory grandmaster emulator with hardware-assisted phase synchronization. | +| **Control waveforms** | Set current, voltage and phase values in the ARStack Studio desktop interface. | +| **Inspect live traffic** | Capture SV and PTP using Studio, Wireshark, ARSAS or DigSubAnalyzer. | +| **Learn with engineering files** | Open SCL files and deploy supported 4I+4V publisher profiles. | -ARStack is made for hands-on experimentation, demonstrations and engineering validation. You can launch the desktop app and explore its controls before connecting hardware; Ethernet SV/PTP output uses the supported ESP32-P4 device. +**M3.H stable lab release (24 October 2026):** an independent capture measured **163,048 consecutive SV frames at 3,999.884 frames/s**, with no missing, duplicate or malformed SV packets and `smpSynch=2` throughout the recorded interval. [View the full measurements](docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md). -## Get started +## Get started in minutes -1. **Install Studio.** Open the [latest stable release](https://github.com/masarray/arstack61850/releases/latest) and download the Windows x64 Setup EXE or Portable ZIP. +1. **[Download ARStack Studio](https://github.com/masarray/arstack61850/releases/latest)** for Windows x64 (Setup EXE or Portable ZIP). +2. Open Studio and try the built-in **4I+4V Quick Start** profile. You can explore the interface before connecting hardware. +3. To transmit real Ethernet frames, connect a compatible **ESP32-P4 Ethernet** board over USB and Ethernet. Use **Configuration → Firmware** to install the firmware bundled with **the same release**. +4. Choose **PTP SOURCE**, set **SMPSYNCH AUTO**, configure voltage/current waveforms, and press **Start**. +5. View the output using Studio capture or Wireshark (`0x88BA` for SV, `0x88F7` for PTP). -2. **Choose a scenario.** Open ARStack Studio and select the built-in **4I+4V** profile, or open a compatible SCL configuration. +**[Full first-run guide →](docs/GETTING_STARTED.md)** · [Installation on Windows and Linux](docs/INSTALL.md) -3. **Connect hardware for live output.** Attach your ESP32-P4 Ethernet board through USB and install the firmware shipped with the *same release* using Studio's Firmware panel. +For active SV/PTP transmission, use an **isolated laboratory Ethernet network**, not an operating substation process bus. -4. **Start your lab traffic.** Connect Ethernet to a dedicated test switch, set your signals and PTP mode, then use **Start / Stop** and inspect the resulting SV/PTP packets. +## Downloads and platforms -Prefer installing from a Windows terminal? The verified installer script follows GitHub's current public stable release: - -```powershell -$script = Join-Path $env:TEMP 'arstack-install.ps1' -Invoke-WebRequest 'https://raw.githubusercontent.com/masarray/arstack61850/main/scripts/install-windows.ps1' -OutFile $script -& $script -``` +| Package | Availability | +| --- | --- | +| Windows Setup EXE | **Available** in the [latest release](https://github.com/masarray/arstack61850/releases/latest) | +| Windows Portable ZIP | **Available** in the [latest release](https://github.com/masarray/arstack61850/releases/latest) | +| Matched ESP32-P4 firmware | **Available** with the same release | +| Linux AppImage | Shown in the [latest release](https://github.com/masarray/arstack61850/releases/latest) when a tested package is published | +| Windows package manager | [WinGet preparation and status](docs/INSTALL.md#windowspackage-manager--winget) | -For Linux x86_64, check [Downloads](https://github.com/masarray/arstack61850/releases/latest) for an AppImage when it is published; the [installation guide](docs/INSTALL.md) also covers Linux command-line setup and WinGet package status. +Every published Windows release includes the application dependencies and a matched firmware package; no C++ build environment is required for ordinary installation. -## See it in action +## Community -The released **M3.H** laboratory milestone captured **163,048 SV frames** in 40.763 seconds at **3,999.884 frames/s**, with continuous `smpCnt` and no missing, duplicate or malformed SV frames in that captured interval. The same test recorded PTP Announce / Sync / Follow_Up traffic. +ARStack is built to make IEC 61850 experiments more accessible to engineers, students and automation enthusiasts around the world. -[View the M3.H engineering results](docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md) · [Explore the product page](https://masarray.github.io/arstack61850/) +- **[Ask a question or report a problem](SUPPORT.md)** — share what you tried and what happened. +- **[Share an interoperability experiment](docs/COMMUNITY_TESTING.md)** — feedback from independent laboratory setups helps improve the project. +- **[Contribute](CONTRIBUTING.md)** — improvements to documentation, software, tests and examples are welcome. -## Developers & contributors +ARStack uses the [GPL-3.0 license](LICENSE). It is a laboratory simulation and engineering tool, not a certified primary measurement or UTC reference device. -ARStack includes a portable C++20 IEC 61850 engine, Qt desktop applications, embedded targets and automated protocol tests. +## For developers -- [Contributing](CONTRIBUTING.md) — propose improvements with reproducible evidence and current-main PRs -- [Security policy](SECURITY.md) — report vulnerabilities privately and handle lab evidence responsibly -- [Technical & developer reference](docs/DEVELOPER_GUIDE.md) — architecture, build commands, implemented modules and lab validation -- [Project direction](NORTH_STAR.md) — product goals and architecture -- [Studio operator guide](apps/arstack_studio/README.md) — usage and firmware handling -- [IEC 61850 simulator](apps/ied_simulator/README.md) — MMS server and model-based engineering workbench -- [Issues and discussions](https://github.com/masarray/arstack61850/issues) — report a problem or share a reproducible test +Want to explore the protocol engine or build from source? Start with the [developer documentation index](docs/DEVELOPER_GUIDE.md). -The application is free and open source under the [GPL-3.0 license](LICENSE). Keep active SV/PTP publishing on a dedicated lab network rather than a live substation LAN. +The source tree includes a portable C++20 IEC 61850 protocol engine, SCL/COMTRADE tooling, MMS and GOOSE foundations, ESP32-P4 firmware and reproducible tests. Capabilities vary by application and release; the [feature matrix](docs/FEATURE_MATRIX.md) separates released tools from libraries and development features. diff --git a/SUPPORT.md b/SUPPORT.md new file mode 100644 index 000000000..651c4996c --- /dev/null +++ b/SUPPORT.md @@ -0,0 +1,19 @@ +# ARStack support + +**[Download the latest version](https://github.com/masarray/arstack61850/releases/latest)** · **[Getting started](docs/GETTING_STARTED.md)** · **[Installation](docs/INSTALL.md)** + +## Something not working? + +Use the [simple bug-report form](https://github.com/masarray/arstack61850/issues/new?template=bug_report.yml). A short description is enough to start. + +Useful details include the Studio release, Windows version, ESP32-P4 firmware build if connected, the exact error shown and the last action you tried. + +**SCL import issue?** Share a screenshot of the validation result and, if permissible, a sanitized SCL/CID/SCD/IID file that reproduces it. Never share confidential installation data. + +**Can't find the board?** Check the USB data cable and serial device selection; use **Configuration → Firmware** to verify the ESP32-P4 and install matching firmware. + +**Can't see SV in Wireshark?** Verify the isolated Ethernet topology, selected capture interface and whether a switch mirror/TAP is needed. + +**Interested in IEC 61850 subscriber testing?** Share results in the [community interoperability guide](docs/COMMUNITY_TESTING.md). + +For a new release, always install the Studio/firmware pairing shipped together. Do not connect active SV/PTP lab output to an operating substation LAN. diff --git a/apps/arstack_studio/README.md b/apps/arstack_studio/README.md index f59ca4e2d..b99c302ee 100644 --- a/apps/arstack_studio/README.md +++ b/apps/arstack_studio/README.md @@ -1,156 +1,52 @@ -# ARStack Studio — Desktop IEC 61850 Lab Tool +# ARStack Studio -ARStack Studio is the native desktop interface for IEC 61850 Sampled Values experiments and ESP32-P4 process-bus simulation. +**Desktop controls for real-time IEC 61850 Sampled Values and PTP lab experiments.** -**[Get the latest release](https://github.com/masarray/arstack61850/releases/latest)** · **[How to install](../../docs/INSTALL.md)** · **[Product website](https://masarray.github.io/arstack61850/)** +[**Download the latest stable version**](https://github.com/masarray/arstack61850/releases/latest) · [**First-run tutorial**](../../docs/GETTING_STARTED.md) · [**Product website**](https://masarray.github.io/arstack61850/) -On Windows, choose **Setup EXE** or **Portable ZIP** from the release page. For Linux, use an **AppImage** when it appears among the verified assets. For Ethernet output, connect the ESP32-P4 board and load its firmware from the **same release**. +ARStack Studio configures the supported 4I+4V publisher profile and controls waveform setpoints. The ESP32-P4 performs real-time Ethernet SV publication and internal PTP source emulation. -Start with a 4I+4V example, choose the timing mode, adjust voltage/current signals, and press **Start**. This software is made for engineers, students and lab experimenters; a GPS reference or protection relay is not needed just to try it. +## Your first experiment -The M3.H release recorded verified 4 kHz SV and local simulated PTP traffic. These measurements belong to that specific release; newer builds have their own release notes and evidence. +1. Install Studio (Windows Setup or Portable) and open **4I+4V Quick Start**. +2. Connect a compatible ESP32-P4 Ethernet development board by USB on an isolated lab Ethernet segment. +3. Use **Configuration → Firmware** to verify the board and install the matching release firmware. +4. Deploy your 4I+4V profile while stopped. +5. Enable **PTP SOURCE**, select **SMPSYNCH AUTO**, set signal magnitude/phase and press **Start**. +6. Observe the generated SV/PTP traffic using Studio Capture or an Ethernet analyzer. -## M3.H laboratory SV + PTP mode +The current M3.H reference publisher supports four current and four voltage channels, one ASDU per frame, a 64-byte sample payload, 4,000 SV frames/s and `smpCnt` 0–3999. Compatible SCL/CID/SCD/IID input can be used for the supported profile. -Set **PTP SOURCE** (`INTERNAL_GM_ONLY`) and **`SMPSYNCH AUTO`** on an isolated Ethernet test bench. PTP continues across operator SV STOP/START; `smpSynch=2` is qualified from local EMAC hardware-TX/phase evidence. This is a **simulated global-grandmaster profile**, not UTC/GNSS calibration. The independent M3.H PCAP recorded **163,048 consecutive SV at 3,999.884 fps over 40.763 s, all `smpSynch=2`, zero malformed/missing/duplicate and 44 PTP Announce + 43 Sync/Follow_Up pairs**. Its intended 120 s capture was interrupted by SV STOP; relay interoperability and alarm-free lock remain untested. +## M3.H measured milestone -[Full released simulator contract, measured capture and quick start](../../docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md). +The 24 October 2026 M3.H public release combines SV publishing with an internal simulated PTP grandmaster and hardware-assisted phase evidence. -## P0 public boundary +An independent lab capture recorded **163,048 consecutive SV frames** in **40.763 seconds**, measured at **3,999.884 frames/s**. No missing, duplicate or malformed SV frames were observed; all captured SV advertised `smpSynch=2`. PTP Announce/Sync/Follow_Up packets were captured alongside SV. -The first release is intentionally narrow and testable: +[Read the complete technical record and firmware identity](../../docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md). -- IEC 61850 Sampled Values, 9-2LE-style **4 current + 4 voltage**; -- wire order `Ia, Ib, Ic, In, Ua, Ub, Uc, Un`; -- one INT32 value plus one 32-bit Quality word for each signal; -- 16 FCDA leaves / 64-byte sample payload; -- **4000 frames/s**, `SmpPerSec`; -- one ASDU per frame; -- explicit `smpCnt` modulus 4000; -- ESP32-P4-ETH target running the ARStack protocol-v1 firmware. +## Firmware installation and recovery -Broader codecs, SCL parsing, COMTRADE support, experimental timing work, or other layouts elsewhere in the repository do **not** expand this P0 deployment boundary. +The Windows package includes the firmware image, manifest/checksum and flashing tool. Use **Configuration → Firmware** to: -## Architecture +1. Select the detected USB serial device. +2. **Verify ESP32-P4**. +3. Choose **Install / Recover** and allow the device to restart. +4. Reconnect and verify the ARStack device identity. -```text -4I+4V Quick Start or compatible SCL/CID/SCD/IID - -> ARIEC61850 SclParser - -> SvPublisherProfileCompiler - -> fail-closed ESP32-P4 deployment classifier - -> Qt SclProfileModel - -> operator Deploy - -> guarded Qt DeviceController - -> ARStack protocol v1 - -> deterministic ESP32-P4 SMV publisher -``` - -The desktop application never becomes the 4000 fps timing source. It configures and supervises the embedded publisher. - -## Normal operator workflow - -1. Open **ARStack Studio**. -2. Choose **4I+4V Quick Start**, or **Open SCL** for a compatible external profile. -3. Connect the ESP32-P4-ETH board. Studio verifies the `ARSTACK identity` handshake and protocol. -4. **Deploy** the validated Class-A 4I+4V profile. -5. Adjust Current / Voltage setpoints as required. -6. **Start** Sampled Values output. -7. **Stop** before changing immutable profile identity fields or before leaving the test setup. - -The Home ribbon exposes explicit `DEVICE`, `FW`, `PROFILE`, and `OUTPUT` state. Disabled Deploy/Start controls explain the blocking condition instead of silently doing nothing. - -Critical START and DEPLOY rules are repeated in the native controller, so keyboard shortcuts or alternate QML paths cannot bypass the public P0 safety boundary. - -## Blank board / firmware recovery - -For a blank board, recovery board, or protocol mismatch, open **Configuration → Firmware**. - -The Firmware Manager uses this fail-closed sequence: - -1. select the USB serial port; -2. **Verify ESP32-P4** using the ROM flasher connection; -3. validate `firmware-manifest.json`; -4. verify the packaged firmware SHA-256; -5. unlock **Install / Recover** only for a verified ESP32-P4; -6. write the single merged recovery image at flash offset `0x0`; -7. reset the chip; -8. reconnect and require the normal ARStack `IDENTIFY` handshake. - -A missing flasher, missing image, malformed manifest, incompatible chip declaration, unsafe flash offset, protocol mismatch, or SHA-256 mismatch keeps installation blocked. - -The redistributable Windows package includes a pinned standalone `espflash`; the operator does **not** need Rust, Python, ESP-IDF, CMake, Qt, or a source checkout. - -## Firmware package contract - -A release package contains: - -```text -ARStackStudio.exe -firmware/ - firmware-manifest.json - arstack-esp32p4-smv-0.1.1.bin - SHA256SUMS.txt -tools/ - espflash.exe -licenses/ - espflash/ -``` - -Manifest schema `arstack.studio.firmware.v1` binds the image to chip `esp32p4`, firmware version `0.1.1` in M3.H, protocol `1`, flash offset `0`, the exact image filename, image SHA-256, ESP-IDF build version, and source commit. - -Protocol `1` is the P0 GUI/firmware capability contract for the supported `SMV-4I4V`, profile deployment, and live setpoint workflow. +Studio and ESP32-P4 firmware must come from the **same public release**. Installing Studio alone does not change the firmware on the board. -## Windows release artifacts +Network SV/PTP transmission is for an isolated test bench; never inject an unsolicited PTP grandmaster or SV source into an operating substation network. -**Current Process Bus Simulator release:** [M3.H Lab Edition](https://github.com/masarray/arstack61850/releases/tag/processbus-sim-m3h-20261024), with matching `ARStack-ProcessBus-M3H-win-x64-setup.exe`, `ARStack-ProcessBus-M3H-win-x64-portable.zip`, and `ARStack-ESP32P4-ProcessBus-M3H-firmware.bin`. Windows Setup and Portable bundle Qt, firmware manifest/hash and a pinned flasher. Build ID `a922e9bdba4e57a4` and source `a922e9bdba4e57a4e88406eb83344455dd8ff142` are the tested public binary identity. M3.H is marked **experimental**, available publicly, and has **10/10 CI workflows PASS**. +## For developers -The historical releases [v0.1.1](https://github.com/masarray/arstack61850/releases/tag/v0.1.1) and [v0.1.0](https://github.com/masarray/arstack61850/releases/tag/v0.1.0) are still available. v0.1.0 recorded an independent 3600-second SV endurance run at `smpSynch=0`, which is **not** M3.H simulated-GM acceptance. +Studio is a Qt 6 / C++20 application. The desktop controller handles device verification, SCL profile validation, and operator commands; real-time frame scheduling occurs in ESP32-P4 firmware. -## Truthfulness boundary - -Phasor and waveform views are generated setpoint previews, not independent Ethernet capture evidence. `smpSynch` remains an embedded synchronization truth and must not be promoted to a synchronization claim without measured evidence. - -CI can prove parser/compiler behavior, firmware manifest/hash enforcement, native START/DEPLOY guards, Qt instantiation, high-DPI launch, firmware build/package generation, Windows portable construction, installer construction, and packaged-app smoke execution. CI cannot prove physical Ethernet waveform or timing behavior. - -For M3.H lab release evidence, see [`../../docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md`](../../docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md). Real-IED acceptance is optional future interoperability testing, not a prerequisite for public simulation use. - -## Development build - -Qt 6.5+ is required with Core, Gui, Qml, Quick, QuickControls2 and SerialPort, together with a C++20 compiler and CMake 3.24+. - -From a Qt-enabled Windows developer shell: +Build from a Qt-enabled shell with CMake 3.24+ and Qt 6.5+: ```powershell cmake -S apps/arstack_studio -B build-arstack-studio -G Ninja -DCMAKE_BUILD_TYPE=Release cmake --build build-arstack-studio --target arstack_studio --parallel -.\build-arstack-studio\arstack_studio.exe -``` - -The repository also provides: - -```powershell -.\apps\arstack_studio\run.cmd -``` - -or: - -```powershell -.\apps\arstack_studio\run-windows.ps1 ``` -These developer launchers are not required by the redistributable release. - -## Automated Studio gates - -`.github/workflows/arstack-studio-qt.yml` checks: - -- native Qt configure/build with warnings as errors; -- bundled 4I+4V reference profile contract; -- native P0 controller fail-closed policy; -- valid firmware bundle acceptance; -- corrupted firmware SHA-256 rejection; -- normal offscreen QML launch; -- high-DPI QML launch; -- Windows launcher PowerShell parsing. - -`.github/workflows/arstack-studio-release.yml` additionally builds the merged ESP32-P4 firmware package and the Windows portable/installer release candidates. +[Developer documentation](../../docs/DEVELOPER_GUIDE.md) · [Embedded architecture](../../embedded/esp32p4_smv_injector/ARCHITECTURE.md) · [Contributing](../../CONTRIBUTING.md) diff --git a/docs/COMMUNITY_TESTING.md b/docs/COMMUNITY_TESTING.md new file mode 100644 index 000000000..7dc48a3d8 --- /dev/null +++ b/docs/COMMUNITY_TESTING.md @@ -0,0 +1,31 @@ +# Share an IEC 61850 interoperability experiment + +Have you tried ARStack with a simulator, analyzer, protection relay or bay controller in your own lab? Your results can help the international IEC 61850 community discover useful configurations and improve ARStack. + +**[Share an interoperability report](https://github.com/masarray/arstack61850/issues/new?template=interoperability.yml)** · [Ask for help](../SUPPORT.md) + +## What is helpful to share? + +- Which **ARStack Studio release** and ESP32-P4 firmware were used? +- What was the lab topology? (For example: ESP32-P4 → isolated switch → analyzer/subscriber.) +- Which SV profile and PTP mode were selected? +- Which device or software was receiving or analyzing the traffic? Make/model/version are helpful but optional. +- What did you observe? Include positive results, unexpected behavior or a capture screenshot. +- Which tests would you like to try next? + +A short description and one screenshot are useful. You do not need to write a formal test report. + +## Capture and privacy + +If you share an SCL/SCD/CID file or packet capture, remove confidential project names, private IP addresses, credentials and other sensitive network information. Only share files you have permission to distribute. + +Each external report describes the **specific setup tested**. Reported results are community observations; the project will clearly identify whether a result was independently reproduced. + +## Experiment ideas + +- Inspect SV `APPID`, `svID`, `smpCnt`, quality and sample counter rollover. +- Compare PTP Announce/Sync/Follow_Up with the received SV stream. +- Test SCL-derived stream selection and publisher configuration. +- Test how a laboratory analyzer or IEC 61850 subscriber interprets the published reference profile. + +Keep active SV/PTP experiments on an isolated lab Ethernet segment, never a live substation process bus. diff --git a/docs/DEVELOPER_GUIDE.md b/docs/DEVELOPER_GUIDE.md index 802bfee2a..721e09120 100644 --- a/docs/DEVELOPER_GUIDE.md +++ b/docs/DEVELOPER_GUIDE.md @@ -1,362 +1,36 @@ -# ARStack61850 — Technical & Developer Reference +# ARStack developer documentation -The [main README](../README.md) is a short, engineer-friendly entry point. This document preserves the full implementation notes, portable C++ library status, test commands and architecture references for contributors. +This index is for engineers who want to study or extend ARStack's source code. If you are new to ARStack, start with the [user-facing quick start](GETTING_STARTED.md). -Maintained source of truth: [NORTH_STAR](../NORTH_STAR.md), [AGENTS.md](../AGENTS.md), and [current release notes](releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md). +## Architecture and project direction ---- +- [Project north star](../NORTH_STAR.md) — overall protocol, execution and hardware design. +- [Project handoff](../PROJECT_HANDOFF.md) — current integration context for maintainers. +- [Studio architecture and operator workflow](../apps/arstack_studio/README.md). +- [ESP32-P4 firmware architecture](../embedded/esp32p4_smv_injector/ARCHITECTURE.md). -## Start here +## Protocol and engineering tools -For a new engineer or AI development thread, read these before making architecture changes: +- [Feature matrix](FEATURE_MATRIX.md) — implemented components and evidence categories. +- [SCL injection workflow](LIVE_INJECTION_WORKFLOW.md). +- [MMS file transfer](MMS_FILE_TRANSFER.md). +- [SCL-assisted MMS connection](SCL_ASSISTED_MMS_CONNECT_PROFILE.md). +- [PTP timing and lab implementation](PTP_LAB_TIMING.md). -1. [`NORTH_STAR.md`](../NORTH_STAR.md) — product thesis, standards, hardware roadmap and definition of success. -2. [`docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md`](../docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md) — latest public process-bus laboratory simulator evidence, setup and release boundary; [`v0.1.0 historical evidence`](../docs/SMV_STUDIO_V0.1.0_PUBLIC_RELEASE.md). -3. [`PROJECT_HANDOFF.md`](../PROJECT_HANDOFF.md) — current repository handoff and separation between released SMV work and active follow-on tracks. -4. [`apps/arstack_studio/README.md`](../apps/arstack_studio/README.md) — operator workflow, firmware recovery and Windows distribution details. -5. [`docs/ONLINE_MODEL_CONNECT_DECISION.md`](../docs/ONLINE_MODEL_CONNECT_DECISION.md) — short decision contract separating full live discovery from SCL-assisted online connect. -6. [`LIVE_DISCOVERY_PROFILE.md`](../LIVE_DISCOVERY_PROFILE.md) — detailed live-discovery and online-model interoperability boundary. -7. [`embedded/esp32p4_smv_injector/ARCHITECTURE.md`](../embedded/esp32p4_smv_injector/ARCHITECTURE.md) — embedded real-time architecture boundaries. -8. [`docs/GLOBAL_BENCHMARK.md`](../docs/GLOBAL_BENCHMARK.md) — vendor-neutral standards/industry research behind the direction. +## Release and test evidence -The project deliberately separates **wire facts**, **configured expectations**, **profile claims**, and **synchronization evidence**. A packet that decodes in a host capture tool is not, by itself, a conformance or timing claim. +- [M3.H release record](releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md). +- [Mainline release/source reconciliation](integration/M3H_MAINLINE_RELEASE_SOURCE_CLOSURE_2026-10-24.md). +- [Historical milestone notes](SMV_STUDIO_V0.1.0_PUBLIC_RELEASE.md). -## Current process-bus simulator milestone +## Building -**M3.H Lab Edition is published publicly** as a stable lab-simulator SV publisher + internal PTP grandmaster emulator. The 40.76-second raw PCAP validates stable lab traffic and hardware-evidence-qualified `smpSynch=2`; it does not prove a subscriber relay would accept or lock without alarms. No external GPS/certification is a dependency for running or releasing this lab simulator. - -The older v0.1.0/v0.1.1 products remain available as historical releases. Further work: independent relay/IED interoperability when hardware is available, deeper 120-second and multi-cycle captures, alternate SV profiles, and optional external time references—not blockers for M3.H public distribution. - -## Lineage and portable core - -The original work began as an incremental C++20 migration of the ARIEC61850 C# protocol stack. The C# implementation remains a behavioral oracle where cross-language equivalence is still useful, while ARStack-specific work adds deterministic RTOS/hardware execution and evidence that a normal desktop stack cannot provide. - -The project is **not yet a complete IEC 61850 replacement stack**. Current -Public Alpha work focuses on a useful read-only MMS engineering client, process -bus codecs/runtime foundations, reproducible cross-language evidence, and an -MCU-safe protocol boundary. - -## Implemented - -### Phase 0 — portable foundation - -- CMake-based C++20 static library. -- BER and MMS data/UTC-time codecs. -- Ethernet/VLAN process-bus framing and classic PCAP support. -- GOOSE and Sampled Values codecs plus offline/runtime supervision foundations. -- SCL read-only parsing and COMTRADE CFG/DAT support for host engineering tools. -- RFC 1006 TPKT, COTP, ISO Session, Presentation, ACSE, and MMS Initiate. -- Confirmed MMS GetNameList, GetVariableAccessAttributes, Read, and Write codecs. -- Read-only MMS FileDirectory/FileOpen/FileRead/FileClose codecs and bounded streaming runtime. -- DataSet directory, InformationReport, RCB state, and report-subscription foundations. - -### Phase 1A–1E — deterministic process-bus core - -- IEC 61850 UTC time and recursive MMS Data/AllData codec. -- Complete GOOSE PDU/frame codecs and offline publisher/subscriber supervision. -- Sampled Values ASDU, multi-ASDU PDU, Ethernet frame, payload, quality, counter, and stream supervision. -- C#-derived synthetic PCAP equivalence evidence. -- Sanitizer, deterministic mutation-smoke, and LLVM libFuzzer coverage. -- Receive-only isolated-laboratory tooling. - -A codec existing in the library does **not** mean the corresponding mutating -operation is enabled in live Phase 4C discovery. - -### Phase 2A — SCL foundation - -- Bounded, dependency-free XML/SCL reader. -- IED, DataSet, FCDA, GOOSE, Sampled Values, report-control, and communication mapping. -- SCL edition detection, type-template resolution, DataSet binding, diagnostics, and SCL/XML fuzzing. - -### Phase 2B — COMTRADE foundation - -- CFG parsing for IEEE C37.111-style 1991/1999/2013 records. -- ASCII, BINARY, BINARY32, and FLOAT32 DAT readers. -- Analog engineering scaling (`a * raw + b`) and packed digital status decoding. -- Timestamp multiplier and multi-rate fallback scheduling. -- Default `Va/Vb/Vc/Vn/Ia/Ib/Ic/In` channel mapping. -- Deterministic SCL Sampled Values to COMTRADE channel binding. -- Read-only `ariec61850_comtrade_inspect` human/JSON CLI. -- C#-derived ASCII and binary fixtures, mutation smoke, sanitizer, and libFuzzer corpus. - -### Phase 3A — MMS transport foundation - -- RFC 1006 TPKT frame encode/decode with strict version, reserved-octet, and length validation. -- Incremental TPKT stream framing for fragmented and coalesced TCP byte streams. -- COTP Connection Request, Connection Confirm, Data, Disconnect Request, and Error TPDU decoding. -- Default C#-compatible CR vector, TPDU-size negotiation, and TSAP selector mirroring. -- COTP Data segmentation by negotiated TPDU size and bounded EOT reassembly. -- Fragment-count, byte-count, and empty non-final fragment abuse guards. -- C#-derived golden vectors, deterministic mutation smoke, sanitizer, and libFuzzer coverage. - -### Phase 3B — MMS association codecs - -- Bounded ISO Session Connect/Accept SPDU and data-transfer profile codecs. -- Presentation CP/CPA context definitions, negotiation results, selectors, and PDV routing. -- ACSE and MMS context negotiation using BER transfer syntax. -- Structured ACSE AARQ/AARE application context, AP-title, AE-qualifier, result, and diagnostic parsing. -- EXTERNAL user-information handling with MMS Initiate retained as an opaque Phase 3C payload. -- Deterministic association-response construction with session and context mirroring. -- Byte-exact C# request/response golden vectors, mutation smoke, sanitizer, and libFuzzer coverage. - -### Phase 4C — live read-only MMS discovery - -`ariec61850_live_discover` currently provides: - -- built-in Windows/POSIX TCP transport; -- MMS association through the implemented OSI stack; -- domain and variable GetNameList discovery; -- named-variable-list/DataSet inventory; -- bounded variable-type probes; -- bounded RCB read evidence; -- LD/LN/DO/DA live-model projection; -- IED identity inference; -- CDC/type heuristics compatible with the C# oracle; -- GOOSE/SV/setting-group/log control-block name inventory; -- dynamic RCB `Bound` / `Unbound` / `NotRead` / `ReadFailed` evidence; -- read-only RCB candidate planning and conservative operational availability. - -The live discovery surface sends only read-only services such as GetNameList, -GetVariableAccessAttributes, GetNamedVariableListAttributes and Read. It does -**not** authorize or perform Write, control, GI, RCB reservation/enable, -dynamic DataSet mutation, or file-service mutation. - -### Phase 4E - MMS file transfer client - -`ariec61850_file_transfer` adds an explicit read-only file-service surface: - -- root/nested FileDirectory with bounded continuation; -- streaming FileOpen/FileRead/FileClose download to a caller-owned sink; -- full signed Integer32 FRSM support; -- exact Confirmed-Error evidence and bounded diagnostics; -- best-effort FileClose that preserves the primary failure; and -- a single oracle-matched rooted-backslash fallback for file-non-existent. - -The default CLI operation is directory listing. Download requires both -`--remote` and `--output`; upload/delete/rename are not implemented. See -`docs/MMS_FILE_TRANSFER.md` for API, safety, CLI, exit-code, and acceptance -details. - -#### Public Alpha A1 model work - -The live model now projects C#-style: - -- `proposedLnTypeId` and `proposedDoTypeId`; -- `typeTemplates`; -- `variableTypeDiscoveries`; -- deterministic IED identity evidence; -- DataSet/RCB/control-block inventory. - -Dynamic report configuration is separated from static model identity with three -fingerprints during migration: - -- `fingerprint` — legacy canonical fingerprint retained for compatibility; -- `structuralFingerprint` — LD/LN/DO/DA + control identities, excluding mutable - DataSet/RCB runtime state; -- `runtimeSnapshotFingerprint` — current DataSet inventory and RCB binding/state. - -For IEDs with dynamic DataSets, a changed runtime fingerprint with an unchanged -structural fingerprint is expected and does not by itself mean the IED model or -firmware changed. - -### Phase 3C — MMS service codecs - -- MMS Initiate request/response and confirmed request/response/error envelopes. -- Invoke-ID routing, ObjectName, GetNameList, GetVariableAccessAttributes, Read, and Write. -- Recursive bounded TypeSpecification and mixed success/failure access results. -- C#-derived vectors, sanitizer, deterministic mutation smoke, and libFuzzer coverage. - -### Phase 4A — offline DataSet and report monitoring - -- DataSet and buffered/unbuffered RCB inventory from supplied GetNameList evidence. -- GetNamedVariableListAttributes request/response and IEC reference normalization. -- Strict InformationReport and OptFlds-driven report-frame decoding. -- RCB attribute read-plan/state mapping with cautious availability confidence. -- Sequence, configuration, DataSet, overflow, and segmentation continuity tracking. -- Bounded offline report monitor and dedicated MMS-reporting fuzz corpus. - -### Hardware process-bus runtime — ESP32-P4 P0/P1/P2 foundation - -The portable core still does not silently open raw sockets or physical IED sessions. **Active process-bus transmission exists only in explicitly enabled hardware/lab targets**, currently `embedded/esp32p4_smv_injector`, and is intended for isolated bench use. - -The current ESP32-P4 target adds: - -- RMII hardware bring-up; -- active Layer-2 SV TX; -- fixed/prebuilt runtime templates; -- deterministic GPTimer cadence; -- separate canonical vs diagnostic stream identities; -- 4I+4V balanced waveform generation; -- once-per-second realtime telemetry; -- a path toward EMAC hardware timestamps and PTP-disciplined synchronization. - -See the embedded README/architecture files and `PROJECT_HANDOFF.md` for the exact current state. - -## Phase 4B: association and report-subscription runtime - -The C++ core now includes a transport-injected MMS association runtime and persistent RCB subscription state machine. It performs COTP, Session, Presentation, ACSE/MMS Initiate, confirmed-service routing, timeout/cancellation handling, RCB re-probe, URCB reservation, `RptEna`, optional GI, InformationReport delivery, and ownership-aware cleanup. See `ASSOCIATION_RUNTIME_PROFILE.md`. - -The core deliberately does not ship a default socket transport in this phase. Applications that intentionally enable live IED access must provide an `MmsByteTransport` implementation. - -## Build - -### Windows - -```powershell -cmake -S . -B build -A x64 -DARIEC61850_WARNINGS_AS_ERRORS=ON -cmake --build build --config Release --parallel -ctest --test-dir build -C Release --output-on-failure -``` - -### Linux +Portable C++20 core: ```bash -cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DARIEC61850_WARNINGS_AS_ERRORS=ON +cmake -S . -B build cmake --build build --parallel ctest --test-dir build --output-on-failure ``` -### Sanitizers - -```bash -CC=clang CXX=clang++ cmake -S . -B build-sanitizers \ - -DCMAKE_BUILD_TYPE=RelWithDebInfo \ - -DARIEC61850_ENABLE_SANITIZERS=ON -cmake --build build-sanitizers --parallel -ctest --test-dir build-sanitizers --output-on-failure -``` - -## Read-only tools - -### Inspect a COMTRADE record - -```powershell -.\build\Release\ariec61850_comtrade_inspect.exe .\record.cfg --json -``` - -### Analyze a saved process-bus PCAP - -```powershell -.\scripts\run-lab-check.ps1 -Pcap .\captures\device-session.pcap -``` - -## Live discovery quick start - -Human summary: - -```powershell -.\build\Release\ariec61850_live_discover.exe 192.168.1.10 102 -``` - -Bounded read-only RCB evidence: - -```powershell -.\build\Release\ariec61850_live_discover.exe ` - 192.168.1.10 102 ` - --timeout-ms 30000 ` - --no-types ` - --no-datasets ` - --max-rcb 50 ` - --rcb-plan dynamic ` - --rcb-availability -``` - -Model JSON with bounded type discovery: - -```powershell -New-Item -ItemType Directory -Force .artifacts\parity | Out-Null - -.\build\Release\ariec61850_live_discover.exe ` - 192.168.1.10 102 ` - --timeout-ms 120000 ` - --no-datasets ` - --no-rcb ` - --max-types 256 ` - --model-json ` - | Set-Content -Encoding utf8 .artifacts\parity\cpp-model.json -``` - -Inspect fingerprint/type evidence: - -```powershell -$model = Get-Content .artifacts\parity\cpp-model.json -Raw | ConvertFrom-Json -$model.fingerprint -$model.structuralFingerprint -$model.runtimeSnapshotFingerprint -$model.typeTemplates.Count -$model.variableTypeDiscoveries.Count -``` - -## Same-IED C# ↔ C++ parity - -The canonical comparator is: - -```powershell -python .\tools\compare_live_model_json.py ` - .artifacts\parity\csharp\ied-model.json ` - .artifacts\parity\cpp-model.json -``` - -Optional evidence: - -```powershell -python .\tools\compare_live_model_json.py ` - .artifacts\parity\csharp\ied-model.json ` - .artifacts\parity\cpp-model.json ` - --types ` - --runtime -``` - -Default pass/fail is structural. Runtime DataSet/RCB drift is informational. -The historical `scripts/compare-live-model-parity.py` path remains only as a -compatibility wrapper around this comparator. - -See `docs/LIVE_MODEL_PARITY.md` for the complete C# export + comparison flow. - -## Embedded direction - -The wire core is being separated from host engineering services. Current target -roles are documented in `docs/EMBEDDED_TARGETS.md`: - -- ESP32-P4-ETH: protocol/performance reference after Public Alpha; -- Waveshare ESP32-S3-POE-ETH-8DI-8DO: real I/O application reference; -- STM32H7/NXP class: later portability/industrialization target. - -The embedded profile already excludes host TCP/live-discovery/PCAP/COMTRADE/SCL -parser tooling and provides a no-RTTI host-simulation gate. The portable MMS -file-service runtime is included through abstract transport/sink boundaries; -its filesystem-backed CLI remains host-only. Sampled Values also -has a caller-owned `encode_into(span)` path so the eventual publisher does not -need to allocate a fresh Ethernet frame buffer per sample. - -## Validation boundary - -Implemented code and green CI do not equal full field acceptance. Current -validation includes GCC/Clang/MSVC builds, deterministic tests, sanitizer/fuzz -workflows, and selected live read-only interoperability evidence. Full -replacement claims still require same-IED C#↔C++ evidence, repeated live cycles, -packet capture review, and multi-vendor physical/simulator interoperability. - -See `docs/FEATURE_MATRIX.md` for a conservative status table and -`PHASE_4C1_LIVE_MODEL_INTEROP.md` for the live-evidence boundary. - -See `REPORTING_PROFILE.md` for the offline report decoder profile, `LAB_INTEROP_CHECKLIST.md` before using a physical IED, and `MIGRATION_CHECKLIST.md` for the detailed progress ledger. - -## North-star reminder - -ARStack is not intended to become a UI-heavy packet viewer with timing bolted on later. The architecture priority is: - -```text -standards truth - > measured hardware evidence - > deterministic execution - > interoperability evidence - > usability - > convenience hacks -``` - -The long-term product family — injector, process-bus analyzer, digital-substation probe and real-time MU/closed-loop simulator — should share the same real-time/evidence core. - -## Public-repository naming policy - -Public project documentation should remain vendor-neutral. Do not add commercial competitor names, product comparisons, logos, screenshots, or vendor marketing references. Standards, protocol identifiers, and technical platform/dependency names required to build or reproduce the project are allowed. +Desktop Studio requirements and developer commands are in the [Studio README](../apps/arstack_studio/README.md). Product installation does not require compiling from source. diff --git a/docs/FEATURE_MATRIX.md b/docs/FEATURE_MATRIX.md index 3c8f1ac48..52c8c1439 100644 --- a/docs/FEATURE_MATRIX.md +++ b/docs/FEATURE_MATRIX.md @@ -1,4 +1,6 @@ -# arstack61850 Feature Matrix +# ARStack61850 — Feature and verification matrix + +**Latest public Process Bus release: [M3.H stable laboratory edition](https://github.com/masarray/arstack61850/releases/latest) (24 October 2026).** For the quickest overview of what you can use today, start with [Getting Started](GETTING_STARTED.md). The detailed table below also covers lower-level libraries and developer features. Status terms are intentionally conservative: @@ -52,8 +54,8 @@ Status terms are intentionally conservative: | GOOSE | Publisher/subscriber runtime foundations | Offline-tested | Physical process-bus acceptance remains pending. | | Sampled Values | PDU / Ethernet encode-decode | Offline-tested | Process-bus work remains separate from MMS control evidence. | | Sampled Values | Caller-owned bounded encode path | Offline-tested | `encode_into(span)` avoids fresh frame-buffer allocation in steady-state publisher design. | -| Sampled Values | Live MCU publisher | Public release / physical accepted | ESP32-P4 ARStack Studio v0.1.0 completed 10/10 Start/Stop cycles and an uninterrupted 3600-second 4000 fps retained run with zero missed slots, TX failures, health reconnects or automatic starts; `smpSynch=0` remains the truthful timing boundary. | -| Desktop product | ARStack Studio / SMV Injector v0.1.0 | Public release | Stable Windows installer + portable package + ESP32-P4 firmware published under tag `v0.1.0`; independent SV spot-check confirmed the bounded 4I+4V / 4000 fps / 1 ASDU / 64-byte profile. | +| Sampled Values | Live MCU publisher | Public release / measured Ethernet | M3.H ESP32-P4 published 4I+4V at 3,999.884 frames/s (163,048 consecutive frames in 40.763 s), no missing/duplicate/malformed frames. Internal laboratory PTP grandmaster and qualified `smpSynch=2` operate together in the accepted capture. Historical v0.1.0 measured 3,600 s SV at `smpSynch=0`. | +| Desktop product | ARStack Studio / Process Bus Simulator M3.H | Public stable lab release | Windows Setup and Portable, matched ESP32-P4 firmware, guided device verification and firmware recovery, 4I+4V controls, PTP SOURCE and SMPSYNCH AUTO. [Release record](releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md). | | SCL | Read-only parser / engineering support | Offline-tested | Host engineering surface; mutable standards-aware export remains later work. | | COMTRADE | CFG/DAT read support | Offline-tested | ASCII/BINARY/BINARY32/FLOAT32 and mapping foundations are implemented. | | PCAP | Read/write/evidence support | Live-proven for C5 evidence | Npcap loopback captures were reconciled with JSON Write lists and retained SHA-256 hashes. | @@ -92,4 +94,4 @@ C1-C5 provide a guarded production control path plus a live evidence harness. Th ## Next parity direction -The next gaps are association-loss and contention evidence for control, remaining MMS file-service parity and fault-record workflows, mutable SCL export/reconstruction, deterministic IED simulator/server integration, and broader application/UI parity. The SMV/ESP32-P4 P0 track is now released as stable `v0.1.0`; future process-bus timing/profile expansion remains a separate follow-on tranche rather than an open blocker for that release. +The next gaps are association-loss and contention evidence for control, remaining MMS file-service parity and fault-record workflows, mutable SCL export/reconstruction, deterministic IED simulator/server integration, and broader application/UI parity. The current SV/PTP public lab release is **M3.H**. Historical v0.1.0 evidence remains documented independently; future profile expansion and external interoperability observations are tracked as separate engineering efforts. diff --git a/docs/GETTING_STARTED.md b/docs/GETTING_STARTED.md new file mode 100644 index 000000000..90b4e5719 --- /dev/null +++ b/docs/GETTING_STARTED.md @@ -0,0 +1,72 @@ +# Your first IEC 61850 Process Bus experiment + +This guide takes you from a fresh Windows installation to inspecting Sampled Values and PTP on a small Ethernet lab. + +**You will learn how to:** open ARStack Studio, generate a 4-current + 4-voltage waveform, start real SV traffic from an ESP32-P4, and inspect PTP alongside SV. + +## What you need + +- A Windows 10/11 x64 PC with [ARStack Studio](https://github.com/masarray/arstack61850/releases/latest). +- For **real Ethernet output**: a compatible ESP32-P4 Ethernet development board, USB data cable, Ethernet cables and an isolated Ethernet switch or direct lab connection. +- Optional: Wireshark or another SV/PTP analyzer. + +You can install and explore the desktop software before purchasing or connecting a board. + +## Step 1 — Install Studio + +1. Open the [latest stable release](https://github.com/masarray/arstack61850/releases/latest). +2. Download the **Windows Setup EXE**. Choose the Portable ZIP if you prefer not to install. +3. Open ARStack Studio. + +The setup includes the desktop dependencies and **matching board firmware**. + +## Step 2 — Connect the ESP32-P4 + +1. Connect the board over **USB data** (not a charge-only cable). +2. Connect the board's Ethernet port to your **isolated test network**. +3. In Studio, connect to the detected serial device. Follow the displayed device identity and firmware status. + +Do not connect the SV publisher or PTP source to an operating substation LAN. + +## Step 3 — Install the matched firmware + +1. In Studio, open **Configuration → Firmware**. +2. Select the ESP32-P4 device and use **Verify ESP32-P4**. +3. Choose **Install / Recover** to install the firmware provided with this Studio release. +4. Allow the board to restart, then reconnect from Studio and confirm device recognition. + +Installing a newer Windows application does **not** update the board firmware automatically. Use the firmware packaged with that release. + +## Step 4 — Generate Sampled Values + +1. Select **4I+4V Quick Start**. +2. Deploy the supported publisher profile while the output is stopped. +3. Choose **PTP SOURCE / internal laboratory grandmaster** and **SMPSYNCH AUTO**. +4. Set current and voltage magnitudes and phase angles in Studio. +5. Press **Start**. The ESP32-P4 sends the real-time SV stream directly from its Ethernet hardware. +6. Press **Stop** when finished. PTP SOURCE can remain active across SV STOP/START. + +The published M3.H reference profile sends 4 current and 4 voltage values at 4,000 SV frames per second. + +## Step 5 — View your Ethernet traffic + +With the PC capture interface connected to the relevant lab segment or switch mirror: + +- Filter **Sampled Values:** `eth.type == 0x88ba` +- Filter **PTP:** `eth.type == 0x88f7` +- Watch the SV `smpCnt` increase and wrap from `3999` to `0`. +- Inspect `svID`, `APPID`, quality fields, and the PTP Announce/Sync/Follow_Up messages. + +A switched network does not automatically forward every multicast frame to the PC capture port. If you cannot see SV, verify the lab capture topology before changing the publisher. + +## Try something different + +Adjust a voltage or current setpoint and observe the generated waveform. Open a compatible SCL/CID/SCD/IID file to inspect and select an SV stream supported by the currently published 4I+4V device profile. + +## Need help? + +Check the [installation guide](INSTALL.md) or [support guide](../SUPPORT.md). If a file does not import or a device will not connect, share the Studio version, the error message and a sanitized example file if possible. + +Want to publish observations from another IEC 61850 tool or device? See the [community testing guide](COMMUNITY_TESTING.md). + +For measured M3.H packet evidence and the exact firmware pairing, see the [release technical record](releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md). diff --git a/docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md b/docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md index e30165105..b2acbecae 100644 --- a/docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md +++ b/docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md @@ -1,59 +1,66 @@ -# ARStack Process Bus Simulator M3.H — Stable Lab Simulator Edition (24 October 2026) +# ARStack Process Bus Simulator M3.H — Stable Lab Edition -**Public stable engineering lab simulator release:** ESP32-P4 Ethernet Sampled Values publisher + IEC/IEEE 61850-9-3-style internal PTP grandmaster, operated through ARStack Studio on Windows. +**24 October 2026 · Public stable release** -This release is intended for isolated process-bus experiments, engineering education, SV/PTP decoder testing, and preparation for interoperability testing with protection relays. It does **not** require an external GPS or relay to operate. It deliberately advertises the **simulated global grandmaster profile** for laboratory purposes. This is a simulation capability, not independently verified UTC accuracy. +Generate IEC 61850 Sampled Values and emulate PTP synchronization using **ARStack Studio and ESP32-P4 Ethernet hardware**. This edition combines a 4-current + 4-voltage SV publisher, internal laboratory PTP grandmaster simulation, and hardware-assisted phase synchronization. -## What works and what was measured +## What's new -* **Sampled Values:** one ASDU, 4 current + 4 voltage, 64-byte 8×(INT32+Quality), APPID 0x4000, SV ID `ARSTACK_SV01`, multicast `01:0C:CD:04:00:01`, 4000 frames/s and `smpCnt=0…3999`. -* **PTP server simulation:** Layer-2 Announce, Sync, hardware-timestamp-derived Follow_Up, 1 Hz cadence, domain 0, EMAC IEEE 1588 timebase with laboratory simulated Global GM / SOURCE profile. Hardware-assisted timestamp and SV phase/MAC observation paths remain enabled. -* **Studio:** 4I+4V scenario controls, explicit START/STOP, PTP SOURCE, `SMPSYNCH AUTO`, waveform/phasor preview, packet capture, verified ESP32-P4 serial identity, and faster automatic USB reconnect without automatic SV START. +- **SV + PTP in one lab setup** — real Ethernet SV at 4,000 frames/s alongside Layer-2 PTP Announce, Sync and Follow_Up. +- **Hardware-assisted synchronization** — ESP32-P4 EMAC timestamp/phase evidence supports `SMPSYNCH AUTO` and qualified lab `smpSynch=2`. +- **Simple desktop control** — adjust 4I+4V magnitudes and phase, use explicit Start/Stop, preview waveforms and capture traffic. +- **Matched firmware included** — Windows Setup and Portable packages contain the ESP32-P4 firmware, firmware manifest and installation utility. -**Independent on-wire capture provided by the engineer, 2026-10-24, M3.H firmware `a922e9bdba4e57a4`:** +## Download and try it -| Evidence | Result | -|---|---| -| Captured SV span | 40.762934 seconds | -| SV packets | 163,048 | -| All `smpSynch` | **2 (100%; no 0 or 1)** | -| Missing/duplicate/reordered `smpCnt` | 0 / 0 / 0 | -| SV malformed / nonzero quality | 0 / 0 | -| Mean SV rate | 3999.884 fps | -| Host timestamp spacing median | 250 microseconds | -| PTP Announce | 44; seq 13–56, no gaps | -| PTP Sync + Follow_Up | 43 + 43; seq 13–55, paired, no gaps | -| PTP interval | 1 Hz approximately | -| Board telemetry | 0 missed, 0 TX failures; same boot and source during capture | -| Earlier M3.G sync flutter | **Not observed in M3.H capture** | +**[Latest stable release and assets](https://github.com/masarray/arstack61850/releases/latest)** · [Windows installation](../INSTALL.md) · [First-run tutorial](../GETTING_STARTED.md) -PCAP SHA-256 `7764c1477c357eb0e08b552073bd5cedd0816d8617c1fd587fff6cdf62e1b190`. The raw engineer capture is not bundled with the public installer. +1. Install ARStack Studio from Windows Setup EXE or use the Portable ZIP. +2. Connect a compatible ESP32-P4 Ethernet board on an isolated lab network. +3. Open **Configuration → Firmware** and install the firmware from **this same release**. Check the device build ID **`a922e9bdba4e57a4`**. +4. Choose **4I+4V Quick Start**, select **PTP SOURCE / INTERNAL_GM_ONLY** and **SMPSYNCH AUTO**, adjust the waveforms and press **Start**. +5. Inspect SV (`0x88BA`) and PTP (`0x88F7`) with Studio Capture, ARSAS, DigSubAnalyzer or Wireshark. -**Capture qualifier:** operator requested 120 s, but SV STOP interrupted capture after ~40.76 s, so Studio correctly marked the *120-second capture request* `INVALID_CAPTURE`. The 40.76 s recorded segment itself is contiguous. The stop cause is not established by that file. This does not demonstrate 120-second endurance or STOP/START endurance. +No external GPS or protection relay is needed for the isolated-lab emulator. **Never connect an unsolicited SV publisher or PTP grandmaster to a live substation process bus.** -## What is intentionally not claimed +## Measured Ethernet results -* Not tested with a connected relay/merging-unit subscriber: **real IED PTP lock, SV acceptance, absence of relay alarms, multi-vendor interoperability, and Pdelay exchange with a peer are unverified** until an IED becomes available. -* Simulated global PTP quality and `smpSynch=2` are **lab-emulation metadata**; without a real timing reference they do not certify UTC/GNSS traceability or sub-microsecond physical accuracy. -* Desktop-host PCAP timestamps are not hardware timestamp calibration. -* Only the supported 4I+4V, 4 kHz SV profile is advertised as implemented on ESP32-P4. +The independent M3.H engineer capture from 24 October 2026 recorded: -## Install / first run +| Measurement | Result | +| --- | --- | +| Continuous SV capture | 40.762934 seconds | +| SV frames | **163,048** | +| Measured SV rate | **3,999.884 frames/s** | +| `smpSynch=2` | **100%** of the recorded SV | +| Missing / duplicate / reordered counters | **0 / 0 / 0** | +| Malformed SV / nonzero quality | **0 / 0** | +| Host-capture spacing median | 250 microseconds | +| PTP Announce | 44 | +| PTP Sync / Follow_Up | 43 / 43 paired | -1. Download Windows Setup EXE (recommended) or Portable ZIP. Both include Qt dependencies, a pinned ESP32-P4 flasher, **matched firmware** and checksums. -2. Connect the ESP32-P4 to USB OTG and Ethernet on an **isolated test network**. Install bundled firmware using **Advanced → Firmware**; verify the board build ID equals **`a922e9bdba4e57a4`**. Merely installing Studio does not update board firmware. -3. Select **PTP Source** / `INTERNAL_GM_ONLY` and **`SMPSYNCH AUTO`**. Leave PTP enabled while stopping/starting SV. Set the 4I+4V amplitudes and phase angles; click **Start**. -4. Use ARStack Studio Capture, ARSAS, DigSubAnalyzer, or Wireshark to inspect multicast SV EtherType `0x88BA` and PTP EtherType `0x88F7`. -5. A successful lab session should have consecutive `smpCnt`, continuous 4 kHz samples, `smpSynch=2` once the on-board phase/MAC evidence is acquired, and paired 1 Hz PTP Announce/Sync/Follow_Up. Stop is intentional and produces no SV packet. +**Reference Ethernet SV profile:** one ASDU, 4I+4V, eight INT32+Quality pairs (64 bytes), APPID `0x4000`, `svID=ARSTACK_SV01`, multicast `01:0C:CD:04:00:01`, `smpCnt=0…3999`. -**Electrical/network caution:** Never connect an unsolicited SV publisher or additional PTP grandmaster to a live substation process bus. Use an isolated VLAN/switch or dedicated laboratory LAN. +PTP source emulation uses Layer-2 Announce/Sync and hardware-timestamp-derived Follow_Up at approximately 1 Hz (domain 0). This is an internally simulated laboratory grandmaster profile; `smpSynch=2` denotes qualified **local emulation evidence**, not external UTC/GNSS calibration. -## Reproducible binary identity +### Measurement and reproduction notes -Software source and firmware image are pinned to **`a922e9bdba4e57a4e88406eb83344455dd8ff142`**, the SHA with **10/10 GitHub CI workflows passing**. Packaging workflow downloads immutable artifacts from successful run **`40061206222`**, verifies their exact checksums, and publishes without recompiling or changing any firmware timing code. +The raw capture is identified by SHA-256 `7764c1477c357eb0e08b552073bd5cedd0816d8617c1fd587fff6cdf62e1b190`. It was provided as independent engineer evidence and is not bundled in the installer. -* Setup EXE SHA-256: `4902e8bf44e78fc55a139ce84a1273f0768ef6fe10000f5d0f85d75d2f6c736e` -* Portable ZIP SHA-256: `ca01424e4102aa65e353737024ebf3258b4d2f614fa60d932c95c353ee228d8f` -* ESP32-P4 BIN SHA-256: `61c508e9bf1357eb964d2b817b260b6a0e985446ddd68de1d45d9467e25c4607` +The operator's requested 120-second capture was interrupted by a **SV STOP** after 40.763 seconds, so the 120-second capture request was marked `INVALID_CAPTURE`. The recorded 40.763-second segment itself is contiguous and valid. This file alone should not be represented as a 120-second endurance record. Extended internal observations and the archived PCAP are different evidence sources. -This is a **public laboratory simulator release**, not a certified MU/PTP reference device. Those are different product categories; external certification is not a release gate for this emulator. +External relay/BCU subscriber acceptance, multi-device PTP behavior and physical UTC time accuracy are outside the measurements in this specific capture. Engineers with external equipment are welcome to [share lab interoperability observations](../COMMUNITY_TESTING.md). + +## Build identity and downloadable artifacts + +The published Studio and firmware pair is pinned to source SHA `a922e9bdba4e57a4e88406eb83344455dd8ff142` (embedded Build ID `a922e9bdba4e57a4`); its source build passed 10/10 CI workflows. Integration to `main` later passed 14/14 candidate workflows through [PR #299](https://github.com/masarray/arstack61850/pull/299), without changing this shipped firmware/Studio source. + +| Artifact | SHA-256 | +| --- | --- | +| `ARStack-ProcessBus-M3H-win-x64-setup.exe` | `4902e8bf44e78fc55a139ce84a1273f0768ef6fe10000f5d0f85d75d2f6c736e` | +| `ARStack-ProcessBus-M3H-win-x64-portable.zip` | `ca01424e4102aa65e353737024ebf3258b4d2f614fa60d932c95c353ee228d8f` | +| `ARStack-ESP32P4-ProcessBus-M3H-firmware.bin` | `61c508e9bf1357eb964d2b817b260b6a0e985446ddd68de1d45d9467e25c4607` | + +The release includes `firmware-manifest.json` and `SHA256SUMS.txt`. The supplied Windows packages include a pinned board flasher; users do not need an embedded build toolchain. + +[Download the latest public release](https://github.com/masarray/arstack61850/releases/latest) · [Help and community testing](../../SUPPORT.md) diff --git a/index.html b/index.html index 13f96bb27..64f8f8ecc 100644 --- a/index.html +++ b/index.html @@ -9,8 +9,8 @@ - - + + @@ -57,9 +57,9 @@

Your own IEC 61850 Process Bus lab.

Real network traffic. Controllable lab scenarios.

A native Windows engineering surface controls an embedded real-time publisher. The sample clock and Ethernet transmission run on the ESP32-P4—not from a Windows application timer.

IEC 61850 Sampled Values

4 current + 4 voltage, 64-byte payload, one ASDU, 4 kHz, sequenced `smpCnt`, configurable waveform magnitudes and phase angles.

PTP Grandmaster simulation

Layer-2 Announce, Sync and hardware-timestamp-derived Follow_Up on an isolated bench. SOURCE retains clock continuity across operator SV STOP / START.

Studio & wire evidence

Windows Qt control, phasor/waveform previews, supervised firmware recovery, USB hotplug detection and independent PCAP capture/reporting.

ARStack StudioWindows setup · USB OTG control
ESP32-P4 EthernetEMAC hardware timestamp + 4 kHz SV grid
PTP · Announce / Sync / Follow_Up
SV · 4I+4V · smpSynch AUTO
-

Independent PCAP: 44 Announce, 43 Sync and 43 Follow_Up, contiguous sequences. M3.H capture was stopped at 40.763 seconds although Studio was configured to request 120 seconds. Physical relay lock and alarm-free SV acceptance remain untested; this is a laboratory emulator, not a UTC-traceable clock reference.

+

Independent PCAP: 44 Announce, 43 Sync and 43 Follow_Up, contiguous sequences. The M3.H reference capture documents measured SV/PTP traffic and locally qualified synchronization in an isolated laboratory setup. Detailed results and measurement context are available in the release technical record.

-

First packet in four steps.

Install StudioUse Windows Setup or Portable; both include the matched firmware and flashing tool.
Flash ESP32-P4Connect USB OTG and install the matching firmware from the release.
Select lab timingUse PTP SOURCE / INTERNAL_GM_ONLY and smpSynch AUTO on an isolated Ethernet LAN.
Start and captureRun 4I+4V, then inspect SV 0x88BA and PTP 0x88F7 in ARSAS, DigSubAnalyzer or Wireshark.

Safety boundary: Never attach the lab PTP GM or SV publisher to a live substation process bus. Use an isolated test LAN or VLAN. ARStack is an open-source experimental engineering tool; no GPS, relay or certification hardware is a prerequisite for use.

+

First packet in four steps.

Install StudioUse Windows Setup or Portable; both include the matched firmware and flashing tool.
Flash ESP32-P4Connect USB OTG and install the matching firmware from the release.
Select lab timingUse PTP SOURCE / INTERNAL_GM_ONLY and smpSynch AUTO on an isolated Ethernet LAN.
Start and captureRun 4I+4V, then inspect SV 0x88BA and PTP 0x88F7 in ARSAS, DigSubAnalyzer or Wireshark.

Safety boundary: Never attach the lab PTP GM or SV publisher to a live substation process bus. Use an isolated test LAN or VLAN. ARStack is an open-source IEC 61850 engineering lab tool; no external GPS or protection relay is needed for the basic simulation.

Download the latest release.

Choose the installer, portable application or matched board firmware from the latest stable GitHub release. Downloads update automatically when a new verified release is published.

Windows Installer

Windows x64 · setup and bundled dependencies

Setup EXE ↗
diff --git a/scripts/check_product_naming.py b/scripts/check_product_naming.py index bd1506fb4..5c55cbfd8 100644 --- a/scripts/check_product_naming.py +++ b/scripts/check_product_naming.py @@ -1,11 +1,13 @@ #!/usr/bin/env python3 -"""Keep public product copy vendor-neutral and prevent legacy competitor naming in code.""" +"""Keep user-facing ARStack copy consistent and vendor-neutral.""" from pathlib import Path import subprocess import sys ROOT = Path(__file__).resolve().parents[1] -# Split literals so this policy file does not itself introduce indexed brand mentions. + +# Keep competitor names out of product copy, without corrupting historical +# interoperability test fixtures and source-code regression evidence. PROHIBITED = ( "ied" + "scout", "omi" + "cron", @@ -14,23 +16,32 @@ ) PUBLIC_PATHS = ( "README.md", + "index.html", + "CONTRIBUTING.md", + "SUPPORT.md", + "docs/GETTING_STARTED.md", + "docs/INSTALL.md", + "docs/COMMUNITY_TESTING.md", + "docs/DEVELOPER_GUIDE.md", "apps/arstack_studio/README.md", "apps/ied_simulator/README.md", - "docs/INSTALL.md", "docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md", ) + tracked = subprocess.run( ["git", "ls-files", "-z"], cwd=ROOT, capture_output=True, check=True ).stdout.split(b"\0") violations = [] +tracked_paths = set() for item in tracked: if not item: continue relative = item.decode("utf-8", errors="surrogateescape") + tracked_paths.add(relative) lower_name = relative.casefold() if PROHIBITED[0] in lower_name: violations.append("filename: " + relative) - if relative not in PUBLIC_PATHS: + if relative not in PUBLIC_PATHS and not relative.startswith(".github/ISSUE_TEMPLATE/"): continue path = ROOT / relative if not path.is_file(): @@ -39,6 +50,11 @@ for term in PROHIBITED: if term in data: violations.append(f"public copy: {relative} contains restricted term") + +for required in ("docs/GETTING_STARTED.md", "docs/COMMUNITY_TESTING.md", "CONTRIBUTING.md", "SUPPORT.md"): + if required not in tracked_paths: + violations.append("missing public entrypoint: " + required) + if violations: sys.stderr.write("REPOSITORY_NAMING_GUARD_FAIL\n" + "\n".join(violations) + "\n") raise SystemExit(1) From d164dfbf56009bf795b457ad9972e0646b3d4dc1 Mon Sep 17 00:00:00 2001 From: Ari Sulistiono Date: Sat, 24 Oct 2026 18:23:14 +0700 Subject: [PATCH 2/2] docs: retain comprehensive developer reference and private security reporting --- .github/ISSUE_TEMPLATE/bug_report.yml | 6 +- CONTRIBUTING.md | 4 +- README.md | 2 +- docs/DEVELOPER_GUIDE.md | 368 ++++++++++++++++++++++++-- 4 files changed, 355 insertions(+), 25 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index 6e87653b5..067b1505b 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -6,7 +6,7 @@ body: - type: markdown attributes: value: | - Thanks for testing ARStack! Short reports are welcome. Please remove confidential details from screenshots and configuration files. + Thanks for testing ARStack! Short reports are welcome. Remove confidential details from screenshots and files. For security vulnerabilities, follow the private reporting process in [SECURITY.md](../../SECURITY.md) rather than creating a public issue. - type: textarea id: problem attributes: @@ -35,9 +35,11 @@ body: attributes: label: Where did it happen? options: - - ARStack Studio + - ARStack Studio (Windows / Linux) - ESP32-P4 firmware / device connection - SCL or engineering file import + - IED Browser / Workbench + - IED Simulator - SV / PTP traffic - Build / installation - Other diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b50b33e90..f6e345a2f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,7 +19,9 @@ You can contribute without writing C++: improve a tutorial, report a reproducibl 4. Attach a screenshot for user-interface changes when possible. 5. Keep public copy **vendor-neutral**, friendly to first-time users, and accurate about supported workflows. -Please do not submit proprietary firmware, confidential substation configurations or credentials. +Never submit proprietary firmware, confidential substation configurations, credentials, or private customer captures. Security vulnerabilities should be reported privately using [SECURITY.md](SECURITY.md), not in public issues. + +Keep new work on current `main` and do not modify immutable release tags or published binaries. For changes involving hardware timing or packet behavior, include measured evidence when available. ## Product and engineering documentation diff --git a/README.md b/README.md index 160e74f3a..13feab8df 100644 --- a/README.md +++ b/README.md @@ -38,7 +38,7 @@ For active SV/PTP transmission, use an **isolated laboratory Ethernet network**, | Windows Portable ZIP | **Available** in the [latest release](https://github.com/masarray/arstack61850/releases/latest) | | Matched ESP32-P4 firmware | **Available** with the same release | | Linux AppImage | Shown in the [latest release](https://github.com/masarray/arstack61850/releases/latest) when a tested package is published | -| Windows package manager | [WinGet preparation and status](docs/INSTALL.md#windowspackage-manager--winget) | +| Windows package manager | [WinGet preparation and status](docs/INSTALL.md#windows-package-manager--winget) | Every published Windows release includes the application dependencies and a matched firmware package; no C++ build environment is required for ordinary installation. diff --git a/docs/DEVELOPER_GUIDE.md b/docs/DEVELOPER_GUIDE.md index 721e09120..dbdc5aee1 100644 --- a/docs/DEVELOPER_GUIDE.md +++ b/docs/DEVELOPER_GUIDE.md @@ -1,36 +1,362 @@ -# ARStack developer documentation +# ARStack61850 — Technical & Developer Reference -This index is for engineers who want to study or extend ARStack's source code. If you are new to ARStack, start with the [user-facing quick start](GETTING_STARTED.md). +The [main README](../README.md) is a short, engineer-friendly entry point. This document preserves the full implementation notes, portable C++ library status, test commands and architecture references for contributors. -## Architecture and project direction +Maintained source of truth: [NORTH_STAR](../NORTH_STAR.md), [AGENTS.md](../AGENTS.md), and [current release notes](releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md). -- [Project north star](../NORTH_STAR.md) — overall protocol, execution and hardware design. -- [Project handoff](../PROJECT_HANDOFF.md) — current integration context for maintainers. -- [Studio architecture and operator workflow](../apps/arstack_studio/README.md). -- [ESP32-P4 firmware architecture](../embedded/esp32p4_smv_injector/ARCHITECTURE.md). +--- -## Protocol and engineering tools +## Start here -- [Feature matrix](FEATURE_MATRIX.md) — implemented components and evidence categories. -- [SCL injection workflow](LIVE_INJECTION_WORKFLOW.md). -- [MMS file transfer](MMS_FILE_TRANSFER.md). -- [SCL-assisted MMS connection](SCL_ASSISTED_MMS_CONNECT_PROFILE.md). -- [PTP timing and lab implementation](PTP_LAB_TIMING.md). +For maintainers and contributors making architecture changes, start with these references: -## Release and test evidence +1. [`NORTH_STAR.md`](../NORTH_STAR.md) — product thesis, standards, hardware roadmap and definition of success. +2. [`docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md`](../docs/releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md) — latest public process-bus laboratory simulator evidence, setup and release boundary; [`v0.1.0 historical evidence`](../docs/SMV_STUDIO_V0.1.0_PUBLIC_RELEASE.md). +3. [`PROJECT_HANDOFF.md`](../PROJECT_HANDOFF.md) — current repository handoff and separation between released SMV work and active follow-on tracks. +4. [`apps/arstack_studio/README.md`](../apps/arstack_studio/README.md) — operator workflow, firmware recovery and Windows distribution details. +5. [`docs/ONLINE_MODEL_CONNECT_DECISION.md`](../docs/ONLINE_MODEL_CONNECT_DECISION.md) — short decision contract separating full live discovery from SCL-assisted online connect. +6. [`LIVE_DISCOVERY_PROFILE.md`](../LIVE_DISCOVERY_PROFILE.md) — detailed live-discovery and online-model interoperability boundary. +7. [`embedded/esp32p4_smv_injector/ARCHITECTURE.md`](../embedded/esp32p4_smv_injector/ARCHITECTURE.md) — embedded real-time architecture boundaries. +8. [`docs/GLOBAL_BENCHMARK.md`](../docs/GLOBAL_BENCHMARK.md) — vendor-neutral standards/industry research behind the direction. -- [M3.H release record](releases/PROCESS_BUS_SIMULATOR_M3H_PUBLIC_20261024.md). -- [Mainline release/source reconciliation](integration/M3H_MAINLINE_RELEASE_SOURCE_CLOSURE_2026-10-24.md). -- [Historical milestone notes](SMV_STUDIO_V0.1.0_PUBLIC_RELEASE.md). +The project deliberately separates **wire facts**, **configured expectations**, **profile claims**, and **synchronization evidence**. A packet that decodes in a host capture tool is not, by itself, a conformance or timing claim. -## Building +## Current process-bus simulator milestone -Portable C++20 core: +**M3.H Lab Edition is published publicly** as a stable lab-simulator SV publisher + internal PTP grandmaster emulator. The 40.76-second raw PCAP validates stable lab traffic and hardware-evidence-qualified `smpSynch=2`; it does not prove a subscriber relay would accept or lock without alarms. No external GPS/certification is a dependency for running or releasing this lab simulator. + +The older v0.1.0/v0.1.1 products remain available as historical releases. Further work: independent relay/IED interoperability when hardware is available, deeper 120-second and multi-cycle captures, alternate SV profiles, and optional external time references—not blockers for M3.H public distribution. + +## Lineage and portable core + +The original work began as an incremental C++20 migration of the ARIEC61850 C# protocol stack. The C# implementation remains a behavioral oracle where cross-language equivalence is still useful, while ARStack-specific work adds deterministic RTOS/hardware execution and evidence that a normal desktop stack cannot provide. + +The project is **not yet a complete IEC 61850 replacement stack**. Current +Public Alpha work focuses on a useful read-only MMS engineering client, process +bus codecs/runtime foundations, reproducible cross-language evidence, and an +MCU-safe protocol boundary. + +## Implemented + +### Phase 0 — portable foundation + +- CMake-based C++20 static library. +- BER and MMS data/UTC-time codecs. +- Ethernet/VLAN process-bus framing and classic PCAP support. +- GOOSE and Sampled Values codecs plus offline/runtime supervision foundations. +- SCL read-only parsing and COMTRADE CFG/DAT support for host engineering tools. +- RFC 1006 TPKT, COTP, ISO Session, Presentation, ACSE, and MMS Initiate. +- Confirmed MMS GetNameList, GetVariableAccessAttributes, Read, and Write codecs. +- Read-only MMS FileDirectory/FileOpen/FileRead/FileClose codecs and bounded streaming runtime. +- DataSet directory, InformationReport, RCB state, and report-subscription foundations. + +### Phase 1A–1E — deterministic process-bus core + +- IEC 61850 UTC time and recursive MMS Data/AllData codec. +- Complete GOOSE PDU/frame codecs and offline publisher/subscriber supervision. +- Sampled Values ASDU, multi-ASDU PDU, Ethernet frame, payload, quality, counter, and stream supervision. +- C#-derived synthetic PCAP equivalence evidence. +- Sanitizer, deterministic mutation-smoke, and LLVM libFuzzer coverage. +- Receive-only isolated-laboratory tooling. + +A codec existing in the library does **not** mean the corresponding mutating +operation is enabled in live Phase 4C discovery. + +### Phase 2A — SCL foundation + +- Bounded, dependency-free XML/SCL reader. +- IED, DataSet, FCDA, GOOSE, Sampled Values, report-control, and communication mapping. +- SCL edition detection, type-template resolution, DataSet binding, diagnostics, and SCL/XML fuzzing. + +### Phase 2B — COMTRADE foundation + +- CFG parsing for IEEE C37.111-style 1991/1999/2013 records. +- ASCII, BINARY, BINARY32, and FLOAT32 DAT readers. +- Analog engineering scaling (`a * raw + b`) and packed digital status decoding. +- Timestamp multiplier and multi-rate fallback scheduling. +- Default `Va/Vb/Vc/Vn/Ia/Ib/Ic/In` channel mapping. +- Deterministic SCL Sampled Values to COMTRADE channel binding. +- Read-only `ariec61850_comtrade_inspect` human/JSON CLI. +- C#-derived ASCII and binary fixtures, mutation smoke, sanitizer, and libFuzzer corpus. + +### Phase 3A — MMS transport foundation + +- RFC 1006 TPKT frame encode/decode with strict version, reserved-octet, and length validation. +- Incremental TPKT stream framing for fragmented and coalesced TCP byte streams. +- COTP Connection Request, Connection Confirm, Data, Disconnect Request, and Error TPDU decoding. +- Default C#-compatible CR vector, TPDU-size negotiation, and TSAP selector mirroring. +- COTP Data segmentation by negotiated TPDU size and bounded EOT reassembly. +- Fragment-count, byte-count, and empty non-final fragment abuse guards. +- C#-derived golden vectors, deterministic mutation smoke, sanitizer, and libFuzzer coverage. + +### Phase 3B — MMS association codecs + +- Bounded ISO Session Connect/Accept SPDU and data-transfer profile codecs. +- Presentation CP/CPA context definitions, negotiation results, selectors, and PDV routing. +- ACSE and MMS context negotiation using BER transfer syntax. +- Structured ACSE AARQ/AARE application context, AP-title, AE-qualifier, result, and diagnostic parsing. +- EXTERNAL user-information handling with MMS Initiate retained as an opaque Phase 3C payload. +- Deterministic association-response construction with session and context mirroring. +- Byte-exact C# request/response golden vectors, mutation smoke, sanitizer, and libFuzzer coverage. + +### Phase 4C — live read-only MMS discovery + +`ariec61850_live_discover` currently provides: + +- built-in Windows/POSIX TCP transport; +- MMS association through the implemented OSI stack; +- domain and variable GetNameList discovery; +- named-variable-list/DataSet inventory; +- bounded variable-type probes; +- bounded RCB read evidence; +- LD/LN/DO/DA live-model projection; +- IED identity inference; +- CDC/type heuristics compatible with the C# oracle; +- GOOSE/SV/setting-group/log control-block name inventory; +- dynamic RCB `Bound` / `Unbound` / `NotRead` / `ReadFailed` evidence; +- read-only RCB candidate planning and conservative operational availability. + +The live discovery surface sends only read-only services such as GetNameList, +GetVariableAccessAttributes, GetNamedVariableListAttributes and Read. It does +**not** authorize or perform Write, control, GI, RCB reservation/enable, +dynamic DataSet mutation, or file-service mutation. + +### Phase 4E - MMS file transfer client + +`ariec61850_file_transfer` adds an explicit read-only file-service surface: + +- root/nested FileDirectory with bounded continuation; +- streaming FileOpen/FileRead/FileClose download to a caller-owned sink; +- full signed Integer32 FRSM support; +- exact Confirmed-Error evidence and bounded diagnostics; +- best-effort FileClose that preserves the primary failure; and +- a single oracle-matched rooted-backslash fallback for file-non-existent. + +The default CLI operation is directory listing. Download requires both +`--remote` and `--output`; upload/delete/rename are not implemented. See +`docs/MMS_FILE_TRANSFER.md` for API, safety, CLI, exit-code, and acceptance +details. + +#### Public Alpha A1 model work + +The live model now projects C#-style: + +- `proposedLnTypeId` and `proposedDoTypeId`; +- `typeTemplates`; +- `variableTypeDiscoveries`; +- deterministic IED identity evidence; +- DataSet/RCB/control-block inventory. + +Dynamic report configuration is separated from static model identity with three +fingerprints during migration: + +- `fingerprint` — legacy canonical fingerprint retained for compatibility; +- `structuralFingerprint` — LD/LN/DO/DA + control identities, excluding mutable + DataSet/RCB runtime state; +- `runtimeSnapshotFingerprint` — current DataSet inventory and RCB binding/state. + +For IEDs with dynamic DataSets, a changed runtime fingerprint with an unchanged +structural fingerprint is expected and does not by itself mean the IED model or +firmware changed. + +### Phase 3C — MMS service codecs + +- MMS Initiate request/response and confirmed request/response/error envelopes. +- Invoke-ID routing, ObjectName, GetNameList, GetVariableAccessAttributes, Read, and Write. +- Recursive bounded TypeSpecification and mixed success/failure access results. +- C#-derived vectors, sanitizer, deterministic mutation smoke, and libFuzzer coverage. + +### Phase 4A — offline DataSet and report monitoring + +- DataSet and buffered/unbuffered RCB inventory from supplied GetNameList evidence. +- GetNamedVariableListAttributes request/response and IEC reference normalization. +- Strict InformationReport and OptFlds-driven report-frame decoding. +- RCB attribute read-plan/state mapping with cautious availability confidence. +- Sequence, configuration, DataSet, overflow, and segmentation continuity tracking. +- Bounded offline report monitor and dedicated MMS-reporting fuzz corpus. + +### Hardware process-bus runtime — ESP32-P4 P0/P1/P2 foundation + +The portable core still does not silently open raw sockets or physical IED sessions. **Active process-bus transmission exists only in explicitly enabled hardware/lab targets**, currently `embedded/esp32p4_smv_injector`, and is intended for isolated bench use. + +The current ESP32-P4 target adds: + +- RMII hardware bring-up; +- active Layer-2 SV TX; +- fixed/prebuilt runtime templates; +- deterministic GPTimer cadence; +- separate canonical vs diagnostic stream identities; +- 4I+4V balanced waveform generation; +- once-per-second realtime telemetry; +- a path toward EMAC hardware timestamps and PTP-disciplined synchronization. + +See the embedded README/architecture files and `PROJECT_HANDOFF.md` for the exact current state. + +## Phase 4B: association and report-subscription runtime + +The C++ core now includes a transport-injected MMS association runtime and persistent RCB subscription state machine. It performs COTP, Session, Presentation, ACSE/MMS Initiate, confirmed-service routing, timeout/cancellation handling, RCB re-probe, URCB reservation, `RptEna`, optional GI, InformationReport delivery, and ownership-aware cleanup. See `ASSOCIATION_RUNTIME_PROFILE.md`. + +The core deliberately does not ship a default socket transport in this phase. Applications that intentionally enable live IED access must provide an `MmsByteTransport` implementation. + +## Build + +### Windows + +```powershell +cmake -S . -B build -A x64 -DARIEC61850_WARNINGS_AS_ERRORS=ON +cmake --build build --config Release --parallel +ctest --test-dir build -C Release --output-on-failure +``` + +### Linux ```bash -cmake -S . -B build +cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DARIEC61850_WARNINGS_AS_ERRORS=ON cmake --build build --parallel ctest --test-dir build --output-on-failure ``` -Desktop Studio requirements and developer commands are in the [Studio README](../apps/arstack_studio/README.md). Product installation does not require compiling from source. +### Sanitizers + +```bash +CC=clang CXX=clang++ cmake -S . -B build-sanitizers \ + -DCMAKE_BUILD_TYPE=RelWithDebInfo \ + -DARIEC61850_ENABLE_SANITIZERS=ON +cmake --build build-sanitizers --parallel +ctest --test-dir build-sanitizers --output-on-failure +``` + +## Read-only tools + +### Inspect a COMTRADE record + +```powershell +.\build\Release\ariec61850_comtrade_inspect.exe .\record.cfg --json +``` + +### Analyze a saved process-bus PCAP + +```powershell +.\scripts\run-lab-check.ps1 -Pcap .\captures\device-session.pcap +``` + +## Live discovery quick start + +Human summary: + +```powershell +.\build\Release\ariec61850_live_discover.exe 192.168.1.10 102 +``` + +Bounded read-only RCB evidence: + +```powershell +.\build\Release\ariec61850_live_discover.exe ` + 192.168.1.10 102 ` + --timeout-ms 30000 ` + --no-types ` + --no-datasets ` + --max-rcb 50 ` + --rcb-plan dynamic ` + --rcb-availability +``` + +Model JSON with bounded type discovery: + +```powershell +New-Item -ItemType Directory -Force .artifacts\parity | Out-Null + +.\build\Release\ariec61850_live_discover.exe ` + 192.168.1.10 102 ` + --timeout-ms 120000 ` + --no-datasets ` + --no-rcb ` + --max-types 256 ` + --model-json ` + | Set-Content -Encoding utf8 .artifacts\parity\cpp-model.json +``` + +Inspect fingerprint/type evidence: + +```powershell +$model = Get-Content .artifacts\parity\cpp-model.json -Raw | ConvertFrom-Json +$model.fingerprint +$model.structuralFingerprint +$model.runtimeSnapshotFingerprint +$model.typeTemplates.Count +$model.variableTypeDiscoveries.Count +``` + +## Same-IED C# ↔ C++ parity + +The canonical comparator is: + +```powershell +python .\tools\compare_live_model_json.py ` + .artifacts\parity\csharp\ied-model.json ` + .artifacts\parity\cpp-model.json +``` + +Optional evidence: + +```powershell +python .\tools\compare_live_model_json.py ` + .artifacts\parity\csharp\ied-model.json ` + .artifacts\parity\cpp-model.json ` + --types ` + --runtime +``` + +Default pass/fail is structural. Runtime DataSet/RCB drift is informational. +The historical `scripts/compare-live-model-parity.py` path remains only as a +compatibility wrapper around this comparator. + +See `docs/LIVE_MODEL_PARITY.md` for the complete C# export + comparison flow. + +## Embedded direction + +The wire core is being separated from host engineering services. Current target +roles are documented in `docs/EMBEDDED_TARGETS.md`: + +- ESP32-P4-ETH: protocol/performance reference after Public Alpha; +- Waveshare ESP32-S3-POE-ETH-8DI-8DO: real I/O application reference; +- STM32H7/NXP class: later portability/industrialization target. + +The embedded profile already excludes host TCP/live-discovery/PCAP/COMTRADE/SCL +parser tooling and provides a no-RTTI host-simulation gate. The portable MMS +file-service runtime is included through abstract transport/sink boundaries; +its filesystem-backed CLI remains host-only. Sampled Values also +has a caller-owned `encode_into(span)` path so the eventual publisher does not +need to allocate a fresh Ethernet frame buffer per sample. + +## Validation boundary + +Implemented code and green CI do not equal full field acceptance. Current +validation includes GCC/Clang/MSVC builds, deterministic tests, sanitizer/fuzz +workflows, and selected live read-only interoperability evidence. Full +replacement claims still require same-IED C#↔C++ evidence, repeated live cycles, +packet capture review, and multi-vendor physical/simulator interoperability. + +See `docs/FEATURE_MATRIX.md` for a conservative status table and +`PHASE_4C1_LIVE_MODEL_INTEROP.md` for the live-evidence boundary. + +See `REPORTING_PROFILE.md` for the offline report decoder profile, `LAB_INTEROP_CHECKLIST.md` before using a physical IED, and `MIGRATION_CHECKLIST.md` for the detailed progress ledger. + +## North-star reminder + +ARStack is not intended to become a UI-heavy packet viewer with timing bolted on later. The architecture priority is: + +```text +standards truth + > measured hardware evidence + > deterministic execution + > interoperability evidence + > usability + > convenience hacks +``` + +The long-term product family — injector, process-bus analyzer, digital-substation probe and real-time MU/closed-loop simulator — should share the same real-time/evidence core. + +## Public-repository naming policy + +Public project documentation should remain vendor-neutral. Do not add commercial competitor names, product comparisons, logos, screenshots, or vendor marketing references. Standards, protocol identifiers, and technical platform/dependency names required to build or reproduce the project are allowed.