diff --git a/README.en.md b/README.en.md index 46d069a..09189b8 100644 --- a/README.en.md +++ b/README.en.md @@ -13,15 +13,19 @@ with low latency. ## 📸 Screenshots -**Dashboard — dozens of mappings, live status, detected ports:** +**Dashboard — dozens of mappings, live status, detected ports and the LAN cluster panel:** ![Dashboard](docs/screenshots/02-dashboard.png) +**LAN cluster — every node's mappings in one table, each row tagged with its host (name + IP):** + +![Cluster](docs/screenshots/06-cluster.png) + | Add mapping (all serial/network options) | In-browser serial console (xterm.js) | |---|---| | ![Mapping form](docs/screenshots/03-add-mapping.png) | ![Console](docs/screenshots/05-console.png) | -| Settings (password · TLS · backup) | Login | +| Settings (password · TLS · cluster · backup) | Login | |---|---| | ![Settings](docs/screenshots/04-settings.png) | ![Login](docs/screenshots/01-login.png) | @@ -225,9 +229,10 @@ bound without TLS. For LAN deployments, use TLS (`admin_ui.tls_*`) and per-mappi ## 🗂️ Configuration & state files -All state lives in the **data dir** (default `data/`): -- `config.json` — admin IP, password hash, all mappings (atomic write; owner-only - perms: 0600 on POSIX, `icacls` owner/SYSTEM/Administrators on Windows). +All state lives in the **data dir** (default `data/`). The data directory is locked to +owner-only (POSIX `0700`; Windows `icacls` owner/SYSTEM/Administrators), so the files +inside aren't readable by other users: +- `config.json` — admin IP, password hash, all mappings (atomic write). - `all.log` — global activity/audit; `audit.log` — config changes. - `logs/.log` — per-mapping history (hourly maintenance: trimmed >15 days / >100 MB). - `tls/` — certificate/key if self-signed is generated. diff --git a/README.md b/README.md index 7778f5a..9bb383a 100644 --- a/README.md +++ b/README.md @@ -14,15 +14,19 @@ web arayüzünden yönetilen, çapraz-platform (Windows + Linux) saf-Python köp ## 📸 Ekran Görüntüleri -**Pano — onlarca eşleme, canlı durum, algılanan portlar:** +**Pano — onlarca eşleme, canlı durum, algılanan portlar ve LAN cluster paneli:** ![Pano](docs/screenshots/02-dashboard.png) +**LAN cluster — ağdaki tüm düğümlerin eşlemeleri tek tabloda, her satırda host (ad + IP):** + +![Cluster](docs/screenshots/06-cluster.png) + | Eşleme ekleme (tüm seri/ağ seçenekleri) | Tarayıcı-içi seri konsol (xterm.js) | |---|---| | ![Eşleme formu](docs/screenshots/03-add-mapping.png) | ![Konsol](docs/screenshots/05-console.png) | -| Ayarlar (parola · TLS · yedek) | Giriş | +| Ayarlar (parola · TLS · cluster · yedek) | Giriş | |---|---| | ![Ayarlar](docs/screenshots/04-settings.png) | ![Giriş](docs/screenshots/01-login.png) | @@ -224,9 +228,10 @@ Raw TCP düz-metindir — güvensiz ağlarda dikkat. Allowed/priority listesinde ## 🗂️ Yapılandırma & durum dosyaları -Tüm durum **veri dizininde** (varsayılan `data/`): -- `config.json` — admin IP, parola hash'i, tüm eşlemeler (atomik yazım; sahibe özel - izinler: POSIX'te 0600, Windows'ta `icacls` ile sahip/SYSTEM/Administrators). +Tüm durum **veri dizininde** (varsayılan `data/`). Veri dizini yalnızca sahibe açık +kilitlenir (POSIX `0700`; Windows'ta `icacls` ile sahip/SYSTEM/Administrators), böylece +içindeki dosyalar başka kullanıcılarca okunamaz: +- `config.json` — admin IP, parola hash'i, tüm eşlemeler (atomik yazım). - `all.log` — global etkinlik/audit; `audit.log` — config değişiklikleri. - `logs/.log` — eşleme başına geçmiş (saatlik bakım: >15 gün veya >100MB kırpılır). - `tls/` — self-signed üretilirse sertifika/anahtar. diff --git a/ROADMAP.md b/ROADMAP.md index e9cacaf..45569ab 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,5 +1,7 @@ # ser2net — Roadmap +> Current release: **v2.5.0**. CI is green across Linux + Windows × Python 3.10–3.13. + ## Shipped ### v1.0 — MVP @@ -29,8 +31,6 @@ - **Admin TLS** via cert/key paths or `openssl` self-signed generation - **Mappings export/import** (JSON, no secrets) + duplicate ---- - ### v2.1a — in-browser serial console - xterm.js terminal over an authenticated WebSocket (`/api/mappings/{id}/console`): read live traffic, or type to the device (net mappings, not read-only) @@ -49,9 +49,7 @@ - **Docker** image + `docker-compose` + headless bind env; **CI** (GitHub Actions: ruff + ubuntu/windows × Python 3.10–3.13); unified cross-platform test runner -### v2.2 — commercial features (Phase 2) -- **REST API** (`/api/v1`): bearer-token JSON API — mapping CRUD, start/stop/restart, - status, ports, OpenAPI 3.0 spec +### v2.2 — Modbus, RBAC, LDAP, MQTT (Phase 2) - **Modbus RTU↔TCP gateway** (`protocol=modbus`): multi-master, bus-locked transactions, txn-id integrity, `0x0B` timeout; reply-unit validation. **Edge mode:** periodic register polling (uint/int/float 16/32, scaling) published to MQTT @@ -61,33 +59,55 @@ group→role mapping, shadow accounts on the RBAC model - **MQTT publishing** (optional `paho-mqtt`): per-mapping serial-line → broker with retained birth/death -- **Client-side virtual COM** recipes (`docs/VIRTUAL-COM.md`); refreshed UI screenshots +- **Client-side virtual COM** recipes (`docs/VIRTUAL-COM.md`) + +### v2.3 — UX & access polish +- **OIDC single sign-on** (authorization-code flow + issuer discovery; claim→role + mapping, shadow accounts) — `app/web/oidc_auth.py`, optional `authlib` +- **REST API token roles** (`admin` / `operator` / `viewer`; `viewer` is read-only) +- **Light / dark theme toggle** (persisted in `localStorage`); xterm fit-to-window addon +- _(i18n was dropped — the UI stays English by decision.)_ + +### v2.4 — packaging & distribution +- **PyInstaller standalone builds** (Windows `.exe`, Linux binary) bundling the optional + MQTT/LDAP/OIDC deps; GitHub Actions release artifacts published on `v*` tags + +### v2.5 — LAN cluster +- **Auto-discovery + unified fleet view**: nodes find each other via HMAC-signed UDP + broadcast beacons (no mDNS) and one node aggregates every node's mappings into a + single **read-only** table, each row tagged with its host (name + IP). Opt-in, off by + default; trust = a shared cluster key. Server-side fan-out to peers' key-guarded + `/api/cluster/local`; the browser only talks to the node it logged into. --- ## Planned -### v2.3 — UX & access polish -- **OIDC / SAML browser-SSO** (next auth step after LDAP; auth-code flow + JWKS) -- **REST API token scopes/roles** (currently a single admin-level token) + per-token -- i18n (TR/EN) for the UI; dark/light theme toggle; xterm fit-to-window addon +### v2.6 — cluster depth & hardening +- **Remote control** from the unified view: start/stop/edit a peer's mappings (the read + path exists; add a key-guarded write endpoint + role check on the target node) +- **Per-node health/uptime** column; surface a node-level "discovery disabled" status in + the UI when the UDP bind fails (today it only logs) +- **Manual peer list** for routed/L3 networks where broadcast doesn't reach (complement + to auto-discovery) +- **Security hardening** (from the v2.5 review): validate/curb peer-advertised IPs before + server-side fetch (SSRF defense-in-depth), optional TLS certificate pinning for peer + fetch, a light rate-limit on `/api/cluster/local`, optional IPv6 (multicast) discovery + +### v2.7 — industrial/IIoT depth +- **Sparkplug B** edge payloads (Modbus register + MQTT plumbing already in place) +- **Modbus write** support (FC 5/6/15/16), per-point MQTT→register control, RTU + inter-frame tuning +- **RS-485 hardware auto-RTS** (`TIOCSRS485`) UI -### v2.4 — packaging & distribution -- ~~PyInstaller standalone builds (Windows `.exe`, Linux binary) bundling the optional - MQTT/LDAP/OIDC deps; GitHub Actions release artifacts on `v*` tags~~ ✅ shipped -- Windows service installer (Shawl); `.deb` / `.rpm` -- Automated multi-platform offline wheelhouse (cp311–cp313 × OS) for source installs +### v2.8 — packaging & migration +- Windows service installer (Shawl); `.deb` / `.rpm` packages +- Automated multi-platform offline wheelhouse (cp310–cp313 × OS) for source installs +- Classic `ser2net.yaml` import for migration -### v2.5 — industrial/IIoT depth -- **Sparkplug B** edge payloads (Modbus register + MQTT plumbing already in place) -- Modbus: write support (FC 5/6/15/16), per-point MQTT→register control, RTU inter-frame - tuning; RS-485 hardware auto-RTS (`TIOCSRS485`) UI -- ~~LAN **cluster dashboard**: nodes auto-discover each other (signed UDP broadcast, - no mDNS) and one node shows every node's mappings in a single read-only table, each - row tagged with its host (name + IP)~~ ✅ shipped (v2.5a) -- Cluster: remote control (start/stop/edit a peer's mappings from one screen) + per-node - health/uptime in the unified view -- classic `ser2net.yaml` import for migration +### Maintenance +- Bump pinned GitHub Actions off Node 20 (deprecated 2026-06-16) to current + `actions/*@v5` to clear the CI/release deprecation warnings ### Icebox / conditional - Thread-per-port serial backend — only if a real Windows high-throughput / @@ -95,7 +115,8 @@ - Per-mapping TLS client certificate verification (mTLS) for data bridges - Pluggable persistent session store -> Note: mDNS/zeroconf advertising has been removed from the roadmap. +> Note: mDNS/zeroconf advertising is intentionally **not** on the roadmap; LAN cluster +> discovery uses signed UDP broadcast instead. --- diff --git a/app/config.py b/app/config.py index ba8cbf1..7fbdb1d 100644 --- a/app/config.py +++ b/app/config.py @@ -554,7 +554,11 @@ def validate(self) -> None: return if not self.key.strip(): raise ConfigError("Cluster is enabled but no shared key is set.") - if not (1 <= int(self.discovery_port) <= 65535): + try: + port = int(self.discovery_port) + except (TypeError, ValueError): + raise ConfigError("Cluster discovery port must be a number.") from None + if not (1 <= port <= 65535): raise ConfigError("Cluster discovery port must be 1..65535.") if self.advertise_ip.strip(): try: diff --git a/app/engine/cluster.py b/app/engine/cluster.py index c39e0da..08d0160 100644 --- a/app/engine/cluster.py +++ b/app/engine/cluster.py @@ -141,8 +141,13 @@ async def start(self) -> None: sock.close() self.log(f"cluster: could not bind UDP {port}: {e} — discovery disabled") return - self._transport, _ = await loop.create_datagram_endpoint( - lambda: _BeaconProtocol(self.handle_datagram), sock=sock) + try: + self._transport, _ = await loop.create_datagram_endpoint( + lambda: _BeaconProtocol(self.handle_datagram), sock=sock) + except OSError as e: + sock.close() # endpoint creation owns the socket only on success + self.log(f"cluster: could not start UDP listener on {port}: {e} — discovery disabled") + return self._running = True self._beacon_task = asyncio.create_task(self._beacon_loop(), name="cluster-beacon") self.log(f"cluster: discovery on UDP {port} as '{socket.gethostname()}' " diff --git a/bootstrap.py b/bootstrap.py index 9306cab..d2164fb 100644 --- a/bootstrap.py +++ b/bootstrap.py @@ -2,7 +2,7 @@ On first run this installs the bundled wheels from ``vendor/wheels/`` into a local ``lib/`` directory (added to ``sys.path`` by ``ser2net.py``), so the app -runs on any machine that merely has a Python 3.11+ interpreter — no internet, +runs on any machine that merely has a Python 3.10+ interpreter — no internet, no virtualenv, no system-wide installs required. It is idempotent: if every dependency already imports, it does nothing. diff --git a/docs/screenshots/01-login.png b/docs/screenshots/01-login.png index bdb7863..495d393 100644 Binary files a/docs/screenshots/01-login.png and b/docs/screenshots/01-login.png differ diff --git a/docs/screenshots/02-dashboard.png b/docs/screenshots/02-dashboard.png index 90c1f36..8a23d7c 100644 Binary files a/docs/screenshots/02-dashboard.png and b/docs/screenshots/02-dashboard.png differ diff --git a/docs/screenshots/03-add-mapping.png b/docs/screenshots/03-add-mapping.png index c123673..956399f 100644 Binary files a/docs/screenshots/03-add-mapping.png and b/docs/screenshots/03-add-mapping.png differ diff --git a/docs/screenshots/04-settings.png b/docs/screenshots/04-settings.png index b6cb16e..3580a8a 100644 Binary files a/docs/screenshots/04-settings.png and b/docs/screenshots/04-settings.png differ diff --git a/docs/screenshots/06-cluster.png b/docs/screenshots/06-cluster.png new file mode 100644 index 0000000..7a3c386 Binary files /dev/null and b/docs/screenshots/06-cluster.png differ diff --git a/tools/_capture_screenshots.py b/tools/_capture_screenshots.py index 9799d71..d011c15 100644 --- a/tools/_capture_screenshots.py +++ b/tools/_capture_screenshots.py @@ -3,9 +3,11 @@ pip install playwright && python -m playwright install chromium python tools/_capture_screenshots.py -Seeds a few (disabled) demo mappings, completes first-run setup, adds an operator -and a viewer so the Users panel is populated, then captures the login, dashboard, -add-mapping form and settings pages. Not part of the app or the test suite. +Starts TWO ser2net nodes that share a cluster key so the dashboard's LAN-cluster +panel shows a real multi-host view, completes first-run setup on the primary node, +adds an operator + viewer (to populate the Users panel), then captures the login, +dashboard (with the cluster panel), add-mapping form, settings and cluster pages. +Not part of the app or the test suite. """ import json import os @@ -15,13 +17,31 @@ import tempfile import time -from playwright.sync_api import sync_playwright - ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +sys.path.insert(0, os.path.join(ROOT, "lib")) +sys.path.insert(0, ROOT) + +from playwright.sync_api import sync_playwright # noqa: E402 +from app.web.auth import hash_password # noqa: E402 + SHOTS = os.path.join(ROOT, "docs", "screenshots") -UI = 8099 +UI = 8099 # primary node (the browser drives this one) +UI_B = 8100 # second cluster node (data source for the fleet view) BASE = f"http://127.0.0.1:{UI}" +CLUSTER_KEY = "demo-cluster-key" + + +def free_port(): + s = socket.socket() + s.bind(("127.0.0.1", 0)) + p = s.getsockname()[1] + s.close() + return p + + +DPORT = free_port() # shared UDP discovery port for both nodes +# Primary node's mappings (the browser node). DEMO_MAPPINGS = [ {"name": "PLC-1", "enabled": False, "serial": {"port": "COM3", "baudrate": 9600}, "network": {"mode": "server", "protocol": "rfc2217", "bind_ip": "0.0.0.0", "port": 4001}}, @@ -36,6 +56,16 @@ "serial": {"port": "COM6", "baudrate": 115200}, "serial_b": {"port": "COM7", "baudrate": 115200}}, ] +# Second node's mappings (shown as a remote host in the fleet table). +DEMO_MAPPINGS_B = [ + {"name": "Robot-arm", "enabled": False, "serial": {"port": "/dev/ttyACM0", "baudrate": 115200}, + "network": {"mode": "server", "protocol": "raw", "bind_ip": "0.0.0.0", "port": 4001}}, + {"name": "Welder-3", "enabled": False, "serial": {"port": "COM4", "baudrate": 9600}, + "network": {"mode": "server", "protocol": "rfc2217", "bind_ip": "0.0.0.0", "port": 4002}}, + {"name": "Weigh-east", "enabled": False, "serial": {"port": "/dev/ttyUSB3", "baudrate": 19200}, + "network": {"mode": "server", "protocol": "raw", "bind_ip": "0.0.0.0", "port": 4003}}, +] + def wait_port(port, timeout=20): end = time.time() + timeout @@ -48,16 +78,32 @@ def wait_port(port, timeout=20): return False -def main(): +def start_node(cfg_obj): tmp = tempfile.mkdtemp(prefix="ser2net_shots_") cfg = os.path.join(tmp, "config.json") with open(cfg, "w") as fh: - json.dump({"admin_ui": {"bind_ip": "127.0.0.1", "port": UI}, "mappings": DEMO_MAPPINGS}, fh) - srv = subprocess.Popen([sys.executable, "ser2net.py", "--no-bootstrap", "--config", cfg], - cwd=ROOT, stdin=subprocess.DEVNULL, - stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + json.dump(cfg_obj, fh) + proc = subprocess.Popen([sys.executable, "ser2net.py", "--no-bootstrap", "--config", cfg], + cwd=ROOT, stdin=subprocess.DEVNULL, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + return proc + + +def main(): + cluster = {"enabled": True, "key": CLUSTER_KEY, "discovery_port": DPORT} + # Bind both nodes to 0.0.0.0 so each is reachable at its auto-detected LAN IP + # (that is the address the other node advertises and fetches). + node_a = start_node({"admin_ui": {"bind_ip": "0.0.0.0", "port": UI}, + "cluster": cluster, "mappings": DEMO_MAPPINGS}) + # Second node is pre-seeded with an admin so its key-guarded /api/cluster/local + # responds immediately (no first-run /setup gate); we never log into it. + node_b = start_node({"admin_ui": {"bind_ip": "0.0.0.0", "port": UI_B}, + "users": [{"username": "admin", "password_hash": hash_password("clusterdemo1"), + "role": "admin", "pwd_version": 1}], + "cluster": cluster, "mappings": DEMO_MAPPINGS_B}) try: - assert wait_port(UI), "server did not start" + assert wait_port(UI), "primary node did not start" + assert wait_port(UI_B), "second node did not start" with sync_playwright() as p: browser = p.chromium.launch() ctx = browser.new_context(viewport={"width": 1280, "height": 900}, @@ -82,17 +128,30 @@ def main(): page.click("form[action='/settings/users'] button[type=submit]") page.wait_for_load_state("networkidle") - # 04 settings + # 04 settings (now includes the LAN cluster section) page.goto(f"{BASE}/settings") page.wait_for_load_state("networkidle") page.screenshot(path=os.path.join(SHOTS, "04-settings.png"), full_page=True) - # 02 dashboard (wait for the mappings table to load) + # 02 dashboard — wait for the cluster panel to discover the second node page.goto(f"{BASE}/") page.wait_for_selector("table.grid") - page.wait_for_timeout(800) + try: + page.wait_for_function( + f"() => document.querySelector('#cluster') && " + f"document.querySelector('#cluster').textContent.includes(':{UI_B}')", + timeout=15000) + except Exception: + print("warning: second node did not appear in the cluster panel " + "(UDP discovery may be blocked in this environment)") + page.wait_for_timeout(600) page.screenshot(path=os.path.join(SHOTS, "02-dashboard.png"), full_page=True) + # 06 cluster — close-up of just the fleet panel + panel = page.query_selector("#cluster") + if panel: + panel.screenshot(path=os.path.join(SHOTS, "06-cluster.png")) + # 03 add-mapping form page.click("text=+ Add mapping") page.wait_for_selector("input[name=mqtt_base_topic]") @@ -106,13 +165,14 @@ def main(): page.screenshot(path=os.path.join(SHOTS, "01-login.png")) browser.close() - print("captured: 01-login, 02-dashboard, 03-add-mapping, 04-settings") + print("captured: 01-login, 02-dashboard, 03-add-mapping, 04-settings, 06-cluster") finally: - srv.terminate() - try: - srv.wait(timeout=5) - except subprocess.TimeoutExpired: - srv.kill() + for proc in (node_a, node_b): + proc.terminate() + try: + proc.wait(timeout=5) + except subprocess.TimeoutExpired: + proc.kill() if __name__ == "__main__":