Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions README.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,19 @@ with low latency.

## 📸 Screenshots

**Dashboard — dozens of mappings, live status, detected ports:**
**Dashboard — dozens of mappings, live status, detected ports and the LAN cluster panel:**

![Dashboard](docs/screenshots/02-dashboard.png)

**LAN cluster — every node's mappings in one table, each row tagged with its host (name + IP):**

![Cluster](docs/screenshots/06-cluster.png)

| Add mapping (all serial/network options) | In-browser serial console (xterm.js) |
|---|---|
| ![Mapping form](docs/screenshots/03-add-mapping.png) | ![Console](docs/screenshots/05-console.png) |

| Settings (password · TLS · backup) | Login |
| Settings (password · TLS · cluster · backup) | Login |
|---|---|
| ![Settings](docs/screenshots/04-settings.png) | ![Login](docs/screenshots/01-login.png) |

Expand Down Expand Up @@ -225,9 +229,10 @@ bound without TLS. For LAN deployments, use TLS (`admin_ui.tls_*`) and per-mappi

## 🗂️ Configuration & state files

All state lives in the **data dir** (default `data/`):
- `config.json` — admin IP, password hash, all mappings (atomic write; owner-only
perms: 0600 on POSIX, `icacls` owner/SYSTEM/Administrators on Windows).
All state lives in the **data dir** (default `data/`). The data directory is locked to
owner-only (POSIX `0700`; Windows `icacls` owner/SYSTEM/Administrators), so the files
inside aren't readable by other users:
- `config.json` — admin IP, password hash, all mappings (atomic write).
- `all.log` — global activity/audit; `audit.log` — config changes.
- `logs/<id>.log` — per-mapping history (hourly maintenance: trimmed >15 days / >100 MB).
- `tls/` — certificate/key if self-signed is generated.
Expand Down
15 changes: 10 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,19 @@ web arayüzünden yönetilen, çapraz-platform (Windows + Linux) saf-Python köp

## 📸 Ekran Görüntüleri

**Pano — onlarca eşleme, canlı durum, algılanan portlar:**
**Pano — onlarca eşleme, canlı durum, algılanan portlar ve LAN cluster paneli:**

![Pano](docs/screenshots/02-dashboard.png)

**LAN cluster — ağdaki tüm düğümlerin eşlemeleri tek tabloda, her satırda host (ad + IP):**

![Cluster](docs/screenshots/06-cluster.png)

| Eşleme ekleme (tüm seri/ağ seçenekleri) | Tarayıcı-içi seri konsol (xterm.js) |
|---|---|
| ![Eşleme formu](docs/screenshots/03-add-mapping.png) | ![Konsol](docs/screenshots/05-console.png) |

| Ayarlar (parola · TLS · yedek) | Giriş |
| Ayarlar (parola · TLS · cluster · yedek) | Giriş |
|---|---|
| ![Ayarlar](docs/screenshots/04-settings.png) | ![Giriş](docs/screenshots/01-login.png) |

Expand Down Expand Up @@ -224,9 +228,10 @@ Raw TCP düz-metindir — güvensiz ağlarda dikkat. Allowed/priority listesinde

## 🗂️ Yapılandırma & durum dosyaları

Tüm durum **veri dizininde** (varsayılan `data/`):
- `config.json` — admin IP, parola hash'i, tüm eşlemeler (atomik yazım; sahibe özel
izinler: POSIX'te 0600, Windows'ta `icacls` ile sahip/SYSTEM/Administrators).
Tüm durum **veri dizininde** (varsayılan `data/`). Veri dizini yalnızca sahibe açık
kilitlenir (POSIX `0700`; Windows'ta `icacls` ile sahip/SYSTEM/Administrators), böylece
içindeki dosyalar başka kullanıcılarca okunamaz:
- `config.json` — admin IP, parola hash'i, tüm eşlemeler (atomik yazım).
- `all.log` — global etkinlik/audit; `audit.log` — config değişiklikleri.
- `logs/<id>.log` — eşleme başına geçmiş (saatlik bakım: >15 gün veya >100MB kırpılır).
- `tls/` — self-signed üretilirse sertifika/anahtar.
Expand Down
73 changes: 47 additions & 26 deletions ROADMAP.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# ser2net — Roadmap

> Current release: **v2.5.0**. CI is green across Linux + Windows × Python 3.10–3.13.

## Shipped

### v1.0 — MVP
Expand Down Expand Up @@ -29,8 +31,6 @@
- **Admin TLS** via cert/key paths or `openssl` self-signed generation
- **Mappings export/import** (JSON, no secrets) + duplicate

---

### v2.1a — in-browser serial console
- xterm.js terminal over an authenticated WebSocket (`/api/mappings/{id}/console`):
read live traffic, or type to the device (net mappings, not read-only)
Expand All @@ -49,9 +49,7 @@
- **Docker** image + `docker-compose` + headless bind env; **CI** (GitHub Actions:
ruff + ubuntu/windows × Python 3.10–3.13); unified cross-platform test runner

### v2.2 — commercial features (Phase 2)
- **REST API** (`/api/v1`): bearer-token JSON API — mapping CRUD, start/stop/restart,
status, ports, OpenAPI 3.0 spec
### v2.2 — Modbus, RBAC, LDAP, MQTT (Phase 2)
- **Modbus RTU↔TCP gateway** (`protocol=modbus`): multi-master, bus-locked
transactions, txn-id integrity, `0x0B` timeout; reply-unit validation. **Edge mode:**
periodic register polling (uint/int/float 16/32, scaling) published to MQTT
Expand All @@ -61,41 +59,64 @@
group→role mapping, shadow accounts on the RBAC model
- **MQTT publishing** (optional `paho-mqtt`): per-mapping serial-line → broker with
retained birth/death
- **Client-side virtual COM** recipes (`docs/VIRTUAL-COM.md`); refreshed UI screenshots
- **Client-side virtual COM** recipes (`docs/VIRTUAL-COM.md`)

### v2.3 — UX & access polish
- **OIDC single sign-on** (authorization-code flow + issuer discovery; claim→role
mapping, shadow accounts) — `app/web/oidc_auth.py`, optional `authlib`
- **REST API token roles** (`admin` / `operator` / `viewer`; `viewer` is read-only)
- **Light / dark theme toggle** (persisted in `localStorage`); xterm fit-to-window addon
- _(i18n was dropped — the UI stays English by decision.)_

### v2.4 — packaging & distribution
- **PyInstaller standalone builds** (Windows `.exe`, Linux binary) bundling the optional
MQTT/LDAP/OIDC deps; GitHub Actions release artifacts published on `v*` tags

### v2.5 — LAN cluster
- **Auto-discovery + unified fleet view**: nodes find each other via HMAC-signed UDP
broadcast beacons (no mDNS) and one node aggregates every node's mappings into a
single **read-only** table, each row tagged with its host (name + IP). Opt-in, off by
default; trust = a shared cluster key. Server-side fan-out to peers' key-guarded
`/api/cluster/local`; the browser only talks to the node it logged into.

---

## Planned

### v2.3 — UX & access polish
- **OIDC / SAML browser-SSO** (next auth step after LDAP; auth-code flow + JWKS)
- **REST API token scopes/roles** (currently a single admin-level token) + per-token
- i18n (TR/EN) for the UI; dark/light theme toggle; xterm fit-to-window addon
### v2.6 — cluster depth & hardening
- **Remote control** from the unified view: start/stop/edit a peer's mappings (the read
path exists; add a key-guarded write endpoint + role check on the target node)
- **Per-node health/uptime** column; surface a node-level "discovery disabled" status in
the UI when the UDP bind fails (today it only logs)
- **Manual peer list** for routed/L3 networks where broadcast doesn't reach (complement
to auto-discovery)
- **Security hardening** (from the v2.5 review): validate/curb peer-advertised IPs before
server-side fetch (SSRF defense-in-depth), optional TLS certificate pinning for peer
fetch, a light rate-limit on `/api/cluster/local`, optional IPv6 (multicast) discovery

### v2.7 — industrial/IIoT depth
- **Sparkplug B** edge payloads (Modbus register + MQTT plumbing already in place)
- **Modbus write** support (FC 5/6/15/16), per-point MQTT→register control, RTU
inter-frame tuning
- **RS-485 hardware auto-RTS** (`TIOCSRS485`) UI

### v2.4 — packaging & distribution
- ~~PyInstaller standalone builds (Windows `.exe`, Linux binary) bundling the optional
MQTT/LDAP/OIDC deps; GitHub Actions release artifacts on `v*` tags~~ ✅ shipped
- Windows service installer (Shawl); `.deb` / `.rpm`
- Automated multi-platform offline wheelhouse (cp311–cp313 × OS) for source installs
### v2.8 — packaging & migration
- Windows service installer (Shawl); `.deb` / `.rpm` packages
- Automated multi-platform offline wheelhouse (cp310–cp313 × OS) for source installs
- Classic `ser2net.yaml` import for migration

### v2.5 — industrial/IIoT depth
- **Sparkplug B** edge payloads (Modbus register + MQTT plumbing already in place)
- Modbus: write support (FC 5/6/15/16), per-point MQTT→register control, RTU inter-frame
tuning; RS-485 hardware auto-RTS (`TIOCSRS485`) UI
- ~~LAN **cluster dashboard**: nodes auto-discover each other (signed UDP broadcast,
no mDNS) and one node shows every node's mappings in a single read-only table, each
row tagged with its host (name + IP)~~ ✅ shipped (v2.5a)
- Cluster: remote control (start/stop/edit a peer's mappings from one screen) + per-node
health/uptime in the unified view
- classic `ser2net.yaml` import for migration
### Maintenance
- Bump pinned GitHub Actions off Node 20 (deprecated 2026-06-16) to current
`actions/*@v5` to clear the CI/release deprecation warnings

### Icebox / conditional
- Thread-per-port serial backend — only if a real Windows high-throughput /
tight-timing need emerges (current asyncio path handles 24+ full-duplex fine)
- Per-mapping TLS client certificate verification (mTLS) for data bridges
- Pluggable persistent session store

> Note: mDNS/zeroconf advertising has been removed from the roadmap.
> Note: mDNS/zeroconf advertising is intentionally **not** on the roadmap; LAN cluster
> discovery uses signed UDP broadcast instead.

---

Expand Down
6 changes: 5 additions & 1 deletion app/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -554,7 +554,11 @@ def validate(self) -> None:
return
if not self.key.strip():
raise ConfigError("Cluster is enabled but no shared key is set.")
if not (1 <= int(self.discovery_port) <= 65535):
try:
port = int(self.discovery_port)
except (TypeError, ValueError):
raise ConfigError("Cluster discovery port must be a number.") from None
if not (1 <= port <= 65535):
raise ConfigError("Cluster discovery port must be 1..65535.")
if self.advertise_ip.strip():
try:
Expand Down
9 changes: 7 additions & 2 deletions app/engine/cluster.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,8 +141,13 @@ async def start(self) -> None:
sock.close()
self.log(f"cluster: could not bind UDP {port}: {e} — discovery disabled")
return
self._transport, _ = await loop.create_datagram_endpoint(
lambda: _BeaconProtocol(self.handle_datagram), sock=sock)
try:
self._transport, _ = await loop.create_datagram_endpoint(
lambda: _BeaconProtocol(self.handle_datagram), sock=sock)
except OSError as e:
sock.close() # endpoint creation owns the socket only on success
self.log(f"cluster: could not start UDP listener on {port}: {e} — discovery disabled")
return
self._running = True
self._beacon_task = asyncio.create_task(self._beacon_loop(), name="cluster-beacon")
self.log(f"cluster: discovery on UDP {port} as '{socket.gethostname()}' "
Expand Down
2 changes: 1 addition & 1 deletion bootstrap.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

On first run this installs the bundled wheels from ``vendor/wheels/`` into a
local ``lib/`` directory (added to ``sys.path`` by ``ser2net.py``), so the app
runs on any machine that merely has a Python 3.11+ interpreter — no internet,
runs on any machine that merely has a Python 3.10+ interpreter — no internet,
no virtualenv, no system-wide installs required.

It is idempotent: if every dependency already imports, it does nothing.
Expand Down
Binary file modified docs/screenshots/01-login.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/02-dashboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/03-add-mapping.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/04-settings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/screenshots/06-cluster.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
102 changes: 81 additions & 21 deletions tools/_capture_screenshots.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@
pip install playwright && python -m playwright install chromium
python tools/_capture_screenshots.py

Seeds a few (disabled) demo mappings, completes first-run setup, adds an operator
and a viewer so the Users panel is populated, then captures the login, dashboard,
add-mapping form and settings pages. Not part of the app or the test suite.
Starts TWO ser2net nodes that share a cluster key so the dashboard's LAN-cluster
panel shows a real multi-host view, completes first-run setup on the primary node,
adds an operator + viewer (to populate the Users panel), then captures the login,
dashboard (with the cluster panel), add-mapping form, settings and cluster pages.
Not part of the app or the test suite.
"""
import json
import os
Expand All @@ -15,13 +17,31 @@
import tempfile
import time

from playwright.sync_api import sync_playwright

ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, os.path.join(ROOT, "lib"))
sys.path.insert(0, ROOT)

from playwright.sync_api import sync_playwright # noqa: E402
from app.web.auth import hash_password # noqa: E402

SHOTS = os.path.join(ROOT, "docs", "screenshots")
UI = 8099
UI = 8099 # primary node (the browser drives this one)
UI_B = 8100 # second cluster node (data source for the fleet view)
BASE = f"http://127.0.0.1:{UI}"
CLUSTER_KEY = "demo-cluster-key"


def free_port():
s = socket.socket()
s.bind(("127.0.0.1", 0))
p = s.getsockname()[1]
s.close()
return p


DPORT = free_port() # shared UDP discovery port for both nodes

# Primary node's mappings (the browser node).
DEMO_MAPPINGS = [
{"name": "PLC-1", "enabled": False, "serial": {"port": "COM3", "baudrate": 9600},
"network": {"mode": "server", "protocol": "rfc2217", "bind_ip": "0.0.0.0", "port": 4001}},
Expand All @@ -36,6 +56,16 @@
"serial": {"port": "COM6", "baudrate": 115200}, "serial_b": {"port": "COM7", "baudrate": 115200}},
]

# Second node's mappings (shown as a remote host in the fleet table).
DEMO_MAPPINGS_B = [
{"name": "Robot-arm", "enabled": False, "serial": {"port": "/dev/ttyACM0", "baudrate": 115200},
"network": {"mode": "server", "protocol": "raw", "bind_ip": "0.0.0.0", "port": 4001}},
{"name": "Welder-3", "enabled": False, "serial": {"port": "COM4", "baudrate": 9600},
"network": {"mode": "server", "protocol": "rfc2217", "bind_ip": "0.0.0.0", "port": 4002}},
{"name": "Weigh-east", "enabled": False, "serial": {"port": "/dev/ttyUSB3", "baudrate": 19200},
"network": {"mode": "server", "protocol": "raw", "bind_ip": "0.0.0.0", "port": 4003}},
]


def wait_port(port, timeout=20):
end = time.time() + timeout
Expand All @@ -48,16 +78,32 @@ def wait_port(port, timeout=20):
return False


def main():
def start_node(cfg_obj):
tmp = tempfile.mkdtemp(prefix="ser2net_shots_")
cfg = os.path.join(tmp, "config.json")
with open(cfg, "w") as fh:
json.dump({"admin_ui": {"bind_ip": "127.0.0.1", "port": UI}, "mappings": DEMO_MAPPINGS}, fh)
srv = subprocess.Popen([sys.executable, "ser2net.py", "--no-bootstrap", "--config", cfg],
cwd=ROOT, stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
json.dump(cfg_obj, fh)
proc = subprocess.Popen([sys.executable, "ser2net.py", "--no-bootstrap", "--config", cfg],
cwd=ROOT, stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
return proc


def main():
cluster = {"enabled": True, "key": CLUSTER_KEY, "discovery_port": DPORT}
# Bind both nodes to 0.0.0.0 so each is reachable at its auto-detected LAN IP
# (that is the address the other node advertises and fetches).
node_a = start_node({"admin_ui": {"bind_ip": "0.0.0.0", "port": UI},
"cluster": cluster, "mappings": DEMO_MAPPINGS})
# Second node is pre-seeded with an admin so its key-guarded /api/cluster/local
# responds immediately (no first-run /setup gate); we never log into it.
node_b = start_node({"admin_ui": {"bind_ip": "0.0.0.0", "port": UI_B},
"users": [{"username": "admin", "password_hash": hash_password("clusterdemo1"),
"role": "admin", "pwd_version": 1}],
"cluster": cluster, "mappings": DEMO_MAPPINGS_B})
try:
assert wait_port(UI), "server did not start"
assert wait_port(UI), "primary node did not start"
assert wait_port(UI_B), "second node did not start"
with sync_playwright() as p:
browser = p.chromium.launch()
ctx = browser.new_context(viewport={"width": 1280, "height": 900},
Expand All @@ -82,17 +128,30 @@ def main():
page.click("form[action='/settings/users'] button[type=submit]")
page.wait_for_load_state("networkidle")

# 04 settings
# 04 settings (now includes the LAN cluster section)
page.goto(f"{BASE}/settings")
page.wait_for_load_state("networkidle")
page.screenshot(path=os.path.join(SHOTS, "04-settings.png"), full_page=True)

# 02 dashboard (wait for the mappings table to load)
# 02 dashboard wait for the cluster panel to discover the second node
page.goto(f"{BASE}/")
page.wait_for_selector("table.grid")
page.wait_for_timeout(800)
try:
page.wait_for_function(
f"() => document.querySelector('#cluster') && "
f"document.querySelector('#cluster').textContent.includes(':{UI_B}')",
timeout=15000)
except Exception:
print("warning: second node did not appear in the cluster panel "
"(UDP discovery may be blocked in this environment)")
page.wait_for_timeout(600)
page.screenshot(path=os.path.join(SHOTS, "02-dashboard.png"), full_page=True)

# 06 cluster — close-up of just the fleet panel
panel = page.query_selector("#cluster")
if panel:
panel.screenshot(path=os.path.join(SHOTS, "06-cluster.png"))

# 03 add-mapping form
page.click("text=+ Add mapping")
page.wait_for_selector("input[name=mqtt_base_topic]")
Expand All @@ -106,13 +165,14 @@ def main():
page.screenshot(path=os.path.join(SHOTS, "01-login.png"))

browser.close()
print("captured: 01-login, 02-dashboard, 03-add-mapping, 04-settings")
print("captured: 01-login, 02-dashboard, 03-add-mapping, 04-settings, 06-cluster")
finally:
srv.terminate()
try:
srv.wait(timeout=5)
except subprocess.TimeoutExpired:
srv.kill()
for proc in (node_a, node_b):
proc.terminate()
try:
proc.wait(timeout=5)
except subprocess.TimeoutExpired:
proc.kill()


if __name__ == "__main__":
Expand Down
Loading