diff --git a/README.en.md b/README.en.md index 09189b8..b5b99b7 100644 --- a/README.en.md +++ b/README.en.md @@ -17,7 +17,7 @@ with low latency. ![Dashboard](docs/screenshots/02-dashboard.png) -**LAN cluster — every node's mappings in one table, each row tagged with its host (name + IP):** +**LAN cluster — every node's mappings in one table; per-node health + remote start/stop:** ![Cluster](docs/screenshots/06-cluster.png) @@ -87,10 +87,13 @@ everything from a **password-protected web UI** — no CLI or hand-edited config - **REST API:** a JSON API (`/api/v1`) for automation — mapping CRUD, start/stop/restart, status and ports; **bearer-token** auth (with a role — `viewer` is read-only); OpenAPI 3.0 (`/api/v1/openapi.json`). The token is generated in Settings. -- **LAN cluster (fleet view):** nodes on the same network **auto-discover** each other - (signed UDP broadcast — no mDNS) and one node shows **every node's mappings in a single - read-only table**, each row tagged with its host (name + IP). Set the **same shared key** - on every node so they trust each other. Off by default; enable under Settings → LAN cluster. +- **LAN cluster (fleet management):** nodes on the same network **auto-discover** each + other (signed UDP broadcast — no mDNS) and one node shows **every node's mappings in a + single table**, each row tagged with its host (name + IP) and each node's **health** + (uptime · version · running/total). Operators can **remotely start/stop/restart** a + peer's mappings from that view. For routed/L3 networks broadcast can't reach, add + **manual peers** (host:port). Trust = the **same shared key** on every node; off by + default, enable under Settings → LAN cluster. - **Deployment:** official **Docker** image + `docker-compose`; **systemd** unit; Linux+Windows × Python 3.10–3.13 **CI** (GitHub Actions). - **MQTT publishing (optional):** publish each mapping's serial lines to an MQTT diff --git a/README.md b/README.md index 9bb383a..4221c27 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ web arayüzünden yönetilen, çapraz-platform (Windows + Linux) saf-Python köp ![Pano](docs/screenshots/02-dashboard.png) -**LAN cluster — ağdaki tüm düğümlerin eşlemeleri tek tabloda, her satırda host (ad + IP):** +**LAN cluster — tüm düğümlerin eşlemeleri tek tabloda; düğüm sağlığı + uzaktan başlat/durdur:** ![Cluster](docs/screenshots/06-cluster.png) @@ -87,11 +87,14 @@ düzenlemeye gerek yok. - **REST API:** otomasyon için JSON API (`/api/v1`) — eşleme CRUD, başlat/durdur/yeniden, durum ve portlar; **bearer-token** kimlik doğrulama (rol seçilebilir — `viewer` salt-okunur); OpenAPI 3.0 (`/api/v1/openapi.json`). Token, Ayarlar'dan üretilir. -- **LAN cluster (filo görünümü):** aynı ağdaki düğümler birbirini **otomatik bulur** +- **LAN cluster (filo yönetimi):** aynı ağdaki düğümler birbirini **otomatik bulur** (imzalı UDP broadcast — mDNS yok) ve bir düğüm **tüm düğümlerin eşlemelerini tek - salt-okunur tabloda** gösterir; her satır hangi bilgisayara ait olduğunu (ad + IP) - belirtir. Düğümlerin birbirine güvenmesi için her birine **aynı paylaşılan anahtar** - girilir. Varsayılan kapalı; Ayarlar → LAN cluster'dan açılır. + tabloda** gösterir; her satır hangi bilgisayara ait olduğunu (ad + IP) ve her düğümün + **sağlığını** (uptime · sürüm · çalışan/toplam) belirtir. Operatörler bu ekrandan + başka bir host'un eşlemelerini **uzaktan başlat/durdur/yeniden başlat** edebilir. + Broadcast'in ulaşmadığı yönlendirilmiş/L3 ağlar için **manuel peer** (host:port) + eklenebilir. Güven = her düğümde **aynı paylaşılan anahtar**; varsayılan kapalı, + Ayarlar → LAN cluster'dan açılır. - **Dağıtım:** resmi **Docker** imajı + `docker-compose`; **systemd** birimi; Linux+Windows × Python 3.10–3.13 **CI** (GitHub Actions). - **MQTT yayınlama (opsiyonel):** seri satırlarını eşleme başına bir MQTT broker'ına diff --git a/ROADMAP.md b/ROADMAP.md index 45569ab..81f795c 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,6 +1,6 @@ # ser2net — Roadmap -> Current release: **v2.5.0**. CI is green across Linux + Windows × Python 3.10–3.13. +> Current release: **v2.6.0**. CI is green across Linux + Windows × Python 3.10–3.13. ## Shipped @@ -75,24 +75,28 @@ ### v2.5 — LAN cluster - **Auto-discovery + unified fleet view**: nodes find each other via HMAC-signed UDP broadcast beacons (no mDNS) and one node aggregates every node's mappings into a - single **read-only** table, each row tagged with its host (name + IP). Opt-in, off by + single read-only table, each row tagged with its host (name + IP). Opt-in, off by default; trust = a shared cluster key. Server-side fan-out to peers' key-guarded `/api/cluster/local`; the browser only talks to the node it logged into. +### v2.6 — cluster depth +- **Remote control** from the unified view: operators start/stop/restart a peer's mappings + (key-guarded `/api/cluster/control` on the target + a session-authed proxy that validates + the peer against a known-address allowlist, so the browser can't aim it anywhere) +- **Per-node health**: uptime · version · running/total, plus an online/offline indicator + and a UI banner when UDP discovery can't bind (manual peers still work) +- **Manual peers** (`host:port`) for routed/L3 networks broadcast can't reach, aggregated + alongside auto-discovered nodes + --- ## Planned -### v2.6 — cluster depth & hardening -- **Remote control** from the unified view: start/stop/edit a peer's mappings (the read - path exists; add a key-guarded write endpoint + role check on the target node) -- **Per-node health/uptime** column; surface a node-level "discovery disabled" status in - the UI when the UDP bind fails (today it only logs) -- **Manual peer list** for routed/L3 networks where broadcast doesn't reach (complement - to auto-discovery) -- **Security hardening** (from the v2.5 review): validate/curb peer-advertised IPs before - server-side fetch (SSRF defense-in-depth), optional TLS certificate pinning for peer - fetch, a light rate-limit on `/api/cluster/local`, optional IPv6 (multicast) discovery +### v2.6.x — cluster hardening (from the v2.5 review) +- Validate/curb peer-advertised IPs before the server-side fetch (SSRF defense-in-depth) +- Optional TLS certificate pinning for peer fetch/control; a light rate-limit on the + key-guarded peer endpoints; optional IPv6 (multicast) discovery +- Remote **edit** of a peer's mapping (today's remote control is start/stop/restart) ### v2.7 — industrial/IIoT depth - **Sparkplug B** edge payloads (Modbus register + MQTT plumbing already in place) diff --git a/app/__init__.py b/app/__init__.py index e69de29..8dff27f 100644 --- a/app/__init__.py +++ b/app/__init__.py @@ -0,0 +1,4 @@ +"""ser2net — serial-to-network bridge with a web config UI.""" + +# Single source of truth for the version (ser2net.py and the web layer import this). +__version__ = "2.6.0" diff --git a/app/config.py b/app/config.py index 7fbdb1d..226ed8b 100644 --- a/app/config.py +++ b/app/config.py @@ -526,6 +526,28 @@ def validate(self) -> None: raise ConfigError(f"OIDC default role must be one of {ROLES} or blank.") +def parse_cluster_peer(entry: str) -> tuple[str, str, int]: + """Parse a manual cluster-peer entry into (scheme, host, port). Accepts + 'host:port' (defaults to http) or 'http(s)://host:port'. Raises ConfigError.""" + s = (entry or "").strip() + scheme = "http" + if "://" in s: + scheme, s = s.split("://", 1) + scheme = scheme.lower() + if scheme not in ("http", "https"): + raise ConfigError(f"Cluster peer scheme must be http or https: {entry}") + host, sep, port_s = s.rpartition(":") + if not sep or not host: + raise ConfigError(f"Cluster peer must be host:port — got: {entry}") + try: + port = int(port_s) + except ValueError: + raise ConfigError(f"Cluster peer port is not a number: {entry}") from None + if not (1 <= port <= 65535): + raise ConfigError(f"Cluster peer port must be 1..65535: {entry}") + return scheme, host, port + + @dataclass class ClusterSettings: """LAN cluster: instances discover each other via signed UDP broadcast beacons @@ -533,17 +555,22 @@ class ClusterSettings: Opt-in: disabled until `enabled` is set AND a shared `key` is configured. The key both signs beacons (so only same-key nodes trust each other) and guards the peer-facing status endpoint. `advertise_ip` overrides the auto-detected LAN IP - that peers use to reach this node's web UI (blank => auto).""" + that peers use to reach this node's web UI (blank => auto). `peers` is a list of + manual 'host:port' entries for routed/L3 networks where UDP broadcast doesn't + reach — they're aggregated alongside auto-discovered peers.""" enabled: bool = False key: str = "" # shared secret (PSK); empty => cluster off discovery_port: int = 41750 # UDP port for broadcast beacons advertise_ip: str = "" # override advertised web-UI IP (blank => auto-detect) + peers: list[str] = field(default_factory=list) # manual 'host:port' peer entries @staticmethod def from_dict(d: dict[str, Any]) -> "ClusterSettings": d = dict(d or {}) known = {f.name for f in dataclasses.fields(ClusterSettings)} - return ClusterSettings(**{k: v for k, v in d.items() if k in known}) + c = ClusterSettings(**{k: v for k, v in d.items() if k in known}) + c.peers = [str(p).strip() for p in (c.peers or []) if str(p).strip()] + return c @property def active(self) -> bool: @@ -566,6 +593,8 @@ def validate(self) -> None: except ValueError: raise ConfigError( f"Cluster advertise IP is not a valid address: {self.advertise_ip}") from None + for entry in self.peers: + parse_cluster_peer(entry) # raises ConfigError on a bad entry @dataclass diff --git a/app/engine/cluster.py b/app/engine/cluster.py index 08d0160..8096629 100644 --- a/app/engine/cluster.py +++ b/app/engine/cluster.py @@ -24,9 +24,11 @@ import socket import ssl import time +import urllib.parse import urllib.request from . import netinfo +from ..config import parse_cluster_peer BEACON_INTERVAL = 5.0 # seconds between outbound beacons PEER_TTL = 20.0 # a peer is dropped if not heard from within this window @@ -60,6 +62,7 @@ def __init__(self, config, logger): self._beacon_task: asyncio.Task | None = None self._peers: dict[str, dict] = {} # id -> {id,name,ip,port,scheme,last_seen} self._running = False + self.discovery_error = "" # non-empty when UDP discovery couldn't start # ----- config helpers ----- @property @@ -124,6 +127,37 @@ def peers(self) -> list[dict]: self._peers = {p["id"]: p for p in live} return sorted(live, key=lambda p: (p["name"].lower(), p["ip"])) + def manual_targets(self) -> list[dict]: + """Configured manual peers (for routed/L3 networks broadcast can't reach), + parsed into fetch targets. Bad entries are skipped (validate() rejects them + at save time, so this only guards a hand-edited config).""" + out = [] + for entry in self.cluster.peers: + try: + scheme, host, port = parse_cluster_peer(entry) + except Exception: + continue + out.append({"scheme": scheme, "ip": host, "port": port, "source": "manual"}) + return out + + def all_targets(self) -> list[dict]: + """Auto-discovered peers + manual peers, deduped by (ip, port). When a manual + peer is also auto-discovered the richer auto entry wins.""" + seen, targets = set(), [] + for p in self.peers(): + seen.add((p["ip"], p["port"])) + targets.append({**p, "source": "auto"}) + for m in self.manual_targets(): + if (m["ip"], m["port"]) not in seen: + seen.add((m["ip"], m["port"])) + targets.append(m) + return targets + + def known_addresses(self) -> set: + """(ip, port) allowlist of peers this node may reach — used to bound the + remote-control proxy so the browser can't aim it at an arbitrary address.""" + return {(t["ip"], int(t["port"])) for t in self.all_targets()} + # ----- lifecycle ----- async def start(self) -> None: if self._running or not self.cluster.active: @@ -139,15 +173,19 @@ async def start(self) -> None: sock.bind(("", port)) except OSError as e: sock.close() - self.log(f"cluster: could not bind UDP {port}: {e} — discovery disabled") + self.discovery_error = f"UDP discovery port {port} unavailable: {e}" + self.log(f"cluster: could not bind UDP {port}: {e} — discovery disabled " + "(manual peers still work)") return try: self._transport, _ = await loop.create_datagram_endpoint( lambda: _BeaconProtocol(self.handle_datagram), sock=sock) except OSError as e: sock.close() # endpoint creation owns the socket only on success + self.discovery_error = f"UDP listener on {port} failed: {e}" self.log(f"cluster: could not start UDP listener on {port}: {e} — discovery disabled") return + self.discovery_error = "" self._running = True self._beacon_task = asyncio.create_task(self._beacon_loop(), name="cluster-beacon") self.log(f"cluster: discovery on UDP {port} as '{socket.gethostname()}' " @@ -203,6 +241,15 @@ def _broadcast_addrs() -> list[str]: out.append(a) return out + @staticmethod + def _peer_ssl_ctx(url: str): + if not url.startswith("https"): + return None + ctx = ssl.create_default_context() + ctx.check_hostname = False + ctx.verify_mode = ssl.CERT_NONE # cluster trust is the shared key, not the cert + return ctx + # ----- peer aggregation (HTTP) ----- async def fetch_peer(self, peer: dict, timeout: float = 2.5) -> dict | None: url = f"{peer['scheme']}://{peer['ip']}:{peer['port']}/api/cluster/local" @@ -210,13 +257,26 @@ async def fetch_peer(self, peer: dict, timeout: float = 2.5) -> dict | None: def _http_get_json(self, url: str, timeout: float) -> dict | None: req = urllib.request.Request(url, headers={"X-Cluster-Key": self.cluster.key}) - ctx = None - if url.startswith("https"): - ctx = ssl.create_default_context() - ctx.check_hostname = False - ctx.verify_mode = ssl.CERT_NONE # cluster trust is the shared key, not the cert try: - with urllib.request.urlopen(req, timeout=timeout, context=ctx) as r: + with urllib.request.urlopen(req, timeout=timeout, context=self._peer_ssl_ctx(url)) as r: + return json.loads(r.read().decode("utf-8")) + except Exception: + return None + + # ----- remote control (HTTP POST to a peer's key-guarded control endpoint) ----- + async def control_peer(self, scheme: str, host: str, port: int, mapping_id: str, + action: str, timeout: float = 4.0) -> dict | None: + url = f"{scheme}://{host}:{port}/api/cluster/control" + data = urllib.parse.urlencode({"mapping_id": mapping_id, "action": action}).encode() + return await asyncio.to_thread(self._http_post_json, url, data, timeout) + + def _http_post_json(self, url: str, data: bytes, timeout: float) -> dict | None: + req = urllib.request.Request( + url, data=data, method="POST", + headers={"X-Cluster-Key": self.cluster.key, + "Content-Type": "application/x-www-form-urlencoded"}) + try: + with urllib.request.urlopen(req, timeout=timeout, context=self._peer_ssl_ctx(url)) as r: return json.loads(r.read().decode("utf-8")) except Exception: return None diff --git a/app/web/routes.py b/app/web/routes.py index 33c795e..39be078 100644 --- a/app/web/routes.py +++ b/app/web/routes.py @@ -17,6 +17,7 @@ import secrets import socket import subprocess +import time from urllib.parse import urlparse from starlette.responses import ( @@ -25,6 +26,7 @@ from starlette.routing import Mount, Route, WebSocketRoute from starlette.staticfiles import StaticFiles +from .. import __version__ from ..config import ( ROLE_RANK, ROLES, ClusterSettings, ConfigError, LdapSettings, MappingConfig, OidcSettings, User, @@ -53,6 +55,18 @@ def _mapping_endpoint(m) -> str: return f"{m.network.bind_ip}:{m.network.port}" +def _fmt_uptime(seconds: float) -> str: + s = int(max(0, seconds)) + d, s = divmod(s, 86400) + h, s = divmod(s, 3600) + m, _ = divmod(s, 60) + if d: + return f"{d}d {h}h" + if h: + return f"{h}h {m}m" + return f"{m}m" + + def build_routes(templates, state, static_dir): def render(request, name, status_code=200, headers=None, **ctx): return templates.TemplateResponse(request, name, ctx, @@ -253,6 +267,7 @@ async def settings_get(request, ok=None, error=None, new_api_token=None): ldap=state.config.ldap, ldap_lib=_module_present("ldap3"), oidc=state.config.oidc, oidc_lib=_module_present("authlib"), cluster=state.config.cluster, instance_id=state.config.instance_id, + cluster_peers_text="\n".join(state.config.cluster.peers), uptime=int(state.started_at)) async def settings_password_post(request): @@ -626,16 +641,18 @@ async def api_status(request): statuses=state.supervisor.all_status(), can_edit=can_edit(request)) - # ---------------- LAN cluster (federated read-only view) ---------------- + # ---------------- LAN cluster (federated view + remote control) ---------------- def local_node_payload() -> dict: - """This node's identity + a status row per mapping — the unit other nodes - aggregate. No secrets: just what the unified dashboard table shows.""" + """This node's identity, health and a status row per mapping — the unit other + nodes aggregate. No secrets: just what the unified dashboard table shows.""" cfg = state.config ip, port, scheme = state.cluster.advertised() statuses = state.supervisor.all_status() - maps = [] + maps, running = [], 0 for m in cfg.mappings: st = statuses.get(m.id) or {} + if st.get("state") == "running": + running += 1 maps.append({ "id": m.id, "name": m.name, "enabled": m.enabled, "kind": _mapping_kind_label(m), "endpoint": _mapping_endpoint(m), @@ -645,7 +662,9 @@ def local_node_payload() -> dict: "bytes_in": st.get("bytes_in", 0), "bytes_out": st.get("bytes_out", 0), }) return {"id": cfg.instance_id, "name": socket.gethostname(), - "ip": ip, "port": port, "scheme": scheme, "mappings": maps} + "ip": ip, "port": port, "scheme": scheme, + "version": __version__, "uptime_s": int(time.time() - state.started_at), + "mapping_count": len(maps), "running_count": running, "mappings": maps} async def cluster_local(request): """Peer-facing: returns this node's mappings to another cluster node that @@ -657,29 +676,86 @@ async def cluster_local(request): return JSONResponse({"error": "forbidden"}, status_code=403) return JSONResponse(local_node_payload()) + def _node_from(target, data) -> dict: + """Build a fleet-table node entry from a fetch target + the peer's payload + (data is None when the peer is unreachable).""" + base = {"self": False, "source": target.get("source", "auto"), + "ip": target["ip"], "port": target["port"], "scheme": target["scheme"]} + if not data: + return {**base, "online": False, "id": target.get("id", ""), + "name": target.get("name", f"{target['ip']}:{target['port']}"), + "uptime": "", "version": "", "mapping_count": 0, "running_count": 0, + "mappings": []} + return {**base, "online": True, "id": data.get("id", target.get("id", "")), + "name": data.get("name", base["ip"]), + "ip": data.get("ip", base["ip"]), "port": data.get("port", base["port"]), + "scheme": data.get("scheme", base["scheme"]), + "uptime": _fmt_uptime(data.get("uptime_s", 0)), "version": data.get("version", ""), + "mapping_count": data.get("mapping_count", len(data.get("mappings", []))), + "running_count": data.get("running_count", 0), + "mappings": data.get("mappings", [])} + async def cluster_status(request): - """Aggregated view for the browser: local node + every discovered peer, + """Aggregated view for the browser: local node + every discovered/manual peer, fetched server-side with the shared key. Renders the unified table.""" if not state.config.cluster.active: - return render(request, "_cluster_body.html", nodes=[]) + return render(request, "_cluster_body.html", nodes=[], can_edit=can_edit(request), + discovery_error="") local = local_node_payload() - nodes = [{**local, "self": True, "online": True}] - peers = state.cluster.peers() - results = await asyncio.gather(*[state.cluster.fetch_peer(p) for p in peers]) \ - if peers else [] - for p, data in zip(peers, results, strict=False): - if data: - nodes.append({ - "id": data.get("id", p["id"]), "name": data.get("name", p["name"]), - "ip": data.get("ip", p["ip"]), "port": data.get("port", p["port"]), - "scheme": data.get("scheme", p["scheme"]), - "self": False, "online": True, "mappings": data.get("mappings", []), - }) - else: - nodes.append({"id": p["id"], "name": p["name"], "ip": p["ip"], - "port": p["port"], "scheme": p["scheme"], - "self": False, "online": False, "mappings": []}) - return render(request, "_cluster_body.html", nodes=nodes) + nodes = [{**local, "self": True, "online": True, "source": "self", + "uptime": _fmt_uptime(local["uptime_s"])}] + targets = state.cluster.all_targets() + results = await asyncio.gather(*[state.cluster.fetch_peer(t) for t in targets]) \ + if targets else [] + for t, data in zip(targets, results, strict=False): + nodes.append(_node_from(t, data)) + return render(request, "_cluster_body.html", nodes=nodes, can_edit=can_edit(request), + discovery_error=state.cluster.discovery_error) + + async def cluster_control(request): + """Peer-facing remote control: another cluster node (presenting the shared key) + asks us to start/stop/restart one of OUR mappings. Key-guarded, no session.""" + cfg = state.config + key = request.headers.get("x-cluster-key", "") + if not (cfg.cluster.active and key and hmac.compare_digest(key, cfg.cluster.key)): + return JSONResponse({"error": "forbidden"}, status_code=403) + form = await request.form() + mid, action = form.get("mapping_id", ""), form.get("action", "") + if action not in ("start", "stop", "restart"): + return JSONResponse({"error": "bad_action"}, status_code=400) + ok, name = await _apply_action(mid, action) + if not ok: + return JSONResponse({"error": name}, status_code=404) + ip = client_ip(request) + state.log(f"cluster: remote {action} of '{name}' from {ip}") + state.audit(ip, f"cluster_remote_{action}", name) + return JSONResponse({"ok": True, "name": name}) + + async def cluster_control_peer(request): + """Browser-facing proxy: an operator on THIS node controls a mapping on a peer. + The peer is validated against the known-address allowlist (so the browser can't + aim this at an arbitrary host), then the call is made with the shared key.""" + if deny := require_role(request, "operator"): + return deny + form = await request.form() + scheme = form.get("scheme", "http") + host, action = form.get("host", ""), form.get("action", "") + mid = form.get("mid", "") + try: + port = int(form.get("port", "0")) + except ValueError: + return PlainTextResponse("Bad port", status_code=400) + if action not in ("start", "stop", "restart"): + return PlainTextResponse("Unknown action", status_code=400) + if (host, port) not in state.cluster.known_addresses(): + return PlainTextResponse("Unknown cluster peer", status_code=403) + res = await state.cluster.control_peer(scheme, host, port, mid, action) + if not res or not res.get("ok"): + return PlainTextResponse("Peer did not accept the command " + "(unreachable or rejected).", status_code=502) + state.log(f"cluster: sent {action} to {host}:{port} for '{res.get('name', mid)}'") + state.audit(client_ip(request), f"cluster_control_{action}", f"{host}:{port}/{res.get('name', mid)}") + return Response("", headers={"HX-Trigger": "refreshCluster"}) async def settings_cluster_post(request): if deny := require_role(request, "admin"): @@ -691,11 +767,13 @@ async def settings_cluster_post(request): port = int(form.get("discovery_port") or 41750) except ValueError: return await settings_get(request, error="Cluster discovery port must be a number.") + peers = [ln.strip() for ln in (form.get("peers") or "").splitlines() if ln.strip()] new = ClusterSettings( enabled=form.get("enabled") == "on", key=(form.get("key") or "").strip(), discovery_port=port, advertise_ip=(form.get("advertise_ip") or "").strip(), + peers=peers, ) async with state.config_lock: old = state.config.cluster @@ -708,7 +786,7 @@ async def settings_cluster_post(request): # apply live: restart discovery so a key/port/enable change takes effect now await state.cluster.stop() await state.cluster.start() - state.log(f"cluster settings updated (enabled={new.enabled})") + state.log(f"cluster settings updated (enabled={new.enabled}, manual_peers={len(peers)})") state.audit(client_ip(request), "cluster_settings", "enabled" if new.active else "disabled") return await settings_get(request, ok="Cluster settings saved.") @@ -814,17 +892,13 @@ async def mapping_delete(request): state.delete_mapping_log(mid) return Response("", headers=_TRIGGER) - async def mapping_action(request): - if deny := require_role(request, "operator"): - return deny - mid = request.path_params["mid"] - action = request.path_params["action"] - if action not in ("start", "stop", "restart"): - return PlainTextResponse("Unknown action", status_code=400) + async def _apply_action(mid, action) -> tuple[bool, str]: + """Start/stop/restart a local mapping by id. Returns (ok, name-or-error). + Shared by the UI action endpoint and the cluster remote-control endpoint.""" async with state.config_lock: m = state.config.get_mapping(mid) if not m: - return PlainTextResponse("Mapping not found", status_code=404) + return False, "Mapping not found" m.enabled = action != "stop" await state.asave() if action == "stop": @@ -833,8 +907,20 @@ async def mapping_action(request): await state.supervisor.restart_mapping(m) else: # start await state.supervisor.apply_mapping(m) - state.log(f"mapping {action}: {m.name}") - state.audit(client_ip(request), f"mapping_{action}", m.name) + return True, m.name + + async def mapping_action(request): + if deny := require_role(request, "operator"): + return deny + mid = request.path_params["mid"] + action = request.path_params["action"] + if action not in ("start", "stop", "restart"): + return PlainTextResponse("Unknown action", status_code=400) + ok, name = await _apply_action(mid, action) + if not ok: + return PlainTextResponse(name, status_code=404) + state.log(f"mapping {action}: {name}") + state.audit(client_ip(request), f"mapping_{action}", name) return Response("", headers=_TRIGGER) routes = [ @@ -869,6 +955,8 @@ async def mapping_action(request): Route("/api/status", api_status), Route("/api/cluster/status", cluster_status), Route("/api/cluster/local", cluster_local), + Route("/api/cluster/control", cluster_control, methods=["POST"]), + Route("/api/cluster/control-peer", cluster_control_peer, methods=["POST"]), Route("/api/ports.json", api_ports_json), Route("/api/ports/refresh", api_ports_refresh, methods=["POST"]), Route("/api/ports/table", api_ports_table), diff --git a/app/web/server.py b/app/web/server.py index f63d0e1..cec8a50 100644 --- a/app/web/server.py +++ b/app/web/server.py @@ -23,9 +23,11 @@ STATIC_DIR = os.path.join(os.path.dirname(__file__), "static") PUBLIC_PREFIXES = ("/static", "/auth/") # /auth/oidc/* is the unauthenticated SSO flow -# /api/cluster/local is authenticated by the shared cluster KEY (checked in the -# handler), not a user session — so it must skip the session gate here. -PUBLIC_PATHS = {"/login", "/setup", "/healthz", "/favicon.ico", "/api/cluster/local"} +# /api/cluster/local + /api/cluster/control are authenticated by the shared cluster +# KEY (checked in the handler), not a user session — so they skip the session gate. +# (control-peer is the browser-facing proxy and keeps the normal session + CSRF.) +PUBLIC_PATHS = {"/login", "/setup", "/healthz", "/favicon.ico", + "/api/cluster/local", "/api/cluster/control"} # High-frequency automatic UI refreshes — not logged to all.log to avoid flooding # the audit trail (the user actions that drive them ARE logged). @@ -63,7 +65,8 @@ async def dispatch(self, request, call_next): # body and are validated inside their handlers — the body must NOT be read # here, since doing so in a BaseHTTPMiddleware drains the stream before the # route handler can parse the form. - if request.method in ("POST", "PUT", "DELETE", "PATCH") and path.startswith("/api/"): + if request.method in ("POST", "PUT", "DELETE", "PATCH") and path.startswith("/api/") \ + and path != "/api/cluster/control": # peer-to-peer call: key-authed, no cookie if not auth.csrf_token_matches(request, request.headers.get("x-csrf-token")): return PlainTextResponse("CSRF validation failed. Reload the page.", status_code=403) diff --git a/app/web/templates/_cluster_body.html b/app/web/templates/_cluster_body.html index 60da862..8940e09 100644 --- a/app/web/templates/_cluster_body.html +++ b/app/web/templates/_cluster_body.html @@ -1,46 +1,57 @@ - +{% if discovery_error %} + +{% endif %} +
- + {% for n in nodes %} {% set hosturl = n.scheme ~ '://' ~ n.ip ~ ':' ~ n.port %} - {% set hostcell %} - {{ n.name }}{% if n.self %} this node{% endif %} -
{{ n.ip }}:{{ n.port }}
- {% endset %} - {% if not n.online %} - - - - - {% elif not n.mappings %} - - - - - {% else %} - {% for m in n.mappings %} - - - - - - - - - - - - {% endfor %} + + + + {% for m in n.mappings %} + + + + + + + + + + + {% endfor %} + {% if n.online and not n.mappings %} + {% endif %} {% else %} - + {% endfor %}
HostMappingTypeEndpointSerialMappingTypeEndpointSerial StateClientsIn / Out
{{ hostcell|safe }}offline / unreachable
{{ hostcell|safe }}no mappings on this node
{{ hostcell|safe }}{{ m.name }}{% if not m.enabled %} disabled{% endif %}{{ m.kind }}{{ m.endpoint }}{{ m.serial }}{{ m.state }}{{ m.client_count }}{{ m.bytes_in }} / {{ m.bytes_out }} - {% if not n.self %}Open{% endif %} -
+ {{ n.name }} + {% if n.self %}this node{% endif %} + {% if n.source == 'manual' %}manual{% endif %} + {{ n.ip }}:{{ n.port }} + {% if n.online %} + · up {{ n.uptime }}{% if n.version %} · v{{ n.version }}{% endif %} · {{ n.running_count }}/{{ n.mapping_count }} running + {% else %} + offline / unreachable + {% endif %} + {% if not n.self and n.online %}Open{% endif %} +
{{ m.name }}{% if not m.enabled %} disabled{% endif %}{{ m.kind }}{{ m.endpoint }}{{ m.serial }}{{ m.state }}{{ m.client_count }}{{ m.bytes_in }} / {{ m.bytes_out }} + {% if can_edit and not n.self %} + {% set vals = '{"scheme":"' ~ n.scheme ~ '","host":"' ~ n.ip ~ '","port":"' ~ n.port ~ '","mid":"' ~ m.id ~ '","action":"' %} + {% if m.enabled %} + + + {% else %} + + {% endif %} + {% endif %} +
no mappings on this node
No cluster nodes discovered yet. Beacons are sent every few seconds.
No cluster nodes yet. Auto-discovery beacons run every few seconds; add manual peers under Settings → LAN cluster for routed networks.
diff --git a/app/web/templates/dashboard.html b/app/web/templates/dashboard.html index 49ad3b5..a0e460e 100644 --- a/app/web/templates/dashboard.html +++ b/app/web/templates/dashboard.html @@ -21,7 +21,7 @@

Serial → Network mappings

Cluster — all nodes on the LAN

read-only · auto-discovered · each row shows its host (name + IP) -
+

Discovering nodes…

diff --git a/app/web/templates/settings.html b/app/web/templates/settings.html index d024356..e224a4a 100644 --- a/app/web/templates/settings.html +++ b/app/web/templates/settings.html @@ -159,18 +159,21 @@

LAN cluster

Discover other ser2net nodes on the local network (signed UDP - broadcast — no mDNS) and show every node's mappings in one read-only table on the - dashboard, each row tagged with its host (name + IP). Set the same shared key - on every node so they trust each other. This node's ID: + broadcast — no mDNS) and show every node's mappings in one table on the dashboard, + each row tagged with its host (name + IP). Set the same shared key on every + node so they trust each other; with the key, operators can start/stop/restart a + peer's mappings from the unified view. This node's ID: {{ instance_id[:8] }}.

- +
+
diff --git a/docs/screenshots/02-dashboard.png b/docs/screenshots/02-dashboard.png index 8a23d7c..42e5d96 100644 Binary files a/docs/screenshots/02-dashboard.png and b/docs/screenshots/02-dashboard.png differ diff --git a/docs/screenshots/03-add-mapping.png b/docs/screenshots/03-add-mapping.png index 956399f..79044ea 100644 Binary files a/docs/screenshots/03-add-mapping.png and b/docs/screenshots/03-add-mapping.png differ diff --git a/docs/screenshots/04-settings.png b/docs/screenshots/04-settings.png index 3580a8a..2715b7e 100644 Binary files a/docs/screenshots/04-settings.png and b/docs/screenshots/04-settings.png differ diff --git a/docs/screenshots/06-cluster.png b/docs/screenshots/06-cluster.png index 7a3c386..a3c87fd 100644 Binary files a/docs/screenshots/06-cluster.png and b/docs/screenshots/06-cluster.png differ diff --git a/ser2net.py b/ser2net.py index 7e74fbe..911edd5 100644 --- a/ser2net.py +++ b/ser2net.py @@ -17,7 +17,9 @@ import os import sys -__version__ = "2.5.0" +# The script's own directory is on sys.path, so the `app` package imports here even +# before the explicit sys.path setup below (and PyInstaller bundles it in frozen builds). +from app import __version__ ROOT = os.path.dirname(os.path.abspath(__file__)) FROZEN = getattr(sys, "frozen", False) # True inside a PyInstaller build diff --git a/tests/test_cluster.py b/tests/test_cluster.py index c27d456..d11c54e 100644 --- a/tests/test_cluster.py +++ b/tests/test_cluster.py @@ -28,6 +28,7 @@ from app.config import AppConfig, ClusterSettings # noqa: E402 from app.engine.cluster import ClusterDiscovery, PEER_TTL, _sign # noqa: E402 +from app.web.auth import hash_password # noqa: E402 def free_port(): @@ -191,9 +192,122 @@ def post(path, fields, header=False): shutil.rmtree(tmp, ignore_errors=True) +def start_node(cfg_obj): + tmp = tempfile.mkdtemp(prefix="ser2net_cl_") + cfg = os.path.join(tmp, "config.json") + with open(cfg, "w") as fh: + json.dump(cfg_obj, fh) + proc = subprocess.Popen([sys.executable, "ser2net.py", "--no-bootstrap", "--config", cfg], + cwd=ROOT, stdin=subprocess.DEVNULL, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + return proc, tmp + + +def test_manual_peer_and_remote_control(): + """Two real nodes linked by a MANUAL peer (no UDP dependency): node A aggregates + node B's mappings + health and can remotely start/stop them with the shared key.""" + key = "rc-key-xyz" + ui_a, ui_b = free_port(), free_port() + # different discovery ports so the nodes DON'T auto-discover each other — the + # manual peer is the only link, making the test deterministic. + dp_a, dp_b = free_port(), free_port() + map_port = free_port() + node_b = {"admin_ui": {"bind_ip": "127.0.0.1", "port": ui_b}, + "users": [{"username": "admin", "password_hash": hash_password("clusteradmin1"), + "role": "admin", "pwd_version": 1}], + "cluster": {"enabled": True, "key": key, "discovery_port": dp_b, + "advertise_ip": "127.0.0.1"}, + "mappings": [{"name": "REMOTE1", "enabled": False, "kind": "net", + "serial": {"port": "/dev/ttyTEST", "baudrate": 9600}, + "network": {"mode": "server", "protocol": "raw", + "bind_ip": "127.0.0.1", "port": map_port}}]} + node_a = {"admin_ui": {"bind_ip": "127.0.0.1", "port": ui_a}, + "cluster": {"enabled": True, "key": key, "discovery_port": dp_a, + "peers": [f"127.0.0.1:{ui_b}"]}} + pa, ta = start_node(node_a) + pb, tb = start_node(node_b) + try: + assert wait_port(ui_a) and wait_port(ui_b), "nodes did not start" + + # admin session on node A + jar = http.cookiejar.CookieJar() + op = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(jar)) + + def csrf(): + return next(c.value for c in jar if c.name == "ser2net_csrf") + + def post_a(path, fields): + req = urllib.request.Request(f"http://127.0.0.1:{ui_a}{path}", + data=urllib.parse.urlencode(fields).encode(), method="POST") + req.add_header("Content-Type", "application/x-www-form-urlencoded") + req.add_header("X-CSRF-Token", csrf()) + try: + return op.open(req, timeout=10).status + except urllib.error.HTTPError as e: + return e.code + + op.open(f"http://127.0.0.1:{ui_a}/setup") + post_a("/setup", {"username": "admin", "password": "adminpass123", + "password2": "adminpass123", "_csrf": csrf()}) + + # node B's mapping id (and confirm it starts disabled) + _, body = raw_request(ui_b, "/api/cluster/local", {"X-Cluster-Key": key}) + bmaps = json.loads(body)["mappings"] + mid = bmaps[0]["id"] + assert bmaps[0]["enabled"] is False, bmaps + + # A's aggregated view fetches B through the manual peer: shows it + health + html = op.open(f"http://127.0.0.1:{ui_a}/api/cluster/status", timeout=10).read().decode() + assert "REMOTE1" in html and "manual" in html, html[:500] + assert ("v2.6" in html or "up " in html), "per-node health (version/uptime) missing" + print("manual peer aggregated with health (version/uptime) + 'manual' tag OK") + + # peer-facing control is key-guarded + code, _ = raw_request(ui_b, "/api/cluster/control", method="POST") + assert code == 403, f"control without key must be 403, got {code}" + + # remote START via A's proxy -> B's mapping becomes enabled + ctrl = {"scheme": "http", "host": "127.0.0.1", "port": str(ui_b), "mid": mid, "action": "start"} + assert post_a("/api/cluster/control-peer", ctrl) == 200, "remote start should be accepted" + for _ in range(20): + _, body = raw_request(ui_b, "/api/cluster/local", {"X-Cluster-Key": key}) + if json.loads(body)["mappings"][0]["enabled"]: + break + time.sleep(0.1) + assert json.loads(body)["mappings"][0]["enabled"] is True, "remote start did not take effect" + print("remote start of a peer's mapping via the unified view OK") + + # remote STOP via A's proxy -> B's mapping becomes disabled + ctrl["action"] = "stop" + assert post_a("/api/cluster/control-peer", ctrl) == 200 + for _ in range(20): + _, body = raw_request(ui_b, "/api/cluster/local", {"X-Cluster-Key": key}) + if not json.loads(body)["mappings"][0]["enabled"]: + break + time.sleep(0.1) + assert json.loads(body)["mappings"][0]["enabled"] is False, "remote stop did not take effect" + print("remote stop of a peer's mapping via the unified view OK") + + # the proxy refuses an address that isn't a known cluster peer (anti-SSRF) + bad = {"scheme": "http", "host": "9.9.9.9", "port": "1", "mid": mid, "action": "start"} + assert post_a("/api/cluster/control-peer", bad) == 403, "unknown peer must be rejected" + print("control proxy rejects an unknown (non-peer) address OK") + + print("\nPASS: manual peers + per-node health + remote control") + finally: + for proc, tmp in ((pa, ta), (pb, tb)): + proc.terminate() + try: + proc.wait(timeout=5) + except subprocess.TimeoutExpired: + proc.kill() + shutil.rmtree(tmp, ignore_errors=True) + + def main(): test_discovery_unit() test_endpoints() + test_manual_peer_and_remote_control() if __name__ == "__main__":