diff --git a/CHANGELOG.md b/CHANGELOG.md index cc464bbb..fd9bd68d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,86 @@ Pithead ships as **one product, one version** — the version lives in the top-l [`VERSION`](VERSION) file and every released image is tagged with it. Releases are cut per the process in [`docs/dev/releasing.md`](docs/dev/releasing.md). +## [1.10.0] - 2026-07-20 + +### Added + +- **One-click remote worker upgrade** (#597). Where the per-worker badge shows and the rig is + editable, Worker Inspect gains an Upgrade rig… button: arm, confirm, and the rig installs the + latest RigForge release itself (rig ≥ v1.11.2 with its default-off `control_upgrade` flag chain + enabled). The intent carries the worker name and confirmed version only; the host runner + re-derives the real target from the RigForge release API over Tor (throttled, cached), resolves + the rig's address and bearer from `config.json`, dials over the LAN, and polls the rig to a + terminal applied / rolled-back / failed with a hard cap. Already-current rigs no-op without + dialing; a rig-side throttle refusal reads as retry-later. Per-rig only — no "upgrade all". + +- **Per-worker RigForge "new version available" badge** (#596). A rig whose reported RigForge + version is older than the latest published release gets a clickable badge in the Workers Alive + table and in Worker Inspect, linking to the release notes — the worker-level twin of the + header's stack-release badge. Notify-only; one hourly, Tor-routed, fail-silent fetch covers the + whole fleet, gated on the same `dashboard.check_for_updates` flag. Rigs that report no version + (plain xmrig, sister API off) show no badge — unknown, not "up to date". + +### Fixed + +- **Removing the control runner no longer strands a sibling checkout's stack (#689).** The + `pithead-control.{path,service}` unit names are global to the host, but a bench box holds + several checkouts at once — and a checkout applying with dashboard control off (or the e2e + harness tearing down) removed whatever units were installed, including the live stack's, + leaving its config editor stuck at "Previewing…" until the next apply. Both removal paths now + check the service unit's `ExecStart` and only touch units owned by the acting checkout, + comparing physical paths so the `current` symlink and the versioned directory it targets + count as the same checkout. + +- **An unedited Save & preview shows zero changes (#695, #696).** On a bundle-deployed box the + Review changes modal reported two changes with nothing edited. First, a path "change" such as + `CLEARNET_STATE_DIR: /srv/code/current/... → /srv/code/pithead-vX.Y.Z/...`: pithead resolved + its own directory with a logical `pwd`, so `.env` paths derived from the checkout dir took the + spelling of whoever invoked it — the deploy symlink interactively, the physical dir under the + control runner's systemd unit — and the same directory diffed against itself. The script now + canonicalizes with `pwd -P`, and `CLEARNET_STATE_DIR` joins its siblings (`CONTROL_DIR`, + `CADDY_LOG_DIR`) as a silent internal path in the change preview. Second, a permanent + "Energy calculator settings updated" row on any box whose `config.json` never set + `dashboard.energy`: the editor round-trips the reference-merged form, so the staged copy + carries the materialized energy defaults, and the preview compared them against the absent + block. The comparison — in the preview row and the commit's audit-key derivation alike — now + merges the reference defaults into both sides, so only a real value change raises the row. + +- **The egress panel no longer reports a phantom clearnet leak for the XvB stats fetch (#701).** + With `xvb.tor: false`, the #170 posture panel and topology view showed the dashboard's XvB + stats connection as a clearnet leak. That fetch is unconditionally routed over Tor (`socks5h`, + #163) — `xvb.tor` gates only the xmrig-proxy donation dial (#166) — so the panel warned about + a leak that cannot happen. The dashboard's XvB stats route is now Tor whenever XvB is enabled, + matching what the code actually does and what `docs/privacy.md` already documented. + +- **The egress panel and network map now list the webhook/ntfy alert sinks (#380).** Both views + derived every dashboard egress except the alert sinks, so a `notifications.tor: false` sink + POSTing to a public endpoint — a real clearnet leak from the host-networked dashboard, which the + egress firewall cannot cover — went uncounted. The new "alert sinks (webhook / ntfy)" entry is + Tor when configured (the default), a counted clearnet leak when Tor is off and any endpoint is + public, and **local** — the LAN carve-out, not a leak — only when every configured endpoint is a + private or loopback IP literal, since a hostname cannot be proven private without a DNS lookup. + +### Changed + +- **The release process requires the targeted end-to-end run.** `docs/dev/releasing.md` now + states that the borrowed-rig `e2e.sh --mode targeted` pass on the release candidate is a + required pre-release gate — `release.sh`'s `--readiness` assessment alone is not enough — and + documents the post-deploy `--check` sweep and its expected parked-bench baseline. Private + bench hostnames in docs, comments, and one harness message are replaced with generic role + names; each box's specifics live in its own `~/README.md`, not the repo. + +### Security + +- **The dashboard's external API fetches are size-capped** (#660). The GitHub release check, the + CoinGecko price feed, and the XvB client's calls (stats, reward estimates, winners, register) + now stream their responses through a shared `bounded_get` helper that cuts the body at 1 MiB, + so a hostile or broken endpoint can no longer make these clients buffer an unbounded payload. + Over-cap reads follow each client's existing failure contract (no result / keep the last good + one). The remaining external GETs — the Tor egress probe, the Healthchecks ping, and the + Telegram `getUpdates` long-poll — ride the same cap; `getUpdates` also caps its batch at 10 + updates so a capped batch can never wedge the poll loop on an offset it cannot advance. + ## [1.9.3] - 2026-07-19 ### Fixed @@ -1053,7 +1133,7 @@ tabbed earnings panel. time out, so the dead-man's-switch pings, the Telegram bot, and XvB stats all stop while mining (onion circuits) keeps working — the stack looks healthy as three features die. A new doctor check makes one request through Tor's SOCKS to a no-content endpoint and WARNs with the fix (restart the - tor container to pick fresh guards) when clearnet exits fail. Found live on pithead-prod after the + tor container to pick fresh guards) when clearnet exits fail. Found live on the production stack after the v1.3.0 deploy. - **A release-box checkout that has run the stack no longer fails `lint-toml` (#421).** taplo globs the filesystem, not the git index, so the generated (git-ignored) `build/tari/config.toml` left by diff --git a/VERSION b/VERSION index 7b0231f5..81c871de 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.9.3 \ No newline at end of file +1.10.0 diff --git a/build/dashboard/mining_dashboard/client/tari/tari_wallet_client.py b/build/dashboard/mining_dashboard/client/tari/tari_wallet_client.py index 0940eba9..6100e6c2 100644 --- a/build/dashboard/mining_dashboard/client/tari/tari_wallet_client.py +++ b/build/dashboard/mining_dashboard/client/tari/tari_wallet_client.py @@ -14,7 +14,7 @@ # The direction/status gate is generous on purpose: p2pool's Tari coinbase is a one-sided output to # wallet_payment_address, and which enum a view-only wallet reports it under # (COINBASE_CONFIRMED vs ONE_SIDED_CONFIRMED vs a plain INBOUND) is the one item pinned to tier-4 -# (gouda). Keeping the accept-set here isolated means tightening it after that check is a one-line +# (the live bench). Keeping the accept-set here isolated means tightening it after that check is a one-line # change. Enum values are the wallet.proto constants (TransactionStatus / TransactionDirection). _DIRECTION_INBOUND = 1 # TRANSACTION_DIRECTION_INBOUND _COINBASE_STATUSES = frozenset( diff --git a/build/dashboard/mining_dashboard/client/xvb_client.py b/build/dashboard/mining_dashboard/client/xvb_client.py index 592d8b13..03337d18 100644 --- a/build/dashboard/mining_dashboard/client/xvb_client.py +++ b/build/dashboard/mining_dashboard/client/xvb_client.py @@ -5,6 +5,7 @@ import requests from mining_dashboard.config.config import TOR_SOCKS_PROXY, XVB_SUBMIT_URL +from mining_dashboard.helper.http import bounded_get from mining_dashboard.helper.utils import parse_hashrate # The four donor tiers XvB publishes an expected per-player reward for. These are exactly the tier @@ -159,7 +160,7 @@ def get_stats(self): # the request carries the wallet, so a clearnet fetch would correlate IP <-> wallet (#163). proxies = {"http": self.tor_proxy, "https": self.tor_proxy} if self.tor_proxy else None try: - response = requests.get(self.url, params=params, timeout=20, proxies=proxies) + response = bounded_get(self.url, params=params, timeout=20, proxies=proxies) if response.status_code == 200: return self._parse_html(response.text) else: @@ -184,7 +185,7 @@ def get_reward_estimates(self): """ proxies = {"http": self.tor_proxy, "https": self.tor_proxy} if self.tor_proxy else None try: - response = requests.get(self.reward_estimate_url, timeout=20, proxies=proxies) + response = bounded_get(self.reward_estimate_url, timeout=20, proxies=proxies) if response.status_code != 200: self.logger.error( f"XvB reward-estimate fetch failed with status code: {response.status_code}" @@ -221,7 +222,7 @@ def get_recent_wins(self): proxies = {"http": self.tor_proxy, "https": self.tor_proxy} if self.tor_proxy else None try: - response = requests.get(self.winners_url, timeout=20, proxies=proxies) + response = bounded_get(self.winners_url, timeout=20, proxies=proxies) if response.status_code != 200: self.logger.error( f"XvB winners fetch failed with status code: {response.status_code}" @@ -275,7 +276,7 @@ def register(self): # wallet, so a clearnet call would correlate IP <-> wallet (#163). proxies = {"http": self.tor_proxy, "https": self.tor_proxy} if self.tor_proxy else None try: - response = requests.get(self.submit_url, params=params, timeout=20, proxies=proxies) + response = bounded_get(self.submit_url, params=params, timeout=20, proxies=proxies) body = (response.text or "").strip() low = body.lower() diff --git a/build/dashboard/mining_dashboard/config/config.py b/build/dashboard/mining_dashboard/config/config.py index 863908c4..1408024b 100644 --- a/build/dashboard/mining_dashboard/config/config.py +++ b/build/dashboard/mining_dashboard/config/config.py @@ -359,6 +359,12 @@ def _nonneg(v): "GITHUB_RELEASES_API", "https://api.github.com/repos/p2pool-starter-stack/pithead/releases/latest", ) +# Latest RigForge release, for the per-worker "new version available" badge (#596). Same check, +# same Tor route, same dashboard.check_for_updates gate as the stack's own release check above. +GITHUB_RIGFORGE_RELEASES_API = os.environ.get( + "GITHUB_RIGFORGE_RELEASES_API", + "https://api.github.com/repos/p2pool-starter-stack/rigforge/releases/latest", +) UPDATE_CHECK_INTERVAL = int(float(os.environ.get("UPDATE_CHECK_INTERVAL", "3600"))) # Donation tier to target (config.json: xvb.donation_level). The XvB raffle picks diff --git a/build/dashboard/mining_dashboard/helper/http.py b/build/dashboard/mining_dashboard/helper/http.py new file mode 100644 index 00000000..cdb8c181 --- /dev/null +++ b/build/dashboard/mining_dashboard/helper/http.py @@ -0,0 +1,62 @@ +"""Bounded reads for every external HTTP fetch (#660). + +Each external client (GitHub release checks #224, the three XvB reads, the CoinGecko price feed +#651, the Tor egress probe, the Healthchecks ping, the Telegram getUpdates long-poll) is +individually tolerant of a bad *parse*, but nothing bounded what got *read*: a hostile or broken +endpoint could hand any of them a multi-GB body and the process would buffer it all before +parsing. ``bounded_get`` streams the body and cuts it at a cap far above any legitimate payload; +over-cap raises a ``requests.RequestException`` subclass, so every caller's existing failure +contract (return ``None`` / keep the last good result) applies unchanged. +""" + +import json + +import requests + +# Generous by orders of magnitude: the largest legitimate payload here (the XvB winners file) is +# well under 100 KiB; the GitHub/CoinGecko JSON bodies are a few KiB. +MAX_RESPONSE_BYTES = 1024 * 1024 + + +class ResponseTooLarge(requests.RequestException): + """Response body exceeded the cap. Subclasses ``RequestException`` so callers' existing + fail-silent handling treats it like any other transport failure.""" + + +class BoundedResponse: + """The slice of ``requests.Response`` the clients actually use: ``status_code``, ``text``, + ``json()``, ``raise_for_status()`` — backed by the capped body.""" + + def __init__(self, status_code, content, encoding): + self.status_code = status_code + self.content = content + self.encoding = encoding + + @property + def text(self): + return self.content.decode(self.encoding or "utf-8", errors="replace") + + def json(self): + return json.loads(self.text) + + def raise_for_status(self): + # HTTPError subclasses RequestException, matching requests' own contract. + if self.status_code >= 400: + raise requests.HTTPError(f"HTTP {self.status_code}") + + +def bounded_get(url, max_bytes=MAX_RESPONSE_BYTES, timeout=20, **kwargs): + """``requests.get`` with a hard response-size cap. + + Streams the body and raises ``ResponseTooLarge`` once it exceeds ``max_bytes``, instead of + buffering an unbounded payload. Passes ``proxies`` / ``params`` / ``headers`` through + unchanged; raises exactly what ``requests.get`` raises otherwise. + """ + with requests.get(url, stream=True, timeout=timeout, **kwargs) as resp: + chunks, size = [], 0 + for chunk in resp.iter_content(chunk_size=65536): + size += len(chunk) + if size > max_bytes: + raise ResponseTooLarge(f"response body exceeded {max_bytes} bytes: {url}") + chunks.append(chunk) + return BoundedResponse(resp.status_code, b"".join(chunks), resp.encoding) diff --git a/build/dashboard/mining_dashboard/service/control_service.py b/build/dashboard/mining_dashboard/service/control_service.py index 3e35e0ad..2b9e4147 100644 --- a/build/dashboard/mining_dashboard/service/control_service.py +++ b/build/dashboard/mining_dashboard/service/control_service.py @@ -262,6 +262,27 @@ def submit_worker_apply(worker, changes, actor="", intent_id=None): return rid +def submit_worker_upgrade(worker, version, actor="", intent_id=None): + """Spool a worker RigForge-upgrade intent (#597). Carries ONLY the worker NAME and the version + the operator confirmed seeing — never a host, port, or token (the host runner resolves the + rig's real address and bearer from workers.list[], exactly like worker-apply), and the version + is a proposal, never a target: the host re-derives the real latest RigForge release over Tor + and refuses a mismatch. Returns the request id (always a UUID).""" + rid = str(uuid.UUID(intent_id)) if intent_id else str(uuid.uuid4()) + request = { + "id": rid, + "action": "worker-upgrade", + "actor": actor, + "worker": worker, + "version": version, + } + tmp = os.path.join(config.CONTROL_REQUESTS_DIR, f".{rid}.tmp") + with open(tmp, "w") as f: + json.dump(request, f) + os.replace(tmp, os.path.join(config.CONTROL_REQUESTS_DIR, f"{rid}.json")) + return rid + + # The config keys the Worker Inspect editor may change — the exact writable allowlist the rig's # control API enforces (rigforge WRITABLE, #236). Validated here (fail-closed, defence in depth), on # the host runner, and finally by the rig itself. NOT writable: identity, filesystem paths, the API diff --git a/build/dashboard/mining_dashboard/service/data_service.py b/build/dashboard/mining_dashboard/service/data_service.py index 7513926e..498a9c3d 100644 --- a/build/dashboard/mining_dashboard/service/data_service.py +++ b/build/dashboard/mining_dashboard/service/data_service.py @@ -41,6 +41,7 @@ DASHBOARD_ENERGY, ENABLE_XVB, GITHUB_RELEASES_API, + GITHUB_RIGFORGE_RELEASES_API, HASHRATE_DROP_MINUTES, HASHRATE_DROP_THRESHOLD_PCT, HOST_IP, @@ -419,6 +420,15 @@ def __init__(self, state_manager, proxy_client, xvb_client): enabled=CHECK_FOR_UPDATES, interval=UPDATE_CHECK_INTERVAL, ) + # RigForge latest-release check (#596): the same flag, throttle and Tor route, pointed at + # the RigForge repo. ONE fleet-wide fetch — the per-worker "rig is behind" verdict is + # derived at the render seam from each rig's live reported version, never stored (#664). + self.rigforge_update_checker = UpdateChecker( + GitHubReleaseClient(GITHUB_RIGFORGE_RELEASES_API, TOR_SOCKS_PROXY), + None, + enabled=CHECK_FOR_UPDATES, + interval=UPDATE_CHECK_INTERVAL, + ) # Live XMR/XTM price feed (#520's auto half): off unless dashboard.energy.price_feed is # set. Same Tor SOCKS route as the update check — CoinGecko only ever sees a Tor exit. self.price_feed = PriceFeed( @@ -543,6 +553,10 @@ def __init__(self, state_manager, proxy_client, xvb_client): # JUST changed — the very upgrade the restored badge advertised. The checker # recomputes it on its own cadence; never resurrect the pre-upgrade banner. loaded_snapshot.pop("update", None) + # Same rule for the fleet-wide RigForge release (#596): with the flag now off, a + # restored `rigforge_release` would keep serving stale per-worker badges until the + # first poll cycle. The checker re-fetches on its cadence; drop it on restore. + loaded_snapshot.pop("rigforge_release", None) self.latest_data.update(loaded_snapshot) self.workers_rejected = bool(self.latest_data.get("workers_rejected", False)) self.miner_released = bool(self.latest_data.get("miner_released", False)) @@ -1335,6 +1349,13 @@ async def run(self): self.latest_data["update"] = await asyncio.to_thread( self.update_checker.maybe_check, time.time() ) + # 8b. The RigForge counterpart (#596): cache the latest RigForge release + # (raw {tag, url}); build_workers derives each rig's badge from it. Written + # unconditionally — the accessor returns None without dialing when the check + # is disabled, so a snapshot-restored release can't outlive a flag flip. + self.latest_data["rigforge_release"] = await asyncio.to_thread( + self.rigforge_update_checker.latest_release_cached, time.time() + ) # 9. Live XMR/XTM prices over Tor (#520) — ONLY when dashboard.energy.price_feed # is set (default off, so the appliance never dials CoinGecko unbidden). The diff --git a/build/dashboard/mining_dashboard/service/egress.py b/build/dashboard/mining_dashboard/service/egress.py index e4343b2b..ab171600 100644 --- a/build/dashboard/mining_dashboard/service/egress.py +++ b/build/dashboard/mining_dashboard/service/egress.py @@ -9,13 +9,23 @@ * The **#270 egress firewall** (``DOCKER-USER``, fail-closed) DROPs non-Tor egress from the *container* subnet — so a container's clearnet route can't actually leave while it's on. * It does **not** cover the **host-networked dashboard** (``network_mode: host``), whose own egress - (XvB stats fetch, update check, Healthchecks ping, Telegram bot) bypasses ``DOCKER-USER`` entirely. - Those rely solely on their SOCKS config — a clearnet route there is a real leak regardless of the - firewall. (All four are Tor-routed by default, so none leak.) + (XvB stats fetch, update check, Healthchecks ping, Telegram bot, price feed, webhook/ntfy alert + sinks) bypasses ``DOCKER-USER`` entirely. Those rely solely on their SOCKS config — a clearnet + route there is a real leak regardless of the firewall. (All are Tor-routed by default, so none + leak.) + +The alert sinks (#380) have one more wrinkle: ``notifications.tor: false`` is a LAN carve-out for +self-hosted endpoints Tor exits can't reach. A POST to a private/loopback IP never leaves your +network, so it routes as *local*, not a clearnet leak. Only IP literals can prove that without a +DNS lookup — a hostname endpoint with Tor off counts as clearnet, honestly, since we can't know +where it resolves. So a connection is a *leak* only when its route is clearnet AND it isn't neutralised by a backstop. """ +import ipaddress +from urllib.parse import urlsplit + from mining_dashboard.config import config TOR = "tor" @@ -30,6 +40,32 @@ def _xvb_route(xvb_enabled, xvb_tor): return TOR if xvb_tor else CLEARNET +def _notify_route(enabled, tor, private): + if not enabled: + return INACTIVE + if tor: + return TOR + return LOCAL if private else CLEARNET + + +def _sinks_all_private(urls): + """True when every configured sink URL targets a private/loopback IP literal — the LAN + carve-out proof. A hostname can't be verified without a DNS lookup (which a pure config + derivation must never do), so any hostname makes this False.""" + hosts = [urlsplit(u).hostname for u in urls if u and u.strip()] + if not hosts: + return False + for host in hosts: + if not host: # malformed URL (no scheme) — unknowable, assume public + return False + try: + if not ipaddress.ip_address(host).is_private: + return False + except ValueError: # not an IP literal — unknowable, assume public + return False + return True + + def compute_egress_posture( *, firewall, @@ -42,9 +78,13 @@ def compute_egress_posture( healthchecks_enabled, telegram_enabled, price_feed_enabled=False, + notify_sinks_enabled=False, + notify_tor=True, + notify_sinks_private=False, ): """Pure derivation of the egress posture from config knobs. Returns ``{components, summary}``.""" xvb = _xvb_route(xvb_enabled, xvb_tor) + sinks = _notify_route(notify_sinks_enabled, notify_tor, notify_sinks_private) # ``firewalled``: is this component's egress on the container subnet the #270 firewall guards? # The dashboard is host-networked, so its own outbound traffic is NOT covered. @@ -97,7 +137,9 @@ def compute_egress_posture( "name": "dashboard", "firewalled": False, # host-networked — bypasses the #270 DOCKER-USER firewall "conns": [ - {"to": "XvB stats (xmrvsbeast.com)", "route": xvb}, # socks5h when on (#163) + # XvB stats fetch — unconditionally socks5h over Tor (#163/#701); xvb.tor only + # governs the xmrig-proxy donation dial above, never this fetch. + {"to": "XvB stats (xmrvsbeast.com)", "route": TOR if xvb_enabled else INACTIVE}, {"to": "update check (github)", "route": TOR}, # socks5h, #224 # Healthchecks.io dead-man's-switch ping — always over Tor when a URL is set (#79). {"to": "Healthchecks.io ping", "route": TOR if healthchecks_enabled else INACTIVE}, @@ -108,6 +150,9 @@ def compute_egress_posture( "to": "price feed (coingecko.com)", "route": TOR if price_feed_enabled else INACTIVE, }, + # Webhook/ntfy alert sinks (#380) — Tor by default; ``notifications.tor: false`` + # to an all-private-IP endpoint set is the LAN carve-out (local, not a leak). + {"to": "alert sinks (webhook / ntfy)", "route": sinks}, ], }, { @@ -162,9 +207,20 @@ def egress_posture_from_config(): healthchecks_enabled=bool(config.HEALTHCHECKS_PING_URL), telegram_enabled=config.TELEGRAM_ENABLED, price_feed_enabled=config.DASHBOARD_ENERGY["price_feed"], + **_notify_knobs(), ) +def _notify_knobs(): + """The #380 alert-sink knobs, shared by both from-config builders.""" + urls = [*config.NOTIFY_WEBHOOK_URLS, config.NTFY_URL] + return { + "notify_sinks_enabled": any(u.strip() for u in urls if u), + "notify_tor": config.NOTIFY_TOR, + "notify_sinks_private": _sinks_all_private(urls), + } + + # --- Stack topology (#170, trust-boundary view) ---------------------------------------- # The egress list above answers "is anything leaking?"; the topology answers "how is the whole # stack wired?" — every component and the route of each link (ingress, egress, internal). Same @@ -215,6 +271,9 @@ def compute_topology( healthchecks_enabled, telegram_enabled, price_feed_enabled=False, + notify_sinks_enabled=False, + notify_tor=True, + notify_sinks_private=False, ): """Pure derivation of the stack topology. Returns ``{nodes, edges, summary}``. @@ -233,8 +292,12 @@ def compute_topology( healthchecks_enabled=healthchecks_enabled, telegram_enabled=telegram_enabled, price_feed_enabled=price_feed_enabled, + notify_sinks_enabled=notify_sinks_enabled, + notify_tor=notify_tor, + notify_sinks_private=notify_sinks_private, ) xvb = _xvb_route(xvb_enabled, xvb_tor) + sinks = _notify_route(notify_sinks_enabled, notify_tor, notify_sinks_private) sidechain = CLEARNET if p2pool_clearnet else TOR rpc = CLEARNET if remote_monero else LOCAL @@ -249,7 +312,8 @@ def compute_topology( # App-level egress. _edge("xmrig-proxy", _ext(xvb), xvb, "XvB donation", "egress"), _edge("dashboard", "tor", TOR, "update check", "egress"), - _edge("dashboard", _ext(xvb), xvb, "XvB stats", "egress"), + # XvB stats fetch — unconditionally Tor (#163/#701); xvb.tor only gates the donation dial. + _edge("dashboard", "tor", TOR if xvb_enabled else INACTIVE, "XvB stats", "egress"), # Healthchecks.io ping — always over Tor when a URL is set (#79). _edge( "dashboard", @@ -274,6 +338,14 @@ def compute_topology( "price feed", "egress", ), + # Webhook/ntfy alert sinks (#380). The LAN carve-out (route ``local``) has no placeable + # node — a LAN appliance isn't in the diagram — so it draws no edge; the shared summary + # still reflects it (as no leak), and the egress list shows the ``local`` route. + *( + [_edge("dashboard", _ext(sinks), sinks, "alert sinks", "egress")] + if sinks != LOCAL + else [] + ), # The Tor hub to the network: SOCKS egress for every daemon + onion-service ingress. _edge("tor", "internet", TOR, "SOCKS + onion circuits", "p2p"), # Internal mesh (hidden until expanded). @@ -318,4 +390,5 @@ def topology_from_config(): healthchecks_enabled=bool(config.HEALTHCHECKS_PING_URL), telegram_enabled=config.TELEGRAM_ENABLED, price_feed_enabled=config.DASHBOARD_ENERGY["price_feed"], + **_notify_knobs(), ) diff --git a/build/dashboard/mining_dashboard/service/healthchecks.py b/build/dashboard/mining_dashboard/service/healthchecks.py index 902921a6..7d8a5393 100644 --- a/build/dashboard/mining_dashboard/service/healthchecks.py +++ b/build/dashboard/mining_dashboard/service/healthchecks.py @@ -34,6 +34,7 @@ HEALTHCHECKS_PING_URL, TOR_SOCKS_PROXY, ) +from mining_dashboard.helper.http import bounded_get logger = logging.getLogger("Healthchecks") @@ -111,7 +112,7 @@ def ping(self): return False try: - resp = requests.get(self.url, timeout=_PING_TIMEOUT_SEC, proxies=self._proxies) + resp = bounded_get(self.url, timeout=_PING_TIMEOUT_SEC, proxies=self._proxies) if 200 <= resp.status_code < 300: if self._last_ping_ok is False: logger.info( diff --git a/build/dashboard/mining_dashboard/service/price_feed.py b/build/dashboard/mining_dashboard/service/price_feed.py index f926b78b..167f4700 100644 --- a/build/dashboard/mining_dashboard/service/price_feed.py +++ b/build/dashboard/mining_dashboard/service/price_feed.py @@ -19,6 +19,8 @@ import requests +from mining_dashboard.helper.http import bounded_get + logger = logging.getLogger("PriceFeed") COINGECKO_SIMPLE_PRICE = "https://api.coingecko.com/api/v3/simple/price" @@ -62,7 +64,7 @@ def fetch(self): return None proxies = {"http": self.tor_proxy, "https": self.tor_proxy} if self.tor_proxy else None try: - resp = requests.get( + resp = bounded_get( COINGECKO_SIMPLE_PRICE, params={ "ids": ",".join(COINGECKO_IDS.values()), diff --git a/build/dashboard/mining_dashboard/service/telegram_commands.py b/build/dashboard/mining_dashboard/service/telegram_commands.py index bd207ed6..c16c2276 100644 --- a/build/dashboard/mining_dashboard/service/telegram_commands.py +++ b/build/dashboard/mining_dashboard/service/telegram_commands.py @@ -17,6 +17,7 @@ TELEGRAM_ENABLED, TOR_SOCKS_PROXY, ) +from mining_dashboard.helper.http import bounded_get from mining_dashboard.helper.utils import ( effective_hashrate, format_disk_size, @@ -40,6 +41,10 @@ # Quiet retry after a failed poll — a Tor-only / offline host can't reach api.telegram.org, so a # persistently-blocked bot backs off instead of hot-looping (and never spams ERROR; #59 discipline). POLL_ERROR_BACKOFF_SECONDS = 15 +# getUpdates batch cap. The offset can only advance after a batch is *parsed*, so a batch that +# trips bounded_get's size cap would be re-fetched forever — bounding the batch keeps the worst +# case (10 updates at Telegram's own per-message field limits) far under the cap instead. +GETUPDATES_LIMIT = 10 # The commands the bot answers. All are read-only status queries — the bot can never change the # stack (start/stop/apply live on the CLI), so a leaked chat can at worst read status, not act. @@ -733,10 +738,11 @@ async def run(self): def _prime_offset(self): """Advance the offset past any pending backlog without acting on it, so a command queued - while the dashboard was down isn't run on startup.""" + while the dashboard was down isn't run on startup. Drains batch by batch: getUpdates + returns at most GETUPDATES_LIMIT updates per call, and returns immediately (timeout 0) + while a backlog remains.""" try: - updates = self._get_updates(0) - if updates: + while updates := self._get_updates(0): self._offset = updates[-1].get("update_id", 0) + 1 except Exception as exc: logger.debug("Telegram offset prime skipped (%s)", type(exc).__name__) @@ -746,13 +752,14 @@ def _get_updates(self, poll_timeout): # Ask Telegram for callback_query updates too when control commands are on — that is how a # tapped inline confirm button arrives (#338); the read-only bot stays messages-only. allowed = '["message","callback_query"]' if self.control_enabled else '["message"]' - params = {"timeout": poll_timeout, "allowed_updates": allowed} + params = {"timeout": poll_timeout, "allowed_updates": allowed, "limit": GETUPDATES_LIMIT} if self._offset is not None: params["offset"] = self._offset url = f"{self._api_base}/bot{self._token}/getUpdates" # The read timeout must outlast Telegram's long-poll hold, or requests aborts the request - # the server is legitimately keeping open; (connect, read) tuple. - resp = requests.get( + # the server is legitimately keeping open; (connect, read) tuple. bounded_get streams, but + # the read timeout still covers the hold: headers only arrive once the hold ends. + resp = bounded_get( url, params=params, timeout=(10, poll_timeout + 10), proxies=self._proxies ) resp.raise_for_status() diff --git a/build/dashboard/mining_dashboard/service/tor_heal.py b/build/dashboard/mining_dashboard/service/tor_heal.py index 107c1847..d069f8ed 100644 --- a/build/dashboard/mining_dashboard/service/tor_heal.py +++ b/build/dashboard/mining_dashboard/service/tor_heal.py @@ -3,7 +3,7 @@ Tor can bootstrap to 100% yet sit on a FAILING GUARD: circuits build but clearnet exits time out at the 60s cutoff, so every Tor-clearnet feature (Healthchecks pings, the Telegram bot, XvB stats) dies at once while mining — onion / already-established circuits — keeps working -and the stack looks healthy. Seen live on pithead-prod after the v1.3.0 deploy: ~6 hours dark +and the stack looks healthy. Seen live in production after the v1.3.0 deploy: ~6 hours dark until a manual ``docker compose restart tor`` reselected guards. The doctor check (v1.3.1) detects this state; this monitor is the heal half. @@ -32,7 +32,7 @@ healed. The restart goes through the same start/stop-only docker-control proxy as the #31 failover. -The manual leg is ``./pithead restart tor``. Real stuck-guard recovery is tier 4 (gouda). +The manual leg is ``./pithead restart tor``. Real stuck-guard recovery is tier 4 (the live bench). """ import asyncio @@ -42,6 +42,7 @@ import requests from mining_dashboard.config.config import TOR_AUTO_HEAL, TOR_SOCKS_PROXY +from mining_dashboard.helper.http import bounded_get logger = logging.getLogger("TorHeal") @@ -100,7 +101,7 @@ def __init__(self, docker_control, enabled=None, probe=None, notify=None, clock= def _probe_egress(): """One SOCKS request through the tor container; True iff a clearnet exit answered.""" try: - requests.get( + bounded_get( PROBE_URL, timeout=PROBE_TIMEOUT_SEC, proxies={"http": TOR_SOCKS_PROXY, "https": TOR_SOCKS_PROXY}, diff --git a/build/dashboard/mining_dashboard/service/update_checker.py b/build/dashboard/mining_dashboard/service/update_checker.py index 92d43fde..f47b6f67 100644 --- a/build/dashboard/mining_dashboard/service/update_checker.py +++ b/build/dashboard/mining_dashboard/service/update_checker.py @@ -1,12 +1,13 @@ """New-release check (#224) — notify-only. -When `dashboard.check_for_updates` is enabled (default OFF), the dashboard periodically asks GitHub +Unless `dashboard.check_for_updates` is disabled (default ON), the dashboard periodically asks GitHub for the latest published release and, if it's newer than the running version, surfaces a header badge linking to it (`build_state` -> `state.update`). It never updates anything — it's a callout so the operator knows to upgrade on their own terms (the one-click upgrade is the separate #59). -Privacy: the check is **opt-in (default off)** and routed over the bridge **Tor SOCKS** (reusing -`TOR_SOCKS_PROXY`, like the XvB stats fetch #163), so enabling it doesn't reveal the host IP to GitHub. +Privacy: the check is **on by default** because it's routed over the bridge **Tor SOCKS** (reusing +`TOR_SOCKS_PROXY`, like the XvB stats fetch #163), so it doesn't reveal the host IP to GitHub; +set `dashboard.check_for_updates` to false to opt out. Every failure path is silent (returns ``None``) so an offline / Tor-only stack just shows no badge. """ @@ -14,6 +15,8 @@ import requests +from mining_dashboard.helper.http import bounded_get + logger = logging.getLogger("UpdateChecker") @@ -53,7 +56,7 @@ def latest_release(self): failure (network, non-200, malformed JSON). Routed through Tor when a proxy is set.""" proxies = {"http": self.tor_proxy, "https": self.tor_proxy} if self.tor_proxy else None try: - resp = requests.get( + resp = bounded_get( self.api_url, timeout=20, proxies=proxies, @@ -83,19 +86,32 @@ def __init__(self, client, running_version, enabled, interval=3600): self.enabled = enabled self.interval = interval self._last = 0.0 + self.release = None self.result = None - def maybe_check(self, now): - """Return the cached ``{available, ...}`` (or ``None``). Performs the (blocking) fetch only - when enabled and the throttle window has elapsed — call via ``asyncio.to_thread``.""" + def latest_release_cached(self, now): + """Return the cached raw ``{tag, url}`` of the latest release (or ``None``). Performs the + (blocking) fetch only when enabled and the throttle window has elapsed — call via + ``asyncio.to_thread``. This is the many-consumers accessor (#596): one throttled fleet-wide + fetch, compared against as many running versions as the caller has.""" if not self.enabled: - self.result = None return None if self._last and (now - self._last) < self.interval: - return self.result + return self.release self._last = now rel = self.client.latest_release() + if rel: + self.release = rel + # On a failed fetch keep the previous release — a blip shouldn't drop a real "update available". + return self.release + + def maybe_check(self, now): + """Return the cached ``{available, ...}`` (or ``None``) for this checker's own running + version. Same throttle/cache contract as ``latest_release_cached``.""" + if not self.enabled: + self.result = None + return None + rel = self.latest_release_cached(now) if rel: self.result = compute_update(self.running, rel["tag"], rel["url"]) - # On a failed fetch keep the previous result — a blip shouldn't drop a real "update available". return self.result diff --git a/build/dashboard/mining_dashboard/web/server.py b/build/dashboard/mining_dashboard/web/server.py index 517b919b..a5b3158c 100644 --- a/build/dashboard/mining_dashboard/web/server.py +++ b/build/dashboard/mining_dashboard/web/server.py @@ -8,6 +8,7 @@ from mining_dashboard.config import config from mining_dashboard.service import audit_service, control_service from mining_dashboard.service.metrics import build_metrics, share_reject_pct +from mining_dashboard.service.update_checker import parse_semver from mining_dashboard.web.prometheus import CONTENT_TYPE as PROMETHEUS_CONTENT_TYPE from mining_dashboard.web.prometheus import render_prometheus from mining_dashboard.web.views import ( @@ -252,6 +253,45 @@ async def handle_worker_apply(request): return web.json_response({"id": rid, **res}) +async def handle_worker_upgrade(request): + """One-click RigForge upgrade for a single rig (#597), via the HOST-side control runner. + + Mirrors the stack's own upgrade (#59) at the per-worker level: the body's version is only what + the operator confirmed seeing (the badge's latest, #596); the host re-derives the real target + from the RigForge release API over Tor and refuses a mismatch, then resolves the rig's address + + bearer from config.json — this container never holds the token and cannot choose what gets + installed. Returns 202 + the request id immediately (a rig build can take minutes); the client + polls /api/control/result. A rig already reporting the requested version short-circuits to a + no-op without spooling — a dial would just burn the rig's own 6h upgrade throttle.""" + _require_control_header(request) + try: + body = await request.json() + except Exception: + raise web.HTTPBadRequest(text="Body must be JSON.") from None + worker = body.get("worker") + version = body.get("version") + if not isinstance(worker, str) or not worker: + raise web.HTTPBadRequest(text="'worker' must be a non-empty string.") + if not isinstance(version, str) or not re.fullmatch(r"v\d+\.\d+\.\d+", version): + raise web.HTTPBadRequest(text="'version' must look like vX.Y.Z.") + data = request.app["latest_data"] or {} + live = next((w for w in data.get("workers", []) if w.get("name") == worker), None) + running = ((live or {}).get("rigforge") or {}).get("version") + # The rig reports bare "1.11.2", the badge proposes tag "v1.11.2" — compare parsed (#596). + if running and parse_semver(running) and parse_semver(running) == parse_semver(version): + return web.json_response( + {"status": "noop", "worker": worker, "note": f"already on {version}"} + ) + try: + rid = control_service.submit_worker_upgrade( + worker, version, request.headers.get("X-Auth-User", "") + ) + except Exception: + logger.exception("Error submitting worker-upgrade") + return web.json_response({"error": "Failed to submit the worker upgrade."}, status=500) + return web.json_response({"id": rid, "status": "pending"}, status=202) + + async def handle_control_result(request): """Client-side polling endpoint for a 202'd preview/commit.""" try: @@ -351,6 +391,9 @@ def create_app(state_manager, latest_data_ref): # writable-key change to its rig. Gated with the rest of the control channel. web.get("/api/worker", handle_worker_detail), web.post("/api/control/worker-apply", handle_worker_apply), + # One-click rig upgrade (#597): spools name + confirmed version only; the host + # re-derives the real target and dials the rig. Same gate as the rest. + web.post("/api/control/worker-upgrade", handle_worker_upgrade), ] ) diff --git a/build/dashboard/mining_dashboard/web/static/components.mjs b/build/dashboard/mining_dashboard/web/static/components.mjs index b7d1d5da..95b27695 100644 --- a/build/dashboard/mining_dashboard/web/static/components.mjs +++ b/build/dashboard/mining_dashboard/web/static/components.mjs @@ -743,6 +743,16 @@ function RigForgeChips({ rf }) { )}`; } +// Per-worker RigForge new-release callout (#596) — the worker-level twin of UpdateBadge. Shown +// only when the server derived that this rig's reported RigForge version is older than the latest +// release (same dashboard.check_for_updates gate). Notify-only — a link to the release notes; the +// one-click rig upgrade is the separate #597. +const RigUpdateBadge = ({ up }) => + up && up.available && up.url + ? html` rf ${up.latest} available ↗` + : null; + // Pool-wide proxy share totals (Issue #82) — a footer under the table. Hidden until the proxy // has reported any shares so it isn't an all-zero line on a fresh start. const ProxyTotals = ({ summary }) => { @@ -812,7 +822,7 @@ function WorkersTable({ workers, summary, ui, onSort, hostIp, stratumPort, onIns w.api_ok === false ? html` api ⚠` : null - }<${RigForgeChips} rf=${w.rigforge} /> + }<${RigForgeChips} rf=${w.rigforge} /><${RigUpdateBadge} up=${w.rigforge_update} /> ${w.ip} ${uptimeCell(w)} ${w.h60_str} diff --git a/build/dashboard/mining_dashboard/web/static/workerview.mjs b/build/dashboard/mining_dashboard/web/static/workerview.mjs index a41f6aab..f62429e0 100644 --- a/build/dashboard/mining_dashboard/web/static/workerview.mjs +++ b/build/dashboard/mining_dashboard/web/static/workerview.mjs @@ -26,11 +26,15 @@ import { const CONTROL_HEADERS = { "Content-Type": "application/json", "X-Pithead-Control": "1" }; const POLL_MS = 2000; const POLL_MAX = 40; // ~80s — the host dials the rig then polls its /status +// ~5 min — covers spool latency + the host runner's own 90s rig-poll cap (#597). A rebuild that +// outlives the cap lands as "accepted"; the badge clears on its own once the rig reports the +// new version, so polling longer here buys nothing. +const UPGRADE_POLL_MAX = 150; // Poll the shared control-result endpoint until a terminal outcome lands, skipping the interim // "running". The apply can briefly out-run the dashboard, so tolerate a transient fetch failure. -async function pollWorkerResult(id) { - for (let i = 0; i < POLL_MAX; i++) { +async function pollWorkerResult(id, max = POLL_MAX) { + for (let i = 0; i < max; i++) { await new Promise((r) => setTimeout(r, POLL_MS)); let res; try { @@ -56,6 +60,10 @@ const STATUS_META = { rolled_back: { cls: "status-bad", label: "Rolled back" }, failed: { cls: "status-bad", label: "Failed" }, error: { cls: "status-bad", label: "Error" }, + // Worker-upgrade extras (#597): a rig already on the target is a calm no-op, and the rig's own + // 6h anti-beacon throttle is retry-later, not a fault (the host runner maps it server-side). + noop: { cls: "status-ok", label: "Already up to date" }, + throttled: { cls: "status-warn", label: "Throttled by the rig — retry later" }, }; function StatusLine({ result }) { @@ -142,6 +150,11 @@ export class WorkerInspect extends Component { jsonError: null, busy: false, result: null, + // One-click rig upgrade (#597): a two-step arm → confirm, its own in-flight flag (a build + // can run minutes) and its own result line, independent of the config editor's. + upgArmed: false, + upgBusy: false, + upgResult: null, }; this.dialogRef = createRef(); } @@ -225,6 +238,27 @@ export class WorkerInspect extends Component { } } + // One-click rig upgrade (#597). POSTs {worker, version} only — the version is the badge's + // latest, a proposal the HOST re-derives and the rig bounds; this client never picks a target. + // 202 means spooled: poll with the long budget (the rig may rebuild its miner, ~10 min). + async upgrade() { + const version = this.state.detail.rigforge_update.latest; + this.setState({ upgArmed: false, upgBusy: true, upgResult: { status: "running" } }); + try { + const res = await fetch("/api/control/worker-upgrade", { + method: "POST", + headers: CONTROL_HEADERS, + body: JSON.stringify({ worker: this.props.name, version }), + }); + let out = await res.json(); + if (res.status === 202 && out.id) out = await pollWorkerResult(out.id, UPGRADE_POLL_MAX); + this.setState({ upgBusy: false, upgResult: out }); + this.load(); // an applied upgrade clears the badge once the rig reports the new version + } catch (e) { + this.setState({ upgBusy: false, upgResult: { status: "error", error: String(e) } }); + } + } + render() { const { phase, detail, error } = this.state; const { name, onClose } = this.props; @@ -243,7 +277,8 @@ export class WorkerInspect extends Component { } renderBody(detail) { - const { mode, tableEdits, editText, jsonError, busy, result } = this.state; + const { mode, tableEdits, editText, jsonError, busy, result, upgArmed, upgBusy, upgResult } = + this.state; const canEdit = detail.control_enabled && detail.editable; return html`
@@ -252,6 +287,36 @@ export class WorkerInspect extends Component { <${InfoCard} label="Hashrate (1m)" value=${detail.hashrate || "—"} /> <${InfoCard} label="RigForge" value=${detail.rigforge ? detail.rigforge.version || "yes" : "—"} />
+ ${ + // This rig runs an older RigForge (#596) — the badge links to the release notes; + // with the control channel on and an operator-set host, the one-click upgrade + // button (#597) appears beside it: arm → confirm → POST → poll (a rig rebuild can + // take ~10 min; the rig rolls back on a build that doesn't come back live). + detail.rigforge_update && + detail.rigforge_update.available && + detail.rigforge_update.url + ? html`

New RigForge release ${detail.rigforge_update.latest} available ↗${ + canEdit && !upgBusy + ? upgArmed + ? html` + ` + : html` ` + : null +}${ + upgBusy + ? html` upgrading — a rebuild can take minutes…` + : null +}

` + : null + } + <${StatusLine} result=${upgResult} /> ${detail.rigforge ? html`<${StatsTable} stats=${detail.rigforge.stats} />` : null}

Edit config

diff --git a/build/dashboard/mining_dashboard/web/views.py b/build/dashboard/mining_dashboard/web/views.py index 63fc72c8..e0c2cf24 100644 --- a/build/dashboard/mining_dashboard/web/views.py +++ b/build/dashboard/mining_dashboard/web/views.py @@ -48,7 +48,7 @@ ) from mining_dashboard.service.egress import egress_posture_from_config, topology_from_config from mining_dashboard.service.metrics import build_metrics, share_reject_pct -from mining_dashboard.service.update_checker import parse_semver +from mining_dashboard.service.update_checker import compute_update, parse_semver from mining_dashboard.version import resolve_version logger = logging.getLogger("WebViews") @@ -834,7 +834,23 @@ def build_system(data): } -def build_workers(workers): +def rigforge_update_for(worker, release): + """The per-worker RigForge new-release callout (#596): ``{available, latest, url}`` or ``None``. + + Derived at the render seam from the rig's live reported version and the fleet-wide cached + latest release — never stored, so it can't outlive its inputs (the #664 lesson: a rig running + X must never badge "X available"). ``compute_update`` normalizes the rig's bare ``1.11.2`` + against the release tag's ``v1.11.2``. No reported version (plain xmrig, sister API off) or no + cached release → ``None``, an honest "unknown", not a false "up to date".""" + if not worker or not release: + return None + version = (worker.get("rigforge") or {}).get("version") + if not version: + return None + return compute_update(version, release.get("tag"), release.get("url")) + + +def build_workers(workers, rigforge_release=None): """Worker rows as data: raw numeric fields (for client-side sorting) alongside their formatted display strings, plus a pool token for the badge. Online first, then by name.""" rows = [] @@ -888,6 +904,8 @@ def build_workers(workers): # RigForge enriched feed (#235): version badge + health/power/tune/watchdog # chips, or None for a plain-xmrig worker (renders nothing extra). "rigforge": _rigforge_display(worker.get("rigforge")), + # {available, latest, url} | None — this rig runs an older RigForge (#596). + "rigforge_update": rigforge_update_for(worker, rigforge_release), } ) except Exception as e: @@ -1541,6 +1559,8 @@ def build_worker_detail(name, data, state_mgr): "status": worker.get("status") if worker else None, "hashrate": format_hashrate(worker.get("h60", 0)) if worker else None, "rigforge": _rigforge_display(worker.get("rigforge")) if worker else None, + # {available, latest, url} | None — this rig runs an older RigForge (#596). + "rigforge_update": rigforge_update_for(worker, (data or {}).get("rigforge_release")), "writable_keys": sorted(WORKER_WRITABLE_KEYS), "last_applied": state_mgr.get_last_applied_worker_config(name), "history": history, @@ -1644,7 +1664,7 @@ def build_state(data, state_mgr, range_arg, window=None, avg_window=DEFAULT_HASH ), "xvb_calc": build_xvb_calc(metrics, state_mgr), "tari": build_tari(data), - "workers": build_workers(data.get("workers", [])), + "workers": build_workers(data.get("workers", []), data.get("rigforge_release")), # Fleet power draw / efficiency and (once a price is set) net profit after power (#260), # with live feed prices (#520) when dashboard.energy.price_feed is on. "energy": build_energy(data.get("workers", []), data.get("prices")), diff --git a/build/dashboard/pyproject.toml b/build/dashboard/pyproject.toml index 0135d541..b6791488 100644 --- a/build/dashboard/pyproject.toml +++ b/build/dashboard/pyproject.toml @@ -7,7 +7,7 @@ name = "mining-dashboard" # Keep in lockstep with the top-level VERSION file — the single source of truth for the stack version # (#44). A shell test (tests/stack/run.sh) fails if these drift; the dashboard *displays* the version # from VERSION (baked in as PITHEAD_VERSION, #58), so this is packaging metadata only. -version = "1.9.3" +version = "1.10.0" description = "Monitoring dashboard and XvB switching engine for Pithead" readme = "README.md" requires-python = ">=3.11" diff --git a/build/dashboard/tests/client/test_xvb_client.py b/build/dashboard/tests/client/test_xvb_client.py index e0d026f2..99360583 100644 --- a/build/dashboard/tests/client/test_xvb_client.py +++ b/build/dashboard/tests/client/test_xvb_client.py @@ -45,7 +45,7 @@ def test_missing_wallet_returns_none(): def test_get_stats_success_parses_html(): client = XvbClient("49abc") resp = MagicMock(status_code=200, text=SAMPLE_HTML) - with patch.object(xvb_mod.requests, "get", return_value=resp) as mock_get: + with patch.object(xvb_mod, "bounded_get", return_value=resp) as mock_get: stats = client.get_stats() assert stats == {"fail_count": 2, "avg_1h": 1500.0, "avg_24h": 3000.0} assert mock_get.call_args.kwargs["params"] == {"address": "49abc"} @@ -56,7 +56,7 @@ def test_get_stats_routes_through_tor_proxy(): # operator's home IP — the request carries the wallet, so a clearnet fetch would correlate them. client = XvbClient("49abc") resp = MagicMock(status_code=200, text=SAMPLE_HTML) - with patch.object(xvb_mod.requests, "get", return_value=resp) as mock_get: + with patch.object(xvb_mod, "bounded_get", return_value=resp) as mock_get: client.get_stats() proxies = mock_get.call_args.kwargs["proxies"] assert proxies["https"].startswith("socks5h://") # socks5h resolves the host via Tor too @@ -66,26 +66,26 @@ def test_get_stats_routes_through_tor_proxy(): def test_get_stats_honours_explicit_proxy(): client = XvbClient("49abc", tor_proxy="socks5h://127.0.0.1:9999") resp = MagicMock(status_code=200, text=SAMPLE_HTML) - with patch.object(xvb_mod.requests, "get", return_value=resp) as mock_get: + with patch.object(xvb_mod, "bounded_get", return_value=resp) as mock_get: client.get_stats() assert mock_get.call_args.kwargs["proxies"]["https"] == "socks5h://127.0.0.1:9999" def test_get_stats_non_200_returns_none(): client = XvbClient("49abc") - with patch.object(xvb_mod.requests, "get", return_value=MagicMock(status_code=503)): + with patch.object(xvb_mod, "bounded_get", return_value=MagicMock(status_code=503)): assert client.get_stats() is None def test_get_stats_network_error_returns_none(): client = XvbClient("49abc") - with patch.object(xvb_mod.requests, "get", side_effect=requests.RequestException("boom")): + with patch.object(xvb_mod, "bounded_get", side_effect=requests.RequestException("boom")): assert client.get_stats() is None def test_get_stats_unexpected_error_returns_none(): client = XvbClient("49abc") - with patch.object(xvb_mod.requests, "get", side_effect=ValueError("kaboom")): + with patch.object(xvb_mod, "bounded_get", side_effect=ValueError("kaboom")): assert client.get_stats() is None @@ -111,7 +111,7 @@ def test_parse_reward_estimates_malformed_or_empty_is_empty_dict(): def test_get_reward_estimates_success_routes_over_tor(): client = XvbClient("49abc") resp = MagicMock(status_code=200, text=SAMPLE_REWARD_TXT) - with patch.object(xvb_mod.requests, "get", return_value=resp) as mock_get: + with patch.object(xvb_mod, "bounded_get", return_value=resp) as mock_get: est = client.get_reward_estimates() assert est["donor_whale"] == 6.17 proxies = mock_get.call_args.kwargs["proxies"] @@ -120,7 +120,7 @@ def test_get_reward_estimates_success_routes_over_tor(): def test_get_reward_estimates_non_200_returns_none(): client = XvbClient("49abc") - with patch.object(xvb_mod.requests, "get", return_value=MagicMock(status_code=503)): + with patch.object(xvb_mod, "bounded_get", return_value=MagicMock(status_code=503)): assert client.get_reward_estimates() is None @@ -136,13 +136,13 @@ def test_get_reward_estimates_unparseable_body_returns_none(): def test_get_reward_estimates_network_error_returns_none(): client = XvbClient("49abc") - with patch.object(xvb_mod.requests, "get", side_effect=requests.RequestException("boom")): + with patch.object(xvb_mod, "bounded_get", side_effect=requests.RequestException("boom")): assert client.get_reward_estimates() is None def test_get_reward_estimates_unexpected_error_returns_none(): client = XvbClient("49abc") - with patch.object(xvb_mod.requests, "get", side_effect=ValueError("kaboom")): + with patch.object(xvb_mod, "bounded_get", side_effect=ValueError("kaboom")): assert client.get_reward_estimates() is None @@ -208,7 +208,7 @@ def test_parse_winners_bounds_hostile_input(): def test_get_recent_wins_success_routes_over_tor(): client = XvbClient(_WIN_WALLET) resp = MagicMock(status_code=200, text=SAMPLE_WINNERS_TXT) - with patch.object(xvb_mod.requests, "get", return_value=resp) as mock_get: + with patch.object(xvb_mod, "bounded_get", return_value=resp) as mock_get: wins = client.get_recent_wins() assert len(wins) == 2 assert mock_get.call_args.args[0].endswith("winners_recent_full_pub.txt") @@ -219,17 +219,17 @@ def test_get_recent_wins_no_wins_is_empty_list_not_none(): # An empty list is a successful "no wins yet" read — distinct from a failed fetch (None). client = XvbClient("49abcSOMEOTHERWALLETxyz99") resp = MagicMock(status_code=200, text=SAMPLE_WINNERS_TXT) - with patch.object(xvb_mod.requests, "get", return_value=resp): + with patch.object(xvb_mod, "bounded_get", return_value=resp): assert client.get_recent_wins() == [] def test_get_recent_wins_failures_return_none(): client = XvbClient(_WIN_WALLET) - with patch.object(xvb_mod.requests, "get", return_value=MagicMock(status_code=503)): + with patch.object(xvb_mod, "bounded_get", return_value=MagicMock(status_code=503)): assert client.get_recent_wins() is None - with patch.object(xvb_mod.requests, "get", side_effect=requests.RequestException("boom")): + with patch.object(xvb_mod, "bounded_get", side_effect=requests.RequestException("boom")): assert client.get_recent_wins() is None - with patch.object(xvb_mod.requests, "get", side_effect=ValueError("kaboom")): + with patch.object(xvb_mod, "bounded_get", side_effect=ValueError("kaboom")): assert client.get_recent_wins() is None @@ -253,20 +253,20 @@ class TestRegister: def test_disabled_when_no_endpoint(self): # Disable sentinel => empty submit_url => never reach out at all. client = XvbClient("49abc", submit_url="") - with patch.object(xvb_mod.requests, "get") as mock_get: + with patch.object(xvb_mod, "bounded_get") as mock_get: assert client.register() == REG_DISABLED mock_get.assert_not_called() def test_missing_wallet_is_invalid(self): client = XvbClient("", submit_url=_SUBMIT) - with patch.object(xvb_mod.requests, "get") as mock_get: + with patch.object(xvb_mod, "bounded_get") as mock_get: assert client.register() == REG_INVALID mock_get.assert_not_called() assert XvbClient("placeholder", submit_url=_SUBMIT).register() == REG_INVALID def test_2xx_is_registered_and_sends_full_wallet(self): client = XvbClient("49fullwalletaddress", submit_url=_SUBMIT) - with patch.object(xvb_mod.requests, "get", return_value=_resp(200, "OK")) as mock_get: + with patch.object(xvb_mod, "bounded_get", return_value=_resp(200, "OK")) as mock_get: assert client.register() == REG_OK # Registration takes the FULL wallet address as ?address=... assert mock_get.call_args.kwargs["params"] == {"address": "49fullwalletaddress"} @@ -276,37 +276,37 @@ def test_already_registered_422_is_idempotent_success(self): # The real steady state for an entered wallet (verified live): 422 + this body. client = XvbClient("49abc", submit_url=_SUBMIT) resp = _resp(422, "ERROR: Wallet Address Already Registered") - with patch.object(xvb_mod.requests, "get", return_value=resp): + with patch.object(xvb_mod, "bounded_get", return_value=resp): assert client.register() == REG_OK def test_invalid_wallet_422(self): client = XvbClient("49abc", submit_url=_SUBMIT) resp = _resp(422, "ERROR: Invalid Wallet Address") - with patch.object(xvb_mod.requests, "get", return_value=resp): + with patch.object(xvb_mod, "bounded_get", return_value=resp): assert client.register() == REG_INVALID def test_no_share_body_is_not_eligible(self): # Best-effort: a body mentioning the PPLNS share => retry quietly, not a failure. client = XvbClient("49abc", submit_url=_SUBMIT) resp = _resp(422, "ERROR: No share in PPLNS window") - with patch.object(xvb_mod.requests, "get", return_value=resp): + with patch.object(xvb_mod, "bounded_get", return_value=resp): assert client.register() == REG_NOT_ELIGIBLE def test_5xx_is_transient_error(self): client = XvbClient("49abc", submit_url=_SUBMIT) - with patch.object(xvb_mod.requests, "get", return_value=_resp(500, "Server error!")): + with patch.object(xvb_mod, "bounded_get", return_value=_resp(500, "Server error!")): assert client.register() == REG_ERROR def test_unrecognised_body_is_error(self): client = XvbClient("49abc", submit_url=_SUBMIT) - with patch.object(xvb_mod.requests, "get", return_value=_resp(418, "teapot")): + with patch.object(xvb_mod, "bounded_get", return_value=_resp(418, "teapot")): assert client.register() == REG_ERROR def test_routes_through_configured_tor_proxy_by_default(self): # #163: the call carries the full wallet, so it ALWAYS rides the bridge Tor SOCKS like # get_stats — a default-constructed client (as main.py builds it) uses TOR_SOCKS_PROXY. client = XvbClient("49abc", submit_url=_SUBMIT) # no tor_proxy => the configured default - with patch.object(xvb_mod.requests, "get", return_value=_resp(200, "OK")) as mock_get: + with patch.object(xvb_mod, "bounded_get", return_value=_resp(200, "OK")) as mock_get: client.register() proxies = mock_get.call_args.kwargs["proxies"] assert proxies["https"] == xvb_mod.TOR_SOCKS_PROXY @@ -315,12 +315,12 @@ def test_routes_through_configured_tor_proxy_by_default(self): def test_network_error_is_transient_error(self): client = XvbClient("49abc", submit_url=_SUBMIT) - with patch.object(xvb_mod.requests, "get", side_effect=requests.RequestException("boom")): + with patch.object(xvb_mod, "bounded_get", side_effect=requests.RequestException("boom")): assert client.register() == REG_ERROR def test_unexpected_error_is_transient_error(self): client = XvbClient("49abc", submit_url=_SUBMIT) - with patch.object(xvb_mod.requests, "get", side_effect=ValueError("kaboom")): + with patch.object(xvb_mod, "bounded_get", side_effect=ValueError("kaboom")): assert client.register() == REG_ERROR diff --git a/build/dashboard/tests/frontend/components.test.mjs b/build/dashboard/tests/frontend/components.test.mjs index 8b1675f1..fcdd7aa0 100644 --- a/build/dashboard/tests/frontend/components.test.mjs +++ b/build/dashboard/tests/frontend/components.test.mjs @@ -603,6 +603,20 @@ test('WorkersTable renders the RigForge version badge + chips when present, noth assert.match(html, /142 W · 86.9 H\/s·W/); }); +test('WorkersTable badges a rig running an older RigForge — and only that rig (#596)', () => { + const s = clone(); + s.workers[0].rigforge_update = { available: true, latest: 'v1.11.2', url: 'https://h/v1.11.2' }; + s.workers[1].rigforge_update = null; // current / plain-xmrig rig -> no badge + const html = renderApp({ state: s }); + assert.match(html, /rf v1\.11\.2 available/); // the accent callout renders + assert.match(html, /A newer RigForge release is available: v1\.11\.2/); // tooltip + assert.equal(html.match(/rf v1\.11\.2 available/g).length, 1); // exactly one rig badged + + const none = clone(); + none.workers.forEach((w) => (w.rigforge_update = null)); + assert.doesNotMatch(renderApp({ state: none }), /RigForge release is available/); +}); + test('Tari status gates the ✔ on a live gRPC channel, never on active-but-dead (#278/#313)', () => { // The ✔ must mean the merge-mine channel is actually up. A dead channel that still reads "active" // must show status-warn and NO check — otherwise a TRANSIENT_FAILURE reads as healthy (#278/#313). diff --git a/build/dashboard/tests/frontend/fixtures/state.json b/build/dashboard/tests/frontend/fixtures/state.json index 307bde57..d8c67535 100644 --- a/build/dashboard/tests/frontend/fixtures/state.json +++ b/build/dashboard/tests/frontend/fixtures/state.json @@ -355,6 +355,7 @@ "level": "ok", "percent": "0%", "total": "0.0", + "unit": "GB", "used": "0.0", "width": "0%" }, @@ -637,6 +638,7 @@ ], "version": null }, + "rigforge_update": null, "status": "online", "uptime": 0, "uptime_str": "0m 0s" @@ -676,6 +678,7 @@ ], "version": null }, + "rigforge_update": null, "status": "offline", "uptime": 0, "uptime_str": "0m 0s" diff --git a/build/dashboard/tests/frontend/workerview.test.mjs b/build/dashboard/tests/frontend/workerview.test.mjs index 7acde8c2..ba7fb4b9 100644 --- a/build/dashboard/tests/frontend/workerview.test.mjs +++ b/build/dashboard/tests/frontend/workerview.test.mjs @@ -233,3 +233,81 @@ test("no applied config versions yet falls back to an explanatory message", () = const out = renderToString(readyInstance({ ...DETAIL, hashrate_by_config: [] }).render()); assert.match(out, /No applied config changes to correlate hashrate against yet/); }); + +// --- One-click rig upgrade (#597) ------------------------------------------------------------- + +const UPG_DETAIL = { + ...DETAIL, + rigforge: { version: "1.11.1", stats: [] }, + rigforge_update: { available: true, latest: "v1.11.2", url: "https://h/v1.11.2" }, +}; + +test("upgrade button gates on rigforge_update + an editable, control-enabled worker (#597)", () => { + assert.match(renderToString(readyInstance(UPG_DETAIL).render()), /Upgrade rig…/); + // Notify-only without an operator-set host or with control off — badge yes, button no. + const noEdit = renderToString(readyInstance({ ...UPG_DETAIL, editable: false }).render()); + assert.match(noEdit, /New RigForge release/); + assert.doesNotMatch(noEdit, /Upgrade rig…/); + const noCtl = renderToString(readyInstance({ ...UPG_DETAIL, control_enabled: false }).render()); + assert.doesNotMatch(noCtl, /Upgrade rig…/); + // No update derived -> no badge, no button. + const current = renderToString(readyInstance({ ...UPG_DETAIL, rigforge_update: null }).render()); + assert.doesNotMatch(current, /Upgrade rig…|New RigForge release/); +}); + +test("arming swaps the button for confirm/cancel; cancel disarms (#597)", () => { + const inst = readyInstance(UPG_DETAIL); + inst.state.upgArmed = true; + const armed = renderToString(inst.render()); + assert.match(armed, /Confirm upgrade/); + assert.match(armed, /Cancel/); + assert.doesNotMatch(armed, /Upgrade rig…/); +}); + +test("upgrade() POSTs {worker, version} and renders the terminal result (#597)", async () => { + const inst = readyInstance(UPG_DETAIL); + let posted = null; + const realFetch = globalThis.fetch; + globalThis.fetch = async (url, opts) => { + if (posted === null && url === "/api/control/worker-upgrade") { + posted = { url, body: JSON.parse(opts.body) }; + return { status: 200, json: async () => ({ status: "noop", note: "already on v1.11.2" }) }; + } + return { ok: true, status: 200, json: async () => UPG_DETAIL }; // the load() refresh + }; + try { + await inst.upgrade(); + await new Promise((r) => setImmediate(r)); // flush the fire-and-forget load() refresh + } finally { + globalThis.fetch = realFetch; + } + assert.equal(posted.url, "/api/control/worker-upgrade"); + assert.deepEqual(posted.body, { worker: "rig1", version: "v1.11.2" }); + assert.equal(inst.state.upgBusy, false); + assert.match(renderToString(inst.render()), /Already up to date/); +}); + +test("terminal statuses render their calm/red variants (#597)", () => { + const inst = readyInstance(UPG_DETAIL); + inst.state.upgResult = { status: "throttled", reason: "throttled: retry after the window" }; + assert.match(renderToString(inst.render()), /Throttled by the rig — retry later/); + inst.state.upgResult = { status: "rolled_back", reason: "miner did not return live" }; + assert.match(renderToString(inst.render()), /Rolled back/); +}); + +// --- RigForge new-release callout (#596) ---------------------------------------------------- + +test("Inspect surfaces the RigForge new-release callout only when the server derived one (#596)", () => { + const behind = { + ...DETAIL, + rigforge: { version: "1.11.1", stats: [] }, + rigforge_update: { available: true, latest: "v1.11.2", url: "https://h/v1.11.2" }, + }; + const out = renderToString(readyInstance(behind).render()); + assert.match(out, /New RigForge release v1\.11\.2 available/); + assert.match(out, /href="https:\/\/h\/v1\.11\.2"/); + + // Current rig / plain xmrig: the server sends null -> no callout, no error. + const current = { ...DETAIL, rigforge_update: null }; + assert.doesNotMatch(renderToString(readyInstance(current).render()), /New RigForge release/); +}); diff --git a/build/dashboard/tests/helper/test_http.py b/build/dashboard/tests/helper/test_http.py new file mode 100644 index 00000000..b99a5554 --- /dev/null +++ b/build/dashboard/tests/helper/test_http.py @@ -0,0 +1,124 @@ +"""Tier 1 — bounded_get (#660): the shared response-size cap for external HTTP fetches.""" + +import re +from pathlib import Path +from unittest.mock import MagicMock, patch + +import pytest +import requests + +from mining_dashboard.helper.http import BoundedResponse, ResponseTooLarge, bounded_get + + +def _streaming_resp(chunks, status_code=200, encoding="utf-8"): + resp = MagicMock(status_code=status_code, encoding=encoding) + resp.iter_content.return_value = iter(chunks) + resp.__enter__.return_value = resp + resp.__exit__.return_value = False + return resp + + +class TestBoundedGet: + def test_under_cap_returns_body(self): + with patch( + "mining_dashboard.helper.http.requests.get", + return_value=_streaming_resp([b'{"tag_nam', b'e": "v1.2.3"}']), + ) as g: + resp = bounded_get("https://example.test/x", timeout=20) + assert resp.status_code == 200 + assert resp.text == '{"tag_name": "v1.2.3"}' + assert resp.json() == {"tag_name": "v1.2.3"} + # The read is streamed, never buffered whole by requests itself. + assert g.call_args.kwargs["stream"] is True + + def test_exactly_at_cap_returns_body(self): + # The cap is strictly greater-than: a body of exactly max_bytes succeeds. + with patch( + "mining_dashboard.helper.http.requests.get", + return_value=_streaming_resp([b"x" * 1024, b"x" * 1024]), + ): + resp = bounded_get("https://example.test/x", max_bytes=2048) + assert resp.content == b"x" * 2048 + + def test_over_cap_raises_request_exception(self): + # ResponseTooLarge subclasses RequestException, so every caller's existing + # fail-silent handling (return None / keep last good) applies unchanged. + with patch( + "mining_dashboard.helper.http.requests.get", + return_value=_streaming_resp([b"x" * 1024] * 3), + ): + with pytest.raises(requests.RequestException): + bounded_get("https://example.test/x", max_bytes=2048) + + def test_stops_reading_at_the_cap(self): + # The over-cap chunk is the last one consumed — the rest of a multi-GB body is never read. + chunks = iter([b"x" * 1024, b"x" * 1024, b"x" * 1024]) + with patch( + "mining_dashboard.helper.http.requests.get", + return_value=_streaming_resp(chunks), + ): + with pytest.raises(ResponseTooLarge): + bounded_get("https://example.test/x", max_bytes=1024) + assert next(chunks, None) is not None # at least one chunk was left unread + + def test_kwargs_pass_through(self): + with patch( + "mining_dashboard.helper.http.requests.get", + return_value=_streaming_resp([b"ok"]), + ) as g: + bounded_get( + "https://example.test/x", + timeout=20, + proxies={"https": "socks5h://tor:9050"}, + params={"a": "b"}, + headers={"User-Agent": "pithead-dashboard"}, + ) + kw = g.call_args.kwargs + assert kw["timeout"] == 20 + assert kw["proxies"] == {"https": "socks5h://tor:9050"} + assert kw["params"] == {"a": "b"} + assert kw["headers"] == {"User-Agent": "pithead-dashboard"} + + def test_bad_encoding_degrades_not_crashes(self): + resp_obj = _streaming_resp([b"\xff\xfe"], encoding=None) + with patch("mining_dashboard.helper.http.requests.get", return_value=resp_obj): + resp = bounded_get("https://example.test/x") + assert isinstance(resp.text, str) # errors="replace", never a decode crash + + def test_raise_for_status_matches_requests_contract(self): + # The Telegram long-poll calls raise_for_status(); an HTTP error must surface as a + # RequestException so the caller's existing catch-and-backoff applies unchanged. + BoundedResponse(200, b"", "utf-8").raise_for_status() # 2xx: no raise + with pytest.raises(requests.RequestException): + BoundedResponse(502, b"", "utf-8").raise_for_status() + + +class TestWiringDriftGuard: + def test_external_clients_route_through_bounded_get(self): + """Every external fetch module goes via bounded_get — any direct requests.( is drift.""" + pkg = Path(__file__).resolve().parents[2] / "mining_dashboard" + external = [ + pkg / "service" / "update_checker.py", + pkg / "service" / "price_feed.py", + pkg / "client" / "xvb_client.py", + pkg / "service" / "tor_heal.py", + pkg / "service" / "healthchecks.py", + ] + direct_call = re.compile(r"requests\.(get|post|put|delete|head|request)\(") + for mod in external: + src = mod.read_text() + hit = direct_call.search(src) + assert hit is None, ( + f"{mod.name}: unbounded {hit.group() if hit else ''} slipped back in" + ) + assert "bounded_get(" in src, f"{mod.name}: no bounded_get call found" + + def test_telegram_get_updates_routes_through_bounded_get(self): + """The getUpdates long-poll is bounded. The module's requests.post( sends (sendMessage, + answerCallbackQuery) keep their own contract — tiny echo bodies of caller-authored + payloads — so only GETs are drift here.""" + pkg = Path(__file__).resolve().parents[2] / "mining_dashboard" + src = (pkg / "service" / "telegram_commands.py").read_text() + hit = re.search(r"requests\.(get|head|request)\(", src) + assert hit is None, f"telegram_commands.py: unbounded {hit.group() if hit else ''}" + assert "bounded_get(" in src diff --git a/build/dashboard/tests/service/test_control_service.py b/build/dashboard/tests/service/test_control_service.py index e9076284..8fb999fb 100644 --- a/build/dashboard/tests/service/test_control_service.py +++ b/build/dashboard/tests/service/test_control_service.py @@ -338,6 +338,21 @@ def test_submit_worker_apply_spools_tokenless_intent(self, spool): # No secret / addressing leaks into the container-writable spool. assert "host" not in req and "port" not in req and "token" not in req + def test_submit_worker_upgrade_spools_name_and_version_only(self, spool): + # #597: same tokenless contract as worker-apply — the version is a proposal the host + # re-derives; the rig's address + bearer never enter the container-writable spool. + rid = control_service.submit_worker_upgrade("rig1", "v1.11.2", actor="admin") + uuid.UUID(rid) + req = json.loads((spool / "requests" / f"{rid}.json").read_text()) + assert req == { + "id": rid, + "action": "worker-upgrade", + "actor": "admin", + "worker": "rig1", + "version": "v1.11.2", + } + assert "host" not in req and "port" not in req and "token" not in req + def test_writable_key_allowlist_has_no_intra_repo_drift(): """#515: the worker writable-key allowlist is hardcoded in THREE places kept in sync only by diff --git a/build/dashboard/tests/service/test_data_service.py b/build/dashboard/tests/service/test_data_service.py index 30c17fb5..3c404a62 100644 --- a/build/dashboard/tests/service/test_data_service.py +++ b/build/dashboard/tests/service/test_data_service.py @@ -495,11 +495,26 @@ def test_restored_snapshot_never_resurrects_the_update_badge(self): sm.load_snapshot.return_value = { "total_live_h15": 5000, "update": {"available": True, "latest": "v1.9.1", "url": "u"}, + # #596: same rule for the fleet-wide RigForge release — restored with the flag now + # off, it would keep serving stale per-worker badges until the first poll cycle. + "rigforge_release": {"tag": "v1.11.2", "url": "u"}, } svc = DataService(sm, MagicMock(), MagicMock()) assert svc.latest_data.get("update") in (None, {}) # never the restored dict + assert svc.latest_data.get("rigforge_release") is None # nor the RigForge one (#596) assert svc.latest_data["total_live_h15"] == 5000 # the rest of the snapshot survives + def test_rigforge_checker_wired_to_the_rigforge_api_under_the_same_flag(self): + # #596 wiring: one fleet-wide RigForge release checker, pointed at the RigForge repo, + # gated on the SAME dashboard.check_for_updates flag as the stack's own check. + sm = MagicMock() + sm.load_snapshot.return_value = None + svc = DataService(sm, MagicMock(), MagicMock()) + assert svc.rigforge_update_checker.client.api_url == ds_mod.GITHUB_RIGFORGE_RELEASES_API + assert "rigforge" in svc.rigforge_update_checker.client.api_url + assert svc.rigforge_update_checker.enabled == svc.update_checker.enabled + assert svc.rigforge_update_checker.client.tor_proxy == svc.update_checker.client.tor_proxy + def test_ignores_non_dict_snapshot(self): sm = MagicMock() sm.load_snapshot.return_value = None diff --git a/build/dashboard/tests/service/test_egress.py b/build/dashboard/tests/service/test_egress.py index ba004457..919fd66a 100644 --- a/build/dashboard/tests/service/test_egress.py +++ b/build/dashboard/tests/service/test_egress.py @@ -8,12 +8,13 @@ LOCAL, TOPOLOGY_NODES, TOR, + _sinks_all_private, compute_egress_posture, compute_topology, ) # The privacy-safe resting config: firewall on, p2pool over Tor, XvB over Tor, local node, no sync, -# healthchecks off (no ping URL configured). +# healthchecks off (no ping URL configured), no alert sinks configured. SAFE = { "firewall": True, "p2pool_clearnet": False, @@ -24,6 +25,9 @@ "remote_monero": False, "healthchecks_enabled": False, "telegram_enabled": False, + "notify_sinks_enabled": False, + "notify_tor": True, + "notify_sinks_private": False, } @@ -64,16 +68,16 @@ def test_p2pool_clearnet_without_firewall_is_a_leak(): assert "exposing your IP" in p["summary"]["label"] -def test_host_networked_dashboard_leaks_despite_firewall(): - # The dashboard's XvB stats fetch is host-networked, so the #270 container firewall can't cover - # it — disabling XvB-over-Tor leaks the host IP even with the firewall on. This is the key nuance. +def test_dashboard_xvb_stats_stays_tor_when_xvb_tor_is_off(): + # xvb.tor gates only the xmrig-proxy donation dial (#166); the dashboard's stats fetch is + # unconditionally socks5h over Tor (#163/#701), so turning xvb.tor off must not show a leak. p = _posture(xvb_tor=False, firewall=True) - assert _conn(p, "dashboard", "XvB stats")["route"] == CLEARNET - assert _conn(p, "dashboard", "XvB stats").get("blocked_by_firewall") is None - # The xmrig-proxy donation dial (a container) IS blocked by the firewall, but the dashboard isn't. + assert _conn(p, "dashboard", "XvB stats")["route"] == TOR + # The donation dial (a container) goes clearnet — but the #270 firewall blocks it. + assert _conn(p, "xmrig-proxy", "XvB donation")["route"] == CLEARNET assert _conn(p, "xmrig-proxy", "XvB donation")["blocked_by_firewall"] is True - assert p["summary"]["leaks"] >= 1 - assert p["summary"]["all_tor"] is False + assert p["summary"]["leaks"] == 0 + assert p["summary"]["all_tor"] is True def test_xvb_disabled_routes_are_inactive(): @@ -110,6 +114,56 @@ def test_price_feed_is_tor_when_enabled_inactive_otherwise(): assert on["summary"]["leaks"] == 0 # Tor-routed, so never a leak +def test_alert_sinks_tor_when_configured_inactive_otherwise(): + # Configuring a webhook/ntfy sink adds a dashboard Tor egress (#380); off → inactive. + assert _conn(_posture(), "dashboard", "alert sinks")["route"] == INACTIVE + on = _posture(notify_sinks_enabled=True) + assert _conn(on, "dashboard", "alert sinks")["route"] == TOR + assert on["summary"]["leaks"] == 0 # Tor-routed, so never a leak + + +def test_alert_sinks_clearnet_public_endpoint_leaks_despite_firewall(): + # notifications.tor=false with a public endpoint: the dashboard is host-networked, so the #270 + # firewall can't cover it — every alert POST exposes the host IP. + p = _posture(notify_sinks_enabled=True, notify_tor=False, firewall=True) + conn = _conn(p, "dashboard", "alert sinks") + assert conn["route"] == CLEARNET + assert conn.get("blocked_by_firewall") is None + assert p["summary"]["leaks"] == 1 + assert p["summary"]["all_tor"] is False + + +def test_alert_sinks_lan_carveout_is_local_not_a_leak(): + # The LAN carve-out: notifications.tor=false with every sink on a private IP. The POST never + # leaves your network — route is local, and it must NOT count toward the leak total. + p = _posture(notify_sinks_enabled=True, notify_tor=False, notify_sinks_private=True) + assert _conn(p, "dashboard", "alert sinks")["route"] == LOCAL + assert p["summary"]["leaks"] == 0 + assert p["summary"]["all_tor"] is True + + +def test_sinks_all_private_requires_ip_literal_proof(): + # Private/loopback IP literals prove the LAN carve-out; hostnames can't (no DNS in a pure + # derivation), so they classify as public — as does an empty or malformed sink set. + assert _sinks_all_private(["http://192.168.1.5/hook"]) is True + assert _sinks_all_private(["http://127.0.0.1:8080/hook", "http://[::1]/ntfy/alerts"]) is True + assert _sinks_all_private(["http://[fc00::1]/hook"]) is True # IPv6 ULA — the v6 LAN case + # The real _notify_knobs shape: webhook configured, NTFY_URL unset ("" must not veto). + assert _sinks_all_private(["http://192.168.1.5/hook", ""]) is True + assert _sinks_all_private(["http://192.168.1.5/hook", "https://ntfy.sh/mytopic"]) is False + assert _sinks_all_private(["http://nas.local/hook"]) is False # hostname — unknowable + assert _sinks_all_private(["http://localhost/hook"]) is False # still a hostname, same rule + assert _sinks_all_private(["http://8.8.8.8/hook"]) is False + assert _sinks_all_private(["http://user@8.8.8.8/hook"]) is False # userinfo can't hide the host + # IPv4-mapped IPv6 targets a public v4 address — must NOT classify private (needs the + # CPython >= 3.11.10 mapped-address rules; pinned here so a runtime downgrade can't unlock it). + assert _sinks_all_private(["http://[::ffff:8.8.8.8]/hook"]) is False + assert _sinks_all_private(["http://100.64.0.1/hook"]) is False # CGNAT/Tailscale — documented + assert _sinks_all_private([]) is False + assert _sinks_all_private(["", " "]) is False + assert _sinks_all_private(["not a url"]) is False + + def test_remote_monerod_rpc_is_clearnet(): assert _conn(_posture(remote_monero=False), "p2pool", "monerod RPC")["route"] != CLEARNET assert _conn(_posture(remote_monero=True), "p2pool", "monerod RPC")["route"] == CLEARNET @@ -188,17 +242,22 @@ def test_topology_clearnet_blocked_by_firewall_is_not_a_leak(): assert topo["summary"]["all_tor"] is True -def test_topology_host_networked_dashboard_xvb_leaks_but_proxy_is_blocked(): +def test_topology_dashboard_xvb_stats_stays_on_the_tor_hub_when_xvb_tor_is_off(): topo = _topo(xvb_tor=False, firewall=True) - # The dashboard's XvB stats fetch is host-networked → the #270 firewall can't cover it. - assert _edge(topo, "dashboard", "internet")["leak"] is True - # The xmrig-proxy XvB dial IS a container → the firewall blocks its clearnet route. + # The dashboard's stats fetch is unconditionally Tor (#163/#701) — it must terminate at the + # tor hub, never bypass to the internet node. + xvb_stats = next(e for e in topo["edges"] if e["label"] == "XvB stats") + assert xvb_stats["to"] == "tor" and xvb_stats["route"] == TOR + assert not any(e["from"] == "dashboard" and e["to"] == "internet" for e in topo["edges"]) + # The xmrig-proxy XvB dial IS clearnet here — a container, so the firewall blocks it. assert _edge(topo, "xmrig-proxy", "internet").get("blocked_by_firewall") is True + assert not any(e.get("leak") for e in topo["edges"]) def test_topology_xvb_disabled_is_inactive_not_a_leak(): topo = _topo(xvb_enabled=False) assert _edge(topo, "xmrig-proxy", "tor")["route"] == INACTIVE + assert next(e for e in topo["edges"] if e["label"] == "XvB stats")["route"] == INACTIVE assert _edge(topo, "dashboard", "tor") # update check still present assert not any(e.get("leak") for e in topo["edges"]) @@ -211,6 +270,24 @@ def test_topology_internal_mesh_is_flagged_and_includes_merge_mining(): assert docker.get("internal") is True +def test_topology_alert_sinks_edge_tracks_the_route(): + # Tor (or unconfigured) → an edge into the tor hub, same as healthchecks. + def sink_edges(topo): + return [e for e in topo["edges"] if e["label"] == "alert sinks"] + + assert sink_edges(_topo(notify_sinks_enabled=True))[0]["route"] == TOR + assert sink_edges(_topo())[0]["route"] == INACTIVE + # Public clearnet endpoint → straight to the internet node, tagged as a real leak. + clearnet = _topo(notify_sinks_enabled=True, notify_tor=False, firewall=True) + (edge,) = sink_edges(clearnet) + assert edge["to"] == "internet" and edge["route"] == CLEARNET and edge["leak"] is True + # LAN carve-out: no placeable node for a LAN appliance, so no edge — but the shared summary + # still reports no leak, so the badge and the egress list stay honest. + local = _topo(notify_sinks_enabled=True, notify_tor=False, notify_sinks_private=True) + assert sink_edges(local) == [] + assert local["summary"]["leaks"] == 0 and local["summary"]["all_tor"] is True + + def test_topology_clearnet_sync_adds_bypass_edge(): topo = _topo(monero_clearnet_sync=True, firewall=False) edge = _edge(topo, "monerod", "internet") @@ -244,6 +321,9 @@ def test_tari_clearnet_sync_surfaces_in_egress_and_topology(): "remote_monero", "healthchecks_enabled", "telegram_enabled", + "notify_sinks_enabled", + "notify_tor", + "notify_sinks_private", ) @@ -321,16 +401,17 @@ def test_firewall_off_counts_every_clearnet_path_as_a_leak(): remote_monero=True, ) clearnet = sum(1 for comp in p["components"] for c in comp["conns"] if c["route"] == CLEARNET) - assert clearnet >= 5 # sidechain, RPC, monero IBD, tari IBD, XvB donation, XvB stats... + assert clearnet >= 5 # sidechain, RPC, monero IBD, tari IBD, XvB donation assert p["summary"]["leaks"] == clearnet assert p["summary"]["blocked_by_firewall"] == 0 assert p["summary"]["all_tor"] is False assert "exposing your IP" in p["summary"]["label"] -def test_firewall_on_blocks_containers_but_not_the_host_dashboard(): - # Same clearnet-everywhere config with the firewall ON: every container path is blocked, leaving - # exactly the host-networked dashboard's XvB stats fetch as the sole real leak (the #270 nuance). +def test_firewall_on_blocks_every_clearnet_path(): + # Same clearnet-everywhere config with the firewall ON: every clearnet path belongs to a + # container, so all are blocked and nothing leaks — the dashboard's own egress is Tor-only + # (#163/#701), so the host-networked firewall bypass has nothing clearnet to expose. p = _posture( firewall=True, p2pool_clearnet=True, @@ -339,8 +420,28 @@ def test_firewall_on_blocks_containers_but_not_the_host_dashboard(): tari_clearnet_sync=True, remote_monero=True, ) - assert p["summary"]["leaks"] == 1 - assert _conn(p, "dashboard", "XvB stats")["route"] == CLEARNET - assert _conn(p, "dashboard", "XvB stats").get("blocked_by_firewall") is None - assert p["summary"]["blocked_by_firewall"] >= 4 - assert p["summary"]["all_tor"] is False + assert p["summary"]["leaks"] == 0 + assert _conn(p, "dashboard", "XvB stats")["route"] == TOR + assert p["summary"]["blocked_by_firewall"] >= 5 + assert p["summary"]["all_tor"] is True + + +# The dashboard clients hard-wired through Tor SOCKS — no knob points any of them at clearnet +# (#163 XvB stats, #224 update check, #79 Healthchecks, #121/#340 Telegram, #520 price feed). +# Scoped by name, not "all dashboard conns", so a future dashboard egress with a legitimate +# clearnet mode (e.g. the #380 alert-sink LAN carve-out) doesn't silently widen this invariant. +_TOR_HARDWIRED = ("XvB stats", "update check", "Healthchecks", "Telegram", "price feed") + + +def test_tor_hardwired_dashboard_clients_never_clearnet_for_any_config(): + # The panel's own #160 lesson: the dashboard bypasses the #270 firewall, so a clearnet route + # here would be a real leak — no knob combination may ever derive one for these clients (#701). + # next()/_conn raise StopIteration if a name drifts, so a rename can't hollow out the sweep. + for cfg in _all_configs(): + p = compute_egress_posture(**cfg) + for name in _TOR_HARDWIRED: + assert _conn(p, "dashboard", name)["route"] != CLEARNET, (cfg, name) + dash_edges = [e for e in compute_topology(**cfg)["edges"] if e["from"] == "dashboard"] + for name in _TOR_HARDWIRED: + edge = next(e for e in dash_edges if name in e["label"]) + assert edge["route"] != CLEARNET, (cfg, name) diff --git a/build/dashboard/tests/service/test_healthchecks.py b/build/dashboard/tests/service/test_healthchecks.py index 0e683cdf..89d80d7a 100644 --- a/build/dashboard/tests/service/test_healthchecks.py +++ b/build/dashboard/tests/service/test_healthchecks.py @@ -59,7 +59,7 @@ def test_no_url_is_a_silent_noop(self, caplog): # A blank ping URL is simply "off" — ping() does nothing, opens no socket, logs nothing. c = _client(ping_url="") with ( - patch.object(hc_mod.requests, "get") as get, + patch.object(hc_mod, "bounded_get") as get, caplog.at_level(logging.DEBUG, logger="Healthchecks"), ): assert c.ping() is False @@ -79,7 +79,7 @@ def test_healthy_ping_hits_the_url(self): # Pure liveness: every ping hits the configured URL (no /fail path — health-aware was # dropped; node-health alerting is the Telegram alerter's job, #121). c = _client() - with patch.object(hc_mod.requests, "get", return_value=_resp(200)) as get: + with patch.object(hc_mod, "bounded_get", return_value=_resp(200)) as get: assert c.ping() is True get.assert_called_once() assert get.call_args.args[0] == "https://hc-ping.com/abc" @@ -92,7 +92,7 @@ def test_404_returns_false_and_does_not_advance_throttle(self, caplog): clock = _Clock(1000.0) c = _client(clock=clock, interval_seconds=60) with ( - patch.object(hc_mod.requests, "get", return_value=_resp(404)), + patch.object(hc_mod, "bounded_get", return_value=_resp(404)), caplog.at_level(logging.WARNING, logger="Healthchecks"), ): assert c.ping() is False @@ -104,18 +104,18 @@ def test_200_still_returns_true_and_advances_throttle(self): # Regression guard: the happy path must keep working once status is checked. clock = _Clock(1000.0) c = _client(clock=clock, interval_seconds=60) - with patch.object(hc_mod.requests, "get", return_value=_resp(200)): + with patch.object(hc_mod, "bounded_get", return_value=_resp(200)): assert c.ping() is True assert c.ping() is False # throttled, since the first ping DID advance the clock clock.t += 61 - with patch.object(hc_mod.requests, "get", return_value=_resp(200)): + with patch.object(hc_mod, "bounded_get", return_value=_resp(200)): assert c.ping() is True def test_two_consecutive_404s_warn_only_once(self, caplog): clock = _Clock(1000.0) c = _client(clock=clock, interval_seconds=60) with ( - patch.object(hc_mod.requests, "get", return_value=_resp(404)), + patch.object(hc_mod, "bounded_get", return_value=_resp(404)), caplog.at_level(logging.WARNING, logger="Healthchecks"), ): assert c.ping() is False @@ -127,9 +127,9 @@ def test_404_then_200_logs_recovery(self, caplog): clock = _Clock(1000.0) c = _client(clock=clock, interval_seconds=60) with caplog.at_level(logging.DEBUG, logger="Healthchecks"): - with patch.object(hc_mod.requests, "get", return_value=_resp(404)): + with patch.object(hc_mod, "bounded_get", return_value=_resp(404)): assert c.ping() is False - with patch.object(hc_mod.requests, "get", return_value=_resp(200)): + with patch.object(hc_mod, "bounded_get", return_value=_resp(200)): assert c.ping() is True recoveries = [r for r in caplog.records if "recovered" in r.message] assert len(recoveries) == 1 @@ -137,7 +137,7 @@ def test_404_then_200_logs_recovery(self, caplog): def test_5xx_also_rejected(self): c = _client() - with patch.object(hc_mod.requests, "get", return_value=_resp(503)): + with patch.object(hc_mod, "bounded_get", return_value=_resp(503)): assert c.ping() is False @@ -146,7 +146,7 @@ def test_ping_passes_socks_proxies(self): # tor_proxy set → the ping rides the bridge Tor SOCKS (host IP hidden from the endpoint). proxy = "socks5h://172.28.0.25:9050" c = _client(tor_proxy=proxy) - with patch.object(hc_mod.requests, "get", return_value=_resp(200)) as get: + with patch.object(hc_mod, "bounded_get", return_value=_resp(200)) as get: assert c.ping() is True assert get.call_args.kwargs["proxies"] == {"http": proxy, "https": proxy} @@ -157,7 +157,7 @@ def test_from_config_always_routes_over_tor(self): patch.object(hc_mod, "TOR_SOCKS_PROXY", "socks5h://172.28.0.25:9050"), ): c = HealthchecksClient.from_config() - with patch.object(hc_mod.requests, "get") as get: + with patch.object(hc_mod, "bounded_get") as get: c.ping() assert get.call_args.kwargs["proxies"]["https"] == "socks5h://172.28.0.25:9050" @@ -166,7 +166,7 @@ class TestThrottle: def test_second_immediate_ping_is_throttled(self): clock = _Clock(1000.0) c = _client(clock=clock, interval_seconds=60) - with patch.object(hc_mod.requests, "get", return_value=_resp(200)) as get: + with patch.object(hc_mod, "bounded_get", return_value=_resp(200)) as get: assert c.ping() is True # first ping goes out assert c.ping() is False # within the interval → skipped get.assert_called_once() @@ -174,7 +174,7 @@ def test_second_immediate_ping_is_throttled(self): def test_ping_again_after_interval_elapses(self): clock = _Clock(1000.0) c = _client(clock=clock, interval_seconds=60) - with patch.object(hc_mod.requests, "get", return_value=_resp(200)) as get: + with patch.object(hc_mod, "bounded_get", return_value=_resp(200)) as get: assert c.ping() is True clock.t += 61 # interval elapsed assert c.ping() is True @@ -183,7 +183,7 @@ def test_ping_again_after_interval_elapses(self): def test_zero_interval_pings_every_call(self): clock = _Clock(1000.0) c = _client(clock=clock, interval_seconds=0) - with patch.object(hc_mod.requests, "get", return_value=_resp(200)) as get: + with patch.object(hc_mod, "bounded_get", return_value=_resp(200)) as get: assert c.ping() is True assert c.ping() is True assert get.call_count == 2 @@ -195,7 +195,7 @@ def test_network_error_is_swallowed_and_retries(self, caplog): c = _client(clock=clock, interval_seconds=60) with ( patch.object( - hc_mod.requests, "get", side_effect=requests.exceptions.ConnectionError("offline") + hc_mod, "bounded_get", side_effect=requests.exceptions.ConnectionError("offline") ) as get, caplog.at_level(logging.DEBUG, logger="Healthchecks"), ): @@ -210,8 +210,8 @@ def test_success_after_failure_advances_throttle(self): clock = _Clock(1000.0) c = _client(clock=clock, interval_seconds=60) with patch.object( - hc_mod.requests, - "get", + hc_mod, + "bounded_get", side_effect=[requests.exceptions.ConnectionError("x"), _resp(200)], ): assert c.ping() is False # failed, no throttle advance diff --git a/build/dashboard/tests/service/test_price_feed.py b/build/dashboard/tests/service/test_price_feed.py index fbc19238..8d8a5db4 100644 --- a/build/dashboard/tests/service/test_price_feed.py +++ b/build/dashboard/tests/service/test_price_feed.py @@ -50,7 +50,7 @@ def test_fetches_both_coins_over_tor(self): c = CoinGeckoClient("USD", tor_proxy="socks5h://t:9050") payload = {"monero": {"usd": 333.97}, "minotari": {"usd": 0.0004}} with patch( - "mining_dashboard.service.price_feed.requests.get", + "mining_dashboard.service.price_feed.bounded_get", return_value=self._resp(200, payload), ) as g: assert c.fetch() == {"xmr": 333.97, "tari": 0.0004} @@ -66,15 +66,13 @@ def test_fetches_both_coins_over_tor(self): def test_non_200_is_silent_none(self): c = CoinGeckoClient("USD") - with patch( - "mining_dashboard.service.price_feed.requests.get", return_value=self._resp(429) - ): + with patch("mining_dashboard.service.price_feed.bounded_get", return_value=self._resp(429)): assert c.fetch() is None def test_network_error_is_silent_none(self): c = CoinGeckoClient("USD") with patch( - "mining_dashboard.service.price_feed.requests.get", + "mining_dashboard.service.price_feed.bounded_get", side_effect=requests.RequestException("offline"), ): assert c.fetch() is None @@ -85,7 +83,7 @@ def test_non_alphabetic_currency_never_dials_out(self): # must mean NO request at all, not a sanitized one. for label in ("US Dollar$", "usd&x=exfil", "a" * 6, ""): c = CoinGeckoClient(label, tor_proxy="socks5h://t:9050") - with patch("mining_dashboard.service.price_feed.requests.get") as g: + with patch("mining_dashboard.service.price_feed.bounded_get") as g: assert c.fetch() is None g.assert_not_called() diff --git a/build/dashboard/tests/service/test_telegram_commands.py b/build/dashboard/tests/service/test_telegram_commands.py index 277a0ea5..9c4c5122 100644 --- a/build/dashboard/tests/service/test_telegram_commands.py +++ b/build/dashboard/tests/service/test_telegram_commands.py @@ -631,30 +631,41 @@ def test_get_updates_parses_results_over_tor(monkeypatch): seen = {} def fake_get(url, params=None, timeout=None, proxies=None): - seen.update(url=url, params=params, proxies=proxies) + seen.update(url=url, params=params, proxies=proxies, timeout=timeout) return _Resp({"ok": True, "result": [{"update_id": 8}]}) - monkeypatch.setattr(tc.requests, "get", fake_get) - assert bot._get_updates(0) == [{"update_id": 8}] + monkeypatch.setattr(tc, "bounded_get", fake_get) + assert bot._get_updates(tc.LONG_POLL_SECONDS) == [{"update_id": 8}] assert "bottok" in seen["url"] and seen["params"]["offset"] == 7 # token + offset forwarded assert seen["proxies"] == {"http": "socks5h://tor:9050", "https": "socks5h://tor:9050"} + # (connect, read) tuple with the read timeout outlasting Telegram's long-poll hold — drop the + # tuple (or shrink the read side) and every legitimate long poll aborts mid-hold (#698). + connect, read = seen["timeout"] + assert read > seen["params"]["timeout"] >= 0 and connect > 0 + # Batch cap: without it, an over-cap batch could never be parsed, so the offset could never + # advance past it and the poll loop would re-fetch it forever (#660 follow-up). + assert seen["params"]["limit"] == tc.GETUPDATES_LIMIT def test_get_updates_not_ok_returns_empty(monkeypatch): bot = _make_bot() - monkeypatch.setattr(tc.requests, "get", lambda *a, **k: _Resp({"ok": False})) + monkeypatch.setattr(tc, "bounded_get", lambda *a, **k: _Resp({"ok": False})) assert bot._get_updates(0) == [] def test_prime_offset_skips_backlog(monkeypatch): + # Backlog spanning two limit-capped batches: prime drains both, then the empty batch stops it. bot = _make_bot() - monkeypatch.setattr( - tc.requests, - "get", - lambda *a, **k: _Resp({"ok": True, "result": [{"update_id": 3}, {"update_id": 9}]}), + batches = iter( + [ + _Resp({"ok": True, "result": [{"update_id": 3}, {"update_id": 9}]}), + _Resp({"ok": True, "result": [{"update_id": 12}]}), + _Resp({"ok": True, "result": []}), + ] ) + monkeypatch.setattr(tc, "bounded_get", lambda *a, **k: next(batches)) bot._prime_offset() - assert bot._offset == 10 # past the last pending update + assert bot._offset == 13 # past the last pending update, across batches def test_prime_offset_swallows_error(monkeypatch): @@ -663,7 +674,7 @@ def test_prime_offset_swallows_error(monkeypatch): def boom(*a, **k): raise OSError("offline") - monkeypatch.setattr(tc.requests, "get", boom) + monkeypatch.setattr(tc, "bounded_get", boom) bot._prime_offset() # must not raise assert bot._offset is None @@ -1093,7 +1104,7 @@ def fake_get(url, params=None, **kw): seen["allowed"] = params["allowed_updates"] return _Resp({"ok": True, "result": []}) - monkeypatch.setattr(tc.requests, "get", fake_get) + monkeypatch.setattr(tc, "bounded_get", fake_get) on._get_updates(0) assert "callback_query" in seen["allowed"] off._get_updates(0) diff --git a/build/dashboard/tests/service/test_tor_heal.py b/build/dashboard/tests/service/test_tor_heal.py index 323fc7df..6f34ee7e 100644 --- a/build/dashboard/tests/service/test_tor_heal.py +++ b/build/dashboard/tests/service/test_tor_heal.py @@ -3,7 +3,7 @@ The decision core is what must be right: the healer restarts tor ONLY when egress is broken past the sustained threshold AND the cooldown has elapsed AND the per-outage restart budget isn't spent. Each guard is pinned separately, so inverting or deleting any of them fails a -test. The actual container restart against a real stuck guard is tier 4 (gouda). +test. The actual container restart against a real stuck guard is tier 4 (the live bench). """ from unittest.mock import patch @@ -282,13 +282,13 @@ async def start(self, *a, **k): class TestProbe: def test_any_http_response_counts_as_egress(self): - with patch("mining_dashboard.service.tor_heal.requests.get") as get: + with patch("mining_dashboard.service.tor_heal.bounded_get") as get: assert TorEgressHealer._probe_egress() is True assert get.call_args.kwargs["proxies"]["https"].startswith("socks5h://") def test_network_failure_is_broken_egress(self): with patch( - "mining_dashboard.service.tor_heal.requests.get", + "mining_dashboard.service.tor_heal.bounded_get", side_effect=requests.ConnectionError("circuit timeout"), ): assert TorEgressHealer._probe_egress() is False diff --git a/build/dashboard/tests/service/test_update_checker.py b/build/dashboard/tests/service/test_update_checker.py index fb7c716f..920f010d 100644 --- a/build/dashboard/tests/service/test_update_checker.py +++ b/build/dashboard/tests/service/test_update_checker.py @@ -48,6 +48,12 @@ def test_major_and_minor_ordering(self): assert compute_update("1.9.9", "v2.0.0", "u")["latest"] == "v2.0.0" assert compute_update("1.2.0", "v1.10.0", "u")["latest"] == "v1.10.0" # 10 > 2, not lexical + def test_bare_rig_version_vs_v_prefixed_tag(self): + # The rig reports bare "1.11.2"; RigForge release tags are "v1.11.2" (#596). Equality + # must hold across the format difference — a current rig never badges its own version. + assert compute_update("1.11.2", "v1.11.2", "u") is None + assert compute_update("1.11.1", "v1.11.2", "u")["latest"] == "v1.11.2" + class TestGitHubReleaseClient: def _resp(self, status=200, payload=None): @@ -59,7 +65,7 @@ def _resp(self, status=200, payload=None): def test_parses_tag_and_url(self): c = GitHubReleaseClient("https://api/releases/latest", tor_proxy="socks5h://t:9050") with patch( - "mining_dashboard.service.update_checker.requests.get", + "mining_dashboard.service.update_checker.bounded_get", return_value=self._resp(200, {"tag_name": "v1.4.0", "html_url": "https://h/v1.4.0"}), ) as g: assert c.latest_release() == {"tag": "v1.4.0", "url": "https://h/v1.4.0"} @@ -72,14 +78,14 @@ def test_parses_tag_and_url(self): def test_non_200_is_silent_none(self): c = GitHubReleaseClient("u") with patch( - "mining_dashboard.service.update_checker.requests.get", return_value=self._resp(404) + "mining_dashboard.service.update_checker.bounded_get", return_value=self._resp(404) ): assert c.latest_release() is None def test_network_error_is_silent_none(self): c = GitHubReleaseClient("u") with patch( - "mining_dashboard.service.update_checker.requests.get", + "mining_dashboard.service.update_checker.bounded_get", side_effect=requests.RequestException("offline"), ): assert c.latest_release() is None @@ -87,7 +93,7 @@ def test_network_error_is_silent_none(self): def test_missing_fields_is_none(self): c = GitHubReleaseClient("u") with patch( - "mining_dashboard.service.update_checker.requests.get", + "mining_dashboard.service.update_checker.bounded_get", return_value=self._resp(200, {"tag_name": "v1.0.0"}), ): # no html_url assert c.latest_release() is None @@ -133,3 +139,30 @@ def test_failed_fetch_keeps_previous_result(self): def test_up_to_date_yields_none(self): uc = UpdateChecker(_FakeClient({"tag": "v0.1.0", "url": "u"}), "0.1.0", enabled=True) assert uc.maybe_check(1000) is None + + +class TestLatestReleaseCached: + """The raw-release accessor (#596): one throttled fleet-wide fetch, many consumers.""" + + def test_disabled_never_calls_and_returns_none(self): + c = _FakeClient({"tag": "v1.11.2", "url": "u"}) + uc = UpdateChecker(c, None, enabled=False) + assert uc.latest_release_cached(1000) is None + assert c.calls == 0 + + def test_returns_raw_release_and_throttles(self): + c = _FakeClient({"tag": "v1.11.2", "url": "https://h/v1.11.2"}) + uc = UpdateChecker(c, None, enabled=True, interval=3600) + assert uc.latest_release_cached(1000) == {"tag": "v1.11.2", "url": "https://h/v1.11.2"} + assert uc.latest_release_cached(1000 + 1800) == { + "tag": "v1.11.2", + "url": "https://h/v1.11.2", + } + assert c.calls == 1 # within the window: cached, no network + uc.latest_release_cached(1000 + 3601) + assert c.calls == 2 + + def test_failed_fetch_keeps_previous_release(self): + uc = UpdateChecker(_FakeClient(None), None, enabled=True, interval=0) + uc.release = {"tag": "v1.11.2", "url": "u"} + assert uc.latest_release_cached(2000)["tag"] == "v1.11.2" # a blip keeps the cache diff --git a/build/dashboard/tests/web/test_server.py b/build/dashboard/tests/web/test_server.py index 295ffc7a..0d11747c 100644 --- a/build/dashboard/tests/web/test_server.py +++ b/build/dashboard/tests/web/test_server.py @@ -626,6 +626,9 @@ async def worker_client(aiohttp_client, control_spool, monkeypatch): "status": "online", "active_pool": "3333", "h60": 5100, + # Bare rig-reported version (#596/#597) — the upgrade noop guard compares it + # (parsed) against the v-prefixed proposal. + "rigforge": {"version": "1.11.0"}, } ] } @@ -709,6 +712,91 @@ async def test_worker_routes_absent_when_control_disabled(self, client): ).status == 404 +class TestWorkerUpgrade: + """The one-click rig upgrade route (#597): spool-only, name + confirmed version, no waiting.""" + + async def test_requires_control_header(self, worker_client): + resp = await worker_client.post( + "/api/control/worker-upgrade", json={"worker": "rig1", "version": "v1.11.2"} + ) + assert resp.status == 403 # CSRF guard + + async def test_malformed_worker_or_version_rejected(self, worker_client): + for body in ( + {"version": "v1.11.2"}, # missing worker + {"worker": "", "version": "v1.11.2"}, # empty worker + {"worker": "rig1"}, # missing version + {"worker": "rig1", "version": "1.11.2"}, # bare — the intent carries the tag form + {"worker": "rig1", "version": "v1.11.2;rm"}, # junk after the tag + ): + resp = await worker_client.post( + "/api/control/worker-upgrade", json=body, headers=CONTROL_HEADERS + ) + assert resp.status == 400, body + + async def test_spools_name_and_version_only_and_returns_202( + self, worker_client, control_spool, monkeypatch + ): + rid = str(uuid.uuid4()) + monkeypatch.setattr(control_service.uuid, "uuid4", lambda: uuid.UUID(rid)) + resp = await worker_client.post( + "/api/control/worker-upgrade", + json={"worker": "rig1", "version": "v1.11.2"}, + headers=CONTROL_HEADERS, + ) + # Always 202 — a rig build can run minutes, so the client polls /api/control/result. + assert resp.status == 202 + body = await resp.json() + assert body["status"] == "pending" and body["id"] == rid + # The intent carries ONLY the worker name + proposed version — never host/port/token. + req = json.loads((control_spool / "requests" / f"{rid}.json").read_text()) + assert req["action"] == "worker-upgrade" + assert req["worker"] == "rig1" and req["version"] == "v1.11.2" + assert "host" not in req and "port" not in req and "token" not in req + assert "changes" not in req + + async def test_non_json_body_is_a_400(self, worker_client): + resp = await worker_client.post( + "/api/control/worker-upgrade", data=b"not json", headers=CONTROL_HEADERS + ) + assert resp.status == 400 + assert "must be JSON" in await resp.text() + + async def test_submit_failure_is_a_500_without_detail(self, worker_client, monkeypatch): + def boom(*a, **k): + raise OSError("spool dir gone") + + monkeypatch.setattr(control_service, "submit_worker_upgrade", boom) + resp = await worker_client.post( + "/api/control/worker-upgrade", + json={"worker": "rig1", "version": "v1.11.2"}, + headers=CONTROL_HEADERS, + ) + assert resp.status == 500 + # The body carries a generic message, never the exception text. + assert "spool dir gone" not in await resp.text() + + async def test_noop_when_rig_already_reports_the_version(self, worker_client, control_spool): + # The fixture rig reports bare "1.11.0"; proposing tag v1.11.0 must short-circuit — + # no spool, no host dial, no burn of the rig's own 6h upgrade throttle. + resp = await worker_client.post( + "/api/control/worker-upgrade", + json={"worker": "rig1", "version": "v1.11.0"}, + headers=CONTROL_HEADERS, + ) + assert resp.status == 200 + assert (await resp.json())["status"] == "noop" + assert list((control_spool / "requests").glob("*.json")) == [] + + async def test_route_absent_when_control_disabled(self, client): + resp = await client.post( + "/api/control/worker-upgrade", + json={"worker": "rig1", "version": "v1.11.2"}, + headers=CONTROL_HEADERS, + ) + assert resp.status == 404 + + class TestWorkerApplyEdgeCases: async def test_apply_bad_body_and_missing_worker(self, worker_client): # Non-JSON body → 400. diff --git a/build/dashboard/tests/web/test_views.py b/build/dashboard/tests/web/test_views.py index cfdfae8d..0872ea02 100644 --- a/build/dashboard/tests/web/test_views.py +++ b/build/dashboard/tests/web/test_views.py @@ -51,6 +51,7 @@ host_display_addr, parse_window, recent_wallet_change, + rigforge_update_for, visible_update, ) @@ -942,6 +943,79 @@ def test_reject_flag_set_on_high_reject_rate(self): assert "10.0%" in row["reject_flag"]["title"] +class TestRigforgeUpdate: + """The per-worker RigForge new-release callout (#596), derived at the render seam.""" + + _REL = {"tag": "v1.11.2", "url": "https://h/v1.11.2"} + _W = {"name": "r", "ip": "10.0.0.1", "status": "online", "active_pool": "3333"} + + def test_behind_rig_gets_the_callout(self): + w = {**self._W, "rigforge": {"version": "1.11.1"}} + out = rigforge_update_for(w, self._REL) + assert out == {"available": True, "latest": "v1.11.2", "url": "https://h/v1.11.2"} + + def test_current_rig_never_badges_its_own_version(self): + # The rig reports bare "1.11.2"; the tag is "v1.11.2" — equality must hold across the + # format difference (the #664 self-consistency guard, per-worker edition). + w = {**self._W, "rigforge": {"version": "1.11.2"}} + assert rigforge_update_for(w, self._REL) is None + + def test_newer_or_unparseable_rig_version_is_none(self): + assert rigforge_update_for({**self._W, "rigforge": {"version": "9.0.0"}}, self._REL) is None + assert ( + rigforge_update_for({**self._W, "rigforge": {"version": "nightly"}}, self._REL) is None + ) + + def test_no_version_or_no_release_is_none(self): + # A plain-:8080 rig reports no version: no badge, not a false "up to date". No cached + # release (check disabled / offline): same. + assert rigforge_update_for(self._W, self._REL) is None + assert rigforge_update_for({**self._W, "rigforge": {"version": "1.11.1"}}, None) is None + + def test_build_workers_attaches_per_row(self): + rows = build_workers( + [ + {**self._W, "name": "behind", "rigforge": {"version": "1.11.1"}}, + {**self._W, "name": "current", "rigforge": {"version": "1.11.2"}}, + {**self._W, "name": "plain"}, + ], + self._REL, + ) + by = {r["name"]: r["rigforge_update"] for r in rows} + assert by["behind"]["latest"] == "v1.11.2" + assert by["current"] is None + assert by["plain"] is None + + def test_build_workers_without_release_attaches_none(self): + rows = build_workers([{**self._W, "rigforge": {"version": "1.11.1"}}]) + assert rows[0]["rigforge_update"] is None + + def test_build_state_feeds_the_cached_release_through(self): + data = _data( + workers=[{**self._W, "rigforge": {"version": "1.11.1"}}], + rigforge_release=self._REL, + ) + st = build_state(data, _state_mgr(), "all") + assert st["workers"][0]["rigforge_update"]["latest"] == "v1.11.2" + + def test_worker_detail_attaches_the_callout(self): + from mining_dashboard.service.storage_service import StateManager + + sm = StateManager(db_path=":memory:") + try: + d = build_worker_detail( + "r", + { + "workers": [{**self._W, "rigforge": {"version": "1.11.1"}}], + "rigforge_release": self._REL, + }, + sm, + ) + finally: + sm.close() + assert d["rigforge_update"]["latest"] == "v1.11.2" + + class TestRejectFlag: """The per-worker reject-rate flag (Issue #82).""" @@ -1923,11 +1997,19 @@ def test_safe_config_emits_a_tor_only_header_badge(self, monkeypatch): assert "Tor-only" in badge["text"] assert st["egress"]["summary"]["all_tor"] is True - def test_host_dashboard_clearnet_leak_emits_a_warning_badge(self, monkeypatch): - # The host-networked dashboard's XvB stats fetch over clearnet leaks despite the firewall — - # the payload must flip the badge to a loud warning and the topology summary to "warn". + def test_xvb_tor_off_stays_a_tor_only_badge(self, monkeypatch): + # xvb.tor gates only the xmrig-proxy donation dial; the dashboard's stats fetch is + # unconditionally Tor (#163/#701), so with the firewall on nothing leaks — badge stays green. _set_egress_config(monkeypatch, XVB_TOR_ENABLED=False) st = build_state(_data(), _state_mgr(), "all") + assert st["badges"][-1]["variant"] == "ok" + assert st["egress"]["summary"]["leaks"] == 0 + + def test_clearnet_leak_emits_a_warning_badge(self, monkeypatch): + # A real leak (clearnet sidechain peers with the firewall down) must flip the badge to a + # loud warning and the topology summary to "warn". + _set_egress_config(monkeypatch, P2POOL_CLEARNET=True, TOR_EGRESS_FIREWALL=False) + st = build_state(_data(), _state_mgr(), "all") badge = st["badges"][-1] assert badge["variant"] == "bad" assert "clearnet egress" in badge["text"] @@ -1947,7 +2029,7 @@ def test_remote_monerod_is_reflected_in_the_payload(self, monkeypatch): ) def test_payload_stays_json_serializable_with_a_leak(self, monkeypatch): - _set_egress_config(monkeypatch, XVB_TOR_ENABLED=False, P2POOL_CLEARNET=True) + _set_egress_config(monkeypatch, P2POOL_CLEARNET=True, TOR_EGRESS_FIREWALL=False) json.dumps(build_state(_data(), _state_mgr(), "all")) diff --git a/build/dashboard/uv.lock b/build/dashboard/uv.lock index 8b7aa0a7..8aacd362 100644 --- a/build/dashboard/uv.lock +++ b/build/dashboard/uv.lock @@ -782,7 +782,7 @@ wheels = [ [[package]] name = "mining-dashboard" -version = "1.9.3" +version = "1.10.0" source = { editable = "." } dependencies = [ { name = "aiofiles" }, diff --git a/build/tari-wallet/entrypoint.sh b/build/tari-wallet/entrypoint.sh index dd15bf12..78b008dc 100755 --- a/build/tari-wallet/entrypoint.sh +++ b/build/tari-wallet/entrypoint.sh @@ -53,7 +53,7 @@ birthday="$(resolve_birthday)" echo "Starting view-only Tari payout wallet (birthday $birthday, base node $BASE_NODE_GRPC) (#462)..." # Point the wallet at the LOCAL base node. The exact config-override key for the base-node peer is -# the one item pinned to tier-4 (gouda) — confirmed there against a live minotari_console_wallet +# the one item pinned to tier-4 (the live bench) — confirmed there against a live minotari_console_wallet # alongside whether the merge-mine coinbase surfaces via GetCompletedTransactions. Passed via the # Tari config env-override convention so it is NON-secret and stays out of argv. export MINOTARI_WALLET__BASE_NODE__GRPC_BASE_NODE_ADDRESS="/dns4/${BASE_NODE_GRPC%%:*}/tcp/${BASE_NODE_GRPC##*:}" diff --git a/docs/architecture.md b/docs/architecture.md index 4e052162..f90114d4 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -36,6 +36,7 @@ flowchart TB HC(["🩺 Healthchecks.io
dead-man's switch"]) XvB(["🎲 XMRvsBeast
pool + stats"]) GitHub(["🐙 GitHub
release check"]) + Coin(["💱 CoinGecko
price feed"]) subgraph stack ["🐳 Pithead"] direction TB @@ -57,6 +58,7 @@ flowchart TB You ==>|HTTPS| Caddy Caddy --> Dashboard Workers ==>|"Stratum 3333"| Proxy + Dashboard ==>|"worker API · LAN"| Workers %% Dashboard internal control + monitoring (never leaves the box) Dashboard -.->|controls| Proxy @@ -67,8 +69,9 @@ flowchart TB Dashboard ==>|"🚨 alerts + commands · 🟢 Tor"| Tor Dashboard ==>|"🔔 webhook/ntfy alerts · 🟢 Tor"| Tor Dashboard ==>|"🩺 liveness ping · 🟢 Tor"| Tor - Dashboard ==>|"📈 XvB stats · 🟢 Tor"| Tor + Dashboard ==>|"📈 XvB stats + raffle · 🟢 Tor"| Tor Dashboard ==>|"🆕 update check · 🟢 Tor"| Tor + Dashboard ==>|"💱 XMR/XTM prices · 🟢 Tor"| Tor Proxy ==>|hashrate| P2Pool Proxy ==>|"hashrate · 🟢 Tor"| Tor @@ -87,13 +90,14 @@ flowchart TB Net -.-> HC Net -.-> XvB Net -.-> GitHub + Net -.-> Coin classDef ext fill:#1e293b,stroke:#64748b,color:#e2e8f0; classDef ctrl fill:#1d4ed8,stroke:#93c5fd,color:#eff6ff; classDef priv fill:#6d28d9,stroke:#c4b5fd,color:#f5f3ff; classDef mine fill:#047857,stroke:#6ee7b7,color:#ecfdf5; - class You,Workers,Net,Telegram,Hooks,HC,XvB,GitHub ext; + class You,Workers,Net,Telegram,Hooks,HC,XvB,GitHub,Coin ext; class Caddy,Dashboard ctrl; class Tor,DockerProxy priv; class Proxy,P2Pool,Monerod,Tari mine; @@ -105,11 +109,16 @@ flowchart TB Reading the diagram: thick arrows carry inbound connections and every path that **leaves the box** — each egress edge is tagged with its route, and **🟢 Tor** means it exits through the Tor daemon (a Tor exit IP, never your host's). Dotted arrows are the dashboard's internal control and monitoring, which -never leave the machine. The dashboard makes five outbound calls — the **Telegram** bot (alerts + -commands), the **webhook/ntfy** alert sinks, the **Healthchecks.io** liveness ping, the **XvB** stats -fetch, and the **GitHub** release check — and all five are Tor-routed, so enabling any of them never -reveals where your stack runs (the webhook/ntfy sinks have a `notifications.tor: false` opt-out for -LAN endpoints Tor can't reach; see [Telegram › Webhook and ntfy sinks](telegram.md#webhook-and-ntfy-sinks)). Node +never leave the machine. The dashboard makes six outbound internet calls — the **Telegram** bot +(alerts + commands), the **webhook/ntfy** alert sinks, the **Healthchecks.io** liveness ping, the +**XvB** calls (stats, raffle registration, winners), the **GitHub** release check, and the opt-in +**CoinGecko** price feed (`dashboard.energy.price_feed`, XMR + XTM spot prices) — and all six are +Tor-routed, so enabling any of them never reveals where your stack runs (the webhook/ntfy sinks have +a `notifications.tor: false` opt-out for LAN endpoints Tor can't reach; see +[Telegram › Webhook and ntfy sinks](telegram.md#webhook-and-ntfy-sinks)). The dashboard also polls +each rig's RigForge API for worker stats, and config applies travel the same path — dialed by the +host-side control runner, so the rig tokens never enter the dashboard container (#185). Both are +direct **LAN** connections to your rigs; they don't route over Tor, so they carry no Tor tag. Node colors group services by role: 🟦 control plane (Caddy, Dashboard), 🟪 privacy and isolation (Tor, Docker socket proxies), and 🟩 the mining core. In remote-node mode the bundled 🟠 Monero node isn't started, and P2Pool talks to your external node instead. diff --git a/docs/configuration.md b/docs/configuration.md index 97a8afd4..b5580950 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -139,7 +139,7 @@ control channel will commit, are unaffected either way. | `dashboard.control.enabled` | `false` _(off)_ | Security-relevant, default off. `true` turns on the dashboard's **Configuration view**: edit `config.json` from the browser, preview the changes, and apply them. The dashboard container never runs `pithead` itself — it writes a typed change request into a spool directory and a root systemd unit on the host (`pithead-control`) validates and applies it (see [Dashboard › Configuration view](dashboard.md#configuration-view)). Fails closed: enabling it without a `dashboard.auth.password` is a validation error, because this channel can change the payout wallet. | | `dashboard.timezone` | `auto` | Timezone for the dashboard's timestamps and charts. `auto` = the host machine's timezone (auto-detected, falling back to `Etc/UTC`); set an IANA name (e.g. `America/Chicago`) to override. | | `dashboard.data_dir` | `auto` | Where the dashboard's database lives. `auto` = `./data/dashboard`, unless the four other `*.data_dir` all point under one parent directory — then the dashboard joins them at `/dashboard`, and the first `upgrade`/`apply` moves data from the old default there automatically (see [Data directories](#data-directories)). | -| `dashboard.check_for_updates` | `true` _(on)_ | The dashboard periodically asks GitHub whether a newer Pithead release exists and, if so, shows a header badge linking to it (e.g. "New release v1.4.0 available"). Notify-only: it never updates anything; you upgrade with `./pithead upgrade` on your own terms. On by default because the check is routed over Tor (the same bridge SOCKS as the XvB fetch, `socks5h` so the DNS lookup goes through Tor too), so GitHub sees a Tor exit, not your IP. It's cached (hourly) and fails silently offline. Set to `false` to opt out entirely. See [Privacy › Runtime egress](privacy.md#runtime-egress). | +| `dashboard.check_for_updates` | `true` _(on)_ | The dashboard periodically asks GitHub whether a newer Pithead release exists and, if so, shows a header badge linking to it (e.g. "New release v1.4.0 available"). Notify-only: it never updates anything; you upgrade with `./pithead upgrade` on your own terms. On by default because the check is routed over Tor (the same bridge SOCKS as the XvB fetch, `socks5h` so the DNS lookup goes through Tor too), so GitHub sees a Tor exit, not your IP. It's cached (hourly) and fails silently offline. The same flag also covers the per-worker [RigForge new-release badge](workers.md#rigforge-new-release-badge) — one more hourly, Tor-routed fetch of the latest RigForge release, compared against every rig's reported version. Set to `false` to opt out of both. See [Privacy › Runtime egress](privacy.md#runtime-egress). | | `dashboard.hashrate_drop_threshold` | `50` | Percent below the recent normal that counts as a hashrate drop for the `hashrate_loss` alert and its chart marker. `50` = fire when total fleet hashrate falls to half its baseline. Raise it to catch smaller dips, lower it to only flag near-total outages. | | `dashboard.hashrate_drop_minutes` | `10` | How many minutes the hashrate must stay below the threshold before the drop is reported — the debounce that keeps a brief blip from pinging you. | | `dashboard.tari_required` | `true` | How much a Tari problem holds up the rest of the stack. Monero is required to mine, so its behavior isn't configurable: a monerod outage always rejects workers (stops `xmrig-proxy` so miners fail over to their backup pools), and the miner is always held until monerod finishes syncing. Tari is only needed for merge-mining, so this one flag decides how much it blocks. `true` (default): a Tari outage also rejects workers, the miner waits for Tari's initial sync too, and a Tari-only (re)sync shows the full-screen Sync view. `false` (non-blocking): keep mining Monero through a Tari outage, start mining as soon as Monero is synced (Tari finishes in the background), and keep the normal dashboard, with a `Tari syncing` indicator, instead of the takeover screen. | diff --git a/docs/dashboard.md b/docs/dashboard.md index 27c0a2dc..27e45dbd 100644 --- a/docs/dashboard.md +++ b/docs/dashboard.md @@ -240,6 +240,11 @@ but its miner isn't, the rig stays in the table with a **miner down** chip rathe offline. Point the rig's descriptor at the enriched feed to turn this on — see [Connecting Miners › RigForge enriched feed](workers.md#rigforge-enriched-feed). +With `dashboard.check_for_updates` on, a rig reporting a RigForge version older than the latest +published release also gets a clickable `rf vX.Y.Z available ↗` badge — the per-worker twin of the +header's new-release badge, notify-only, linking to the RigForge release notes. See +[Connecting Miners › RigForge new-release badge](workers.md#rigforge-new-release-badge). + Each rig shows accepted and rejected share counts (invalid shares folded into the rejected column as `3 (+2 inv)` when present). A rig whose reject rate climbs past ~5% gets a red **⚠** flag next to its rejected count — a rig submitting stale or bad shares (bad overclock, flaky network, clock drift) @@ -286,6 +291,15 @@ To make a rig editable, give it `host`, `token`, and (unless it's the default `8 in its [`workers.list[]`](configuration.md#configuration-reference) descriptor. Without a host, or without a token, the rig isn't a write target and the panel says so. +When the rig's [new-release badge](#workers-alive) shows and the rig is editable, an **Upgrade +rig…** button appears beside it: arm it, confirm, and the rig upgrades its own RigForge to the +latest release — the per-worker twin of the stack's one-click upgrade. The rig may rebuild its +miner (about ten minutes when the XMRig pin changed) and rolls itself back if the miner doesn't +come back live. The panel shows the outcome (applied / rolled back / failed); a repeat click inside +the rig's own six-hour upgrade window reads as "throttled — retry later", not an error. See +[Connecting Miners › One-click rig upgrade](workers.md#one-click-rig-upgrade) for what the rig +must enable and how the target is derived. + How it stays safe: - **The dashboard never holds the rig's token.** It spools the worker name and the change into the diff --git a/docs/dev/release-server.md b/docs/dev/release-server.md index cfe8bbf3..5ac434ce 100644 --- a/docs/dev/release-server.md +++ b/docs/dev/release-server.md @@ -217,20 +217,21 @@ tests/integration/run.sh --host you@server --dir pithead --readiness ## Bench allocation and the rig lock -Ten boxes are shared between this repo's tier-4 harness, RigForge's release gates, and +The bench boxes are shared between this repo's tier-4 harness, RigForge's release gates, and production mining. Two automations cycling the same rig's services corrupt each other's results -(the 2026-07-10 miner-0 incident: an operator "fixed" a service an e2e run had deliberately -stopped), so ownership is static and every run takes a kernel lock. +(a real 2026-07 incident: an operator "fixed" a service an e2e run had deliberately stopped), +so ownership is static and every run takes a kernel lock. -Static allocation — each box states its owner in `/etc/bench-role`: +Static allocation — each box states its owner in `/etc/bench-role`, and each box's own +`~/README.md` carries its specifics (hostname, role, data roots): -| Box | Owner | Use | +| Box role | Owner | Use | |---|---|---| -| miner-0 | RigForge | `e2e-real` / tune gates. Pithead never touches its services. | -| miner-1, miner-2 | Pithead | Tier-4 loaner rigs: `e2e.sh` repoints one at the test bench for a run, then reverts it. Verify `systemctl is-active xmrig` after any remote restart. | -| miner-3 … miner-7 | Production | Mining only. No test traffic. | -| gouda | Pithead | Test bench + release box (the tier-4 target). | -| pithead-prod | Production | Production stack; deploys only. | +| RigForge bench rig | RigForge | `e2e-real` / tune gates. Pithead never touches its services. | +| Loaner rigs (two) | Pithead | Tier-4: `e2e.sh` repoints one at the test bench for a run, then reverts it. Verify `systemctl is-active xmrig` after any remote restart. | +| Fleet rigs | Production | Mining only. No test traffic. | +| Test bench | Pithead | Test bench + release box (the tier-4 target). | +| Production host | Production | Production stack; deploys only. | The run lock. Both harnesses take a `flock` on `/var/lock/rig-e2e.lock` before the first service-touching action and hold it on an inherited FD for the whole run, so the kernel releases diff --git a/docs/dev/releasing.md b/docs/dev/releasing.md index 2b1649a0..ddb5f7a1 100644 --- a/docs/dev/releasing.md +++ b/docs/dev/releasing.md @@ -110,6 +110,31 @@ required, blocking pre-release gate. A release must not be promoted or published matrix is green against the real Monero + Tari nodes. This is what makes every published version a single, validated bundle. +Two runs of the matrix are required, and the automated one is the smaller of the two: + +1. `release.sh` stage 2 runs the non-destructive `--readiness` assessment against the live + stack (via `RELEASE_INTEGRATION_ARGS`). It proves the box is fit to cut from — it does not + mine, restart anything, or touch a rig. +2. Before cutting, run the targeted end-to-end matrix on the release candidate with a borrowed + loaner rig: + + ```bash + BENCH_HOST= MINER_HOST= tests/integration/e2e.sh --mode targeted + ``` + + This deploys the candidate to the bench's dedicated e2e checkout, repoints the rig at it + (under the [rig lock](release-server.md#bench-allocation-and-the-rig-lock), with automatic + restore of both), and proves what `--readiness` cannot: a real miner mining through the + stack, the lifecycle phase (restart, apply secret-preservation, node-down failover), and + fail-closed auth. If the release's diff touches the worker or control-descriptor path, add + the `--rigforge-control` legs (needs a rig with its control API enabled). The readiness + gate alone does not satisfy this requirement; abort the release on any failure. + +After deploying the published release to the bench, run the non-destructive live sweep as the +closing check: `tests/integration/run.sh --local --dir --check`. On a bench with no +miners connected, exactly two failures are expected — `workers online` and `stratum total +hashes` — anything else is a regression. + ## Signed releases Every promoted image digest and the install bundle carry a cosign key signature diff --git a/docs/dev/testing-strategy.md b/docs/dev/testing-strategy.md index d3c027e8..b4c738c9 100644 --- a/docs/dev/testing-strategy.md +++ b/docs/dev/testing-strategy.md @@ -143,6 +143,8 @@ it — the worker-API **auth model is the #315 none/name/token matrix**, not the | Worker Inspect edit lands on the rig: `editable` true, a `max_temp_c` nudge via `/api/control/worker-apply` hits the rig's `/status` + records history, reverted | live rig control API on | 4 ▶ (`run.sh --rigforge-control`, #513) | | Rig-side edit reflects: a direct rig control-API change shows in the enriched feed; a `config.json` hand-edit shows in the masked prefill | live rig + direct dial | 4 ▶ (`run.sh --rigforge-control`, #516) | | Control-apply auto-rollback (rigforge#236): a hashrate-tanking change is recorded `rolled_back` in the worker-apply result + per-worker history | live rig + fault-injection | 4 ▶ (`run.sh --rigforge-control`, #517; operator supplies `IT_RIG_ROLLBACK_CHANGES`) | +| Per-worker RigForge new-release badge (#596): one fleet-wide latest-release fetch (hourly throttle, disabled = never dials) compared against each rig's reported `version`, bare-vs-`v`-prefixed SemVer | `dashboard.check_for_updates` + rig-reported version | 1 ✅ (`test_update_checker` comparison/throttle/no-dial; `test_views` + node tests for the per-worker plumbing) · 4 (deferred — live badge on a real rig, owed to the #597 gouda loaner session) | +| One-click rig upgrade (#597): the #596 badge proposes a version, the host re-derives the target from the RigForge release API over Tor and dials the rig's `/upgrade`; refusals (non-latest, GitHub unreachable/no tag, old rig < v1.11.2 non-202), the anti-beacon throttle, and the poll-cap timeout→accepted fallback | `/api/control/worker-upgrade` + host runner + badge render | 1 ✅ (`test_server.py` + `test_control_service.py`, `tests/stack` #597 cases, `workerview.test.mjs`) · 4 (deferred — gouda loaner: badge → click → applied → badge clears, repeat-click throttle, old-rig refusal) | | Stratum auth accept/reject: matching `pass` mines, wrong/missing `pass` rejected, rotation | live proxy `--access-password` | 4 (deferred — a headless xmrig login probe, real proxy binary) | | Dev-fee independence (#173): proxy `--donate-level` and rig `DONATION` both honored | live proxy + rig | 4 (deferred) | diff --git a/docs/operations.md b/docs/operations.md index d83ca6ef..2672e6df 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -134,6 +134,12 @@ on, and remove them when it is off: - `pithead-control.service` — a root oneshot running `pithead control-run-pending` from the install directory. Fixed command, no parameters from the container. +The unit names are global to the host, so removal is ownership-checked: a checkout with the flag +off only removes units whose `ExecStart` points at itself, comparing physical paths so the +`current` symlink and the versioned directory it targets count as the same checkout. Another +checkout on the same box (an e2e harness, a bundle smoke test) therefore cannot delete the live +stack's runner and strand its queued requests. + The runner dispatches exactly three actions: `apply --dry-run --porcelain` (preview), `apply -y` (commit), and a release upgrade — the dashboard's [Upgrade button](dashboard.md#upgrading-from-the-dashboard), for which the runner re-derives the diff --git a/docs/privacy.md b/docs/privacy.md index 816f037c..8525f70c 100644 --- a/docs/privacy.md +++ b/docs/privacy.md @@ -77,7 +77,7 @@ What the running stack sends to the internet, connection by connection. | Dashboard **XvB raffle registration** (#263) | `xmrvsbeast.com` | your Monero **wallet** (no longer your IP) | ✅ Tor (`socks5h`, same path as the stats fetch) | on, only if XvB enabled; fires once you have a PPLNS share | `XVB_ENABLED=false`, or set `XVB_SUBMIT_URL` to a disable sentinel (`off`), to stop it | | Dashboard **XvB winners fetch** (raffle-wins display) | `xmrvsbeast.com` | nothing — the winners file is public and the request carries no wallet | ✅ Tor (`socks5h`, same path as the stats fetch) | on, only if XvB enabled | `XVB_ENABLED=false` stops it | | **XvB donation mining** (only while donating) | `na.xmrvsbeast.com:4247` via Tor | — | ✅ **Tor** (per-pool `socks5`, DNS proxy-side) by default (#166) | on while donating | opt out with `xvb.tor: false` (exposes IP for max yield); `xvb.enabled: false` stops it entirely | -| Dashboard **update check** (#224) | `api.github.com` | nothing about you — GitHub sees a **Tor exit**, not your IP | ✅ Tor (`socks5h`) | **on** | `dashboard.check_for_updates: false` to opt out; cached, fails silently offline | +| Dashboard **update check** (#224, plus the per-worker RigForge badge #596) | `api.github.com` | nothing about you — GitHub sees a **Tor exit**, not your IP; the RigForge check sends no rig versions, it only reads the latest release | ✅ Tor (`socks5h`) | **on** | `dashboard.check_for_updates: false` opts out of both; cached, fails silently offline | | **Caddy** TLS (dashboard HTTPS) | local only | — | n/a — `tls internal`, **no ACME / no external CA** | on | clean (no egress) | | **Telegram** bot (#121) | `api.telegram.org` | nothing about you — Telegram sees a **Tor exit**, not your IP | ✅ **always** Tor (`socks5h`, #340) | **off** | opt-in; both the alert sends and the command poll ride Tor | | Dashboard **Healthchecks** ping (#79) | `hc-ping.com` (or self-hosted) | nothing about you — the endpoint sees a **Tor exit**, not your IP | ✅ **always** Tor (`socks5h`) | opt-in (set `healthchecks.ping_url`; off until set) | the ping URL must be Tor-reachable (hosted, public, or an onion self-hosted instance) — there is no clearnet mode | @@ -88,6 +88,12 @@ What the running stack sends to the internet, connection by connection. resolved on the clearnet either. The host-networked dashboard reaches the bridge's Tor SOCKS at `172.28.0.25:9050`. +The dashboard's egress panel classifies the alert-sink carve-out by what it can prove from the +config alone: with `notifications.tor: false`, the sinks show as **local** — not a counted leak — +only when every configured endpoint is a private or loopback IP literal, since such a POST never +leaves your network. A hostname can't be proven private without a DNS lookup, so a hostname +endpoint with Tor off counts as **clearnet**, a real leak. + --- ## Build / setup-time egress diff --git a/docs/workers.md b/docs/workers.md index 17cc6011..a179a8da 100644 --- a/docs/workers.md +++ b/docs/workers.md @@ -322,6 +322,47 @@ control port. [TODO: verify upstream — confirm the `rigforge.control` mirror ships in RigForge; until then this parses to nothing and the row stays `accepted`.] +#### RigForge new-release badge + +When `dashboard.check_for_updates` is on, the dashboard also compares each rig's reported RigForge +version against the latest published RigForge release and badges the rigs that are behind — in the +Workers Alive table and in Worker Inspect, next to the version. The badge is notify-only and links +to the release notes; upgrading the rig is still done on the rig. + +One fetch covers the whole fleet: the release check is hourly, rides the same Tor route as the +stack's own new-release check, and fails silent offline (no badge, no error). The same +`dashboard.check_for_updates` flag gates both checks — off means neither dials GitHub. + +Only rigs on the enriched `8081` feed report a version. A plain-xmrig rig on `8080` has no version +to compare, so it shows no badge — that reads as "unknown", not "up to date". + +#### One-click rig upgrade + +With the control channel on (`dashboard.control.enabled`) and the rig editable (an operator-set +`host` + `token` in `workers.list[]`), the badge gains an **Upgrade rig…** button in +[Worker Inspect](dashboard.md#worker-inspect): arm, confirm, and the rig checks out and installs +the latest RigForge release itself — the per-worker twin of the stack's one-click upgrade. + +The rig side must opt in: RigForge's `control_upgrade` capability (default off, on top of its +`control` flag, bearer token, and source pin) and RigForge **v1.11.2 or newer** — earlier versions +refuse legitimate upgrades on a fresh clone. See +[RigForge › ADR 0002](https://github.com/p2pool-starter-stack/rigforge/blob/main/docs/adr/0002-remote-upgrade.md) +for the rig's own guard chain (monotonic version, tag must be an ancestor of `main`, rollback when +the miner doesn't come back live, and a six-hour throttle between attempts). + +What the dashboard sends is a proposal, not a target: the host-side runner re-derives the real +latest release from the RigForge release API over Tor and refuses a mismatch, then resolves the +rig's address and bearer from `config.json` — the container never holds the token and cannot choose +what gets installed. The dial to the rig itself rides the mining LAN, like every control-path dial. +A rig already on the latest release short-circuits to a no-op without dialing, so the rig's +six-hour window isn't burned; a repeat click inside that window surfaces as retry-later. + +An upgrade can rebuild the rig's miner (about ten minutes when the XMRig pin changed). The runner +polls the rig to a terminal outcome with a hard cap; past the cap the panel reports the upgrade +still running and the badge clears on its own once the rig's next poll reports the new version. +Upgrades are per-rig only — there is deliberately no "upgrade all" button, so one click can never +stagger rebuilds across the whole farm. + --- ## New to mining? Start with RigForge diff --git a/pithead b/pithead index 27c1bc99..90f2d96d 100755 --- a/pithead +++ b/pithead @@ -86,7 +86,11 @@ if [ "${BASH_SOURCE[0]}" != "${0}" ]; then _STACK_SOURCED=1; fi if [ "$_STACK_SOURCED" = "0" ]; then # Always operate from the directory containing this script, so the stack can be managed from # anywhere (./pithead, an absolute path, cron, systemd, ...). All paths below are relative to it. - SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + # -P (#695): resolve symlinks so every $PWD-derived .env value (CLEARNET_STATE_DIR, CONTROL_DIR, + # ...) renders the same physical path however pithead is invoked — an interactive apply through + # the `current ->` deploy symlink and the systemd control runner on the real dir used to render + # different strings for the same directory, so an unedited preview showed a path "change". + SCRIPT_DIR="$(cd -P "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)" cd "$SCRIPT_DIR" || error "Cannot enter the script directory: $SCRIPT_DIR" # Absolute path to this script — run_chain (#94) re-invokes it once per chained step. PITHEAD_SELF="$SCRIPT_DIR/$(basename "${BASH_SOURCE[0]}")" @@ -4467,8 +4471,8 @@ describe_change() { msg="Dashboard configuration editing disabled (#33) — the control runner units are removed; the dashboard container is recreated." fi ;; - CONTROL_DIR | CADDY_LOG_DIR) - # Fixed paths under ./data — internal, change only when the checkout moves. + CLEARNET_STATE_DIR | CONTROL_DIR | CADDY_LOG_DIR) + # Fixed paths under ./data — internal, change only when the checkout moves (#695). msg="" ;; DASHBOARD_ONION_ENABLED) @@ -5071,9 +5075,13 @@ control_approval_gate() { # # (#349) without a third dry-run. Names only, never values. dashboard.energy (#504) is # config.json-only, so it never appears in the env porcelain — fold a synthetic DASHBOARD_ENERGY # name into the list when that block changed, else an energy-only commit would audit no key. + # Reference defaults merged into both sides (#696), same as the preview leg: the editor + # round-trips the reference-merged form, and materialized defaults are not a change. local keys keys=$(porcelain_keys "$porcelain") - if ! jq -e --slurpfile live "$CONFIG_FILE" '(.dashboard.energy // {}) == ($live[0].dashboard.energy // {})' "$staged" >/dev/null 2>&1; then + if ! jq -e --slurpfile live "$CONFIG_FILE" --slurpfile ref "$REFERENCE_CONFIG" \ + '(($ref[0].dashboard.energy // {}) + ($live[0].dashboard.energy // {})) + == (($ref[0].dashboard.energy // {}) + (.dashboard.energy // {}))' "$staged" >/dev/null 2>&1; then keys="${keys:+$keys }DASHBOARD_ENERGY" fi printf '%s' "$keys" @@ -5172,7 +5180,13 @@ control_preview() { # # arms Apply and the commit lands it in config.json. The approval gate allowlists exactly # this config.json-only block; any OTHER config.json-only delta still refuses (see # control_approval_gate). INFO never flips destructive, so the existing verdict stands. - if ! jq -e --slurpfile live "$CONFIG_FILE" '(.dashboard.energy // {}) == ($live[0].dashboard.energy // {})' "$staged" >/dev/null 2>&1; then + # Compare with the reference defaults merged into BOTH sides (#696): the editor round-trips + # the reference-merged form, so on a config.json that never set dashboard.energy the staged + # copy carries the materialized defaults — an absent block and explicit defaults are the + # same settings, not a change. + if ! jq -e --slurpfile live "$CONFIG_FILE" --slurpfile ref "$REFERENCE_CONFIG" \ + '(($ref[0].dashboard.energy // {}) + ($live[0].dashboard.energy // {})) + == (($ref[0].dashboard.energy // {}) + (.dashboard.energy // {}))' "$staged" >/dev/null 2>&1; then result=$(printf '%s' "$result" | jq '.changes += [{flag:"INFO",key:"dashboard.energy",msg:"Energy calculator settings (dashboard.energy) — electricity price / currency / XMR price updated."}]') fi control_write_result "$cdir/results" "$id" "$result" @@ -5742,6 +5756,184 @@ control_worker_apply() { # control_audit "$auditf" "$id" "$actor" "worker-apply" "accepted" } +control_worker_upgrade() { # + # One-click RigForge upgrade for a single rig (#597) — fuses the two existing templates: + # control_worker_apply's rig resolution/guards/dial (address + bearer from the HOST config, + # never the intent) and control_upgrade's throttled host-side target re-derivation over Tor + # (the container proposes a version; GitHub decides the real target; a mismatch is refused). + # The rig bounds whatever tag we send with its own monotonic + ancestry guards and rolls back + # a build that doesn't come back live — rollback coverage is rig-side (rigforge#322). + local file="$1" id="$2" actor="$3" cdir="$4" + local results="$cdir/results" auditf="$cdir/audit/control.log" + control_audit "$auditf" "$id" "$actor" "worker-upgrade" "started" + _wu_reject() { # — refused before dialing the rig; nothing changed + control_write_result "$results" "$id" "$(jq -n --arg e "$1" '{status:"rejected",error:$e,ts:(now|floor)}')" + control_audit "$auditf" "$id" "$actor" "worker-upgrade" "rejected" + } + _wu_fail() { # — the dial started and did not complete + control_write_result "$results" "$id" "$(jq -n --arg e "$1" '{status:"failed",error:$e,ts:(now|floor)}')" + control_audit "$auditf" "$id" "$actor" "worker-upgrade" "failed" + } + local worker proposed + worker=$(jq -r '.worker // ""' "$file") + # Same charset pin as worker-apply: the name is a config.json lookup key. LC_ALL=C so [!-~] + # is the printable-ASCII BYTE range regardless of the host locale (#185's UTF-8 lesson). + if ! printf '%s' "$worker" | LC_ALL=C grep -qE '^[!-~]{1,128}$'; then + _wu_reject "malformed or missing 'worker' name in the request." + return 0 + fi + proposed=$(jq -r '.version // ""' "$file") + if ! printf '%s' "$proposed" | grep -qE '^v[0-9]+\.[0-9]+\.[0-9]+$'; then + _wu_reject "malformed or missing 'version' in the upgrade request." + return 0 + fi + # Resolve the rig's ADDRESS + BEARER from the HOST's config.json — never the intent (same + # fail-closed contract as worker-apply: only an operator-set host can be a target). + local host cport token + host=$(jq -r --arg n "$worker" "$WORKER_LIST_JQ"'worker_list[] | select(.name == $n) | .host // ""' "$CONFIG_FILE" 2>/dev/null | head -1) + cport=$(jq -r --arg n "$worker" "$WORKER_LIST_JQ"'worker_list[] | select(.name == $n) | .control_port // 8082' "$CONFIG_FILE" 2>/dev/null | head -1) + token=$(jq -r --arg n "$worker" "$WORKER_LIST_JQ"'worker_list[] | select(.name == $n) | .token // ""' "$CONFIG_FILE" 2>/dev/null | head -1) + if [ -z "$host" ]; then + _wu_reject "worker '$worker' has no configured host in workers.list[] (or the deprecated dashboard.workers[]) — set host + control_port + token to upgrade it." + return 0 + fi + if ! printf '%s' "$host" | grep -qE '^[A-Za-z0-9._-]{1,253}$'; then + _wu_reject "worker '$worker' has an invalid host." + return 0 + fi + if ! printf '%s' "$cport" | grep -qE '^[0-9]{1,5}$' || [ "$cport" -lt 1 ] || [ "$cport" -gt 65535 ]; then + _wu_reject "worker '$worker' has an invalid control_port." + return 0 + fi + if [ -z "$token" ]; then + _wu_reject "worker '$worker' has no token in workers.list[] (or the deprecated dashboard.workers[]) — the rig's control API is bearer-mandatory." + return 0 + fi + # Per-drain budget: an upgrade blocks the single-threaded root runner on the rig's build + # (minutes, vs seconds for worker-apply), so exactly ONE dials per drain. v1 is per-worker + # only — no "upgrade all" — and a real fleet upgrade is one rig at a time by design. + if [ "${CONTROL_WU_BUDGET:-0}" -le 0 ]; then + _wu_reject "another worker upgrade is already in this cycle — retry in a moment." + return 0 + fi + CONTROL_WU_BUDGET=$((CONTROL_WU_BUDGET - 1)) + # Host-side re-derivation of the target from the RigForge release API over Tor — load-bearing: + # the rig deliberately computes no "latest" itself (ADR 0002 D4), it bounds the tag we send. + # The derived tag is cached for 10 minutes and the dial itself is stamp-throttled to one per + # 10 minutes (claimed BEFORE the dial, control_upgrade's anti-beacon lesson): a compromised + # container flooding well-formed intents costs at most one GitHub/Tor egress per window, + # while a legitimate rig-after-rig fleet upgrade reuses the cached tag. + local tagf="$cdir/staged/.rigforge-latest-tag" stampf="$cdir/staged/.rigforge-latest-stamp" tag="" + if [ -n "$(find "$tagf" -mmin -10 2>/dev/null)" ]; then + tag=$(cat "$tagf" 2>/dev/null) + fi + if ! printf '%s' "$tag" | grep -qE '^v[0-9]+\.[0-9]+\.[0-9]+$'; then + if [ -n "$(find "$stampf" -mmin -10 2>/dev/null)" ]; then + _wu_reject "a RigForge release lookup was attempted less than 10 minutes ago and no usable tag is cached — retry in a few minutes." + return 0 + fi + touch "$stampf" + local prefix socks rel + prefix=$(env_get NETWORK_PREFIX 2>/dev/null) + [ -n "$prefix" ] || prefix="172.28.0" + socks="${prefix}.25:9050" + if ! rel=$(curl -fsS --max-time 60 --socks5-hostname "$socks" \ + -H 'Accept: application/vnd.github+json' \ + "https://api.github.com/repos/p2pool-starter-stack/rigforge/releases/latest" 2>/dev/null); then + _wu_reject "could not reach the GitHub release API over Tor — nothing was changed. Check './pithead doctor' and retry." + return 0 + fi + tag=$(printf '%s' "$rel" | jq -r '.tag_name // ""' 2>/dev/null) + if ! printf '%s' "$tag" | grep -qE '^v[0-9]+\.[0-9]+\.[0-9]+$'; then + _wu_reject "the GitHub release API returned no usable RigForge release tag — nothing was changed." + return 0 + fi + printf '%s' "$tag" >"$tagf" + fi + if [ "$proposed" != "$tag" ]; then + _wu_reject "requested version $proposed is not the latest published RigForge release ($tag) — reload the dashboard and retry." + return 0 + fi + control_write_result "$results" "$id" "$(jq -n --arg w "$worker" --arg v "$tag" '{status:"running",worker:$w,version:$v,ts:(now|floor)}')" + # POST the upgrade to the rig's control API — direct LAN dial like worker-apply, NOT Tor. The + # body carries the HOST-derived tag only; the token rides one header, never the URL or result. + local url="http://$host:$cport/upgrade" bodyf="$cdir/staged/.$id.body" code + if ! code=$(curl -sS -o "$bodyf" -w '%{http_code}' --max-time 15 \ + -H "Authorization: Bearer $token" -H "Content-Type: application/json" \ + --data "$(jq -n --arg v "$tag" '{version:$v}')" "$url" 2>/dev/null); then + rm -f "$bodyf" + _wu_fail "could not reach worker '$worker' control API at $host:$cport — nothing was changed." + return 0 + fi + if [ "$code" != "202" ]; then + local rig_err + # Rig-supplied text is attacker-influenceable (a compromised rig / LAN MITM); cap it. + rig_err=$(jq -r '.error // ""' "$bodyf" 2>/dev/null | head -c 500) + rm -f "$bodyf" + _wu_reject "worker '$worker' refused the upgrade (HTTP $code): ${rig_err:-no detail}." + return 0 + fi + local change_id + change_id=$(jq -r '.change_id // ""' "$bodyf" 2>/dev/null | head -c 64) + rm -f "$bodyf" + # Poll the rig's /status for THIS change_id's terminal outcome. The cap is a deliberate + # trade: a no-rebuild upgrade (the common case — git checkout + restart) reaches terminal in + # well under 90s, while a pin-change rebuild (~10 min) times out to "accepted" below and the + # badge (#596) clears on its own when the rig reports the new version. Polling the full build + # would hand a hostile/hung rig 12 minutes of the single-threaded root drain per intent + # (sec-review finding) — 90s keeps the stall bound in worker-apply's envelope. There is no + # in-progress status (rigforge#320): a non-matching change_id just means pending. Terminals + # are applied / rolled_back / failed; a rig-side throttle refusal arrives as + # failed+"throttled" and is surfaced as retry-later, not an error. The cap is overridable + # (CONTROL_WU_POLL_CAP) so the stack tests can prove the timeout→accepted fallback in seconds. + local sbody scode status reason deadline=$((SECONDS + ${CONTROL_WU_POLL_CAP:-90})) + while [ "$SECONDS" -lt "$deadline" ]; do + sleep 5 + sbody="$cdir/staged/.$id.status" + if ! scode=$(curl -sS -o "$sbody" -w '%{http_code}' --max-time 10 \ + -H "Authorization: Bearer $token" "http://$host:$cport/status" 2>/dev/null); then + rm -f "$sbody" + continue + fi + [ "$scode" = "200" ] || { + rm -f "$sbody" + continue + } + # Only trust a status whose change_id matches ours — the rig may still be showing a + # PREVIOUS change's terminal state (no in-progress status, rigforge#320). + if [ "$(jq -r '.change_id // ""' "$sbody" 2>/dev/null)" != "$change_id" ]; then + rm -f "$sbody" + continue + fi + status=$(jq -r '.status // ""' "$sbody") + case "$status" in + applied | rolled_back | failed) + # reason is rig-supplied (attacker-influenceable); cap it before it is stored/rendered. + reason=$(jq -r '.reason // ""' "$sbody" | head -c 500) + rm -f "$sbody" + # The rig collapses its refusals into failed+free-text (rigforge#320); its 6h + # anti-beacon throttle is retry-later, not a fault — give it its own status so the + # dashboard renders it calm instead of red. + if [ "$status" = "failed" ] && printf '%s' "$reason" | grep -qi "throttl"; then + status="throttled" + fi + control_write_result "$results" "$id" "$(jq -n --arg s "$status" --arg c "$change_id" --arg w "$worker" --arg v "$tag" --arg r "$reason" \ + '{status:$s,change_id:$c,worker:$w,version:$v,reason:(if $r=="" then null else $r end),ts:(now|floor)}')" + control_audit "$auditf" "$id" "$actor" "worker-upgrade" "$status" + return 0 + ;; + esac + rm -f "$sbody" + done + # Accepted but no terminal status inside the cap — the upgrade is running on the rig. Record + # accepted, not failure: the badge (#596) clears on its own when the rig's next summary poll + # reports the new version ('applied' echoes no version, rigforge#320 — the summary is the + # confirmation of record either way). + control_write_result "$results" "$id" "$(jq -n --arg c "$change_id" --arg w "$worker" --arg v "$tag" \ + '{status:"accepted",change_id:$c,worker:$w,version:$v,note:"upgrade still running on the rig — check the rig if the badge has not cleared in a while",ts:(now|floor)}')" + control_audit "$auditf" "$id" "$actor" "worker-upgrade" "accepted" +} + control_process_request() { # local file="$1" cdir="$2" id action actor size # Refuse a symlinked / non-regular claimed file (graft #437): a symlink dropped in requests/ @@ -5785,6 +5977,7 @@ control_process_request() { # commit) control_commit "$id" "$actor" "$cdir" ;; upgrade) control_upgrade "$file" "$id" "$actor" "$cdir" ;; worker-apply) control_worker_apply "$file" "$id" "$actor" "$cdir" ;; + worker-upgrade) control_worker_upgrade "$file" "$id" "$actor" "$cdir" ;; restart | apply) control_lifecycle "$action" "$id" "$actor" "$cdir" ;; *) control_write_result "$cdir/results" "$id" "$(jq -n '{status:"rejected",error:"unknown action",ts:(now|floor)}')" @@ -5823,6 +6016,9 @@ control_run_pending() { # the runner on a network round-trip, so cap how many dial per drain (the rest reject with a retry # hint). control_worker_apply reads + decrements this in the same shell. CONTROL_WA_BUDGET=5 + # Worker-upgrade budget (#597): an upgrade blocks the runner on a rig build (minutes), so + # exactly one runs per drain; the rest reject with a retry hint. + CONTROL_WU_BUDGET=1 names=$(cd "$cdir/requests" 2>/dev/null && ls -1tr -- *.json 2>/dev/null) || true if [ -z "$names" ]; then log "No pending control requests." @@ -5852,9 +6048,34 @@ control_run_pending() { provision_control_runner() { [ "$OS_TYPE" == "Linux" ] || return 0 command -v systemctl >/dev/null 2>&1 || return 0 - local unit_dir="/etc/systemd/system" + local unit_dir="${PITHEAD_UNIT_DIR:-/etc/systemd/system}" if [ "${DASHBOARD_CONTROL_ENABLED:-false}" != "true" ]; then if [ -e "$unit_dir/pithead-control.path" ] || [ -e "$unit_dir/pithead-control.service" ]; then + # The unit names are box-global but a box can hold several checkouts (release bench: + # live stack + e2e harness + bundle-smoke tmp dirs). Only remove units whose ExecStart + # points at THIS checkout — deleting a sibling's runner strands its dashboard control + # requests unprocessed (the editor hangs at "Previewing…" until that stack's next + # apply/upgrade reinstalls the units). + # Ownership compares PHYSICAL paths: one checkout has two spellings — the `current` + # symlink and the versioned dir it points at (production units carry the versioned + # spelling). A literal $PWD compare would call our own unit foreign and never remove + # it. If the unit's dir is gone, resolve the deepest existing ancestor and keep the + # rest verbatim; an unparseable ExecStart is foreign (fail safe, leave it alone). + if [ -e "$unit_dir/pithead-control.service" ]; then + local owner_dir dir tail + owner_dir=$(sed -n 's|^ExecStart=\(/.*\)/pithead control-run-pending$|\1|p' \ + "$unit_dir/pithead-control.service" | head -n 1) + dir="$owner_dir" tail="" + while [ -n "$dir" ] && [ "$dir" != "/" ] && [ ! -d "$dir" ]; do + tail="/$(basename "$dir")$tail" + dir=$(dirname "$dir") + done + if [ -z "$owner_dir" ] || + [ "$(cd "$dir" 2>/dev/null && pwd -P)$tail" != "$(pwd -P)" ]; then + log "Leaving the dashboard control runner units alone (#33) — they belong to another checkout." + return 0 + fi + fi log "Removing the dashboard control runner units (#33)..." sudo systemctl disable --now pithead-control.path >/dev/null 2>&1 || true sudo rm -f "$unit_dir/pithead-control.path" "$unit_dir/pithead-control.service" @@ -5862,9 +6083,11 @@ provision_control_runner() { fi return 0 fi - # Already installed for this checkout — keep the routine apply sudo-free. - if grep -qs "PathExistsGlob=$CONTROL_DIR/requests/\*.json" "$unit_dir/pithead-control.path" && - grep -qs "ExecStart=$PWD/pithead control-run-pending" "$unit_dir/pithead-control.service"; then + # Already installed for this checkout — keep the routine apply sudo-free. (-F: both paths + # are literals — versioned dirs carry dots (pithead-v1.9.3), and the glob star must not + # read as a regex repeat.) + if grep -qsF "PathExistsGlob=$CONTROL_DIR/requests/*.json" "$unit_dir/pithead-control.path" && + grep -qsF "ExecStart=$PWD/pithead control-run-pending" "$unit_dir/pithead-control.service"; then return 0 fi log "Installing the dashboard control runner (systemd path unit, #33)..." diff --git a/tests/integration/lib.sh b/tests/integration/lib.sh index 94910291..7ad9dddc 100644 --- a/tests/integration/lib.sh +++ b/tests/integration/lib.sh @@ -304,7 +304,7 @@ rig_lock() { # rig_lock [shared] echo "rig busy ($(cat "$hf" 2>/dev/null || echo unknown)) — waiting..." >&2 flock $mode 9 else - echo "miner-0 busy: $(cat "$hf" 2>/dev/null || echo unknown). Retry with RIG_LOCK_WAIT=1 to queue." >&2 + echo "rig busy: $(cat "$hf" 2>/dev/null || echo unknown). Retry with RIG_LOCK_WAIT=1 to queue." >&2 exit 75 # EX_TEMPFAIL — callers can tell "busy, retry later" from a real failure fi fi diff --git a/tests/integration/run.sh b/tests/integration/run.sh index b2b37327..0d2df159 100755 --- a/tests/integration/run.sh +++ b/tests/integration/run.sh @@ -1401,19 +1401,38 @@ _onion_reachable_external() { rx "$snippet" | grep -q "PROBE-OK" } -# Reap the root pithead-control systemd units on the box, unconditionally and idempotently (#477). +# Reap the root pithead-control systemd units THIS checkout installed, idempotently (#477). # The hardening phase installs pithead-control.{path,service} to exercise the #33 spool; the restore # apply is supposed to remove them, but that removal runs EARLY in apply (provision_control_runner) — # before container recreation + the tor restart — so a restore apply that dies partway (render/preflight # failure, or wait_status_ok timing out mid-apply) leaves the ROOT path unit watching the control spool # past the phase and beyond. A later apply is convergent and would clean it, but only if it runs. This -# teardown mirrors provision_control_runner's removal branch (pithead:5242) and runs regardless of the -# restore apply's exit code. No-ops where there's no systemd (macOS/dev). Returns non-zero ONLY if a -# unit survives (e.g. sudo unavailable) so the caller can warn loudly instead of silently passing. +# teardown mirrors provision_control_runner's removal branch and runs regardless of the restore +# apply's exit code. Units owned by ANOTHER checkout are left in place and count as success: the +# unit names are box-global, and on a shared bench the live stack's runner uses them too — reaping +# it strands that dashboard's control requests (config editor stuck at "Previewing…"). rx runs in +# $IT_REMOTE_DIR, so the snippet's working dir is this run's checkout on the box. Ownership +# compares PHYSICAL paths, mirroring the provision_control_runner branch: one checkout has two +# spellings (the `current` symlink vs the versioned dir production units carry). No-ops where +# there's no systemd (macOS/dev). Returns non-zero ONLY if a unit WE own survives (e.g. sudo +# unavailable) so the caller can warn loudly instead of silently passing. _remove_control_units() { rx ' command -v systemctl >/dev/null 2>&1 || exit 0 ud=/etc/systemd/system + if [ -e "$ud/pithead-control.service" ]; then + owner_dir=$(sed -n "s|^ExecStart=\(/.*\)/pithead control-run-pending\$|\1|p" \ + "$ud/pithead-control.service" | head -n 1) + dir=$owner_dir tail="" + while [ -n "$dir" ] && [ "$dir" != "/" ] && [ ! -d "$dir" ]; do + tail="/$(basename "$dir")$tail" + dir=$(dirname "$dir") + done + if [ -z "$owner_dir" ] || + [ "$(cd "$dir" 2>/dev/null && pwd -P)$tail" != "$(pwd -P)" ]; then + exit 0 + fi + fi if [ -e "$ud/pithead-control.path" ] || [ -e "$ud/pithead-control.service" ]; then sudo systemctl disable --now pithead-control.path >/dev/null 2>&1 || true sudo rm -f "$ud/pithead-control.path" "$ud/pithead-control.service" @@ -1957,7 +1976,7 @@ run_rigforge_control() { # The write path resolves the rig's host + token from workers.list[] (#506), or the deprecated # dashboard.workers[] fallback if that's what the box's baseline already carries. Use the - # baseline's descriptor if it already pins this rig's host (the gouda/prod case, whichever shape); + # baseline's descriptor if it already pins this rig's host (the live-box case, whichever shape); # otherwise inject one into workers.list[] from --rig-host + IT_RIG_TOKEN (local-only, so the token # never leaves the bench). No source for either -> skip: we won't drive an edit we can't address, # nor mutate what we can't restore. diff --git a/tests/stack/run.sh b/tests/stack/run.sh index b75862a4..468b5983 100755 --- a/tests/stack/run.sh +++ b/tests/stack/run.sh @@ -39,8 +39,10 @@ run_sourced() { ) } -# A throwaway sandbox dir, cleaned on exit. -SANDBOX="$(mktemp -d)" +# A throwaway sandbox dir, cleaned on exit. Physical path (#695): pithead canonicalizes its +# own directory with pwd -P, so a sandbox spelled through a symlink (macOS /var -> /private/var) +# would render .env paths that no longer string-match the $SANDBOX-based assertions. +SANDBOX="$(cd "$(mktemp -d)" && pwd -P)" trap 'rm -rf "$SANDBOX"' EXIT # A fake docker that records calls and answers the few queries setup/apply make. @@ -4424,6 +4426,21 @@ assert_contains "PITHEAD_CONFIG_FILE override is honoured" "$out" "37890" # nano out="$(cd "$C" && DOCKER_LOG="$CTRL_LOG" PATH="$C/bin:$PATH" ./pithead apply --dry-run --porcelain 2>/dev/null)" assert_eq "without the override, config.json shows no changes" "$out" "" +echo "== black-box: symlink-invoked stack renders physical paths (#695) ==" +# A stack managed through a deploy symlink (`current -> pithead-vX.Y.Z`) must render the same +# .env as one managed from the physical dir: SCRIPT_DIR resolves with pwd -P, so an unedited +# preview through the symlink shows zero changes and an apply never rewrites the $PWD-derived +# paths (CLEARNET_STATE_DIR & co.) to the symlink spelling. +ln -sfn "$C" "$SANDBOX/current-link" +out="$(cd "$SANDBOX/current-link" && DOCKER_LOG="$CTRL_LOG" PATH="$C/bin:$PATH" ./pithead apply --dry-run --porcelain 2>/dev/null)" +assert_rc "dry-run through the symlink exits 0" "$?" "0" +assert_eq "unedited preview through the symlink shows zero changes (#695)" "$out" "" +out="$(cd "$SANDBOX/current-link" && DOCKER_LOG="$CTRL_LOG" PATH="$C/bin:$PATH" ./pithead apply -y 2>&1)" +assert_rc "apply through the symlink succeeds" "$?" "0" +assert_contains "clearnet state dir keeps the physical path" "$(cat "$C/.env")" "CLEARNET_STATE_DIR=$C/data/clearnet-state" +assert_not_contains "the symlink spelling never reaches .env" "$(cat "$C/.env")" "current-link" +rm -f "$SANDBOX/current-link" + echo "== black-box: apply --dry-run is read-only re: node credential generation (#556) ==" # Direct CLI leg: a fresh/hand-edited local-node config with placeholder/empty creds must not have # config.json rewritten by a --dry-run preview — the read-only contract #556 reported broken @@ -4904,6 +4921,26 @@ assert_eq "energy cost landed in config.json" "$(jq -r '.dashboard.energy.cost_p assert_eq "energy currency landed in config.json" "$(jq -r '.dashboard.energy.currency' "$C/config.json")" "EUR" assert_contains "energy commit audits the synthetic key name (#504)" "$(grep '"action":"commit","status":"applied"' "$AUDIT" | tail -n 1)" "DASHBOARD_ENERGY" +# Unedited editor round-trip (#696): the form serves the reference-merged config and posts the +# merged document back, so a save with NO edits must preview as zero changes. The live energy +# block above is partial — the merge materializes the remaining reference defaults (tari_price, +# price_feed) into the staged copy, and defaults against an absent value are the same settings, +# not an "Energy calculator settings updated" row. +UUIDE="55555555-5555-4555-8555-555555555555" +jq -s --arg id "$UUIDE" '{id:$id, action:"preview", actor:"admin", + config:((.[0] | del(._docs)) * .[1])}' "$ROOT/config.reference.json" "$C/config.json" >"$REQS/$UUIDE.json" +run_pending >/dev/null +assert_eq "unedited merged round-trip previews" "$(jq -r '.status' "$RESULTS/$UUIDE.json" 2>/dev/null)" "previewed" +assert_eq "unedited merged round-trip shows zero changes (#696)" "$(jq -r '.changes | length' "$RESULTS/$UUIDE.json" 2>/dev/null)" "0" +# Audit leg of the same contract: committing that unedited round-trip must not record a phantom +# DASHBOARD_ENERGY key — the gate's audit comparison merges the reference defaults too (#696). +printf '{"id":"%s","action":"commit","actor":"admin"}\n' "$UUIDE" >"$REQS/$UUIDE.json" +run_pending >/dev/null +assert_eq "unedited merged round-trip commits" "$(jq -r '.status' "$RESULTS/$UUIDE.json" 2>/dev/null)" "applied" +assert_not_contains "unedited commit audits no phantom DASHBOARD_ENERGY key (#696)" \ + "$(grep '"action":"commit","status":"applied"' "$AUDIT" | tail -n 1)" "DASHBOARD_ENERGY" +rm -f "$RESULTS/$UUIDE.json" "$STAGED/$UUIDE.json" + # NEGATIVE — the #504 security teeth: an energy edit BUNDLED with a change that is NOT on the env # allowlist (monero.rpc_lan_access -> MONERO_RPC_BIND) must be REFUSED. The energy exemption must # not become a carrier for other config: the gate re-derives the env change set host-side and the @@ -5857,7 +5894,7 @@ upg455_fail=$( assert_not_contains "failed upgrade does NOT move the current pointer (#455)" "$upg455_fail" "symlink" echo "== black-box: deploy-box layout (#455) ==" -# A sandboxed source-checkout install whose chain data dirs share one root — the prod/gouda +# A sandboxed source-checkout install whose chain data dirs share one root — the live deploy-box # layout. Proves the default resolution, the apply-time migration, and the upgrade-time # symlink end to end through the real CLI (docker/sudo stubbed). L="$SANDBOX/boxroot/pithead-v9.9.9" @@ -6074,6 +6111,317 @@ assert_eq "worker-apply accept path records the rig's changed_keys" \ assert_contains "worker-apply accept is audited as applied" \ "$(cat "$WA3/audit/control.log")" '"action":"worker-apply","status":"applied"' +# --------------------------------------------------------------------------- +echo "== control channel: worker upgrade fails closed (#597) ==" +# control_worker_upgrade fuses the worker-apply template (rig resolved from the HOST config, never +# the intent) with the stack-upgrade template (host-side target re-derivation, throttled). These are +# the pre-dial fail-closed guards. +WU="$SANDBOX/ctrl597" +mkdir -p "$WU/staged" "$WU/results" "$WU/audit" +cat >"$WU/config.json" <<'EOF' +{ "workers": { "list": [ + { "name": "rig1", "host": "10.0.0.9", "control_port": 8082, "token": "tok-rig1" }, + { "name": "rig2", "host": "10.0.0.8" } +] } } +EOF +wu_case() { #