Audit README claims against shipped behaviour (start: Tailscale, Kubernetes, Fleet)

[mcdays94]

Why

README and project description make factual claims about feature behaviour. Over time, either (a) the code evolved but the README didn't, or (b) claims were written at design-time and shipped implementation is narrower than described. This is a tracker issue for systematically auditing those claims and either:

• Tightening the README text to match current behaviour, or
• Filing implementation issues to close the gap where the claim is aspirational but fixable

Claims already spotted (starting points)

1. Tunnels — Tailscale capabilities

README line 143:

Tailscale: Full peer graph with online status, IPs, OS, relay regions, TX/RX bytes, exit node status

Actual behaviour (from internal/collector/tunnels.go):

• When tailscale status --json succeeds with the host daemon: all those fields are populated. Claim is true. ✅
• When the CLI falls back to plain-text parser (version skew between bundled CLI and host daemon — a specific documented case in the code comment, line 162): only IP, hostname, online state, and OS are captured. Relay, TxBytes, RxBytes, Tags, DNSName, MagicDNS are silently missing. ⚠️
• When the socket is unreachable (common Unraid-plugin install without bind-mount): self is nil, dashboard renders nothing at all (see the bug filed separately at Tunnels widget silently drops Tailscale when BackendState != Running (hint never surfaced) #243). ⚠️

Proposed update: add a qualifier — "Full peer graph when the host daemon is reachable via JSON; plain-text fallback (version-skew case) captures a reduced subset."

2. Tunnels — detection coverage claim

README line 142:

Cloudflared: Tunnel status, connection count, ingress routes — detects both host binary and Docker containers
Tailscale: ... — detects both host binary and Docker containers

Actual behaviour:

• Detection is gated on the string "cloudflare" or "tailscale" appearing in the container name OR image (see tunnels.go:65 and :207). A user running Tailscale as a sidecar with a non-standard name (ts-vpn, mullvad-ts, headscale-connector) won't be detected. Similar for Cloudflared.
• Host-binary detection requires the binary to be accessible in the NAS Doctor container's PATH — which means either the Dockerfile bundles it (we bundle both tailscale and cloudflared is NOT bundled, let me verify) or the user custom-builds.

Actually let me re-check: Dockerfile line 34 bundles tailscale but NOT cloudflared. So the "detects both host binary and Docker containers" claim is stronger for Tailscale than Cloudflared. Worth clarifying.

3. Kubernetes claim

README line 71:

Kubernetes: Cluster monitoring for k8s, k3s, EKS, GKE, AKS — nodes (status, disk usage, pod capacity), pods grouped by node with namespace breakdown, deployments, services, PVCs, warning events. In-cluster auto-detection + external token auth

Needs verification:

• Does the k8s section actually render on the dashboard in the shipped build? AGENTS.md has no recent mentions of k8s features. Could be a shipped feature that works silently or a pre-v0.9 feature that still works. Needs a live test on a k3s cluster (user has k3s-gitops).
• Are EKS / GKE / AKS genuinely tested, or is the claim based on "uses kubectl API, should work with any k8s"?

4. Fleet tunnel auto-detection

README line 191:

Auto-detect connection type: LAN (private IP) vs public hostname with tunnel detection (Cloudflare, Tailscale)

Needs verification: does the fleet "Add Server" UI actually distinguish these? Surface-level inspection hasn't confirmed.

5. Prometheus metrics count

README line 567:

All metrics prefixed with nasdoctor_. Full list:
<details><summary>Expand metric list (80+ metrics)</summary>

Claim: 80+ metrics. Also line 203:

Prometheus | Scrape /metrics — 90+ gauges for system, disk, SMART, Docker, UPS, ZFS, services, tunnels, Proxmox, Kubernetes, findings

Two different counts (80+ vs 90+). Actual count from /metrics endpoint should be source of truth. Update both to match.

6. Backblaze failure rate data

Multiple places cite "Backblaze Q4-2025 data, 337k+ drives". Verify the analyzer actually uses this data vintage (not older cached values).

How to tackle

One way: grill-me on this issue to confirm scope + priority, then file individual implementation/docs tickets per gap. Another way: batch-fix the documentation (lighter) and file only the code-change tickets for actual feature gaps.

Not urgent. Priority v0.9.8 / v0.10.0 alongside the other docs-hygiene tasks.

Related

• Tailscale-specific detection+rendering bug: Tunnels widget silently drops Tailscale when BackendState != Running (hint never surfaced) #243 (filed separately; the README claim is the symptom of it being undocumented)
• v0.9.8 theme: "observability + cache-invalidation hygiene + honest defaults" — this audit extends the theme to "honest documentation"

Owner / next step

No one assigned yet. When someone picks this up, suggested approach: open this issue and walk the README top-to-bottom, ticking off each claim against a live snapshot API response from a real install, filing sub-issues for each mismatch.

[mcdays94]

Progress update (v0.9.8 session, 2026-04-23):

Addressed in #243 (merged via PR #245):

• Tailscale peer-graph claim (README line 143) — tightened with socket-mount qualifier and plain-text fallback caveat
• Tailscale 'detects Docker containers' claim (README line 142) — tightened, with env-var configurable container names documented

Split into AFK sub-tickets:

• README claim audit: Cloudflared 'detects host binary' is misleading (binary not bundled) #251 — Cloudflared 'detects host binary' claim (binary not bundled; host-binary detection silently fails in default install)
• README claim audit: Prometheus metric count disagrees with itself (80+ vs 90+) #252 — Prometheus metric count reconciliation (README says 80+ in one place, 90+ in another, template says 30+)

Remaining on this tracker (investigation-heavy, not AFK):

• Kubernetes claim verification (requires live test on a k3s cluster)
• Fleet auto-detect-tunnel-type claim verification
• Backblaze data vintage check