Skip to content

Add Netgear CM1000/CM1000v2 support and per-modem module structure - #1

Closed
mphuff wants to merge 13 commits into
masterfrom
claude/netgear-modem-monitoring-7fnxj9
Closed

Add Netgear CM1000/CM1000v2 support and per-modem module structure#1
mphuff wants to merge 13 commits into
masterfrom
claude/netgear-modem-monitoring-7fnxj9

Conversation

@mphuff

@mphuff mphuff commented Jul 22, 2026

Copy link
Copy Markdown
Owner

Summary

Adds monitoring for the Netgear CM1000 / CM1000v2 DOCSIS 3.1 modem alongside the existing Arris SB8200, selectable via modem_model = cm1000. To avoid confusing the two vendors inside an "arris"-named file, modem-specific logic is split into a new src/modems/ package, with arris_stats.py reduced to a generic driver.

Changes

Per-modem module structure (src/modems/)

  • common.py — shared HTTP plumbing (HEADERS, http_get).
  • arris_sb8200.py — the existing token-cookie login + BeautifulSoup table parsing, moved out of arris_stats.py unchanged.
  • netgear_cm1000.py — HTTP Basic Auth + regex extraction of the CM1000's pipe-delimited JavaScript tagValueList strings (the CM1000 does not serve static tables). Parses both DOCSIS 3.0 (QAM) and 3.1 (OFDM/OFDMA) channels and strips units into the same stats shape the SB8200 produces.
  • __init__.py — a small registry (get_modem, SUPPORTED).

Each module exposes the same interface — USES_LOGIN_TOKEN / login / get_html / parse — and arris_stats.py selects one via the registry and delegates. The generic config loader, poll loop, and InfluxDB writer are unchanged in behavior.

Reuse

  • The CM1000 writes to the same downstream_statistics / upstream_statistics measurements, so the existing SB8200 Grafana dashboard works without changes.
  • No new config keys were needed — the generic modem_url / modem_auth_required / modem_username / modem_password / modem_model settings cover it, so config.ini / Dockerfile / default_config are untouched and the config-consistency tests stay green.

Docs & tests

  • New CLAUDE.md documenting the architecture, the modem-module interface, the config-consistency invariant, and how to add a new modem.
  • README section covering CM1000 setup (http://192.168.100.1/DocsisStatus.htm, Basic Auth, admin/password).
  • New unit test for the CM1000 JavaScript parser. All tests pass.

Notes / follow-up

  • Netgear firmware revisions vary slightly in how DocsisStatus.htm is formatted. The parser targets the common CM1000-family layout and strips units defensively; if a specific firmware returns no stats, run with --debug and capture the modem's page source so field offsets can be tuned.
  • The Eero mesh sits behind the modem and has no local stats API (cloud-only), so it is out of scope here — DOCSIS stats come entirely from the CM1000v2.

Test plan

  • bash tests/run_tests.sh — all 4 tests pass
  • Module compiles in both run contexts (script from src/, package import from repo root)
  • Validate the CM1000 parser against a real CM1000v2 status page

🤖 Generated with Claude Code

Generated by Claude Code

claude added 3 commits July 22, 2026 03:31
@claude
Add Netgear CM1000/CM1000v2 support and per-modem module structure
90c95dd 
Introduce monitoring for the Netgear CM1000 / CM1000v2 DOCSIS 3.1 modem
alongside the existing Arris SB8200, selectable via modem_model=cm1000.

To keep the two vendors from being confused in an "arris"-named file, split
modem-specific logic into a src/modems/ package. Each modem module owns its
auth flow and status-page parsing behind a small shared interface
(USES_LOGIN_TOKEN / login / get_html / parse), while arris_stats.py stays a
generic driver (config, poll loop, InfluxDB writer) that dispatches via a
registry.

- src/modems/common.py: shared HEADERS + http_get helper
- src/modems/arris_sb8200.py: token-cookie login + BeautifulSoup table parsing
- src/modems/netgear_cm1000.py: HTTP Basic Auth + regex extraction of the
  pipe-delimited JavaScript tagValueList strings (DOCSIS 3.0 QAM and 3.1
  OFDM/OFDMA channels), stripping units into the same stats shape
- CM1000 reuses the existing InfluxDB schema and SB8200 Grafana dashboard
- Add a unit test for the CM1000 JS parser; document setup in README
- Add CLAUDE.md describing architecture, config-consistency invariants, and
  how to add a new modem

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bKShZ5y7RFZzqfbkn6WZq
@claude
Correct CM1000v2 auth and parsing against real modem output
8f2818c 
Rework the Netgear module to match a CM1000v2 verified locally:

- Auth is a webToken form login, not HTTP Basic Auth: GET /GenieLogin.asp to
  scrape the (unquoted) webToken + cookies, POST /goform/GenieLogin, then fetch
  /DocsisStatus.asp with the resulting session. login() now returns an
  authenticated requests.Session. Success is content-sniffed since the modem
  serves a 200 + JS redirect stub when unauthenticated.
- Fix the status-page URL to DocsisStatus.asp.
- Parse the live server-rendered tables by id (dsTable/usTable/d31dsTable/
  d31usTable) covering DOCSIS 3.0 and 3.1 OFDM/OFDMA, with the correct column
  offsets (extra Unerrored-Codewords column downstream; no symbol-rate column
  upstream). Skip un-provisioned "Not Locked" channels (channel_id 0).
- Drop the JavaScript tagValueList parser: on this firmware those strings hold
  stale placeholder data, so parsing them would emit bogus points.
- Rewrite the unit test around a faithful slice of the real DocsisStatus.asp,
  and update README/CLAUDE.md to describe the form login and table parsing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bKShZ5y7RFZzqfbkn6WZq
@claude
Add optional CM1000v2 event-log monitoring
277150b 
Adds an opt-in capability (poll_event_log, default False) to scrape the
Netgear CM1000v2 DOCSIS event log (EventLog.asp) and write it to a separate
InfluxDB measurement for Grafana annotations.

- netgear_cm1000: get_event_log_html() fetches EventLog.asp on the existing
  authenticated session; parse_event_log() extracts the events, which live only
  in the page's JS InitTagValue() pseudo-XML string (<docsDevEventTable>), and
  splits the level into name + priority.
- arris_stats: send_events_to_influx() writes docsis_events points (tags:
  level/priority/event_id; fields: count/description), timestamped with each
  event's first-seen time so re-polling overwrites rather than duplicates. The
  driver calls it only when poll_event_log is set and the modem module exposes
  parse_event_log, so other modems are unaffected and failures never break the
  stats loop.
- Add poll_event_log to config.ini / Dockerfile / default_config in lockstep;
  add unit tests for the event-log parser (including a Critical T3 event and the
  missing-string case); document in README/CLAUDE.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bKShZ5y7RFZzqfbkn6WZq
@mphuff mphuff mentioned this pull request Jul 22, 2026
Merged
3 tasks
claude and others added 10 commits July 24, 2026 23:42
@claude
Add 'file' destination as a no-InfluxDB fallback
4a30619 
Allow destination = file to append each poll's data (and event-log entries) to
file_path as JSON Lines, so the tool can run without a time-series database.

- Factor point-building out of the InfluxDB writers into build_stats_points() /
  build_event_points(), reused by send_to_file() / send_events_to_file().
- Add file_path config key (default modem_stats.jsonl) to default_config /
  config.ini / Dockerfile in lockstep; wire destination == 'file' into the loop.
- Add a unit test for the file writer; document in README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bKShZ5y7RFZzqfbkn6WZq
@claude
Add Docker Compose local stack + spec (InfluxDB + Grafana)
82d9454 
Provide a ready-to-run local monitoring stack and document how to operate it.

- docker-compose.yml: InfluxDB 1.8 + Grafana OSS with named volumes, healthcheck,
  restart: unless-stopped, and an auto-provisioned Grafana datasource. The scraper
  is included as an optional, commented-out service.
- grafana/provisioning/datasources/influxdb.yml: auto-wires Grafana -> InfluxDB.
- .env.example: Compose settings (Grafana password, DB name).
- deploy/com.arris-stats.plist: launchd agent template to keep the host-side
  scraper running (RunAtLoad + KeepAlive).
- docs/local-stack.md: full spec -- architecture, the Docker-Desktop-on-Mac
  networking gotcha (why the scraper runs on the host to reach the modem on the
  home LAN), keep-it-running steps, and carveouts for splitting Grafana or
  InfluxDB onto another host later.
- README: link the spec; .gitignore: local log / jsonl output.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bKShZ5y7RFZzqfbkn6WZq
@claude
Add CM1000v2 Grafana dashboard (auto-provisioned)
fee5c18 
- grafana/provisioning/dashboards/cm1000.json: modern (schemaVersion 39) dashboard
  for Grafana 11-13 with per-channel downstream power, downstream SNR/MER, upstream
  power, a non_negative_derivative "new uncorrectables" rate panel, and an event-log
  table + critical-event annotations from docsis_events.
- grafana/provisioning/dashboards/dashboards.yml: file provider so it auto-loads.
- Pin the provisioned InfluxDB datasource uid to 'influxdb' so the dashboard can
  reference it deterministically.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bKShZ5y7RFZzqfbkn6WZq
@claude
Re-auth on expired session; add heartbeat monitoring
5c46089 
Two operational fixes for long-running local deployments:

- Session expiry is no longer fatal. When a login-token modem (CM1000/SB8200)
  returns no HTML with an existing credential -- almost always an expired
  session (the CM1000 serves a redirect stub) -- the driver now re-authenticates
  in place and retries before falling through to the exit_on_html_error path.
  Previously a routine timeout exited the process.

- Add optional heartbeat_url (default None). After each successful poll the
  scraper pings the URL; point it at a dead-man's-switch (e.g. Healthchecks.io,
  or an AWS API Gateway->Lambda->SNS endpoint) to get alerted when the monitor
  stops running. Uses stdlib urllib; failures are logged and ignored so an
  unreachable heartbeat never disrupts the loop.

Add heartbeat_url to config.ini / Dockerfile / default_config in lockstep;
add a heartbeat no-op test; document heartbeat setup in README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bKShZ5y7RFZzqfbkn6WZq
@claude
Add LaunchDaemon (boot-time) variant for the scraper
7d3656c 
Add deploy/com.arris-stats.daemon.plist to run the scraper at system boot as a
launchd daemon (before/without login), for a dedicated monitor Mac -- an
alternative to the login-time LaunchAgent. Runs as a background service and drops
to the owning user via UserName. Document the login-vs-boot choice in
docs/local-stack.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bKShZ5y7RFZzqfbkn6WZq
@mphuff
Remove local sensitive files
ec8e612
@claude
Add systemd service for running the scraper on Linux
16050c8 
Add deploy/arris-stats.service so the scraper can run as a boot-time systemd
service on Linux (Mint/Ubuntu) -- the Linux analog of the macOS launchd files.
Runs as the invoking user, restarts on failure, logs to the journal. Document
the Linux path in docs/local-stack.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bKShZ5y7RFZzqfbkn6WZq
@mphuff
Merge remote-tracking branch 'origin/claude/netgear-eventlog-monitori…
5b4272c 
…ng' into claude/netgear-eventlog-monitoring
@claude
Fix tests/Docker after config.ini -> config.ini.example rename
1ffd0bb 
The tracked config is now the git-ignored-real / tracked-template split, so:

- test_config_file now reads src/config.ini.example and checks that it lists the
  same KEYS as default_config (values may differ -- it's a filled-in CM1000
  template), instead of asserting exact value equality against the old
  src/config.ini.
- Dockerfile copies config.ini.example -> config.ini at build so
  `--config config.ini` still works in the image (ENV keeps overriding at
  runtime; users can -v mount their own config.ini).
- CLAUDE.md config-consistency section updated for the template split.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bKShZ5y7RFZzqfbkn6WZq
@mphuff
Merge pull request #2 from mphuff/claude/netgear-eventlog-monitoring
2261d34 
Add optional CM1000v2 event-log monitoring
@mphuff mphuff mentioned this pull request Jul 28, 2026
Merged
3 tasks
@mphuff mphuff closed this Aug 5, 2026
@mphuff
mphuff deleted the claude/netgear-modem-monitoring-7fnxj9 branch August 5, 2026 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mphuff @claude