Skip to content

Install a verified Ookla CLI for adaptive SQM - #1093

Open
Jason-Morcos wants to merge 5 commits into
Ozark-Connect:devfrom
Moballo-LLC:agent/fix-sqm-speedtest-bootstrap
Open

Install a verified Ookla CLI for adaptive SQM#1093
Jason-Morcos wants to merge 5 commits into
Ozark-Connect:devfrom
Moballo-LLC:agent/fix-sqm-speedtest-bootstrap

Conversation

@Jason-Morcos

@Jason-Morcos Jason-Morcos commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Problem

Adaptive SQM currently pipes the packagecloud repository bootstrap into a shell, removes any existing speedtest package, and installs an unpinned candidate with apt. That can fail when UniFi OS changes its Debian base, mutates a vendor-managed package surface, and makes the calibration binary drift independently of Network Optimizer.

Fix

  • install the official Ookla archive directly into /data/network-optimizer/bin
  • pin the same 1.2.0.84 (ea6b6773cf) build selected by the existing packagecloud install path
  • centralize the reviewed archive and executable SHA-256 values so installation, calibration, and status checks share one source of truth
  • select reviewed aarch64, armhf, or x86_64 artifacts and verify both archive and executable SHA-256 values
  • bound the HTTPS download, extract in a private staging directory, and atomically activate the validated executable
  • log a failed Ookla installation and continue deploying the SQM scripts and cron entries; calibration remains fail-closed until the managed binary validates
  • invoke and report health for that exact path/build without removing a UniFi package
  • report jq separately alongside the existing bc dependency status

Validation

  • full Release solution build against current dev: succeeded
  • SQM test project: 272 passed, including best-effort installation, checksum-source, managed invocation, and generated Bash syntax coverage
  • focused Web deployment-status test: passed
  • git diff --check: passed

GitHub did not schedule a new Actions run for this head because the upstream workflow currently filters pull-request events to main, while this pull request targets dev. The unfiltered Web suite is not claimed as locally green on macOS: two existing Vodafone AES-CCM tests are unsupported on this host, and the remaining run exceeded the bounded local test window.

No gateway package, SQM script, cron entry, service, or live installation was changed while validating this fix.

@Jason-Morcos
Jason-Morcos force-pushed the agent/fix-sqm-speedtest-bootstrap branch from 438c172 to 6f24e66 Compare August 3, 2026 04:11
@tvancott42

Copy link
Copy Markdown
Collaborator

Couple things need fixing here, and then I can work this in for sure. The Ookla CLI install shouldn't fail the whole install script, we want it to log and continue as it does today with the apt install attempt.

The other thing, let's centralize the hashes here:

Location Hashes Notes
src/NetworkOptimizer.Sqm/ScriptGenerator.cs L115-126 (boot script, Section 1) archive + binary, 3 arches Only place the archive hashes are needed
src/NetworkOptimizer.Sqm/ScriptGenerator.cs L320-323 (calibration script) binary, 3 arches Re-checked before every calibration run
src/NetworkOptimizer.Web/Services/SqmDeploymentService.cs L127 (status command) binary, 3 arches Inlined into the one-line shell string

@tvancott42
tvancott42 changed the base branch from main to dev September 6, 2026 22:05
tvancott42 and others added 5 commits September 7, 2026 10:16
…scrub

Dragging the playhead showed one set of figures a second after settling, then
another two seconds later. The card debounces seeks, but an answer already in
flight for an earlier position was rendered when it landed, then noticed the
playhead had moved and asked again. A click seeks once, so it never showed it.

- The card cancels the in-flight answer when the playhead (or the selected
  WANs, or their rate at the playhead) changes, and discards an answer that
  completes after cancellation instead of rendering it.
- The page's historic snapshot carries a sequence number, so a slow older seek
  cannot overwrite a newer one.
- The WAN line reads the rate the tiles carry rather than the answer's echo of
  it, and a snapshot landing after the card asked re-answers the split, which
  is allocated against that rate at the playhead.
The lag after a seek was not the debounce and not InfluxDB (every historic
query there answers in 10-40 ms). It was three things stacked:

- At the playhead the card asked the console for two DPI reports (the quarter
  hour and the 24 h exclusion window) before looking at the measured WAN, and
  the console gate spaces those calls two seconds apart. On a site the gateway
  agent measures the split never reads them. Measured first now, and a covered
  site skips the DPI, the history, and first-seen entirely. An uncovered site
  fetches all three as before.
- The map's historic fetch asked device_health for every node with a MAC, one
  at a time, and clients write none: around a hundred empty round trips per
  seek. Infrastructure nodes only now.
- The page renders on its own 3 s tick before a seek's snapshot lands, so the
  card asked with the previous instant's WAN rate, then cancelled and asked
  again when the snapshot arrived. The page now stamps the instant its rate
  describes and the card waits for it to match the playhead before asking.
  Debounce cut from 400 to 150 ms; the map already spaces seeks 500 ms apart.

The service's catch blocks swallowed a cancelled query as a per-stage miss and
the request ran on to a degraded answer nobody would see; they let our own
cancellation through now. A playhead answer logs its stage timings at Debug.
Static/bench-verified on the UCG-Fiber, the first 6.0.x image for that line:
qca-ssdk.ko .text byte-identical to the live-verified 6.0.5 SSDK, every other
common kernel module .text-identical, vermagic unchanged, all boot-tweak
userland present including MongoDB (unifi-perf-tweaks docs/compat-6.0.7.md).
The UXG ceiling stays at 6.0.5.
@Jason-Morcos
Jason-Morcos force-pushed the agent/fix-sqm-speedtest-bootstrap branch from 6f24e66 to 6f400e1 Compare September 7, 2026 18:06
@Jason-Morcos

Copy link
Copy Markdown
Contributor Author

Addressed both requests in 6f400e1:

  • The managed Ookla installer now runs as a best-effort operation. Any prerequisite, download, checksum, extraction, or activation failure is logged and the boot script continues creating the SQM scripts and cron entries. The later calibration script still fails closed before changing TC rates unless the exact managed binary validates.
  • The six reviewed hashes now have one source of truth in ScriptGenerator; the boot installer, calibration script, and deployment-status command interpolate those constants instead of duplicating literals.

I also refreshed the change onto current dev. The full Release solution builds, all 272 SQM tests pass, the focused Web deployment-status test passes, and generated Bash syntax is covered. GitHub did not schedule a new check because the current workflow filters pull-request events to main while this PR targets dev; the PR description records that limitation and the bounded macOS Web-suite result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants