Skip to content

Test suite rebuild - #11322

Open
NomDeTom wants to merge 33 commits into
meshtastic:developfrom
NomDeTom:test-suite-rebuild
Open

Test suite rebuild#11322
NomDeTom wants to merge 33 commits into
meshtastic:developfrom
NomDeTom:test-suite-rebuild

Conversation

@NomDeTom

@NomDeTom NomDeTom commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Native test suite: per-suite isolation, leftover-state reporting, and the coverage that would have caught it

Longitudinal PR off develop, phased, with a commit per stage. Test and harness work plus two small production changes, each isolated and explained below.

Why

test_admin_radio failed four assertions that had nothing to do with the code they were testing. The cause was 22 suites upstream.

Every native suite that constructs a NodeDB loads and saves ~/.portduino/default/prefs/nodes.proto, config.proto, channels.proto, module.proto, device.proto, warm.dat, transmit_history.dat — and nothing ever cleared it. test_nodedb_blocked deliberately fills the database with MAX_NUM_NODES - 2 favourited nodes to test the protected-node cap; that is correct, and it is what the test is for. An incidental removeNodeByNum() in the next test persisted it. Twenty-two suites later test_admin_radio loaded a full, all-protected database in which getOrCreateMeshNode() can only return NULL, so the node under test was never created and the protected-bit assertions read false.

Three things made this expensive:

  • The poison outlived the run. A filtered rerun of test_admin_radio alone kept failing, in a different worktree, with a fresh .pio — the state is keyed on $HOME.
  • Per-run isolation would not have fixed it. A full run from a completely empty $HOME reproduces it, because the poisoning is generated within one run.
  • Nothing looked at what a suite left behind. There was no signal at all until a distant suite failed for reasons that looked unrelated.

CI never saw it, and not because CI is careful: its area_rules happen to run admin first, while PlatformIO's local discovery is reverse-alphabetical and runs it last. Neither order was chosen. CI was green by accident, and would have broken the moment anyone reordered the areas.

A test run could also rewrite a real meshtasticd node database on the same machine.

What landed

Phase 0 — one canonical node cap

The native cap was stated in four places that disagreed, and that disagreement already caused a wrong diagnosis: the cap was computed as 248 from mesh-pb-constants.h, so a genuinely saturated 200-node database looked arithmetically impossible. The real cap is 198.

On portduino MAX_NUM_NODES is not a compile-time constant at all — the variant defines it as portduino_config.MaxNodes, resolved at runtime, default 200, settable per host via General: MaxNodes. variant.h is reached before mesh-pb-constants.h, so that header's ARCH_PORTDUINO branch never fires and its plausible-looking 250 is dead code.

#error-guarding that dead branch found a real defect. Eight translation units reach mesh-pb-constants.h without configuration.hSerialConsole.cpp, StreamAPI.cpp, PacketAPI.cpp, ServerAPI.cpp, PiWebServer.cpp, ServiceEnvelope.cpp, MeshtasticOTA.cpp and test/TestUtil.cpp — so each was compiling with a different MAX_NUM_NODES, and therefore a different PACKETHISTORY_MAX, than the rest of the build. Each now includes configuration.h first. It cannot be included from mesh-pb-constants.h itself: that reaches SerialConsole.h via DebugConfiguration.h and closes a cycle.

Also names the bare 250 in getMaxNodesAllocatedSize() as NODEDB_MIGRATION_LOAD_CEILING — a decode allowance for files from larger-cap firmware, not a cap — and fixes the doc that named the wrong source and a "10-250" range wrong for native.

Phase 1 — per-suite scratch $HOME, and leftovers as an outcome

bin/pio-test-isolate.sh runs each suite in its own scratch $HOME, registered as test_testing_command for env:native and env:coverage so a bare pio test and CI get the same boundary, not just bin/run-tests.sh. Mutation inside a suite is free; carrying state across a suite boundary is now impossible by construction rather than by policy.

Overriding HOME around the already-built binary rather than around pio also sidesteps the blocker that a bare HOME= breaks pio's own ~/.platformio/penv/bin/pio lookup.

Leftovers are reported on a second axis, PASS/FAIL × CLEAN/DIRTY, because an unintended write has no matching assertion by definition — nobody writes TEST_ASSERT for a save they do not know is happening. Five decisions, each because the obvious implementation is brittle enough to get deleted:

  1. The harness asserts it, not the test, so it applies to every suite without the author opting in.
  2. Only the set of changed paths is asserted, never contents. Hashes answer "did this change?" and nothing more. Content baselines over protobuf bytes are snapshot tests — add a field to NodeInfoLite and every recorded hash churns, which is how snapshot suites become an --update-all ritual and then noise.
  3. Granularity follows the state flag, so the two ship together. Per-test by default (TestUtil redefines RUN_TEST to checkpoint after each test, naming the exact test that dirtied things); suite boundary for state=per-suite, where carrying state across test cases is the declared behaviour.
  4. A declared write that does not happen is MISSING, reported separately from DIRTY. It catches silently-broken persistence — the shape of the upstream TAK config bug, where a has_ flag was never set so the save wrote nothing and no test noticed. A warning for now, since some declared writes are conditional.
  5. Guard the guard. state_assert_empty() refuses to run a suite against a sandbox that is not empty — otherwise the after-diff measures against the wrong baseline and reports CLEAN while meaning nothing — and bin/test-state-check.sh drives the real wrapper with fixtures asserting CLEAN / CLEAN / DIRTY / MISSING plus both directions of the empty assertion.

Declarations live in one central test/state-manifest.tsv — suite, flags, mandatory reason — so every opt-out is visible in one diffable list; per-suite files hide growth. The run prints how many suites declare non-default handling, so the number creeping up is visible without anyone auditing the file. --write-manifest proposes entries for a human to paste and justify; it never applies them and neither does CI, because an auto-accepted baseline is the same rot as an auto-updated snapshot.

Graded AMBER, not RED: with isolation in place, DIRTY means "undeclared", not "dangerous", and a check that lands red on day one gets switched off.

Also: a signal name from the runner is not a crash. exit(UNITY_END()) returns the failure count, and PlatformIO renders a non-zero exit code as a POSIX signal — 4 failures prints SIGILL, 5 prints SIGTRAP, and the suite is reported [ERRORED]. That cost hours of hunting a memory bug that did not exist, on an env that carries no sanitizer at all. The runner now says so inline.

Phase 2 — test_admin_radio fixtures per test

setUp() did if (!nodeDB) nodeDB = new NodeDB(); and never deleted it, so 83 of 85 tests shared one never-reset database and never restored config, owner, devicestate or channelFile. The fixture that does restore them was opt-in and armed by exactly two tests. The comment claiming the rest "set their own config/region state and are unaffected" was not true — the admin handlers under test write all four.

Every test now goes through the fixture. All 85 passed, so nothing was silently relying on the shared state. It costs about 7% of the suite's runtime, worth paying to write the phase-3 tests against a clean fixture rather than 83 tests' residue.

Phase 3 — the coverage

  • test_fscommon_getfiles, 8 tests. getFiles() runs on every phone sync and nothing asserted its cap, depth limit, wasLimited paths, overlong-path rejection or capacity release. All eight describe today's behaviour and pass on arrival — fix(phone-api): forward-port manifest-scan skip + bounded getFiles (#10754, #10757) to develop #10778 already landed the real fix — which is the point: this is the baseline a later change has to leave alone.
  • Three admin node-DB tests. set_favorite_node, set_ignored_node and toggle_muted_node persist a NodeInfoLite bit and must not reconfigure the radio. Pure characterization — they pass on develop — but that reconfigure is the path implicated in the WisMesh Tag favourite crash, and develop asserted nothing about it.
  • menuHandler::toggleNodeMuted() extracted from its banner-callback lambda, which is why nothing in MenuHandler.cpp was reachable from a test. Behaviour-neutral by construction.

Phase 4 — randomised order, last and deliberately

Randomising an order-dependent suite set does not find bugs so much as convert a silent pass into intermittent red, and the first instinct is to revert the randomisation rather than fix the coupling. Phases 1–2 removed the coupling; this keeps it removed.

--shuffle / --seed <n>, seeded from HEAD by default: one order per commit, so a red is replayable and attributable to the diff instead of flaky. The seed is printed at the start and carried into the RESULT: line, so a verdict is replayable from that line alone; the full order prints on failure, because for an order-dependent failure the order is the diagnostic. The shuffle is a Fisher-Yates over a MINSTD generator rather than awk's rand(), whose sequence differs between gawk and mawk — a seed that does not reproduce the same order on another machine is not a seed.

CI shuffles its area order the same way, seeded from GITHUB_SHA and printed with the command to replay it locally.

Lessons learnt along the way

  • PlatformIO ignores the order of -f flags. list_test_names() walks test_dir with os.walk(); filters only select. So per-suite isolation did not need one invocation per suite — a test_testing_command wrapper gets it in a single invocation, which is the faster mechanism (measured: 54s vs 68s for three suites, ~4.7s of pio startup per extra invocation). But randomised order does need per-suite invocations. That ~4.7s/suite is the real price of phase 4, and it is worth stating rather than discovering later.
  • ARDUINO is defined on portduino, so it is the wrong guard for host-only test code. The state-checkpoint code silently compiled to a no-op stub until it was guarded on ARCH_PORTDUINO.

Results

Run Verdict
-e native, pristine HOME (run 1) AMBER 16 suite(s) left undeclared shared stateall 44 suites PASS
-e native, normal HOME (run 2, after populating the manifest) GREEN 44/44 suites passed, all CLEAN
-e native --shuffle (seed 809606438) GREEN 44/44, all CLEAN
-e native --seed 20260801 GREEN 44/44, all CLEAN
-e coverage (informational) RED 2 failedpre-existing, not this branch

The 16 undeclared suites on run 1 match the measured baseline exactly. Each now carries a manifest entry with a reason, taken from --write-manifest output rather than guessed.

The coverage failures are pre-existing and unrelated: test_packet_signing's B11/B12, which fail on this host under ASan on develop too. They are a latent uninitialised-read/UB in the PKI encode path that this host's ASan exposes and CI's older runtime does not; ruled out as prefs-related by running from a pristine HOME. 43 of 44 suites pass under coverage.

Not in this PR

Excluded Why
reserve() outside the __cpp_exceptions guard; file.name() null guard Production changes; stacked separately so the FS suite here can show they alter nothing observable. Neither is coverable natively — exceptions are on, so the #else branch is not compiled, and no in-tree backend returns a null name.
Narrowing the mute toggle's saveToDisk() mask The defect phase 3c characterizes. Today one NodeInfoLite bit rewrites all five segments, and the test asserts that deliberately, with a comment naming it. When the narrowing lands the assertion is expected to change — that diff is the point.

🤝 Attestations

  • I have tested that my proposed changes behave as described.
  • I have tested that my proposed changes do not cause any obvious regressions on the following devices:
    • Heltec (Lora32) V3
    • LilyGo T-Deck
    • LilyGo T-Beam
    • RAK WisBlock 4631
    • Seeed Studio T-1000E tracker card
    • Other (please specify below)

Bench test suite only.

Summary by CodeRabbit

  • New Features
    • Native deployments support runtime-configured node capacities with validation for unsupported values.
    • Node muting now persists reliably and safely ignores unknown nodes.
  • Bug Fixes
    • Standardized node and packet ID formatting in menus and diagnostic logs.
    • Improved test isolation and detection of undeclared filesystem changes.
  • Tests
    • Added coverage for file enumeration, node controls, defaults, and shared-state handling.
    • Added reproducible suite shuffling and clearer failure diagnostics.
  • Documentation
    • Expanded native testing guidance and clarified node-capacity behavior.

Also finishes the node-ID log-format cleanup started in #10798. That PR documented the convention (0x%08x in logs, !%08x in user-facing display) but did not retro-apply it, leaving 22 call sites on the bare %08x form. Those are corrected here, and a note-level trunk linter (node-id-format) now holds the line so it cannot drift again.

@NomDeTom
NomDeTom requested a review from thebentern August 1, 2026 12:02
@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The pull request adds native test isolation, deterministic suite ordering, filesystem-state verdicts, expanded native coverage, runtime Portduino node-capacity validation, centralized node mute handling, and node-ID and Unity-exit linting.

Changes

Native testing infrastructure

Layer / File(s) Summary
Filesystem state isolation and classification
bin/lib/test-state.sh, bin/pio-test-isolate.sh, test/TestUtil.*, variants/native/portduino/platformio.ini, bin/test-state-check.sh
Native suites use isolated homes. Filesystem changes and surviving processes are recorded and classified.
Deterministic ordering and verdict reporting
.github/workflows/test_native.yml, bin/lib/shuffle.sh, bin/run-tests.sh, test/README.md, AGENTS.md
The workflow and runner support seeded ordering, replay commands, state-aware verdicts, preserved diagnostics, and sanitizer reporting.
Native suite coverage and termination
test/test_admin_radio/test_main.cpp, test/test_default/test_main.cpp, test/test_fscommon_getfiles/test_main.cpp, test/test_*/**, test/native-suite-count
Tests cover node metadata, traffic scaling, filesystem enumeration, and explicit Unity process termination. The canonical suite count is 44.

Platform configuration and diagnostics

Layer / File(s) Summary
Runtime capacity and configuration includes
src/mesh/mesh-pb-constants.h, src/mesh/NodeDB.h, src/platform/portduino/ConfigCheck.cpp, docs/node_info_stores.md, src/**
Portduino capacity comes from configuration. NodeDB retains a separate 250-node migration ceiling and validates excessive MaxNodes values. Configuration headers precede dependent headers.
Node formatting and menu behavior
src/graphics/draw/MenuHandler.*, src/mesh/PacketHistory.cpp, src/modules/*.cpp, test/test_admin_radio/test_main.cpp
Node mute changes use toggleNodeMuted(). Node and packet identifiers use consistent hexadecimal formatting.
Node-ID and Unity-exit linting
bin/lint-node-id-format.sh, bin/lint-unity-exit.sh, .github/node-id-format-allowlist.txt, .trunk/trunk.yaml, bin/test-lint-unity-exit.sh
Non-blocking linters detect bare node-ID formats and bare UNITY_END() calls. Self-tests cover Unity-exit detection.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

Suggested labels: cleanup, github_actions

Suggested reviewers: thebentern

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 44.83% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately identifies the main change as a rebuild of the native test suite, although it omits specific implementation details.
Description check ✅ Passed The description clearly covers motivation, implementation phases, results, exclusions, and testing status, including the required device-testing limitation.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@NomDeTom
NomDeTom requested a review from Copilot August 1, 2026 12:02
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

⚡ Try this PR in the Web Flasher

Note

Building this pull request… the flash button, badges and supported-board
list will appear here automatically once CI finishes.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Not ready to approve

Two small correctness issues were found in new code paths (log ID formatting in MenuHandler::toggleNodeMuted() and regex-based MISSING detection in bin/lib/test-state.sh) that should be fixed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

This PR hardens the native/portduino unit-test harness against persisted $HOME state leakage by isolating each suite in its own scratch home, reporting undeclared filesystem writes as an outcome axis, and adding targeted regression/characterization coverage (FS manifest bounds, admin node-metadata saves, and menu mute toggle). It also fixes a portduino-specific build hazard where MAX_NUM_NODES could silently differ per translation unit when mesh-pb-constants.h was included without configuration.h.

Changes:

  • Add per-suite $HOME isolation via a PlatformIO test_testing_command wrapper, plus shared-state reporting (CLEAN/DIRTY/MISSING) driven by a central test/state-manifest.tsv.
  • Rebuild/expand native coverage: new test_fscommon_getfiles suite, new Default::getConfiguredOrDefaultMsScaled(..., TrafficType) tests, and per-test fixture isolation + new admin/menu metadata tests in test_admin_radio.
  • Enforce correct portduino node-cap semantics by #error-guarding the ARCH_PORTDUINO branch in mesh-pb-constants.h and fixing include order in affected translation units; extract menuHandler::toggleNodeMuted() for testability.
File summaries
File Description
variants/native/portduino/platformio.ini Registers per-suite isolation wrapper for native and coverage test envs.
test/TestUtil.h Redefines RUN_TEST to checkpoint test-state after each Unity test.
test/TestUtil.cpp Adds optional per-test filesystem change attribution (portduino-only).
test/test_fscommon_getfiles/test_main.cpp New suite covering bounded getFiles() behavior and capacity-release idiom.
test/test_default/test_main.cpp Adds tests for region-throttle overload of getConfiguredOrDefaultMsScaled.
test/test_admin_radio/test_main.cpp Makes globals fixture per-test; adds admin metadata + menu mute toggle tests.
test/state-manifest.tsv Central allowlist of deliberate per-suite persisted writes + reasons.
test/README.md Updates guidance: isolation, CLEAN/DIRTY, shuffling, and failure interpretation.
test/native-suite-count Bumps canonical suite count (43 → 44).
src/SerialConsole.cpp Ensures configuration.h is included before headers relying on portduino caps.
src/platform/esp32/MeshtasticOTA.cpp Same include-order fix for portduino cap correctness.
src/mqtt/ServiceEnvelope.cpp Same include-order fix for portduino cap correctness.
src/mesh/StreamAPI.cpp Same include-order fix for portduino cap correctness.
src/mesh/raspihttp/PiWebServer.cpp Same include-order fix for portduino cap correctness (in portduino-only TU).
src/mesh/NodeDB.h Names NODEDB_MIGRATION_LOAD_CEILING and clarifies it is not the runtime cap.
src/mesh/mesh-pb-constants.h #error if ARCH_PORTDUINO reaches a compile-time MAX_NUM_NODES definition.
src/mesh/api/ServerAPI.cpp Same include-order fix for portduino cap correctness.
src/mesh/api/PacketAPI.cpp Same include-order fix for portduino cap correctness.
src/graphics/draw/MenuHandler.h Exposes toggleNodeMuted() for test reachability.
src/graphics/draw/MenuHandler.cpp Extracts mute toggle logic into toggleNodeMuted().
docs/node_info_stores.md Updates documentation to reflect native/portduino runtime node cap semantics.
bin/test-state-check.sh Adds self-test fixtures for the shared-state checker.
bin/run-tests.sh Adds shared-state grading, suite shuffling, seed replay, and manifest proposals.
bin/pio-test-isolate.sh New wrapper: per-suite scratch $HOME + leftover-state classification/retention.
bin/lib/test-state.sh New shared helpers for fingerprinting/classifying suite filesystem changes.
AGENTS.md Updates native test-failure debugging guidance (CLEAN/DIRTY axis, signals, shuffle).
.github/workflows/test_native.yml Adds deterministic CI area-order shuffle with replayable seed controls.
.github/copilot-instructions.md Documents new isolation/state-reporting model and portduino cap semantics.
Review details
  • Files reviewed: 24/28 changed files
  • Comments generated: 1
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Comment thread src/graphics/draw/MenuHandler.cpp

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🧹 Nitpick comments (2)
src/mesh/mesh-pb-constants.h (1)

115-123: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Shorten the new C++ comments.

The new explanations exceed the repository's two-line limit and repeat the same runtime-cap rationale. Keep each comment to one or two lines. Move detailed background to docs/node_info_stores.md if needed.

  • src/mesh/mesh-pb-constants.h#L115-L123: reduce the include-order explanation to one or two lines and retain only why configuration.h must precede this header.
  • src/mesh/NodeDB.h#L24-L29: reduce the migration-ceiling explanation to one or two lines.
  • src/mesh/NodeDB.h#L528-L533: reduce the load-ceiling calculation explanation to one or two lines.

As per coding guidelines, keep C++ comments minimal and avoid restating code behavior.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/mesh/mesh-pb-constants.h` around lines 115 - 123, Shorten the comments at
src/mesh/mesh-pb-constants.h:115-123, src/mesh/NodeDB.h:24-29, and
src/mesh/NodeDB.h:528-533 to one or two lines each. In the mesh-pb-constants.h
comment, retain only that configuration.h must be included first for the runtime
MAX_NUM_NODES definition; in NodeDB.h, retain only the essential
migration-ceiling and load-ceiling calculation context. Remove repeated
rationale and move any necessary background to docs/node_info_stores.md.

Source: Coding guidelines

test/TestUtil.cpp (1)

69-107: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider std::filesystem instead of manual dirent/stat recursion.

fileFingerprint and walk reimplement a recursive directory walk with opendir/readdir/lstat. Since this code only compiles under ARCH_PORTDUINO (host builds), std::filesystem::recursive_directory_iterator and std::filesystem::file_size/hashing over an ifstream would remove the manual pointer management and work uniformly across the Linux/macOS/Windows native targets this project already builds for.

This is optional: the current code is correct as written.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/TestUtil.cpp` around lines 69 - 107, Optionally replace the manual POSIX
traversal in walk with std::filesystem::recursive_directory_iterator, preserving
relative regular-file keys and skipping directories or inaccessible entries as
the current implementation does. Update fileFingerprint to use standard C++ file
I/O, retaining the existing hash algorithm and return behavior for unreadable
files; limit the change to the host-only ARCH_PORTDUINO code path.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/test_native.yml:
- Around line 262-265: Update the seed initialization in the workflow step
around seed_input so the suite_order_seed input is passed through the step’s env
configuration rather than interpolated into the shell script. Read that
environment variable as a quoted shell value, preserving the existing
empty-check and explicit-seed behavior while preventing shell metacharacters
from being executed.

In `@bin/pio-test-isolate.sh`:
- Around line 82-95: Replace the `paste -sd'; ' -` join in the `PER_TEST_DETAIL`
assignment with a delimiter-safe approach that separates every listed undeclared
path using the exact `"; "` sequence. Preserve the existing five-item truncation
and additional-count suffix behavior.

In `@bin/run-tests.sh`:
- Around line 216-230: Eliminate the duplicated MINSTD Fisher-Yates
implementation by extracting shuffle_suites into a shared helper while
preserving bin/run-tests.sh as the canonical behavior. Update bin/run-tests.sh
at lines 216-230 to source/invoke that helper, and replace the inline awk block
in .github/workflows/test_native.yml at lines 275-286 with the same shared
helper call so both paths remain byte-identical.

In `@bin/test-state-check.sh`:
- Around line 19-22: Update the directory setup in test-state-check.sh so the cd
to ROOT_DIR is guarded and the script exits immediately with a clear error if
changing directories fails. Preserve the existing SCRIPT_DIR and ROOT_DIR
resolution.

In `@docs/node_info_stores.md`:
- Around line 64-66: Update the capacity documentation in
docs/node_info_stores.md so the property matrix includes the ESP32-S3 100-node
tier or explicitly identifies its values as representative, while preserving
consistency with the platform table. Document that the host default is 200 nodes
and that wasm_config_apply() overrides portduino_config.MaxNodes to 80 for WASM,
including the affected footnote or platform notes.

In `@src/graphics/draw/MenuHandler.cpp`:
- Around line 2934-2947: Update the LOG_INFO call in
menuHandler::toggleNodeMuted to format nodeNum as a lowercase, zero-padded
hexadecimal identifier with the required 0x prefix, using the `0x%08x` format.

In `@src/mesh/NodeDB.h`:
- Around line 532-533: Update the MaxNodes validation before the size_t
conversion used by getMaxNodesAllocatedSize and the loadCeiling calculation:
clamp or reject positive values above the intended runtime maximum so they
cannot become an oversized MAX_NUM_NODES value. Preserve the existing handling
for nonpositive values and ensure loadCeiling is computed only from the bounded
node count.

---

Nitpick comments:
In `@src/mesh/mesh-pb-constants.h`:
- Around line 115-123: Shorten the comments at
src/mesh/mesh-pb-constants.h:115-123, src/mesh/NodeDB.h:24-29, and
src/mesh/NodeDB.h:528-533 to one or two lines each. In the mesh-pb-constants.h
comment, retain only that configuration.h must be included first for the runtime
MAX_NUM_NODES definition; in NodeDB.h, retain only the essential
migration-ceiling and load-ceiling calculation context. Remove repeated
rationale and move any necessary background to docs/node_info_stores.md.

In `@test/TestUtil.cpp`:
- Around line 69-107: Optionally replace the manual POSIX traversal in walk with
std::filesystem::recursive_directory_iterator, preserving relative regular-file
keys and skipping directories or inaccessible entries as the current
implementation does. Update fileFingerprint to use standard C++ file I/O,
retaining the existing hash algorithm and return behavior for unreadable files;
limit the change to the host-only ARCH_PORTDUINO code path.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 93a54d1c-46fe-4d41-a289-95de317310d3

📥 Commits

Reviewing files that changed from the base of the PR and between d0e9d02 and a797bc4.

⛔ Files ignored due to path filters (1)
  • test/state-manifest.tsv is excluded by !**/*.tsv
📒 Files selected for processing (27)
  • .github/copilot-instructions.md
  • .github/workflows/test_native.yml
  • AGENTS.md
  • bin/lib/test-state.sh
  • bin/pio-test-isolate.sh
  • bin/run-tests.sh
  • bin/test-state-check.sh
  • docs/node_info_stores.md
  • src/SerialConsole.cpp
  • src/graphics/draw/MenuHandler.cpp
  • src/graphics/draw/MenuHandler.h
  • src/mesh/NodeDB.h
  • src/mesh/StreamAPI.cpp
  • src/mesh/api/PacketAPI.cpp
  • src/mesh/api/ServerAPI.cpp
  • src/mesh/mesh-pb-constants.h
  • src/mesh/raspihttp/PiWebServer.cpp
  • src/mqtt/ServiceEnvelope.cpp
  • src/platform/esp32/MeshtasticOTA.cpp
  • test/README.md
  • test/TestUtil.cpp
  • test/TestUtil.h
  • test/native-suite-count
  • test/test_admin_radio/test_main.cpp
  • test/test_default/test_main.cpp
  • test/test_fscommon_getfiles/test_main.cpp
  • variants/native/portduino/platformio.ini

Comment thread .github/workflows/test_native.yml Outdated
Comment thread bin/pio-test-isolate.sh
Comment thread bin/run-tests.sh Outdated
Comment thread bin/test-state-check.sh
Comment thread docs/node_info_stores.md
Comment thread src/graphics/draw/MenuHandler.cpp
Comment thread src/mesh/NodeDB.h
@NomDeTom NomDeTom added tech debt Code or lib references that are not up to date or propper standards bugfix Pull request that fixes bugs labels Aug 1, 2026
@NomDeTom
NomDeTom marked this pull request as ready for review August 1, 2026 20:47
@NomDeTom
NomDeTom requested a review from Copilot August 1, 2026 20:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Not ready to approve

The new shared-state fingerprinting and suite discovery use GNU-only tooling (find -printf, md5sum), which is not available by default on macOS despite the repo shipping a native-macos environment, so the harness should be made portable (or explicitly gated) before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details
  • Files reviewed: 30/36 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
bin/lint-node-id-format.sh (1)

53-58: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Tighten looks_like_id()'s broad keyword fallback.

The last four alternatives in looks_like_id() match any occurrence of "packet", "node", "sender", or "relay" anywhere in the accumulated statement. In files like PacketHistory.cpp, every log message is prefixed with "Packet History - ...", so this fallback matches almost every statement, regardless of whether the specific %08x argument is actually an ID.

This contradicts the design rationale stated in the header comment: a false positive costs more than a miss. Tighten the fallback to match ID-shaped tokens near the message, not the presence of a generic domain word.

♻️ Proposed tightening
 	function looks_like_id(s) {
 		return (s ~ /(->|\.)(num|from|to|id|dest|sender|relay_node|next_hop)[^A-Za-z0-9_]/) ||
 		       (s ~ /[Nn]ode[Nn]um/) || (s ~ /nodeId/) || (s ~ /getFrom[ \t]*\(/) ||
-		       (s ~ /[^A-Za-z0-9_]sender[^A-Za-z0-9_]/) ||
-		       (s ~ /[Nn]ode/) || (s ~ /[Pp]acket/) || (s ~ /[Ss]ender/) || (s ~ /[Rr]elay/)
+		       (s ~ /[^A-Za-z0-9_]sender[^A-Za-z0-9_]/) ||
+		       (s ~ /relay_?[Nn]ode/) || (s ~ /relayed_by/) || (s ~ /ourRelayID/)
 	}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@bin/lint-node-id-format.sh` around lines 53 - 58, Update the final broad
keyword alternatives in looks_like_id() so packet, node, sender, and relay only
trigger when they occur in an ID-shaped token or relevant identifier context
near the message, rather than anywhere in the accumulated statement. Preserve
the existing specific patterns and the header comment’s preference for avoiding
false positives.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@bin/lib/shuffle.sh`:
- Around line 15-18: Update shuffle_suites to return immediately when $# is
zero, before invoking printf, so an empty suite list produces no blank output;
preserve the existing shuffling behavior when suite names are provided.

In `@bin/run-tests.sh`:
- Around line 280-291: Update verdict_red() to aggregate the “test cases:”
summaries from every shuffled suite in $LOG when $SHUFFLE is set, rather than
selecting only the final matching line. Preserve the existing last-line behavior
for non-shuffled runs and ensure the RED report reflects failures from all suite
invocations.

---

Nitpick comments:
In `@bin/lint-node-id-format.sh`:
- Around line 53-58: Update the final broad keyword alternatives in
looks_like_id() so packet, node, sender, and relay only trigger when they occur
in an ID-shaped token or relevant identifier context near the message, rather
than anywhere in the accumulated statement. Preserve the existing specific
patterns and the header comment’s preference for avoiding false positives.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 71fdce5b-bafc-40fa-b796-c8196b756e38

📥 Commits

Reviewing files that changed from the base of the PR and between a797bc4 and ffe460f.

📒 Files selected for processing (16)
  • .github/node-id-format-allowlist.txt
  • .github/workflows/test_native.yml
  • .trunk/trunk.yaml
  • bin/lib/shuffle.sh
  • bin/lib/test-state.sh
  • bin/lint-node-id-format.sh
  • bin/pio-test-isolate.sh
  • bin/run-tests.sh
  • bin/test-state-check.sh
  • docs/node_info_stores.md
  • src/graphics/draw/MenuHandler.cpp
  • src/mesh/PacketHistory.cpp
  • src/mesh/mesh-pb-constants.h
  • src/modules/NodeInfoModule.cpp
  • src/modules/PositionModule.cpp
  • src/platform/portduino/ConfigCheck.cpp
🚧 Files skipped from review as they are similar to previous changes (6)
  • bin/test-state-check.sh
  • docs/node_info_stores.md
  • bin/pio-test-isolate.sh
  • .github/workflows/test_native.yml
  • bin/lib/test-state.sh
  • src/graphics/draw/MenuHandler.cpp

Comment thread bin/lib/shuffle.sh
Comment thread bin/run-tests.sh Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.trunk/trunk.yaml:
- Around line 82-94: Update bin/lint-unity-exit.sh so its UNITY_END() scanning
preserves /* ... */ block-comment state across physical lines and ignores
occurrences inside comments. Accumulate and analyze continued multi-line calls,
including cases where UNITY_END() appears within an existing exit( invocation,
so valid wrapped calls are not reported. Enable the unity-exit rule in
trunk.yaml only after the scanner handles these statement-aware cases.

In `@bin/lint-unity-exit.sh`:
- Around line 36-59: The Unity scanner around bare_unity_end and its diagnostic
loop must become stateful and match-aware. Track multi-line comment and
string-literal state across AWK records, recognize logical statements spanning
lines such as split exit(UNITY_END()), and preserve detection of
whitespace-separated bare calls without matching ignored text. Update reporting
to emit one diagnostic for each actual match at its recorded column rather than
once per physical line with a fixed column, and add fixtures covering these
cases.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d8edbb0c-f71b-49a7-b19e-bc0ce351e6d2

📥 Commits

Reviewing files that changed from the base of the PR and between ffe460f and 2a4f9cc.

📒 Files selected for processing (13)
  • .github/copilot-instructions.md
  • .trunk/trunk.yaml
  • AGENTS.md
  • bin/lib/shuffle.sh
  • bin/lib/test-state.sh
  • bin/lint-unity-exit.sh
  • bin/pio-test-isolate.sh
  • bin/run-tests.sh
  • bin/test-state-check.sh
  • test/README.md
  • test/test_meshpacket_serializer/test_serializer.cpp
  • test/test_mqtt/MQTT.cpp
  • test/test_serial/SerialModule.cpp
🚧 Files skipped from review as they are similar to previous changes (4)
  • AGENTS.md
  • bin/pio-test-isolate.sh
  • bin/run-tests.sh
  • .github/copilot-instructions.md

Comment thread .trunk/trunk.yaml
Comment thread bin/lint-unity-exit.sh Outdated
NomDeTom added 15 commits August 2, 2026 03:39
The native node cap was stated in four places that disagreed, and the disagreement
already caused a wrong diagnosis: a saturated 200-node database looked arithmetically
impossible because the cap had been read as 248, computed from a header that does not
apply on this platform. The real value is 198.

On portduino MAX_NUM_NODES is not a compile-time constant at all - the variant defines
it as `portduino_config.MaxNodes`, resolved at runtime, default 200 and settable per
host with `General: MaxNodes`. variant.h is reached before mesh-pb-constants.h, so that
header's ARCH_PORTDUINO branch never fires and its plausible-looking 250 is dead code.

- #error-guard the dead branch rather than leave a wrong number where people grep. The
  guard found a real defect: seven translation units reach mesh-pb-constants.h without
  configuration.h (SerialConsole.cpp, StreamAPI.cpp, PacketAPI.cpp, ServerAPI.cpp,
  PiWebServer.cpp, ServiceEnvelope.cpp, MeshtasticOTA.cpp, and test/TestUtil.cpp), so
  each was compiling with a different MAX_NUM_NODES - and therefore a different
  PACKETHISTORY_MAX - than the rest of the build. Each now includes configuration.h
  first. It cannot be included from mesh-pb-constants.h itself: that reaches
  SerialConsole.h through DebugConfiguration.h and closes a cycle.
- Name the bare 250 in getMaxNodesAllocatedSize() NODEDB_MIGRATION_LOAD_CEILING. It is a
  decode allowance for files written by larger-cap firmware, not a cap, and it read like
  one.
- Fix docs/node_info_stores.md, which named the wrong source and a "10-250" range that
  is wrong for native, and the copilot-instructions tunables line that said "portduino
  250".
Native suites shared one directory. Every suite that constructs a NodeDB loads and
saves ~/.portduino/default/prefs/ - nodes.proto, config.proto, channels.proto,
module.proto, device.proto, warm.dat, transmit_history.dat - and nothing cleared it,
so state leaked suite -> suite within a run and run -> every run after it. A test run
could also rewrite a real meshtasticd node database on the same machine.

Per-run isolation does not fix this: the leak is generated inside a single run, so the
boundary has to be per suite.

bin/pio-test-isolate.sh runs each suite in its own scratch $HOME, registered as
test_testing_command for env:native and env:coverage so a bare `pio test` and CI get the
same boundary, not just bin/run-tests.sh. It runs the binary unchanged and exits with its
exit code, so PlatformIO's pass/fail is untouched. Overriding HOME here rather than
around `pio` also sidesteps the blocker that a bare HOME= breaks pio's own
~/.platformio/penv/bin/pio lookup.

Leftovers are reported as a second axis, PASS/FAIL x CLEAN/DIRTY, because an unintended
write has no matching assertion by definition - nobody writes TEST_ASSERT for a save they
do not know is happening. The harness asserts it from outside, so it applies to every
suite without the author opting in.

- Only the *set of changed paths* is asserted, never contents. Hashes answer the boolean
  "did this change?" and nothing more; content baselines over protobuf bytes would churn
  on every NodeInfoLite field added, which is how snapshot suites become noise.
- Deliberate writes are declared in test/state-manifest.tsv - one central file, suite /
  flags / mandatory reason. run-tests.sh prints the opt-out count on every run.
- Granularity follows the state flag, so the two ship together: per-test by default
  (TestUtil redefines RUN_TEST to checkpoint after each test, naming the exact test that
  dirtied things), suite boundary for state=per-suite, where carrying state across test
  cases is the declared behaviour.
- A declared write that does NOT happen is reported as MISSING, not folded into DIRTY. It
  catches silently broken persistence; a warning for now, since some are conditional.
- Graded AMBER, not RED. With isolation in place DIRTY means "undeclared", not
  "dangerous", and a check that lands red on day one gets switched off.

Guard the guard, both halves: state_assert_empty() refuses to run a suite against a
sandbox that is not empty (otherwise the after-diff measures against the wrong baseline
and reports CLEAN while meaning nothing), and bin/test-state-check.sh drives the real
wrapper with fixtures asserting CLEAN / CLEAN / DIRTY / MISSING plus both directions of
the empty assertion. A checker that silently matches everything would otherwise pass
forever.

--write-manifest proposes entries for a human to paste and justify; it never applies
them, and neither does CI.
A native suite ends in exit(UNITY_END()), and UNITY_END() returns the failure count.
PlatformIO's native runner reads that non-zero exit code as a POSIX signal number, so
four failures print "Program received signal SIGILL", five print "SIGTRAP", and the suite
is classified [ERRORED] rather than [FAILED].

There is no crash. The signal name tracks the failure count and nothing else - it moved
SIGILL -> SIGTRAP when a diagnostic probe added a fifth failure - and it cost hours of
hunting a memory bug that did not exist, on an env (native) that carries no sanitizer at
all. It also explains the phantom extra test case in the totals: the runner adds a
synthetic entry for the signal it thinks it saw.

run-tests.sh now says so inline whenever a signal line appears, and the three
agent-facing docs say it too.
setUp() did `if (!nodeDB) nodeDB = new NodeDB();` and never deleted it, so 83 of the 85
tests shared one never-reset database and never restored config, owner, devicestate or
channelFile. The fixture that does restore them was opt-in and armed by exactly two
tests. The setUp comment claiming the rest "set their own config/region state and are
unaffected" was not true - the admin handlers under test write all four globals.

Route every test through the fixture instead: setUp saves the globals and installs a
fresh NodeDB, tearDown restores and deletes it. The two tests that armed it themselves no
longer need to.

All 85 pass, so nothing was silently relying on the shared state. It costs about 7% of
the suite's runtime (a NodeDB construction is a loadFromDisk plus, with a region set, key
generation) - worth paying to write the phase 3 tests against a clean fixture rather than
83 tests' residue.

Also cap the per-test attribution in the run summary at five entries; the full list stays
in the suite's sandbox.
getFiles() runs on every phone sync via STATE_SEND_FILEMANIFEST, and nothing asserted any
of its bounding behaviour. It does execute unasserted from test_stream_api's handshakes,
but the cap, the depth limit, the wasLimited paths, overlong-path rejection and capacity
release were all unguarded.

Eight tests, all describing what the code does today: today's code is already correct
here, since meshtastic#10778 landed the by-reference collectFiles(), the 64-entry cap, the strlcpy
bounds and the swap-idiom release. They pass on arrival, which is the point - this is the
baseline a later change has to leave alone.

Two things they do not cover, and cannot:

- Moving reserve() outside the __cpp_exceptions guard. Exceptions are on natively, so the
  #else branch is not compiled. The suite's job there is to prove that change alters
  nothing observable.
- The file.name() null guard. No in-tree backend returns null; the guard is defensive.

The manifest-release test pins the swap idiom rather than calling
PhoneAPI's releaseFilesManifest(), which is file-local. It asserts capacity() == 0, not
just size() == 0 - a size-only check passes on clear(), which is the bug meshtastic#7924 shipped.

Suite count 43 -> 44, recounted against the directories rather than copied.
set_favorite_node, set_ignored_node and toggle_muted_node each persist a NodeInfoLite bit
and nothing else. MeshService::reloadConfig() gates its region re-derivation and
configChanged notification on saveWhat & (SEGMENT_CONFIG | SEGMENT_CHANNELS), so a
SEGMENT_NODEDATABASE-only save already skips the live radio reconfigure.

Pure characterization - all three pass on develop. Worth pinning because that reconfigure
is the path implicated in the WisMesh Tag favourite-node crash, and develop asserts
nothing about it: widening the saveWhat mask or reordering the check would currently go
unnoticed.

Ported from the config-save series along with ConfigChangedCounter (an Observer<void *>
counting configChanged notifications, the only externally visible signal that the reload
branch was taken) and TEST_NODE_NUM. They join the existing suite, so no suite-count
change.
The node menu's mute action was inline in a banner-callback lambda, and that lambda only
ever runs via screen->showOverlayBanner() - which is why nothing in MenuHandler.cpp was
reachable from a test. Lift the `selected == Mute` branch into
menuHandler::toggleNodeMuted(uint32_t) and call it from the lambda.

Behaviour-neutral by construction: same statements, same order, same bare saveToDisk().
The null check moves into the function, so the call site no longer needs its own lookup.
Verified by the native build and suite; the byte-identical-image check on a
headroom-constrained nRF52 board was not run locally - CI's firmware-size comment covers
it.

Three tests come with it, all describing today's behaviour:

- the bit flips both ways and no configChanged fires (develop never calls reloadConfig on
  this path);
- an unknown node is a no-op rather than a write;
- and the segment mask. Flipping one NodeInfoLite bit currently rewrites all five
  segments via bare saveToDisk(). That is asserted deliberately, with the comment naming
  it as characterization of a known defect: a pending fix narrows it to
  SEGMENT_NODEDATABASE, and when it lands this assertion is expected to change, which
  makes the improvement visible in the diff instead of silent.

saveToDisk() is not virtual, so the mask is observed through its effect - remove the five
prefs files, toggle, and see which reappear.
test/native-suite-count is the registered total and is machine-checked against test/test_*
on every full run and by the suite-count-check CI job. Every other statement of the count
is a copy that drifts: copilot-instructions said 12, AGENTS.md said 19, and the real
number is 44.

Replace both literals with a pointer to the file, say explicitly that no document should
state the count as a literal, and reframe the two suite listings as descriptions rather
than inventories - they carry per-suite information the count does not, so they stay, but
nothing should infer completeness from their length. Register the new FS suite in both.
Landed last, deliberately. Randomising an order-dependent suite set does not find bugs so
much as convert a silent pass into intermittent red, and the first instinct is to revert
the randomisation rather than fix the coupling. Phases 1-2 removed the coupling; this
keeps it removed.

Both runners previously hid order dependence behind a fixed order that happened to differ
between them, and neither order was chosen: CI's area rules put admin first, PlatformIO's
local discovery is reverse alphabetical and put it last. CI was green by accident.

- bin/run-tests.sh --shuffle / --seed <n>. The seed defaults to HEAD's short SHA: one
  order per commit, so a red is replayable and attributable to the diff instead of flaky,
  while the project keeps exploring orders. Printed at the start and carried into the
  RESULT line, so a verdict is replayable from that line alone; the full order is printed
  on failure, because for an order-dependent failure the order is the diagnostic.
- The shuffle is a Fisher-Yates over a MINSTD generator rather than awk's rand(), whose
  sequence differs between gawk and mawk. A seed that does not reproduce the same order on
  another machine is not a seed.
- Shuffling needs one `pio test -f <suite>` invocation per suite - PlatformIO orders by
  its own os.walk() over test/ and filters only select - which measures at about 4.7s per
  suite of extra startup.
- CI shuffles its area order, seeded from GITHUB_SHA and printed with the command to
  replay it locally. Intra-area order stays PlatformIO's; controlling it there would mean
  per-suite invocations, which is a cost worth deciding separately.

Also records the 16 measured entries in test/state-manifest.tsv, each with its reason,
taken from a full run's --write-manifest output rather than guessed.
getConfiguredOrDefaultMsScaled(configured, default, nodes, TrafficType) is the overload
every telemetry and position module actually calls, and nothing referenced TrafficType
anywhere under test/. All four of its behaviours were unguarded: the no-region guard, the
throttle <= 1 short-circuit, the multiply, and the 64-bit overflow clamp.

The throttles are real, not hypothetical - EU_866 carries PROFILE_LITE, which sets both
positionThrottle and telemetryThrottle to 10, so a change here moves broadcast spacing in
that region by an order of magnitude.

Each test pins numOnlineNodes at the congestion threshold and uses ROUTER, which never
congestion-scales, so the coefficient is 1 and the throttle is the only variable. The
overflow case needs a base above INT32_MAX/10, hence three days rather than one.
Shuffling the area order on every run - including pull_request - would turn a
contributor's PR red for an ordering they did not choose, which is how a randomisation
gets reverted instead of the coupling being fixed. That is the exact dynamic the ordering
work was sequenced last to avoid, and the previous commit walked straight into it.

- pull_request keeps the fixed declared area order.
- push and schedule shuffle, seeded from the commit SHA: deterministic per commit,
  printed, attributable, and never blocking someone else's PR.
- A suite_order_seed input on workflow_call and workflow_dispatch overrides both, so a
  specific failing order can be replayed anywhere, including on a PR.

The run log prints which mode it took, the resulting order, and the local command to
replay it.
The seed is reachable through workflow_call, which callers can pass programmatically. The
workflow_dispatch copy tripped checkov's "workflow_dispatch inputs MUST be empty" rule,
and suppressing it was not worth it: replaying a specific order is a local operation, and
the run log already prints the exact bin/run-tests.sh command to do it.
RadioInterface.cpp documents the rule: 0x%08x in logs, !%08x in user-facing
display. MenuHandler held every remaining exception - seven logs printing bare
%08X, and two display labels doing the same.

Repo-wide there are now no bare %08X node IDs left in log calls.
suite_order_seed and github.event_name were spliced into the run: script as
${{ }} text, so a value carrying shell metacharacters would execute as code on
the runner rather than being read as data. semgrep (run-shell-injection) and
zizmor (template-injection) both flag it.

Both now arrive as environment variables and are read as "$VAR".
bin/run-tests.sh and test_native.yml each carried a byte-identical copy of the
MINSTD Fisher-Yates awk. The workflow prints "replay locally: ./bin/run-tests.sh
--shuffle --seed $seed" after a shuffled CI run, and that instruction is only
true while the two agree - drift would be announced by a replay quietly
reproducing a different order than the one that failed.

Extract shuffle_suites() to bin/lib/shuffle.sh and source it from both.
Permutations verified identical across seeds before and after the move.
NomDeTom added 16 commits August 2, 2026 03:39
Three defects in the new harness:

state_classify() matched declarations two different ways - state_path_declared()
for "undeclared", a hand-rolled regex for "missing". Interpolating an entry into
an ERE also let a metacharacter in a manifest name match a file that is not the
declared one. Both directions now go through the one helper.

`paste -sd'; '` does not join with "; ": with -s, paste cycles through a
multi-character delimiter one character per join, so paths rendered as
"a;b c;d e". Replaced with an awk join.

test-state-check.sh ran on after a failed cd instead of stopping (SC2164).

./bin/test-state-check.sh: 6/6 fixtures pass, MISSING included.
MaxNodes was validated only for <= 0. Any positive value, including a typo'd or
pasted-in one, propagates to MAX_NUM_NODES and scales both the node DB and the
nodes.proto decode ceiling - failing at boot with no obvious cause.

The ceiling is a sanity bound, not a capability limit; raise it if a host
genuinely needs more.
The property matrix omitted the ESP32-S3 100-node flash tier that the platform
table above it lists, and neither mentioned that the WASM build overrides
MaxNodes to 80 in wasm_config_apply().
The ARCH_PORTDUINO #error assumed it was unreachable in a normal build. It is
not: the vendored device-ui sources include this header without configuration.h,
which broke both native-tft docker builds.

Include configuration.h here instead, ahead of every compile-time default -
variant.h overrides MAX_RX_TOPHONE as well as MAX_NUM_NODES, so placing it lower
in the file just moves the divergence to a redefinition. The #error stays as a
backstop for the case where that include genuinely stops providing the cap.

Verified with the native env's own flags: a TU including only this header now
compiles, normal-order use of both macros compiles, and NodeDB.cpp compiles.
Marked artificial: nothing in the node DB fails at 16001. 16000 sits just under
the 16384 (128 x 128) population where HopScalingModule saturates its sampling
denominator and starts dropping nodes, so a host inside the bound still gets
meaningful hop recommendations.
RadioInterface.cpp documents the convention - 0x%08x in logs, !%08x in display -
but nothing enforced it, which is how the MenuHandler cluster drifted. 22 call
sites in PacketHistory, NodeInfoModule and PositionModule are still off it.

A trunk linter rather than a CI grep job, because trunk checks changed files:
new violations get flagged without a 22-site cleanup landing in an unrelated PR.
Modelled on the existing too-many-defined definition.

Scoped to values it can tell are IDs - an ID-shaped argument (->num, .from,
getNodeNum) or message text naming one. A 32-bit hex that is not an ID is out of
scope, so the CRC32 logs in ethOTA.cpp are correctly ignored.

Emits "note", trunk's only non-blocking level: "warning" and "info" both exit
non-zero and would gate CI, which is not what a log-format nit deserves. The
pre-existing sites are line-scoped in the allowlist, so a new bad call in those
same files is still caught.
The seeded allowlist made the rule green by declaring the backlog acceptable.
Empty it instead, so the 22 pre-existing sites are reported and get cleaned up
by whoever next edits those files.

Costs nothing to do: the rule emits "note", so these are non-blocking either
way. The allowlist stays for its real purpose - a value the linter misreads as
an ID.
Clears the 22 sites the node-id-format linter reports, so the rule starts from
zero rather than from a backlog nobody can see - trunk suppresses pre-existing
findings by default, so left alone these would not have surfaced on edit the way
an empty allowlist implies.

Format strings only; no argument or control flow changes. The !%08x
user-facing display forms are deliberately untouched - that is the other half of
the same convention.
run-tests.sh fused build and run in a single pio invocation, so whichever suite
PlatformIO's directory walk reached first absorbed the entire src compile and
reported it as its own duration. On a real run that made a 0.03s suite report
13m21s, and hid the build cost from every other number in the summary.

Do what .github/workflows/test_native.yml already does: one --without-testing
build pass, then run with --without-building. Measured on a full 44-suite run -
the build is now a single reported figure and 968 test cases execute in 1.9s,
with no suite above 0.084s.

Build output goes to its own log rather than $LOG: the outcome regexes match
"error:" and "[ERRORED]", so a compiler diagnostic sharing that file would read
as a test failure.

Both red paths now keep the log they quote from. $LOG and the build log are
mktemps the EXIT trap removes, so the three grepped lines were previously all
anyone ever saw - and the cause is usually further up than the first [FAILED].
bin/pio-test-isolate.sh already keeps a failing or DIRTY suite's sandbox and log
under .pio/test-state/<suite>/. What was missing is the cross-suite view: $LOG is
a mktemp the EXIT trap deletes, so run-tests.sh quoted three grepped lines from a
file that no longer existed by the time anyone looked.

Preserve it as .pio/build/<env>/test-failure.log from both red paths - including
"no success summary found", which said "see log" while preserving nothing, and
which is exactly the case where the build died before any suite ran and so left
no per-suite sandbox either.

Cleared at the start of every run, so a green run cannot leave a red one's log
lying around looking current.
A shuffled run is one `pio test` invocation per suite, all appending to the
same log, so the log carries one PlatformIO "N test cases:" summary per suite.
verdict_red() took `tail -1`, which reports whatever the LAST suite did: a
failure in suite 3 printed a "0 failed" summary from suite 44 directly under
"RED - failures detected:".

Sum the summaries instead. A single summary line - every unshuffled run - is
passed through verbatim, so the familiar output is byte-identical.

The patterns are passed to the awk helper as strings rather than /regex/
literals: awk evaluates a regex literal in argument position as `$0 ~ /re/`,
so the callee would receive 0 or 1 and silently sum garbage.
`printf '%s\n' "$@"` with no arguments still writes one empty line, and both
callers read shuffle_suites through mapfile, so an empty suite list arrived as
a single suite named "". Return before the printf when there is nothing to
shuffle.
The native harness is a Linux tool: bash 4+ (mapfile), GNU coreutils and GNU
find (-printf, md5sum, -executable). Most of that predates this branch -
mapfile and both find predicates are already on develop - but none of it was
written down, so the requirement was there to be discovered rather than read.

Refuse to start on a non-Linux uname instead of degrading. On a BSD userland
this would not fail cleanly: it would mis-hash the sandbox and mis-read the
suite list, and still print a verdict. A state check that silently measures
the wrong thing is worse than one that declines to run.

Carrying a per-host fallback was the alternative, and it buys a second code
path that nothing in CI exercises. bin/test-native-docker.sh already exists
for macOS and non-Linux hosts, and the native-macos PlatformIO env is a build
target for meshtasticd, not a test host - the isolation wrapper is registered
for env:native and env:coverage only.

Documented in the script header, test/README.md, and both agent docs.
Four sites across three suites ended on a bare UNITY_END(). That ends the
reporting, not the suite: setup() returns, the runtime goes on calling loop(),
and the process runs forever. PlatformIO does not notice - it reports a suite
from its Unity output, not from process exit - so the suite passes, the run
goes green, and the binary stays resident. Thirteen of them had accumulated on
one dev box, the oldest 19 hours old.

The costs are quiet by construction:

- the per-suite sandbox is deleted underneath a live process, so its
  CLEAN/DIRTY verdict describes what the suite had written when the harness
  stopped looking, not what it left behind;
- .gcda coverage and LeakSanitizer's report both flush from atexit handlers,
  so a suite that never exits contributes no coverage and gets no leak check;
- each survivor pins its own deleted 94 MB binary, which du cannot see.

Three of the four sites are the #else of an architecture guard, which is the
easiest one to get wrong - it looks like there is nothing to clean up.
test_serial and test_mqtt each had a correct exit(UNITY_END()) in their live
branch, so a "does this file call exit() anywhere" check passes them both.

test/README.md gets a section on it, since the skeleton showing the right
shape had not stopped this happening four times.
A suite that never exits was invisible: PlatformIO reports a suite from its
Unity output, so the run stayed green while the binary kept running. Two
checks, because they fail differently.

Runtime, in bin/pio-test-isolate.sh: the sandbox $HOME is mktemp-unique per
suite, so any process still holding it is a survivor of that suite. Matching
on the environment rather than a remembered PID identifies one whatever its
parentage - a fork, a grandchild, a process already reparented to init - none
of which a $! comparison catches. Reaped before the after-fingerprint is
taken, so that fingerprint measures a tree nobody is still writing to, and so
a run cannot leave processes accumulating on the host. Recorded as a sixth
summary column and graded AMBER: the tests did pass, but the CLEAN verdict and
the coverage were measured under a false assumption.

Author-time, as bin/lint-unity-exit.sh, wired into trunk at "note" like
node-id-format: every UNITY_END() must be wrapped in exit(). The rule is per
occurrence, and that is the point - a file-level "calls exit() somewhere"
check passes test_serial and test_mqtt, which have a correct one in their live
branch and a bare one in the #else. Running it over the tree turned up
test_mqtt, which the file-level pass had missed.

It allows `int rc = UNITY_END(); ...; exit(rc)`, used by test_packet_signing
to restore globals between the summary and the exit. That is where the rule
gives ground: capturing and never exiting would leak and is not flagged.
Flagging a correct idiom would push someone to "fix" working code.

bin/test-state-check.sh gains a survivor fixture, asserting the wrapper both
reports and reaps - a detector that only reports leaves the host accumulating
processes, which is half the harm. 8/8.
The rule judged one physical line at a time, which reports two kinds of correct
code as bare:

    /* a comment that happens to
       mention UNITY_END() */          <- interior lines were never stripped

    exit(
        UNITY_END());                  <- exit( and the macro never met

On a probe of both, two of three findings were wrong. This is a note-level rule
whose whole job is advice, and bin/lint-node-id-format.sh already says why that
matters: a false positive costs more than a miss. One that cries wolf gets
ignored, and the real finding goes with it.

Carry /* ... */ state across lines and accumulate logical statements before
testing, with a 12-line cap so one unclosed call cannot swallow the rest of the
file - the same structure lint-node-id-format.sh uses, so the two custom linters
in bin/ work alike rather than each having its own idea.

Verified both directions: the develop-era sources still produce the same four
findings, the fixed tree produces none, and a probe covering block-comment
interiors, wrapped exit(), line comments, return UNITY_END() and capture-then-
exit reports only the genuinely bare calls - including a complete block comment
followed by real bare code on the same line, which the state machine has to
keep live.

Reported by CodeRabbit on meshtastic#11322.
@NomDeTom
NomDeTom force-pushed the test-suite-rebuild branch from 2a4f9cc to 92624a0 Compare August 2, 2026 02:40

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
bin/run-tests.sh (1)

443-443: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Aggregate the failure count in the RED fallback line for shuffled runs.

This line takes tail -1 of every "[0-9]+ failed" match in $LOG. In shuffled mode, each suite appends its own pio test output to the same $LOG (Lines 290-303), so this reproduces the exact bug already fixed for the "test cases:" summary via summarise_test_cases() (Lines 366-390): if an earlier suite fails and a later suite also fails with a different count, this machine-readable RESULT: line reports only the last matching count, not the true total. This directly works against the PR's stated goal of clarifying Unity failure-count reporting.

Use the same aggregation approach as summarise_test_cases() here.

🐛 Proposed fix
-	echo "RESULT: RED $(grep -oE '[0-9]+ failed' "$LOG" | tail -1 || echo 'build/crash error')"
+	total_failed=$(grep -oE '[0-9]+ failed' "$LOG" | awk '{s+=$1} END{print s+0}')
+	if ((total_failed > 0)); then
+		echo "RESULT: RED $total_failed failed"
+	else
+		echo "RESULT: RED build/crash error"
+	fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@bin/run-tests.sh` at line 443, Update the RED result reporting around the
existing RESULT line to aggregate all failure counts from LOG, matching the
summation behavior implemented by summarise_test_cases(). Preserve the existing
build/crash error fallback when no failure count is found, and ensure shuffled
runs report the total failures across all suites rather than only the final
match.
🧹 Nitpick comments (3)
test/TestUtil.cpp (1)

140-184: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚖️ Poor tradeoff

Add direct coverage for the new checkpoint/diff logic.

testStateCheckpoint, walk, fileFingerprint, and suiteFromPath are new, non-trivial logic (recursive traversal, FNV-1a hashing, add/modified/removed diffing, suite-name parsing). No test in the provided files exercises this C++ path directly; bin/test-state-check.sh only covers the separate shell-side state_classify logic. Consider a small dedicated suite that sets MESHTASTIC_TEST_STATE_REPORT/HOME to a scratch directory, calls testStateCheckpoint across synthetic file changes, and asserts the emitted added/modified/removed lines.

As per path instructions, test/**/*: Add or update tests for behavior changes when suites are added or removed.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/TestUtil.cpp` around lines 140 - 184, Add direct C++ tests for the
checkpoint logic centered on testStateCheckpoint, using a scratch HOME and
MESHTASTIC_TEST_STATE_REPORT environment, synthetic files, and multiple calls to
verify added, modified, and removed report lines. Also cover suiteFromPath
parsing and the walk/fileFingerprint behavior through these checkpoints, and
place or update tests under the applicable test suite so added or removed suites
are handled.

Source: Path instructions

bin/lint-unity-exit.sh (1)

29-30: 🩺 Stability & Availability | 🔵 Trivial

Do not report scanner failures as clean scans.

The shell ignores the awk exit status, and Line 110 always returns 0. A read or parser failure can therefore produce no diagnostic while satisfying Trunk's success_codes: [0].

Keep findings non-blocking, but emit a Trunk-formatted diagnostic for scanner failures or expose a separate failure status. This contract is defined by .trunk/trunk.yaml Lines 82-94.

Also applies to: 107-110

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@bin/lint-unity-exit.sh` around lines 29 - 30, Update the scanner flow in the
shell script so read or awk/parser failures are not treated as clean scans.
Preserve non-blocking findings, but propagate scanner failure status or emit a
Trunk-formatted diagnostic consistent with the configured success-code contract,
and ensure the script’s final status reflects scanner failures instead of always
returning success.
test/test_fscommon_getfiles/test_main.cpp (1)

1-8: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Trim comments to the guideline's one-to-two-line limit.

Three comment blocks in this file exceed the repository's comment-length rule: the file header (Lines 1-8, 7 comment lines), the FSCom-path rationale (Lines 20-22, 3 lines), and the capacity-release rationale (Lines 202-205, 4 lines). The content is useful context, but the guideline caps comments at one or two lines and asks for minimal, non-restating comments.

Condense each block to its core rationale in one or two lines.

As per path instructions, "Keep code comments minimal—one or two lines maximum—and comment only when explaining non-obvious rationale; do not restate straightforward code."

Also applies to: 20-22, 202-205

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/test_fscommon_getfiles/test_main.cpp` around lines 1 - 8, Condense the
file header comment to one or two lines summarizing the getFiles() coverage and
bounded-walk rationale. Also shorten the FSCom-path rationale and
capacity-release rationale comments to one or two lines each, preserving only
their non-obvious context and removing behavior restatements.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@bin/lint-unity-exit.sh`:
- Around line 44-59: Rewrite strip_comments as a stateful lexical scanner that
recognizes escaped string literals, character literals, and raw-string literals,
preserving their contents so tokens such as UNITY_END() and /* do not affect
comment state. Replace the greedy block-comment match with one-pair-at-a-time
scanning while maintaining in_block across lines, and add fixtures covering both
false positives inside literals and missed real calls after literal or
block-comment content.
- Around line 74-79: Update bare_unity_end() so wrapper exemptions are
token-aware: only recognize a standalone exit(UNITY_END()) and capture-then-exit
usage, excluding identifiers such as myexit and operators such as ==, !=, >=,
and +=. Remove the return UNITY_END() exemption, since return does not terminate
the runner, while preserving detection of other bare UNITY_END() calls.

In `@bin/run-tests.sh`:
- Around line 290-303: Add "${PASSTHRU[@]}" to both PlatformIO test invocations
inside the shuffled loop in the SHUFFLE path, covering the QUIET and tee
branches. Preserve the existing suite filter, logging, and return-code handling
while ensuring all collected user flags are forwarded for every suite.
- Around line 78-80: Update the startup directory change in bin/run-tests.sh by
guarding the cd "$ROOT_DIR" command so failure terminates the script
immediately; preserve the existing ROOT_DIR calculation and ensure later
relative-path operations cannot run from an unintended working directory.

In `@bin/test-state-check.sh`:
- Around line 91-130: Update the survivor PID lookup in the test block around
the `survivor_dir` and `leaked_pid` checks so its glob matches the fixture’s
`$HOME/../survivor.pid` location under the sandbox home directory. Preserve the
existing verification that the reported survivor is actually reaped, or
alternatively change the fixture’s PID output location to match the current
lookup.

---

Outside diff comments:
In `@bin/run-tests.sh`:
- Line 443: Update the RED result reporting around the existing RESULT line to
aggregate all failure counts from LOG, matching the summation behavior
implemented by summarise_test_cases(). Preserve the existing build/crash error
fallback when no failure count is found, and ensure shuffled runs report the
total failures across all suites rather than only the final match.

---

Nitpick comments:
In `@bin/lint-unity-exit.sh`:
- Around line 29-30: Update the scanner flow in the shell script so read or
awk/parser failures are not treated as clean scans. Preserve non-blocking
findings, but propagate scanner failure status or emit a Trunk-formatted
diagnostic consistent with the configured success-code contract, and ensure the
script’s final status reflects scanner failures instead of always returning
success.

In `@test/test_fscommon_getfiles/test_main.cpp`:
- Around line 1-8: Condense the file header comment to one or two lines
summarizing the getFiles() coverage and bounded-walk rationale. Also shorten the
FSCom-path rationale and capacity-release rationale comments to one or two lines
each, preserving only their non-obvious context and removing behavior
restatements.

In `@test/TestUtil.cpp`:
- Around line 140-184: Add direct C++ tests for the checkpoint logic centered on
testStateCheckpoint, using a scratch HOME and MESHTASTIC_TEST_STATE_REPORT
environment, synthetic files, and multiple calls to verify added, modified, and
removed report lines. Also cover suiteFromPath parsing and the
walk/fileFingerprint behavior through these checkpoints, and place or update
tests under the applicable test suite so added or removed suites are handled.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: fa394e84-30b0-40be-86cb-fdb0255a12e5

📥 Commits

Reviewing files that changed from the base of the PR and between 2a4f9cc and 92624a0.

⛔ Files ignored due to path filters (1)
  • test/state-manifest.tsv is excluded by !**/*.tsv
📒 Files selected for processing (39)
  • .github/copilot-instructions.md
  • .github/node-id-format-allowlist.txt
  • .github/workflows/test_native.yml
  • .trunk/trunk.yaml
  • AGENTS.md
  • bin/lib/shuffle.sh
  • bin/lib/test-state.sh
  • bin/lint-node-id-format.sh
  • bin/lint-unity-exit.sh
  • bin/pio-test-isolate.sh
  • bin/run-tests.sh
  • bin/test-state-check.sh
  • docs/node_info_stores.md
  • src/SerialConsole.cpp
  • src/graphics/draw/MenuHandler.cpp
  • src/graphics/draw/MenuHandler.h
  • src/mesh/NodeDB.h
  • src/mesh/PacketHistory.cpp
  • src/mesh/StreamAPI.cpp
  • src/mesh/api/PacketAPI.cpp
  • src/mesh/api/ServerAPI.cpp
  • src/mesh/mesh-pb-constants.h
  • src/mesh/raspihttp/PiWebServer.cpp
  • src/modules/NodeInfoModule.cpp
  • src/modules/PositionModule.cpp
  • src/mqtt/ServiceEnvelope.cpp
  • src/platform/esp32/MeshtasticOTA.cpp
  • src/platform/portduino/ConfigCheck.cpp
  • test/README.md
  • test/TestUtil.cpp
  • test/TestUtil.h
  • test/native-suite-count
  • test/test_admin_radio/test_main.cpp
  • test/test_default/test_main.cpp
  • test/test_fscommon_getfiles/test_main.cpp
  • test/test_meshpacket_serializer/test_serializer.cpp
  • test/test_mqtt/MQTT.cpp
  • test/test_serial/SerialModule.cpp
  • variants/native/portduino/platformio.ini
🚧 Files skipped from review as they are similar to previous changes (31)
  • src/modules/PositionModule.cpp
  • .trunk/trunk.yaml
  • src/modules/NodeInfoModule.cpp
  • src/mesh/raspihttp/PiWebServer.cpp
  • .github/node-id-format-allowlist.txt
  • src/graphics/draw/MenuHandler.h
  • src/mesh/api/ServerAPI.cpp
  • variants/native/portduino/platformio.ini
  • src/platform/portduino/ConfigCheck.cpp
  • bin/lib/shuffle.sh
  • src/mqtt/ServiceEnvelope.cpp
  • src/mesh/api/PacketAPI.cpp
  • src/SerialConsole.cpp
  • src/mesh/PacketHistory.cpp
  • test/test_meshpacket_serializer/test_serializer.cpp
  • src/mesh/NodeDB.h
  • test/test_default/test_main.cpp
  • test/test_serial/SerialModule.cpp
  • src/mesh/mesh-pb-constants.h
  • AGENTS.md
  • test/test_mqtt/MQTT.cpp
  • .github/copilot-instructions.md
  • test/test_admin_radio/test_main.cpp
  • src/platform/esp32/MeshtasticOTA.cpp
  • bin/lint-node-id-format.sh
  • test/TestUtil.h
  • src/mesh/StreamAPI.cpp
  • test/native-suite-count
  • docs/node_info_stores.md
  • .github/workflows/test_native.yml
  • src/graphics/draw/MenuHandler.cpp

Comment thread bin/lint-unity-exit.sh Outdated
Comment thread bin/lint-unity-exit.sh Outdated
Comment thread bin/run-tests.sh Outdated
Comment thread bin/run-tests.sh
Comment thread bin/test-state-check.sh
Second round of review findings on the same scanner, all confirmed by direct
test before changing anything. Six defects, one root cause: layered regexes
cannot tokenise C++.

False positives (correct code reported):
  - UNITY_END() inside a string literal read as code

False negatives (real leaks missed):
  - a string containing "/*" opened comment state and swallowed later lines
  - greedy .* removed everything between two block comments on one line,
    taking a bare call with it
  - myexit(UNITY_END()) matched the exit() exemption as a substring
  - x == UNITY_END() and total += UNITY_END() matched the assignment exemption

Replaced with a character-level scan carrying comment state, and token-bounded
exemptions: exit must be a whole identifier, and the capture form must be a
plain `=`. Raw string literals are still not modelled - there are none under
test/, and delimiter tracking for a case that does not occur would be untested
code guarding untested code, so it is documented rather than guessed at.

Also drops the `return UNITY_END()` exemption. It only terminates from main(),
there is no main() under test/, and from a helper it just returns a count.

bin/test-lint-unity-exit.sh pins all fifteen cases, every false positive and
false negative found in review among them. The rule has been wrong twice in a
way that looked fine by inspection; it needed a self-test more than it needed
another careful reading.

Two further findings in the same review:

  - bin/run-tests.sh dropped PASSTHRU in shuffled mode, so `--shuffle -vvv`
    built verbosely and then ran quietly. The shuffled loop now forwards
    EXTRA_ARGS, which is PASSTHRU minus the -f pair it supplies per suite.
  - bin/run-tests.sh did not guard `cd "$ROOT_DIR"`.

And one that did not reproduce: the survivor fixture's glob does find the pid
file (verified with the lookup instrumented - the earlier failure was an
artifact of running the script from /tmp, where SCRIPT_DIR cannot resolve).
The assertion was still weak, because an empty pid took the "not running"
branch and passed vacuously. It now fails if the pid was never recorded, and
finds the file by search rather than assuming a directory depth.

Reported by CodeRabbit on meshtastic#11322.
@NomDeTom
NomDeTom requested a review from Copilot August 2, 2026 10:09

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
bin/lint-unity-exit.sh (1)

109-126: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Restore the per-occurrence diagnostic contract. The scanner records one macro position and emits one result per accumulated statement. The self-test only checks whether output exists, so it cannot prevent this regression.

  • bin/lint-unity-exit.sh#L109-L126: evaluate each UNITY_END() occurrence independently and emit its own line and column when it is bare.
  • bin/test-lint-unity-exit.sh#L31-L47: assert exact finding counts and locations, including mixed legal and bare calls in one statement.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@bin/lint-unity-exit.sh` around lines 109 - 126, Update the statement scanner
around the UNITY_END detection in bin/lint-unity-exit.sh (lines 109-126) to
evaluate every UNITY_END() occurrence independently and emit its own line and
column only when that occurrence is bare, including mixed legal and bare calls
within one statement. Strengthen the self-test in bin/test-lint-unity-exit.sh
(lines 31-47) to assert exact finding counts and reported locations rather than
merely checking that output exists.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@bin/test-lint-unity-exit.sh`:
- Around line 31-47: Enhance the expect() helper in the lint test to accept and
validate the expected diagnostic count and location details, including line and
column, rather than only checking whether output exists. Update existing cases
with those expectations and add a fixture covering exit(UNITY_END());
UNITY_END(); on one line, asserting both diagnostics and their locations.

---

Outside diff comments:
In `@bin/lint-unity-exit.sh`:
- Around line 109-126: Update the statement scanner around the UNITY_END
detection in bin/lint-unity-exit.sh (lines 109-126) to evaluate every
UNITY_END() occurrence independently and emit its own line and column only when
that occurrence is bare, including mixed legal and bare calls within one
statement. Strengthen the self-test in bin/test-lint-unity-exit.sh (lines 31-47)
to assert exact finding counts and reported locations rather than merely
checking that output exists.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: bf0640f2-298a-47df-bec2-a0c742e53ad1

📥 Commits

Reviewing files that changed from the base of the PR and between 92624a0 and 7a4aead.

📒 Files selected for processing (4)
  • bin/lint-unity-exit.sh
  • bin/run-tests.sh
  • bin/test-lint-unity-exit.sh
  • bin/test-state-check.sh
🚧 Files skipped from review as they are similar to previous changes (2)
  • bin/test-state-check.sh
  • bin/run-tests.sh

Comment thread bin/test-lint-unity-exit.sh Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Human review recommended

It makes broad, workflow-integrated changes to the test harness/CI execution model that warrant a final human validation pass in CI to confirm PlatformIO runner behavior and end-to-end compatibility.

Review details
  • Files reviewed: 33/41 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

The self-test only asked "did the linter say anything", so it could not have
caught a wrong line, a wrong column, or a missing second finding. Fixtures now
assert the exact diagnostics as line:col, and the first run of that assertion
found two real problems.

The caret pointed at the wrong occurrence. For `exit(UNITY_END()); UNITY_END();`
the verdict was right but the column was 17 - the wrapped call - because the
scanner stripped terminating forms out of the whole statement and then reported
the first occurrence it had seen. Two bare calls on one line reported once.

Judged per occurrence now, by looking back through whitespace at what wraps it,
so both the count and the caret are right. That also needed a position map from
strip_noncode(): removing a comment or collapsing a literal shifts every later
column, and counting occurrences in the raw line does not recover it either -
TEST_MESSAGE("... UNITY_END() ..."); UNITY_END(); has two occurrences in the raw
text and one in the code.

Four of the expected columns I wrote by hand were also wrong, off by one. The
linter was right in every case; the assertions were not. They are computed from
the fixture text now rather than pasted from output, because a baseline accepted
from the tool it is testing asserts nothing.

17 fixtures, including the two-on-one-line case from review and its mirror.

Reported by CodeRabbit on meshtastic#11322.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes bugs tech debt Code or lib references that are not up to date or propper standards

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants