Skip to content

Add verify diagnostics CLI with validation tests#14

Open
MRhuang1106 wants to merge 8 commits into
Andyyook:mainfrom
MRhuang1106:main
Open

Add verify diagnostics CLI with validation tests#14
MRhuang1106 wants to merge 8 commits into
Andyyook:mainfrom
MRhuang1106:main

Conversation

@MRhuang1106

@MRhuang1106 MRhuang1106 commented Jun 30, 2026

Copy link
Copy Markdown

Summary

Adds tools/verify_diagnostics.py, a CLI for validating diagnostic JSON reports and referenced .logd artifacts before opening a PR.

Changes

  • Adds argparse CLI support for diagnostic report validation.
  • Supports verbose output, JSON output, and threshold checks.
  • Validates diagnostic JSON structure, module counts, module status values, logd path shape, chunk ordering, decrypt command presence, and missing artifact references.
  • Adds defensive subprocess handling for missing executables, timeouts, non-zero exits, and OS errors.
  • Adds focused unittest coverage for valid reports, structural errors, threshold failures, JSON CLI output, and subprocess error handling.
  • Adds diagnostic artifacts under diagnostic/ for this PR.

Testing

python -m unittest discover -s tests
Ran 6 tests
OK

Also attempted local diagnostic build commands on Windows:

python build.py
python build.py -m backend

The full build could not complete in this environment because required toolchains such as npm/cargo/git are not installed. The captured local diagnostic output is committed as:

  • diagnostic/build-b58b0052.json
  • diagnostic/build-b58b0052.logd

Checklist

  • Relevant modules affected by these changes build locally
  • Tests pass locally
  • Diagnostic build log is committed in this PR
  • Documentation has been updated, if applicable
  • Configuration or schema changes are documented, if applicable
  • No generated build artifacts are committed, except the required diagnostic build log
  • Changes are scoped to the PR purpose and avoid unrelated cleanup
  • Security, privacy, and error-handling implications have been considered

  • I would like to request that my diagnostic build log is removed before merging

@MRhuang1106

Copy link
Copy Markdown
Author

Added the requested diagnostic artifacts to PR #14: diagnostic/build-b58b0052.json and diagnostic/build-b58b0052.logd. Unit tests pass locally with python -m unittest discover -s tests. Full python build.py was attempted, but this Windows environment is missing several required toolchains, so the diagnostic files include the captured local failure output.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant