Skip to content

[FEATURE] Update CLI with grouped logs download feature - #82

Merged
oxesoft merged 2 commits into
project-chip:v2.15-cli-developfrom
greens:feature/grouped_logs
May 15, 2026
Merged

[FEATURE] Update CLI with grouped logs download feature#82
oxesoft merged 2 commits into
project-chip:v2.15-cli-developfrom
greens:feature/grouped_logs

Conversation

@greens

@greens greens commented May 13, 2026

Copy link
Copy Markdown
Contributor

What changed

This PR implements grouped log archive downloads, a feature that exists on the web UI (and the API), but was not yet available in the CLI:

image

I've broken out the log fetching functionality into a separate group under the test-run-execution command:
image

but also maintained the existing syntax/functionality in case there were users relying on it:
image

Testing

Testing existing syntax for backwards-compatability:
image

New syntax for simple log download:
image

And new syntax for the new feature (with and without specifying output file):
image

@CLAassistant

CLAassistant commented May 13, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the test_run_execution CLI command into a command group, introducing a new log subcommand and support for downloading grouped logs as ZIP archives. The changes include updates to the OpenAPI specification, the client generation script, and the CLI implementation to handle binary data responses. Review feedback identifies several improvement opportunities: sanitizing filenames derived from execution titles to prevent filesystem errors or path traversal, explicitly specifying UTF-8 encoding when writing text logs, and correcting type hints for optional parameters in the internal log-fetching functions.

Comment thread th_cli/commands/test_run_execution.py Outdated
Comment thread th_cli/commands/test_run_execution.py Outdated
Comment thread th_cli/commands/test_run_execution.py Outdated
Comment thread th_cli/commands/test_run_execution.py Outdated

@antonio-amjr antonio-amjr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @greens,

Great work here, specially using groups for the test_run_execution command (we still lack the group structure for some commands).

I have just one concern here. Since the CLI API is autogenerated to reflect the backend CLI, the openapi.json file should be close to the backend's.

That said, please mind that although this update would work, someone could update the openapi.json using the backend's version or use the URL approach (e.g. ~/certification-tool/cli/scripts/generate_client.sh -i http://<TEST_HARNESS_IP>/api/v1/openapi.json) and end up undoing this unless this update is also added to Backend structure.

Let me know if that makes sense or if you have any ideas for the API generation approach.
I recently fixed and changed the auto API generation, so let me know any point of improvement in this process.

@greens

greens commented May 14, 2026

Copy link
Copy Markdown
Contributor Author

@antonio-amjr Thanks for the notes. I went through and opened a new PR against the backend that should ensure the generated JSON matches what we've got here: project-chip/certification-tool-backend#313

It's simply an artifact of using the StreamingResponse that results in FastAPI not including the response metadata in its generated JSON.

@oxesoft
oxesoft merged commit 6e60ffd into project-chip:v2.15-cli-develop May 15, 2026
3 checks passed
oxesoft added a commit that referenced this pull request May 25, 2026
* [Fix] Handling Empty Usage Tag In CLI's PICS Parser (#43)

* PICS parser logic update: verifying if the 'usage' tag exists, ignoring if not.

* Gemini's suggestion: Improving conditional logic

* Removing unnecessary attribution line

* Fix _send_prompt_response calls and added timeout for large upload (#44)

* Fix _send_prompt_response calls and added timeout for large upload

* Update th_cli/test_run/prompt_manager.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Code review - Gemini

* Added constants values

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Show friendly message to unsupported TWO WAY TALK test execution using CLI (#46)

* Add Checking for  ffmpeg and improve abort-testing command

* Webrtc working

* Changes related to two way talk

* Updated ffmpeg parameters

* Deleted unwated config file

* changes after black command

* Update th_cli/commands/abort_testing.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update th_cli/th_utils/ffmpeg_converter.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update th_cli/test_run/websocket.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Code review gemini

* Code review gemini

* Removed extra space in warning message from shared_constant file

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Fixing the CLI Run Test Logic to handle project config and user config (#47)

* Fixing the CLI Run Test Logic to handle project config and user config

* Gemini Suggestions: Fixing Path creation and adding except print to the user.

* Removed Error column from test execution list (#49)

* [Feature] Adding the Node ID to CLI Output (#48)

* Adding the Node ID to CLI output

* Updating ChipServerInfo attribute type

Plus minor fixes

* Fix cli unit tests not working (#50)

* Fix CLI unit tests not working

* Removed Error column from test execution list (#49)

* Updated conftest and models

* Update th_cli/commands/test_run_execution.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [FIX] CLI PICS Parser And Prompt (#51)

* Adding Message Prompt Request type and fixing PICS XML parser

* Fixing some flake8 issues

* Fixing isort issue

* Gemini's Suggestion: Remove unnecessary timeout exception logic

* Fix test_run_execution command (#52)

* Fix test_run_execution command

* Update tests/test_test_run_execution.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update tests/test_test_run_execution.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update th_cli/commands/test_run_execution.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Added support to project-id in test_run_execution command

* Black formating

* Code review

* Code review

* Code review

* code review

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [Feature] Printing Manual Pairing Code (#53)

* Printing the Manual Pairing Code to CLI's test execution output

* Gemini Suggestion: Improving logic and fixing f strings

* Fixing comment

Adding missing parameter for the method's comment

Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>

---------

Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>

* Adding the test parameters to the models to be used in the CLI's project config (#55)

Also, the logic to print the chipserver info was fixed to always show the Node ID, even without the manual pairing code

* Enhanced the th-cli available-tests command with new formatting options (#54)

* Improve CLI available-tests command

* Code formating

* Minor code changes

* Code review - gemini code assistant

* Increase websocket message size (#56)

* Increase websocket message size

* Create constant for WEBSOCKET_MAX_MESSAGE_SIZE

* Improve  FFmpeg Error Handling in Video Stream Tests (#57)

* Add message for ffmpeg not found

* Code review - gemini

* Code review

* Code review

* [Feature] Swap CLI Properties Config With JSON (#58)

* Swapping the Properties config file with the JSON format

* Fixing lint alerts

* New Line At File's End

Gemini Suggestion: adding a new line to the default_config.json file as per recommendation. This can prevent som e text processing tools errors.

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Implement CLI Support for Push AV Tests  (#59)

* Initial code

* Stable version

* Added unit tests

* Minor changes

* Code formating and isort

* Updated unit test

* Code review

* Add comment regarding disabling SSL verification

* Removed unused import

* Removed unused imports

* Minor changes

* Fixed f-string

* Removed unecessary init files

* Code review - minor changes

* [Feature] Double-Dash Extra Arguments (#60)

* Implement the double dash extra arguments feature

Also including the Unit Tests

* Fixing ouput message

* Improving parser logic

* Fixing lint errors

* Improving parser logic for subsequent --

* Fixing the README Notes

* Fixing CLI run_tests command conditional for extra arguments

* Fixing ChipServer info display when using no project ID or config options (#61)

* Fix WebSocket close handshake error for long-running test executions (#63)

* Fix WebSocket close handshake error for long-running test executions in the CLI

* Update th_cli/test_run/websocket.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [FIX] Updating CLI API Client Generator (#62)

* Updating and Fixing CLI API Client Generator

Updating tool version, files, scripts to support ARM64 hosts and generate successfully

* Re-enable black formatter to fix several files structure

* Adding postprocessing script to fix return type of api_response.py file

Also, re-enabling the formatters in the end of the generation

* Returning the TimeoutException logic but now adding the import accordingly

* Updating shell scripts to handle host architecture type and new flag options

* Fixing licence header generation with mustache files

* Added NFC_WIFI and THREAD_MESHCOP

* [Fix] CLI Project Create and Default Config (#67)

* Fixing default config file and usage for project creation/update

Also, fixing script file extension for run_pytest

* Gemini Suggestion: Adding new line at end of file

* Update regex pattern for test ID validation (#66)

Edited the regex pattern, so that Tests such as TC-I-XXX work with the CLI, which threw an error before.

* [Feature] Replace OpenAPI Generator with Datamodel-Code-Generator (#65)

* New CLI API Client Generator

Exchanging OpenAPI-Generator with Datamodel-Code-Generator

* Removing Openapi-generator files and adding README

* Fixing parameter placeholder from project command

* Fixing the Project Update command to handle empty names

* Fixing Project Update Unit Tests

* Fixing script's README file

* Project update improvements and minor fixes

* Implementing some TODOs with missing logic for headers and multipart fields (file or data)

* Fixing the README title

* [Feature] Refactoring CLI Project Command (#68)

* Updating CLI project command to use Click's group feature with subcommads

* Simplifying logic using a SyncAPI generator and a contextmanager decorator

* Fix test case comparison logic (#69)

* Fix test case comparison logic

* Update th_cli/utils.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Rever code formating

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Add missing image_handler.py and image_verification.html for snapshot camera verification                                                (#71)

* Fixed Screenshot camera verification

* Fixed copyright year

* Code review Gemini-code-assist

* Code review Gemini-code-assist

* Fix regex pattern to support additional valid test ID formats   (#73)

* Fix regex to support single-segment and underscore-category test IDs (TC-I-X, TC-BR-X, TC-MCORE_FS-X.X)

* Update tests/test_validation.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Feature/909 Improve CLI unit tests code coverage for camera files (#72)

* Fixed Screenshot camera verification

* Fixed copyright year

* Code review Gemini-code-assist

* Code review Gemini-code-assist

* Improve CLI test coverage across camera files and minor static analysis violations

* Undo cov-fail-under change

* black formating

* [FIX] Run_Tests Command Configuration Logic (#74)

* Fixing run_tests command configuration logic

* Gemini review: improving logic

* CLI  to support two way talk tests (#75)

* Initial implementation to support two way talk

* Some code improvements

* Impreved the user experience by only resume with the test case execution only when the CLI detects that the browser was opened

* Improved code comments

* Update th_cli/test_run/camera/two_way_talk_handler.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update th_cli/commands/run_tests.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Code review  - gemini code assist::Use dependency injection to pass the handler instance through the call stack

* Code review

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [Feature] Add PICS Parameter To CLI Projects (#78)

* Adding pics parameters to CLI's project commands

* Fixing project update unit test

* Updating run-test command to use the configured pics with none was provided

* Fetching default project when no project ID was provided

* Improving project manipulation in run-test command

* [Fix] Project Create Command With No PICS (#79)

* Fixing project create command when PICS is not provided

* Gemini Suggestion: Removing unecessary global variable. Using PICS object directly

* [FIX] Updating CLI To Handle Temporary PICS (#80)

* Updating CLI to handle PICS during execution to be temporary (not persisted to the project in DB)

* Fixing else conditional from run_tests.py file

* Fixing openapi.json Create Cli Test Run Execution description

* [FEATURE] Update CLI with grouped logs download feature (#82)

* Update CLI with grouped logs download feature

* allow endpoints to return binary data

* Fix(cli): unique per-run log filename with timestamp + mode=w (#984) (#83)

---------

Co-authored-by: antonio-amjr <116589331+antonio-amjr@users.noreply.github.com>
Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>
Co-authored-by: Antonio Melo Jr. <a_junior@apple.com>
Co-authored-by: Romulo Quidute Filho <rquidute@apple.com>
Co-authored-by: ali gruenhaupt <yesilbas.ma@gmail.com>
Co-authored-by: Steven Green <sgreen@csa-iot.org>
Co-authored-by: abhisheksingh-esp <abhishek.singh@espressif.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants