Merge V2.15-develop into main - #317
Conversation
* Updated package-dependency-list.txt * Updated missing dependencies
* Reverting the BR_VARIANT option to the otbr start script * Adding TODO to the script * Adding missing comment char * Updating Script Comment Year Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Adding switch parameters options to the script --------- Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com>
* replace erro with warning * Update test_collections/matter/sdk_tests/support/python_testing/models/python_test_parser.py Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> * black error fix * pr review --------- Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>
* Fixing intermittent problem with Python's Base Manager Creating the manager now only once in the SDK Container Init * Refactoring and updating the unit tests with related mocks
* Updated python_testing references after sdk refactoring * Removed no wanted file changes * Updated matter_testing references * Fixed line too long * Fixed code - black * Updated SDK SHA * Removed commented code
* Isolate SDK docker pull script * Changes after code review --------- Co-authored-by: Romulo Quidute Filho <rquidute@apple.com>
* Isolate SDK docker pull script * Rebasing * Apply suggestions from code review Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> * Move call tp update-pull-sdk-docker-image.sh from setup.sh to update-sample-apps.sh * Changes after code review * Fixed CI issues * Code review fixes * Fixed unit tests * Fixed mypy --------- Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> Co-authored-by: Romulo Quidute Filho <rquidute@apple.com>
* Implemented changes for custom python test * code review * Code review * Code reviewd * Fixed code violations * Chanegs after code review * Remove unused import
* Fixed some python tests not listed * code review
* Implement support to export project config * Fix mypy * Changed the returned schema and service endpoint name * Added import project config * Apply suggestions from code review Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> * code review --------- Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>
* Added support to negated PICS * Fixed flake * Code review. No consider negatives PICS * Update app/pics_applicable_test_cases.py * Code review
* Remove unused file * Updated cspell
* Fix CI issues from v2.12 branch * Fixed merge * Fix test case * Updated SDK_DOCKER_IMAGE url * Updated SDK_DOCKER_IMAGE url
…ing (#173) * Add support for configuring terms and conditions in device commissioning This commit introduces a new `enhanced_setup_flow` configuration property to the test harness backend, allowing device manufacturers to define the terms and conditions (T&C) that must be acknowledged and set during device commissioning. These values are critical for supporting the Terms and Conditions Certification Test Suite. Key changes include: - Added the `EnhancedSetupFlowConfig` class to `DutConfig`, which allows specifying the T&C version and user response required during commissioning. - Updated `generate_command_arguments` in `utils.py` to append the necessary arguments for T&C validation and the device's commissioning method. - Adjusted model validation to ensure the `enhanced_setup_flow` field is processed correctly within `DutConfig`. These changes ensure the test harness can enforce manufacturer-specific T&C values during the commissioning process, aligning with the broader requirements of T&C validation. * * Ran ./scripts/format.sh * Update test_collections/matter/test_environment_config.py Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Update app/user_prompt_support/uploaded_file_support.py Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Update test_collections/matter/sdk_tests/support/performance_tests/utils.py Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Update test_collections/matter/sdk_tests/support/python_testing/models/utils.py Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Update test_collections/matter/sdk_tests/support/python_testing/models/utils.py Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Update test_collections/matter/sdk_tests/support/python_testing/__init__.py Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Revert changes to test_collections/matter/sdk_tests/support/performance_tests/utils.py * Restyle test_environment_config.py * test: Add in-test-commissioning-method parameter for manual commissioning validation Add new parameter to control commissioning flow during tests, allowing manual validation of the commissioning process instead of auto-commissioning. This enables test cases to verify terms and conditions acknowledgements during the commissioning flow. - Add in-test-commissioning-method parameter to test flags - Support manual commissioning flow validation - Enable testing of T&C acknowledgement scenarios * Add new field "enhanced_setup_flow" to exception message * Cherry-pick test_python_parser.py from origin/main * Cherry-pick python_test_parser.py from origin/main * Revert "Cherry-pick python_test_parser.py from origin/main" This reverts commit 5d65ce5. * Revert "Cherry-pick test_python_parser.py from origin/main" This reverts commit 2cd99b7. * Removed broken test `test_python_parser.py` * Revert "Removed broken test `test_python_parser.py`" This reverts commit 48c0e32. --------- Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com>
Updated Node.js installation source from setup_18.x to setup_20.x to align with Ubuntu 22.04 base image requirements
* First implementation * Implement prompt and copy files from/to contanier * Unit tests * Updated SKA SHA * Unit test typo fixes * Mypy flake8 fixes * Mypy fixes * Update test_collections/matter/sdk_tests/support/python_testing/models/utils.py Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> * Update test_collections/matter/sdk_tests/support/utils.py Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> * Code review * Review fixes * Remove unused import --------- Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>
* Add logs for project config * Updated cspell.json * Update app/test_engine/test_runner.py Co-authored-by: hiltonlima <116589806+hiltonlima@users.noreply.github.com> * Fixed unit test * Unit test * Unit test --------- Co-authored-by: hiltonlima <116589806+hiltonlima@users.noreply.github.com>
…n logging (#290) * Implemented configuration for enabling/disabling container information logging * Code review - Gemini
…#291) * Fixes for Python tests being loaded twice during Test Harness startup * Code review
….1-beta1+winter2026 version
* Initial code for thread support * Code formating and isort * Apply suggestion from @gemini-code-assist[bot] Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Apply suggestion from @gemini-code-assist[bot] Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Apply suggestion from @gemini-code-assist[bot] Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Code review - gemini-code-assistant * Code review - gemini-code-assistant * Code review - gemini-code-assistant * Code review flake8 - gemini-code-assistant * Code review mypy --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Adding nfc-wifi to the backend * Keeping Flake8 happy * Making another tool happy * improvents according to Gemini * refactor updates throughout necessary files * reverting change in test_suite.py * updates according to Gemini * updates to keep tools happy * tooling update
* Updated all references from Thread to Thread_meshcop * Code formating * Updated SDK_SHA
* Fixing argument generation when using NFC commissioning Dut config shouldn't pass discrminator, passcode, manual code or qr-code when using NFC pairing mode * Fixing unit tests * Improving logic and exchangin data structure from list to set
…sses imported from additional Matter SDK modules. (#300) * Fix python script parser * Code review - gemini * Code review - adjust endswith for TestBase in python script parser * Code review - mypy fix
* Improved logic to load python test scripts * Added test_python_test_initialization.py unit test file * Update copyright year * Code review - mypy/flake8 * Code review - mypy/flake8 * Code review - mypy/flake8 * Code review - flake8 * Update test_collections/matter/sdk_tests/support/python_testing/list_python_tests_classes.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update test_collections/matter/sdk_tests/support/python_testing/list_python_tests_classes.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Code review - gemini code assist * Code review --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Fix json-arg not working * Code review- Gemini Code assist * Code review- Gemini Code assist * Code review- Gemini Code assist * Added more unit test in order to covert more scenarios
* Updating infrastructure to handle temporary PICS for Test Run Executions * Fixing test_run_executions.py docstring * Improving return expression in the test_run.py file * Fixing the related unit tests * Removing unnecessary import
…n their base class is defined in support_modules (#306) * Consider using extra dir(SDK files) when discovering custom test cases * Code review - Google code assist * Fix black
* Remove commissioning prompt from no commissioning suite * Fix mypy * Code review
) * Fix TC_DD_3_24 hang on user prompt timeout in _show_prompt_request. When wait_for_user_input times out, response_str is None so nothing was sent to the test socket, leaving the SDK process blocked on input() forever. * Update test_collections/matter/sdk_tests/support/python_testing/models/test_case.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Code review --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
CommissionDeviceTest has been moved from matter.testing.commissioning to matter.testing.CommissioningPreTest in the Matter SDK. The old path no longer exports this symbol, which causes --get-test-info to fail and the Python test list to be empty in the UI. Fixes #981
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive updates to the test harness, including project export/import functionality, CLI-driven test executions with configuration overrides, and real-time Python test logging. It expands communication capabilities with new WebSocket endpoints for video relay and WebRTC signaling, optimizes test discovery through shared container sessions, and updates the Matter SDK integration. Technical feedback highlights critical bugs in the new Docker file-transfer utilities regarding tar archive handling, an exception shadowing issue in the PICS processing logic that prevents proper error catching, and a concurrency bottleneck in the video relay service caused by a hardcoded UDP port.
| class FileNotFoundError(PlatformTestError): | ||
| """Exception raised when test file is not found""" | ||
|
|
||
| pass |
There was a problem hiding this comment.
The custom FileNotFoundError exception shadows the built-in Python FileNotFoundError. Since the custom exception does not inherit from the built-in one, the except FileNotFoundError: block at line 81 will fail to catch the exception raised by open() when a file is missing. This results in the built-in exception propagating unhandled by this specific block. It is recommended to rename the custom exception to avoid shadowing.
| class FileNotFoundError(PlatformTestError): | |
| """Exception raised when test file is not found""" | |
| pass | |
| class PlatformFileNotFoundError(PlatformTestError): | |
| """Exception raised when test file is not found""" | |
| pass |
| except FileNotFoundError: | ||
| raise FileNotFoundError(f"File {platform_tests_file} not found") |
There was a problem hiding this comment.
This block should catch the built-in FileNotFoundError and raise the custom one defined earlier (after renaming it to avoid shadowing).
| except FileNotFoundError: | |
| raise FileNotFoundError(f"File {platform_tests_file} not found") | |
| except FileNotFoundError: | |
| raise PlatformFileNotFoundError(f"File {platform_tests_file} not found") |
| stream, _ = container.get_archive(str(container_file_path)) | ||
| with open( | ||
| f"{str(destination_path)}/{destination_file_name}", | ||
| "wb", | ||
| ) as f: | ||
| for d in stream: | ||
| f.write(d) |
There was a problem hiding this comment.
The copy_file_from_container method writes the raw tar stream returned by container.get_archive directly to the destination file. Docker's get_archive API returns a tar archive, so the resulting file on the host will be a tarball containing the requested file, rather than the file's actual content. The stream should be processed using the tarfile module to extract the specific file content before writing it to the host.
| with tarfile.open(f"{host_file_path}", mode="r") as tar_in: | ||
| with tarfile.open(fileobj=tar_stream, mode="w") as tar_out: | ||
| for member in tar_in.getmembers(): | ||
| # Put the file with the expected name | ||
| member.name = destination_container_path.name | ||
| tar_out.addfile(member, tar_in.extractfile(member)) |
There was a problem hiding this comment.
The copy_file_to_container method attempts to open the host file as a tar archive (tarfile.open(..., mode="r")). If the host file is a regular file (e.g., a .json or .txt file), this will raise a tarfile.ReadError. To copy a regular file, the method should create a new tar archive in memory and add the file to it before calling put_archive.
| sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0) | ||
| sock.settimeout(1.0) | ||
| sock.bind((UDP_SOCKET_INTERFACE, UDP_SOCKET_PORT)) |
There was a problem hiding this comment.
The video relay logic binds to a hardcoded UDP port (5000). This implementation prevents multiple concurrent video streams because only one socket can bind to a specific port at a time. If multiple clients attempt to access the video stream simultaneously, subsequent connections will fail with an 'Address already in use' error. Consider using dynamic port allocation or a port pool.
|
Closing in favor of a merge branch that resolves squash-artifact conflicts against main. See replacement PR. |
Summary
v2.15-developintomain, covering the v2.15+spring2026 release line (Matter 1.6 SVE) plus subsequent fixes.Test plan
v2.15+spring2026version is reported correctly