Merge V2.15-develop into main by oxesoft · Pull Request #318 · project-chip/certification-tool-backend

oxesoft · 2026-05-22T18:37:13Z

Summary

Promotes the v2.15+spring2026 release line (Matter 1.6 SVE) plus subsequent fixes from v2.15-develop into main.
Highlights: Matter 1.6 SVE v2.15-beta3.1, NFC commissioning support (Wi-Fi and Thread), Camera/WebRTC features, Python test framework improvements, and recent fixes (Fix ModuleNotFoundError - matter.testing.CommissioningPreTest #316, Fix ImportError for CommissionDeviceTest in test_harness_client #312, Fix TC_DD_3_24 hang on user prompt timeout in _show_prompt_request #308).

Notes on conflicts

main HEAD is a squash merge of release content that already exists on v2.15-develop as individual commits, so the true merge base is far back at Merge V2.11+fall2024 into main #174 (V2.11). A direct v2.15-develop -> main PR produced spurious conflicts.
This branch was built from origin/main, then v2.15-develop was merged in. The merge commit's tree is identical to v2.15-develop's tree, with both main HEAD (54ea96f) and v2.15-develop HEAD (2ac562d) preserved as parents.

Test plan

CI passes on the PR
Smoke test the Test Harness against a representative DUT
Verify v2.15+spring2026 version is reported correctly

Replaces #317.

* Updated package-dependency-list.txt * Updated missing dependencies

* Reverting the BR_VARIANT option to the otbr start script * Adding TODO to the script * Adding missing comment char * Updating Script Comment Year Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Adding switch parameters options to the script --------- Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com>

* replace erro with warning * Update test_collections/matter/sdk_tests/support/python_testing/models/python_test_parser.py Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> * black error fix * pr review --------- Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>

* Fixing intermittent problem with Python's Base Manager Creating the manager now only once in the SDK Container Init * Refactoring and updating the unit tests with related mocks

* Updated python_testing references after sdk refactoring * Removed no wanted file changes * Updated matter_testing references * Fixed line too long * Fixed code - black * Updated SDK SHA * Removed commented code

…" (#166) This reverts commit 316a934.

* Isolate SDK docker pull script * Changes after code review --------- Co-authored-by: Romulo Quidute Filho <rquidute@apple.com>

* Isolate SDK docker pull script * Rebasing * Apply suggestions from code review Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> * Move call tp update-pull-sdk-docker-image.sh from setup.sh to update-sample-apps.sh * Changes after code review * Fixed CI issues * Code review fixes * Fixed unit tests * Fixed mypy --------- Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> Co-authored-by: Romulo Quidute Filho <rquidute@apple.com>

* Implemented changes for custom python test * code review * Code review * Code reviewd * Fixed code violations * Chanegs after code review * Remove unused import

…-app script (#169)

* Fixed some python tests not listed * code review

* Implement support to export project config * Fix mypy * Changed the returned schema and service endpoint name * Added import project config * Apply suggestions from code review Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> * code review --------- Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>

* Added support to negated PICS * Fixed flake * Code review. No consider negatives PICS * Update app/pics_applicable_test_cases.py * Code review

* Remove unused file * Updated cspell

* Fix CI issues from v2.12 branch * Fixed merge * Fix test case * Updated SDK_DOCKER_IMAGE url * Updated SDK_DOCKER_IMAGE url

…ing (#173) * Add support for configuring terms and conditions in device commissioning This commit introduces a new `enhanced_setup_flow` configuration property to the test harness backend, allowing device manufacturers to define the terms and conditions (T&C) that must be acknowledged and set during device commissioning. These values are critical for supporting the Terms and Conditions Certification Test Suite. Key changes include: - Added the `EnhancedSetupFlowConfig` class to `DutConfig`, which allows specifying the T&C version and user response required during commissioning. - Updated `generate_command_arguments` in `utils.py` to append the necessary arguments for T&C validation and the device's commissioning method. - Adjusted model validation to ensure the `enhanced_setup_flow` field is processed correctly within `DutConfig`. These changes ensure the test harness can enforce manufacturer-specific T&C values during the commissioning process, aligning with the broader requirements of T&C validation. * * Ran ./scripts/format.sh * Update test_collections/matter/test_environment_config.py Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Update app/user_prompt_support/uploaded_file_support.py Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Update test_collections/matter/sdk_tests/support/performance_tests/utils.py Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Update test_collections/matter/sdk_tests/support/python_testing/models/utils.py Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Update test_collections/matter/sdk_tests/support/python_testing/models/utils.py Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Update test_collections/matter/sdk_tests/support/python_testing/__init__.py Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> * Revert changes to test_collections/matter/sdk_tests/support/performance_tests/utils.py * Restyle test_environment_config.py * test: Add in-test-commissioning-method parameter for manual commissioning validation Add new parameter to control commissioning flow during tests, allowing manual validation of the commissioning process instead of auto-commissioning. This enables test cases to verify terms and conditions acknowledgements during the commissioning flow. - Add in-test-commissioning-method parameter to test flags - Support manual commissioning flow validation - Enable testing of T&C acknowledgement scenarios * Add new field "enhanced_setup_flow" to exception message * Cherry-pick test_python_parser.py from origin/main * Cherry-pick python_test_parser.py from origin/main * Revert "Cherry-pick python_test_parser.py from origin/main" This reverts commit 5d65ce5. * Revert "Cherry-pick test_python_parser.py from origin/main" This reverts commit 2cd99b7. * Removed broken test `test_python_parser.py` * Revert "Removed broken test `test_python_parser.py`" This reverts commit 48c0e32. --------- Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com>

Updated Node.js installation source from setup_18.x to setup_20.x to align with Ubuntu 22.04 base image requirements

* First implementation * Implement prompt and copy files from/to contanier * Unit tests * Updated SKA SHA * Unit test typo fixes * Mypy flake8 fixes * Mypy fixes * Update test_collections/matter/sdk_tests/support/python_testing/models/utils.py Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> * Update test_collections/matter/sdk_tests/support/utils.py Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> * Code review * Review fixes * Remove unused import --------- Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>

* Add logs for project config * Updated cspell.json * Update app/test_engine/test_runner.py Co-authored-by: hiltonlima <116589806+hiltonlima@users.noreply.github.com> * Fixed unit test * Unit test * Unit test --------- Co-authored-by: hiltonlima <116589806+hiltonlima@users.noreply.github.com>

…sses imported from additional Matter SDK modules. (#300) * Fix python script parser * Code review - gemini * Code review - adjust endswith for TestBase in python script parser * Code review - mypy fix

* Improved logic to load python test scripts * Added test_python_test_initialization.py unit test file * Update copyright year * Code review - mypy/flake8 * Code review - mypy/flake8 * Code review - mypy/flake8 * Code review - flake8 * Update test_collections/matter/sdk_tests/support/python_testing/list_python_tests_classes.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update test_collections/matter/sdk_tests/support/python_testing/list_python_tests_classes.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Code review - gemini code assist * Code review --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Fix json-arg not working * Code review- Gemini Code assist * Code review- Gemini Code assist * Code review- Gemini Code assist * Added more unit test in order to covert more scenarios

* Updating infrastructure to handle temporary PICS for Test Run Executions * Fixing test_run_executions.py docstring * Improving return expression in the test_run.py file * Fixing the related unit tests * Removing unnecessary import

…n their base class is defined in support_modules (#306) * Consider using extra dir(SDK files) when discovering custom test cases * Code review - Google code assist * Fix black

* Remove commissioning prompt from no commissioning suite * Fix mypy * Code review

) * Fix TC_DD_3_24 hang on user prompt timeout in _show_prompt_request. When wait_for_user_input times out, response_str is None so nothing was sent to the test socket, leaving the SDK process blocked on input() forever. * Update test_collections/matter/sdk_tests/support/python_testing/models/test_case.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Code review --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

CommissionDeviceTest has been moved from matter.testing.commissioning to matter.testing.CommissioningPreTest in the Matter SDK. The old path no longer exports this symbol, which causes --get-test-info to fail and the Python test list to be empty in the UI. Fixes #981

# Conflicts: # test_collections/matter/sdk_tests/support/performance_tests/models/test_suite.py

gemini-code-assist

Code Review

This pull request introduces several enhancements and optimizations to the test harness, including performance test support with summary report generation, optimized Python test initialization, and real-time logging capabilities. The database schema and API were updated to handle execution-specific configurations and PICS overrides, while a new THREAD_MESHCOP pairing mode was added. Feedback from the review identifies several opportunities to improve code robustness, such as adding timeouts to HTTP requests, using value equality for status codes, URL-encoding parameters, and ensuring safe attribute access and explicit file encoding.

rquidute and others added 30 commits October 22, 2024 21:11

Updated SDK SHA - revert

d0238f3

Changed SDK_DOCKER_TAG to f2e5de7_latest

25f7581

Make all TH dependencies not static (#164)

2caab2e

* Updated package-dependency-list.txt * Updated missing dependencies

Add TC-IDM-10.5 test case to mandatory suite (#153)

851ad45

Check user prompt answer FAILED for DUT commissioning (#155)

018d273

Handle Skip test fir old script python test cases (#154)

6ebfcf8

[FIX] Intermittent Tests Start Error With No Information (#157)

151057f

* Fixing intermittent problem with Python's Base Manager Creating the manager now only once in the SDK Container Init * Refactoring and updating the unit tests with related mocks

Update python_testing references after SDK refactoring (#162)

13c7911

* Updated python_testing references after sdk refactoring * Removed no wanted file changes * Updated matter_testing references * Fixed line too long * Fixed code - black * Updated SDK SHA * Removed commented code

Revert "[FIX] Intermittent Tests Start Error With No Information (#157)…

c6f8a12

…" (#166) This reverts commit 316a934.

01 - Isolate SDK docker pull script (#142)

85887ab

* Isolate SDK docker pull script * Changes after code review --------- Co-authored-by: Romulo Quidute Filho <rquidute@apple.com>

Implemented changes for custom python test (#167)

9d79b81

* Implemented changes for custom python test * code review * Code review * Code reviewd * Fixed code violations * Chanegs after code review * Remove unused import

Update update-sample-app script to copy apps from new origin (#148)

77def30

Fix suite names for custom python tests (#168)

5b50541

Fix move update-pull-sdk-docker-image script content to update-sample…

90531ab

…-app script (#169)

Fixed some python tests not listed (#172)

73a13e9

* Fixed some python tests not listed * code review

[FIX] Added support to negated PICS (#170)

a2a87e5

* Added support to negated PICS * Fixed flake * Code review. No consider negatives PICS * Update app/pics_applicable_test_cases.py * Code review

Merge branch 'v2.11+fall2024' into spring2025

0bb4f89

Fixes after merge

6ca0fb5

Remove unused file (#175)

3574d8e

* Remove unused file * Updated cspell

Fix ci issues v 2 12 (#176)

dcc8436

* Fix CI issues from v2.12 branch * Fixed merge * Fix test case * Updated SDK_DOCKER_IMAGE url * Updated SDK_DOCKER_IMAGE url

chore: upgrade Node.js from v18.x to v20.x (#177)

20db0c0

Updated Node.js installation source from setup_18.x to setup_20.x to align with Ubuntu 22.04 base image requirements

Implement rename test run execution name (#180)

a7c0075

Fix the checking for storage-path (#183)

5fe46f6

rquidute and others added 16 commits February 24, 2026 11:19

Fix the Python test script parser to correctly identify base test cla…

06379b4

…sses imported from additional Matter SDK modules. (#300) * Fix python script parser * Code review - gemini * Code review - adjust endswith for TestBase in python script parser * Code review - mypy fix

Version v2.15-beta2.2+spring2026

28fa53b

Fix json-arg not working (#304)

3ab803b

* Fix json-arg not working * Code review- Gemini Code assist * Code review- Gemini Code assist * Code review- Gemini Code assist * Added more unit test in order to covert more scenarios

Release Matter 1.6 SVE (v2.15-beta3+spring2026)

1919605

Release Matter 1.6 SVE version 2 (v2.15-beta3.1+spring2026)

deda8ea

Fix custom sideloaded test scripts not appearing in the test list whe…

30adcd1

…n their base class is defined in support_modules (#306) * Consider using extra dir(SDK files) when discovering custom test cases * Code review - Google code assist * Fix black

Remove commissioning prompt from no commissioning tests (#307)

cafea0d

* Remove commissioning prompt from no commissioning suite * Fix mypy * Code review

Make pcscd --disable-polkit started unconditionally (#309)

64038f8

Annotate grouped log download for api generation (#313)

1794f7f

v2.15+spring2026 version (#315)

ba754d2

Fix ModuleNotFoundError - matter.testing.CommissioningPreTest (#316)

2ac562d

Merge V2.15-develop into main

d4a73f6

# Conflicts: # test_collections/matter/sdk_tests/support/performance_tests/models/test_suite.py

oxesoft requested review from antonio-amjr, hiltonlima and rquidute May 22, 2026 18:38

gemini-code-assist Bot reviewed May 22, 2026

View reviewed changes

antonio-amjr approved these changes May 22, 2026

View reviewed changes

rquidute approved these changes May 22, 2026

View reviewed changes

hiltonlima approved these changes May 25, 2026

View reviewed changes

This was referenced May 25, 2026

Missing timeout #319

Closed

Fix comparison #320

Closed

Missing URL encode #321

Closed

Missing default value can lead to AttributeError #322

Closed

Missing encoding #323

Closed

oxesoft merged commit 5c1b9ac into main May 25, 2026
7 of 8 checks passed

oxesoft deleted the merge/v2.15-develop-into-main branch May 25, 2026 12:32

Navigation Menu

Uh oh!

Merge V2.15-develop into main - #318

Merge V2.15-develop into main#318
oxesoft merged 164 commits into
mainfrom
merge/v2.15-develop-into-main

oxesoft commented May 22, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

Uh oh!

Conversation

oxesoft commented May 22, 2026

Summary

Notes on conflicts

Test plan

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants