[Bug] TC-DD-3.23 and TC-DD-3.24 passed in THGUI, but failing in CLI

[Pradeep-grl]

Describe the bug

While executing TC-DD-3.23 and TC-DD-3.24 in CLI, the commissioning step fails in both test cases after reading the payload. The failure occurs because the controller is unable to discover/reach the DUT over mDNS after the device joins the Thread network, resulting in a CASE session establishment timeout.

However, both test cases(TC-DD-3.23 and TC-DD-3.24) passed successfully when validated through THGUI.

Steps to reproduce the behavior

1. Mount the NFC Reader (HID OMNIKEY) and DUT (STM32WBA65I-DK1 board) to the controller.
2. Place the NFC reader and the DUT NFC antenna physically close to each other.
3. Reset the DUT (STM32WBA65I-DK1 board) by pressing the blue Reset button.
4. Navigated to the CLI scripts directory: cd certification-tool/cli/scripts
5. Create the project: th-cli project create --name <Project name>
6. Navigate to certification-tool/cli/th_cli path and Update the configuration as mentioned below:
   "int-arg": "NFC_Reader_index:0"
7. Run the test in CLI:
   th-cli run-tests --tests-list TC_DD_3_23 --project-id <id> -c default_config.json -n <Name of the test run execution>
th-cli run-tests --tests-list TC_DD_3_24 --project-id <id> -c default_config.json -n <Name of the test run execution>

Expected behavior

Both TC-DD-3.23 and TC-DD-3.24 test cases should pass without any errors.

Log files

THGUI Pass Logs:

TC-DD-3.23 Logs:
Controller Log: TC-DD-3_23_2026_05_26_07_40_38.log
DUT Log: TC-DD-3_23-THGUI-DUT_log.txt

TC-DD-3.24 Logs:
Controller Log: TC-DD-3_24_2026_05_26_07_48_21.log
DUT Log: TC-DD-3_24-THGUI-DUT_log.txt

Failure CLI Logs:

TC-DD-3.23 Logs:
Controller Log: TC-DD-3_23.log
DUT Log: TC-DD-3_23-CLI-Failure-DUT_log.txt

TC-DD-3.24 Logs:
Controller Log: TC-DD-3_24_2026_05_26_07_48_21.log
DUT Log: TC-DD-3_24-CLI-Failure-DUT_log.txt

Screenshots

Environment

Version: v2.15+spring2026
Sha: 2ac562d

[Pradeep-grl]

@rquidute, Could you please take a look at this issue.

[rquidute]

Hi @Pradeep-grl, could you please double check the controller logs for TC-DD-3.24? It seems it is the same for both GUI and CLI.

[Pradeep-grl]

Apologies, @rquidute . I mistakenly uploaded the failure CLI controller log for TC-DD-3.24 incorrectly. Please find the correct log attached below.

Failure CLI Log:

TC-DD-3.24 Controller Log: TC-DD-3_24.log

THGUI Pass Log:

TC-DD-3.24 Controller Log: TC-DD-3_24_2026_05_26_07_48_21.log

[rquidute]

Thank you for providing the logs @Pradeep-grl . After careful analysis we've identified a likely the root cause.

It's not a CLI vs GUI issue — it's a project configuration difference.

• The GUI project has the thread section configured as:
  "thread": {"operational_dataset_hex": "0e08...fd3c..."}

• The CLI project has:
  "thread": {"rcp_serial_path": "/dev/ttyACM0", "dataset": {...}, "operational_dataset_hex": "0e08...fd3c..."}

To confirm: could you run the CLI test using a config with only operational_dataset_hex in the thread section (same shape as the GUI project), and share the result? This should reproduce the GUI pass behaviour in CLI.

Once you confirm the CLI is now passing with that config, we can proceed with the appropriate fix on our side

[Pradeep-grl]

Hi @rquidute ,
Thanks for analyzing the issue and identifying the root cause. Today, I validated both test cases under different scenarios:

1. First, I validated the test cases in THGUI, and both test cases passed successfully. Please find the attached controller logs below:
   THGUI Pass Logs:
   TC-DD-3.23: TC-DD-3_23_2026_05_27_10_02_58.log
   TC-DD-3.24: TC-DD-3_24_2026_05_27_10_07_53.log

2. Then, I navigated to certification-tool/cli/th_cli and created a new configuration file named nfc_config.json. I copied the entire UI configuration into this file and validated both test cases in CLI using this nfc_config.json. Both test cases passed successfully. Please find the attached controller logs below.
   CLI Pass Logs:
   TC-DD-3.23: TC-DD-3_23.log
   TC-DD-3.24: TC-DD-3_24.log

3. Next, I navigated to certification-tool/cli/th_cli and updated the Thread section in the existing default_config.json with "Operational_dataset_hex". I then validated both test cases in CLI using this updated default_config.json, and both test cases passed successfully. Please find the attached controller logs below.
   CLI Pass Logs:
   TC-DD-3.23: TC-DD-3_23.log
   TC-DD-3.24: TC-DD-3_24.log

4. I navigated to certification-tool/cli/th_cli and created another configuration file named copydefault_config.json. I copied the complete contents of default_config.json into this file and validated both test cases in CLI using this copydefault_config.json. Both test cases passed successfully. Please find the attached controller logs below.
   CLI Pass Logs:
   TC-DD-3.23: TC-DD-3_23.log
   TC-DD-3.24: TC-DD-3_24.log

All the above scenarios were tested using the same project.

5. As mentioned earlier in the issue, I then created a new project under certification-tool/cli/scripts. Using the previously working copydefault_config.json, I attempted to validate the test cases again in CLI. In this scenario, both test cases failed during the commissioning step after reading the payloads.
   From the controller logs, I observed that the CLI project contains:
   "thread": {"rcp_serial_path": "/dev/ttyACM0", "dataset": {...}, "operational_dataset_hex": "0e08...fd3c..."}
   Please find the attached failure controller logs below.
   TC-DD-3.23: TC-DD-3_23.log
   TC-DD-3.24: TC-DD-3_24.log

[rquidute]

Thank you @Pradeep-grl for the thorough validation across all scenarios — this is exactly what we needed to confirm the root cause.
We will proceed with opening a PR with this fix.

[rquidute]

@Pradeep-grl Can we assume as long as you are using operational_dataset_hexin project config the OTRB was started earlier outside TH, right?

Could you please test it locally using the steps below?

1. Update the backend submodule to the fix branch:
   cd certification-tool/backend
   git fetch origin
   git checkout fix/1004-thread-autoconfig-prefer-operational-dataset-hex

2. Get the new backend commit SHA:
   git rev-parse HEAD

3. Update the backend SHA in docker-compose.yml:
   Open certification-tool/docker-compose.yml and update the backend image tag with the SHA obtained in the previous step.

4. Rebuild the backend image:
   cd ..
   ./scripts/build.sh backend

5. Restart the Test Harness:
   ./scripts/stop.sh
   ./scripts/start.sh