From d8dad7606eb09adf72508572704cbbea6fb8f20f Mon Sep 17 00:00:00 2001 From: Lukasz Towarek Date: Sun, 21 Jun 2026 18:50:13 +0200 Subject: [PATCH 1/4] Enable UBSan for telemetry/web_server QEMU tests Scopes UndefinedBehaviorSanitizer to the component under test in the telemetry and web_server QEMU test apps, per ESP-IDF's documented pattern (project-wide UBSan grows code/data size too much to fit on-target). No new CI job needed - the existing QEMU matrix entries already build and run these test apps. Part of #87. Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_01QwTBvENKGMczMo9x7nAMrg --- CONTRIBUTING.md | 7 +++++++ car/components/telemetry/test_apps/CMakeLists.txt | 7 ++++++- car/components/web_server/test_apps/CMakeLists.txt | 5 +++++ 3 files changed, 18 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9ef3aff..5d9374a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -515,6 +515,13 @@ Each test scope can target a different execution environment. Choose based on wh Target testing is the authoritative environment. Host and QEMU environments accelerate iteration and expand CI coverage; they do not replace on-target validation. +The `telemetry` and `web_server` QEMU test apps build their component under test with +[UndefinedBehaviorSanitizer](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/fatal-errors.html#undefined-behavior-sanitizer-ubsan-checks) +enabled (`-fsanitize=undefined -fno-sanitize=shift-base`, scoped to that one component via +`idf_component_get_property`/`target_compile_options` in the test app's `CMakeLists.txt`). +UBSan is not enabled for the production firmware build — it grows code/data size too much to +fit on-target — so this is the only place it currently runs. + #### Excluding test files in QEMU mode When a test app supports both QEMU and hardware, prefer excluding hardware-only test files at the CMake level over scattering `#ifdef` guards inside test cases. Use a Kconfig boolean (for example `CONFIG_FOO_TEST_QEMU_MODE`) defined in `main/Kconfig.projbuild`, set it in `sdkconfig.defaults.qemu`, and conditionalize the file list in `main/CMakeLists.txt`: diff --git a/car/components/telemetry/test_apps/CMakeLists.txt b/car/components/telemetry/test_apps/CMakeLists.txt index 8a716d6..2a4f255 100644 --- a/car/components/telemetry/test_apps/CMakeLists.txt +++ b/car/components/telemetry/test_apps/CMakeLists.txt @@ -4,4 +4,9 @@ set(EXTRA_COMPONENT_DIRS "../.." "../../DFRobot_AXP313A/DFRobot_AXP313A/esp_idf" set(COMPONENTS main) include($ENV{IDF_PATH}/tools/cmake/project.cmake) -project(telemetry_test) \ No newline at end of file +project(telemetry_test) + +# Scope UBSan to the component under test (see CONTRIBUTING.md); enabling it +# project-wide grows code/data size too much to fit on-target. +idf_component_get_property(telemetry_lib telemetry COMPONENT_LIB) +target_compile_options(${telemetry_lib} PRIVATE "-fsanitize=undefined" "-fno-sanitize=shift-base") \ No newline at end of file diff --git a/car/components/web_server/test_apps/CMakeLists.txt b/car/components/web_server/test_apps/CMakeLists.txt index 78cfeac..2ccdc51 100644 --- a/car/components/web_server/test_apps/CMakeLists.txt +++ b/car/components/web_server/test_apps/CMakeLists.txt @@ -3,3 +3,8 @@ set(EXTRA_COMPONENT_DIRS "../.." "../../DFRobot_AXP313A/DFRobot_AXP313A/esp_idf" set(COMPONENTS main) include($ENV{IDF_PATH}/tools/cmake/project.cmake) project(web_server_test) + +# Scope UBSan to the component under test (see CONTRIBUTING.md); enabling it +# project-wide grows code/data size too much to fit on-target. +idf_component_get_property(web_server_lib web_server COMPONENT_LIB) +target_compile_options(${web_server_lib} PRIVATE "-fsanitize=undefined" "-fno-sanitize=shift-base") From 06088de1dd972ecf1a71a77cf39459e25297468d Mon Sep 17 00:00:00 2001 From: Lukasz Towarek Date: Mon, 22 Jun 2026 00:05:21 +0200 Subject: [PATCH 2/4] Make UBSan opt-in for telemetry/web_server QEMU tests UBSan was previously baked into both test apps' CMakeLists.txt unconditionally, so every default QEMU functional test run also paid the sanitizer's instrumentation cost and conflated "does the feature work" with "is there UB". Gate it behind a CONFIG__TEST_UBSAN Kconfig boolean (default n), set via a new sdkconfig.defaults.ubsan overlay, following the existing CONFIG__TEST_QEMU_MODE pattern. The default test-telemetry/test-web_server CI jobs build without it; new test-telemetry-ubsan/test-web_server-ubsan jobs combine the new overlay with sdkconfig.defaults.qemu to run the same QEMU tests with UBSan enabled, as a separate, dedicated step. Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_01QwTBvENKGMczMo9x7nAMrg --- .github/workflows/cpp-car.yml | 14 ++++++++++++-- CONTRIBUTING.md | 14 +++++++++----- car/components/telemetry/test_apps/CMakeLists.txt | 11 +++++++---- .../telemetry/test_apps/main/Kconfig.projbuild | 8 ++++++++ .../telemetry/test_apps/sdkconfig.defaults.ubsan | 1 + car/components/web_server/test_apps/CMakeLists.txt | 11 +++++++---- .../web_server/test_apps/main/Kconfig.projbuild | 8 ++++++++ .../web_server/test_apps/sdkconfig.defaults.ubsan | 1 + 8 files changed, 53 insertions(+), 15 deletions(-) create mode 100644 car/components/telemetry/test_apps/sdkconfig.defaults.ubsan create mode 100644 car/components/web_server/test_apps/sdkconfig.defaults.ubsan diff --git a/.github/workflows/cpp-car.yml b/.github/workflows/cpp-car.yml index 933de0f..902cf3a 100644 --- a/.github/workflows/cpp-car.yml +++ b/.github/workflows/cpp-car.yml @@ -64,6 +64,16 @@ jobs: qemu: true sdkconfig_overlay: sdkconfig.defaults.qemu pytest_pattern: pytest_web_server_qemu.py + - name: test-telemetry-ubsan + path: car/components/telemetry/test_apps + qemu: true + sdkconfig_overlay: sdkconfig.defaults.qemu;sdkconfig.defaults.ubsan + pytest_pattern: pytest_telemetry_qemu.py + - name: test-web_server-ubsan + path: car/components/web_server/test_apps + qemu: true + sdkconfig_overlay: sdkconfig.defaults.qemu;sdkconfig.defaults.ubsan + pytest_pattern: pytest_web_server_qemu.py container: image: ghcr.io/ltowarek/dust-mite-cpp-devcontainer @@ -86,9 +96,9 @@ jobs: uses: actions/cache@v5 with: path: ${{ github.workspace }}/.ccache - key: ccache-${{ runner.os }}-idf-${{ matrix.path }}-${{ github.run_id }} + key: ccache-${{ runner.os }}-idf-${{ matrix.name }}-${{ github.run_id }} restore-keys: | - ccache-${{ runner.os }}-idf-${{ matrix.path }}- + ccache-${{ runner.os }}-idf-${{ matrix.name }}- ccache-${{ runner.os }}-idf- - name: Build diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5d9374a..ea57cd2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -515,12 +515,16 @@ Each test scope can target a different execution environment. Choose based on wh Target testing is the authoritative environment. Host and QEMU environments accelerate iteration and expand CI coverage; they do not replace on-target validation. -The `telemetry` and `web_server` QEMU test apps build their component under test with +The `telemetry` and `web_server` QEMU test apps can build their component under test with [UndefinedBehaviorSanitizer](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/fatal-errors.html#undefined-behavior-sanitizer-ubsan-checks) -enabled (`-fsanitize=undefined -fno-sanitize=shift-base`, scoped to that one component via -`idf_component_get_property`/`target_compile_options` in the test app's `CMakeLists.txt`). -UBSan is not enabled for the production firmware build — it grows code/data size too much to -fit on-target — so this is the only place it currently runs. +(`-fsanitize=undefined -fno-sanitize=shift-base`, scoped to that one component via +`idf_component_get_property`/`target_compile_options` in the test app's `CMakeLists.txt`). This +is opt-in, gated behind a `CONFIG__TEST_UBSAN` Kconfig boolean (default `n`) set via +the `sdkconfig.defaults.ubsan` overlay — the default `test-telemetry`/`test-web_server` CI jobs +build and run without it, and a separate `test-telemetry-ubsan`/`test-web_server-ubsan` CI job +combines that overlay with `sdkconfig.defaults.qemu` to build and run the same QEMU tests with +UBSan enabled. UBSan is not enabled for the production firmware build — it grows code/data size +too much to fit on-target. #### Excluding test files in QEMU mode diff --git a/car/components/telemetry/test_apps/CMakeLists.txt b/car/components/telemetry/test_apps/CMakeLists.txt index 2a4f255..b25af7e 100644 --- a/car/components/telemetry/test_apps/CMakeLists.txt +++ b/car/components/telemetry/test_apps/CMakeLists.txt @@ -6,7 +6,10 @@ set(COMPONENTS main) include($ENV{IDF_PATH}/tools/cmake/project.cmake) project(telemetry_test) -# Scope UBSan to the component under test (see CONTRIBUTING.md); enabling it -# project-wide grows code/data size too much to fit on-target. -idf_component_get_property(telemetry_lib telemetry COMPONENT_LIB) -target_compile_options(${telemetry_lib} PRIVATE "-fsanitize=undefined" "-fno-sanitize=shift-base") \ No newline at end of file +# Opt-in via CONFIG_TELEMETRY_TEST_UBSAN (see CONTRIBUTING.md); scoped to the +# component under test since enabling UBSan project-wide grows code/data size +# too much to fit on-target. +if(CONFIG_TELEMETRY_TEST_UBSAN) + idf_component_get_property(telemetry_lib telemetry COMPONENT_LIB) + target_compile_options(${telemetry_lib} PRIVATE "-fsanitize=undefined" "-fno-sanitize=shift-base") +endif() \ No newline at end of file diff --git a/car/components/telemetry/test_apps/main/Kconfig.projbuild b/car/components/telemetry/test_apps/main/Kconfig.projbuild index 0f4ef8d..5bfc5c6 100644 --- a/car/components/telemetry/test_apps/main/Kconfig.projbuild +++ b/car/components/telemetry/test_apps/main/Kconfig.projbuild @@ -8,4 +8,12 @@ config TELEMETRY_TEST_QEMU_MODE binary runs under QEMU. Hardware-dependent test cases report TEST_IGNORE instead of crashing on missing peripherals. +config TELEMETRY_TEST_UBSAN + bool "Build telemetry with UndefinedBehaviorSanitizer" + default n + help + Compiles the telemetry component under test with -fsanitize=undefined. + Opt-in: not part of the default build, only enabled by the dedicated + UBSan CI job/sdkconfig overlay. + endmenu diff --git a/car/components/telemetry/test_apps/sdkconfig.defaults.ubsan b/car/components/telemetry/test_apps/sdkconfig.defaults.ubsan new file mode 100644 index 0000000..f75e953 --- /dev/null +++ b/car/components/telemetry/test_apps/sdkconfig.defaults.ubsan @@ -0,0 +1 @@ +CONFIG_TELEMETRY_TEST_UBSAN=y diff --git a/car/components/web_server/test_apps/CMakeLists.txt b/car/components/web_server/test_apps/CMakeLists.txt index 2ccdc51..079e315 100644 --- a/car/components/web_server/test_apps/CMakeLists.txt +++ b/car/components/web_server/test_apps/CMakeLists.txt @@ -4,7 +4,10 @@ set(COMPONENTS main) include($ENV{IDF_PATH}/tools/cmake/project.cmake) project(web_server_test) -# Scope UBSan to the component under test (see CONTRIBUTING.md); enabling it -# project-wide grows code/data size too much to fit on-target. -idf_component_get_property(web_server_lib web_server COMPONENT_LIB) -target_compile_options(${web_server_lib} PRIVATE "-fsanitize=undefined" "-fno-sanitize=shift-base") +# Opt-in via CONFIG_WEB_SERVER_TEST_UBSAN (see CONTRIBUTING.md); scoped to the +# component under test since enabling UBSan project-wide grows code/data size +# too much to fit on-target. +if(CONFIG_WEB_SERVER_TEST_UBSAN) + idf_component_get_property(web_server_lib web_server COMPONENT_LIB) + target_compile_options(${web_server_lib} PRIVATE "-fsanitize=undefined" "-fno-sanitize=shift-base") +endif() diff --git a/car/components/web_server/test_apps/main/Kconfig.projbuild b/car/components/web_server/test_apps/main/Kconfig.projbuild index acd1197..28fdeb8 100644 --- a/car/components/web_server/test_apps/main/Kconfig.projbuild +++ b/car/components/web_server/test_apps/main/Kconfig.projbuild @@ -8,4 +8,12 @@ config WEB_SERVER_TEST_QEMU_MODE test binary runs under QEMU. Hardware-dependent test cases report TEST_IGNORE instead of blocking on missing peripherals. +config WEB_SERVER_TEST_UBSAN + bool "Build web_server with UndefinedBehaviorSanitizer" + default n + help + Compiles the web_server component under test with -fsanitize=undefined. + Opt-in: not part of the default build, only enabled by the dedicated + UBSan CI job/sdkconfig overlay. + endmenu diff --git a/car/components/web_server/test_apps/sdkconfig.defaults.ubsan b/car/components/web_server/test_apps/sdkconfig.defaults.ubsan new file mode 100644 index 0000000..f82640a --- /dev/null +++ b/car/components/web_server/test_apps/sdkconfig.defaults.ubsan @@ -0,0 +1 @@ +CONFIG_WEB_SERVER_TEST_UBSAN=y From 79754b738d5b8a1aabf393fd4a9cf2491c0c04e0 Mon Sep 17 00:00:00 2001 From: Lukasz Towarek Date: Mon, 22 Jun 2026 00:09:55 +0200 Subject: [PATCH 3/4] Run UBSan as a dedicated CI job, not extra build matrix entries Adding -ubsan variants to the build job's existing matrix mixed a sanitizer run in among feature builds and hardware test builds that have nothing to do with it. Give UBSan its own job, mirroring clang-format/clang-tidy: a dedicated job with a matrix scoped only to the opted-in test apps, building each with sdkconfig.defaults.qemu; sdkconfig.defaults.ubsan and running its QEMU tests. Move the CONTRIBUTING.md description into the quality-checks section alongside clang-format/clang-tidy, generalized to "a QEMU test app can opt in" rather than naming the two current adopters, and focused on how to wire a new one up. Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_01QwTBvENKGMczMo9x7nAMrg --- .github/workflows/cpp-car.yml | 75 +++++++++++++++++++++++++++++------ CONTRIBUTING.md | 34 +++++++++++----- 2 files changed, 85 insertions(+), 24 deletions(-) diff --git a/.github/workflows/cpp-car.yml b/.github/workflows/cpp-car.yml index 902cf3a..0d8f2ab 100644 --- a/.github/workflows/cpp-car.yml +++ b/.github/workflows/cpp-car.yml @@ -64,16 +64,6 @@ jobs: qemu: true sdkconfig_overlay: sdkconfig.defaults.qemu pytest_pattern: pytest_web_server_qemu.py - - name: test-telemetry-ubsan - path: car/components/telemetry/test_apps - qemu: true - sdkconfig_overlay: sdkconfig.defaults.qemu;sdkconfig.defaults.ubsan - pytest_pattern: pytest_telemetry_qemu.py - - name: test-web_server-ubsan - path: car/components/web_server/test_apps - qemu: true - sdkconfig_overlay: sdkconfig.defaults.qemu;sdkconfig.defaults.ubsan - pytest_pattern: pytest_web_server_qemu.py container: image: ghcr.io/ltowarek/dust-mite-cpp-devcontainer @@ -96,9 +86,9 @@ jobs: uses: actions/cache@v5 with: path: ${{ github.workspace }}/.ccache - key: ccache-${{ runner.os }}-idf-${{ matrix.name }}-${{ github.run_id }} + key: ccache-${{ runner.os }}-idf-${{ matrix.path }}-${{ github.run_id }} restore-keys: | - ccache-${{ runner.os }}-idf-${{ matrix.name }}- + ccache-${{ runner.os }}-idf-${{ matrix.path }}- ccache-${{ runner.os }}-idf- - name: Build @@ -187,8 +177,67 @@ jobs: run: ./scripts/run_clang_tidy.sh working-directory: car + ubsan: + needs: build-docker + permissions: + contents: read + actions: read + + runs-on: ubuntu-latest + + strategy: + matrix: + include: + - name: telemetry + path: car/components/telemetry/test_apps + pytest_pattern: pytest_telemetry_qemu.py + - name: web_server + path: car/components/web_server/test_apps + pytest_pattern: pytest_web_server_qemu.py + + container: + image: ghcr.io/ltowarek/dust-mite-cpp-devcontainer + options: --user root + credentials: + username: ${{ github.repository_owner }} + password: ${{ secrets.GITHUB_TOKEN }} + + defaults: + run: + shell: bash + + steps: + - name: Checkout + uses: actions/checkout@v4 + with: + submodules: recursive + + - name: Cache ccache + uses: actions/cache@v5 + with: + path: ${{ github.workspace }}/.ccache + key: ccache-${{ runner.os }}-idf-ubsan-${{ matrix.name }}-${{ github.run_id }} + restore-keys: | + ccache-${{ runner.os }}-idf-ubsan-${{ matrix.name }}- + ccache-${{ runner.os }}-idf-ubsan- + + - name: Build + env: + CCACHE_DIR: ${{ github.workspace }}/.ccache + SDKCONFIG_DEFAULTS: sdkconfig.defaults;sdkconfig.defaults.qemu;sdkconfig.defaults.ubsan + run: | + source $IDF_PATH/export.sh + idf.py build + working-directory: ${{ matrix.path }} + + - name: Run QEMU tests + run: | + source $IDF_PATH/export.sh + pytest ${{ matrix.pytest_pattern }} --embedded-services idf,qemu -v + working-directory: ${{ matrix.path }} + ci-status-cpp-car: - needs: [build-docker, build, clang-format, clang-tidy] + needs: [build-docker, build, clang-format, clang-tidy, ubsan] if: always() runs-on: ubuntu-latest steps: diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ea57cd2..c8f8d30 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -452,6 +452,29 @@ tolerates that link failure and only requires `compile_commands.json` to be gene `clang-tidy` CI job is blocking (part of `ci-status-cpp-car`); since `WarningsAsErrors` is empty, it only fails on a genuine clang-tidy tool error, not on findings. +#### UBSan + +A QEMU test app can opt in to building its component under test with +[UndefinedBehaviorSanitizer](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/fatal-errors.html#undefined-behavior-sanitizer-ubsan-checks). +Enable it by adding a `CONFIG__TEST_UBSAN` Kconfig boolean (default `n`) to the test +app's `main/Kconfig.projbuild`, a matching `sdkconfig.defaults.ubsan` overlay that sets it to +`y`, and guarding the sanitizer flags in the test app's `CMakeLists.txt`: + +```cmake +if(CONFIG__TEST_UBSAN) + idf_component_get_property(_lib COMPONENT_LIB) + target_compile_options(${_lib} PRIVATE "-fsanitize=undefined" "-fno-sanitize=shift-base") +endif() +``` + +Scoping the flags to just the component under test (rather than the whole app) follows ESP-IDF's +documented pattern — enabling UBSan project-wide grows code/data size too much to fit on-target, +which is also why it is never enabled for the production firmware build. A dedicated `ubsan` CI +job builds and runs each opted-in test app with `sdkconfig.defaults.ubsan` layered on top of +`sdkconfig.defaults.qemu`, using its own matrix (separate from the `build` job's matrix) so the +default test apps keep building without the sanitizer. The `ubsan` CI job is blocking (part of +`ci-status-cpp-car`). + ### Car test types Test scope and execution environment are independent choices. Scope determines what is under test; environment determines where it runs. @@ -515,17 +538,6 @@ Each test scope can target a different execution environment. Choose based on wh Target testing is the authoritative environment. Host and QEMU environments accelerate iteration and expand CI coverage; they do not replace on-target validation. -The `telemetry` and `web_server` QEMU test apps can build their component under test with -[UndefinedBehaviorSanitizer](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/fatal-errors.html#undefined-behavior-sanitizer-ubsan-checks) -(`-fsanitize=undefined -fno-sanitize=shift-base`, scoped to that one component via -`idf_component_get_property`/`target_compile_options` in the test app's `CMakeLists.txt`). This -is opt-in, gated behind a `CONFIG__TEST_UBSAN` Kconfig boolean (default `n`) set via -the `sdkconfig.defaults.ubsan` overlay — the default `test-telemetry`/`test-web_server` CI jobs -build and run without it, and a separate `test-telemetry-ubsan`/`test-web_server-ubsan` CI job -combines that overlay with `sdkconfig.defaults.qemu` to build and run the same QEMU tests with -UBSan enabled. UBSan is not enabled for the production firmware build — it grows code/data size -too much to fit on-target. - #### Excluding test files in QEMU mode When a test app supports both QEMU and hardware, prefer excluding hardware-only test files at the CMake level over scattering `#ifdef` guards inside test cases. Use a Kconfig boolean (for example `CONFIG_FOO_TEST_QEMU_MODE`) defined in `main/Kconfig.projbuild`, set it in `sdkconfig.defaults.qemu`, and conditionalize the file list in `main/CMakeLists.txt`: From 0d924e003234a9049d710b6cf8cb8cb45aead8e0 Mon Sep 17 00:00:00 2001 From: Lukasz Towarek Date: Mon, 22 Jun 2026 00:10:18 +0200 Subject: [PATCH 4/4] Use full name for UndefinedBehaviorSanitizer (UBSan) heading Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_01QwTBvENKGMczMo9x7nAMrg --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c8f8d30..ca77a27 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -452,7 +452,7 @@ tolerates that link failure and only requires `compile_commands.json` to be gene `clang-tidy` CI job is blocking (part of `ci-status-cpp-car`); since `WarningsAsErrors` is empty, it only fails on a genuine clang-tidy tool error, not on findings. -#### UBSan +#### UndefinedBehaviorSanitizer (UBSan) A QEMU test app can opt in to building its component under test with [UndefinedBehaviorSanitizer](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/fatal-errors.html#undefined-behavior-sanitizer-ubsan-checks).