Skip to content

Commit dbffe4d

Browse files
committed
Update specification of policies for cmake tests
1 parent f3fd655 commit dbffe4d

4 files changed

Lines changed: 83 additions & 3 deletions

File tree

AGENTS.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,33 @@ Keep ROS-related changes confined to `ros2/` plus the documented root helpers, d
1111
<!-- ros2-overlay-end -->
1212
For python: Use python standard >= 3.12, matplotlib is the backend for most plots, but for images PIL and opencv are also used. For any statistics-like plot prefer seaborn, my default choice. Use pytorch for machine learning applications, supported by sklearn. Function names beings with Capital letter, snake case, methods not. Classes Similarly. Internal methods (not public API) must start with _, local scope variables end with _. All methods of classes shall start with small letter. Prefer dataclasses instead of dicts and enums instead of Literals if more than two entries. Type Hints Must Always Be Present. Onnx Export Compatibility Is Generally Required. When Writing New Classes Or Functions, A Runnable Example Should Always Be Present With Output To Show Results.
1313
For C++/CUDA: C++17 and C++20 are the core standards. CUDA mainly >12.6. Answers should be on point without too many digressions, technical (for intermediate and advanced users) but simple enough to explain the concepts. Prefer using concepts over SFINAE. Unit tests using Catch2. Check files to see convention of names. Prefer Classes over structs.
14+
15+
### CMake and derived-project test policy
16+
17+
Do not copy template-conformance CMake verifiers into a derived project merely
18+
because the donor template has them. In particular, do not register tests that
19+
recursively configure and rebuild the same derived project inside its ordinary
20+
CTest suite when a fresh configure/build/install/consumer command or CI job
21+
already proves the contract.
22+
23+
For a derived project:
24+
25+
- prefer Catch2 or pytest for project runtime behavior;
26+
- validate CMake options, headless/full feature matrices, installation,
27+
packaging, and external consumers through explicit fresh out-of-tree
28+
acceptance commands owned by local CI;
29+
- use disposable consumer projects outside the normal test build when nested or
30+
installed consumption must be proven;
31+
- add a permanent CMake-script test only when it is lightweight, target-owned,
32+
isolates behavior unavailable through an existing target/test, and does not
33+
recursively rebuild the project;
34+
- never import `VerifyTemplateProject*` or other donor self-validation tests as
35+
product tests.
36+
37+
The template repository may retain broader conformance tests because it owns
38+
generic generation and tailoring behavior. That exception does not make those
39+
tests part of the derived-project contract.
40+
1441
For MATLAB: Use classes a lot also in MATLAB, with a python style, but do it only when it makes sense. Functions in MATLAB are often more efficient. Evaluate whether it makes sense to have stateful implementation. Use "self" instead of "obj". All variables names must specify the datatype of the variable since MATLAB does not (hungarian notation). The following list applies: d for double, f for float, b for bool, str for struct and not for strings, char for strings and chars, ui8 for uint8, i8 for int8. All the other integers are similar to the latter. Specify "obj" as prefix if an object, cell if a cell, table if a table; "bus_" if a Simulink bus. The names are always in Pascal case including the prefix, for instance ui8MyVariable. Never nest functions definitions within other functions, always do them separate or at most in the same file (after the main function implementation). Add them as local in the same function file only when not re-used elsewhere, otherwise prefer a single implementation. Function names and static methods of classes starts with Capital letter. Local functions names ends with underscore meaning "private". Names of variables must be explicative and tell what the variable does. Short names are not allowed unless "very local in scope". Use underscore for those variables and preferably Tmp within the name. For codes that are intended to be algorithms of some kind (e.g. not plots or things to run on the host PC), make them always MATLAB codegen safe (especially if codegen directive is used). In that case names should be limited to 31 chars. Add the same template of doc to functions as below and always specify arguments-end block for input and output:
1542
%% SIGNATURE
1643
%

doc/developments/derived_project_upgrade_agent_guidelines.md

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,9 @@ Before any edit:
118118
- record generated artifacts or package-consumer checks that define success;
119119
- identify user-facing APIs, options, names, and workflows that must remain
120120
stable;
121-
- add a focused red guard first when the donor delta fixes a reproducible bug.
121+
- add a focused red guard first when the donor delta fixes a reproducible bug,
122+
but do not turn a one-time feature-matrix or consumer acceptance build into a
123+
recursive test in the target's ordinary CTest suite.
122124

123125
After the update:
124126

@@ -131,6 +133,18 @@ After the update:
131133
- test installed or packaged consumption when the update touches exports,
132134
metadata, headers, or dependency flow.
133135

136+
Keep template conformance and target acceptance separate:
137+
138+
- do not copy donor `tests/cmake/VerifyTemplateProject*` files into a derived
139+
repository;
140+
- do not register recursive configure/build/install tests in ordinary target
141+
CTest when explicit fresh acceptance commands or CI already own the contract;
142+
- use disposable nested/build-tree/install-tree consumers outside the normal
143+
test build;
144+
- add a permanent CMake-script test only when it is lightweight, target-owned,
145+
non-recursive, and covers behavior unavailable through Catch2, pytest, an
146+
existing target, or the acceptance matrix.
147+
134148
If the baseline is already red, record the exact failure set before proceeding.
135149
The update may continue only when the failure is unrelated and the user agrees.
136150
Do not report the target as green; report that no new regression was introduced
@@ -252,6 +266,8 @@ considered.
252266
- Confirm which donor tests are template-only and which contracts must survive
253267
in a derived project.
254268
- Do not treat template-validation workflows as project workflows.
269+
- Translate applicable donor CMake conformance into target acceptance commands;
270+
do not copy recursive verifier scripts into target CTest.
255271

256272
### Stage 1 - Baseline the target and capture tailoring
257273

@@ -272,7 +288,9 @@ considered.
272288
- List each applicable donor delta and its classification.
273289
- Name exact target files and line/semantic anchors.
274290
- Define allowed and frozen surfaces.
275-
- Add red-green tests before implementation where practical.
291+
- Add red-green tests before implementation where practical. For build-system
292+
feature matrices and consumer behavior, use a recorded failing acceptance
293+
command rather than a permanent recursive CTest.
276294
- Order changes so every commit candidate is internally coherent and testable.
277295
- Obtain user approval when the plan changes architecture, dependencies, public
278296
API, release behavior, or established tailoring.

doc/template_usage.md

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,21 @@ Only the main project configures documentation, tests, examples, wrappers, and g
134134

135135
## Tests
136136

137-
Use Catch2 for compiled tests, pytest for Python tests, and CTest as the common runner. Put compiled tests in `test*.cpp` or `test*.cu` files and Python tests in `test*.py` files. Add narrow CMake-script tests under `tests/cmake/` when the behavior is about configuration, installation, generated files, CI YAML, or documentation output rather than runtime C++ behavior.
137+
Use Catch2 for compiled tests, pytest for Python tests, and CTest as the common
138+
runner. Put compiled tests in `test*.cpp` or `test*.cu` files and Python tests in
139+
`test*.py` files.
140+
141+
The template repository uses `tests/cmake/` for template-owned conformance,
142+
tailoring, and generation checks. Do not copy those verifiers into a tailored
143+
project. A derived project should prove configuration, feature matrices,
144+
installation, packaging, and external consumption with explicit fresh
145+
out-of-tree commands in its acceptance/CI matrix. Do not make ordinary CTest
146+
recursively configure and rebuild the same project when CI already owns that
147+
behavioral gate.
148+
149+
A permanent derived-project CMake-script test is justified only when it is
150+
lightweight, project-specific, cannot be covered by an existing runtime target
151+
or acceptance command, and does not recursively rebuild the project.
138152

139153
Run focused checks during development. Prefer `ctest --test-dir <build>` so the
140154
same command works from the repository root, local scripts, and CI jobs:
@@ -202,6 +216,9 @@ The script replaces `template_project::logging` with the required project namesp
202216

203217
It also removes the root CMake hook for the template MATLAB regression helper and rewrites `tests/CMakeLists.txt` so only starter project unit tests remain registered.
204218

219+
This cleanup boundary is intentional: template CMake conformance tests remain
220+
owned by the donor/testfield validation harness, not by the tailored product.
221+
205222
### Workflow materialization
206223

207224
The runnable `.github/workflows/*.yml` files in this repository validate the

doc/testing_and_ci.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,24 @@ Compiled tests are discovered from `test*.cpp` and `test*.cu` files and run with
1616
Catch2. Python tests are discovered from `test*.py` files and registered as
1717
normal CTest entries that execute `python -m pytest -q <test-file>`.
1818

19+
### Template conformance versus derived-project acceptance
20+
21+
The template's default CTest suite includes conformance checks for generic
22+
tailoring, generated files, workflows, and package behavior. Those checks are
23+
template-maintainer infrastructure and are removed by normal tailoring.
24+
25+
Do not reproduce them as recursive CMake tests in a derived project. In
26+
particular, ordinary derived-project CTest must not configure and rebuild the
27+
same project again merely to cover an option combination, headless build,
28+
installation, package archive, or consumer build. Put those gates in explicit
29+
fresh out-of-tree acceptance commands or CI jobs, where build ownership,
30+
prerequisites, logs, and artifacts are visible.
31+
32+
Permanent CMake-script tests in a derived project are limited to lightweight,
33+
project-owned checks that cannot be expressed through Catch2, pytest, an
34+
existing build target, or the acceptance matrix. They must not import
35+
`VerifyTemplateProject*` scripts or create nested full-project builds.
36+
1937
Catch2 remains the unit-test framework for the core C++ and CUDA project. Tests
2038
inside ROS packages are the deliberate exception: they use
2139
`ament_cmake_gtest` so test targets and results participate in the ament/colcon

0 commit comments

Comments
 (0)