Implement automatic discovery of platforms and devices with testdefs.h inheritance for flexible testing

[Copilot]

This PR addresses the testing inflexibility issue where device and platform combinations were manually maintained and often incomplete. The testing system now automatically discovers all available platforms from lib/node_types and all devices from devices.ini, generating appropriate test combinations with platform-specific test definitions using the new testdefs.h infrastructure.

Problem

The existing testing approach had several limitations:

• Device and platform lists were hardcoded in tests/conf_data.py
• New devices added to devices.ini were not automatically tested
• New platforms added to lib/node_types required manual test configuration updates
• Platform-specific test pins (IOT_TEST_*) were not properly utilized
• Test coverage was incomplete and maintenance-heavy

Solution

Auto-Discovery System with testdefs.h Integration

Created tests/auto_discovery.py with functions to automatically:

• Discover platforms: Scans lib/node_types/ directory for all available platforms (20 found vs 14 manual)
• Parse devices: Reads devices.ini to extract all device definitions (40 found vs 22 manual)
• Read test definitions: Extracts IOT_TEST_* definitions from each platform's testdefs.h files
• Handle inheritance: Implements folder-based inheritance following base directory symlinks (mirrors bin/prepare_build_dir logic)
• Generate syntax: Creates appropriate test code for each device/platform combination
• Platform compatibility: Intelligently determines device-platform compatibility based on available test definitions

Inheritance-Based Test Definition System

The system now uses the new testdefs.h infrastructure with proper inheritance:

// Inheritance examples:
// wemos_d1_mini → nodemcu (inherits D1, D2, D3, D4 pins)
// m5stickc_plus → m5stickc → esp32 (combines platform-specific + base definitions)
// esp32_c3_devkitm_1 → esp32 (inherits numeric pins + neopixel settings)

Intelligent Test Generation

The system generates platform-appropriate test syntax using inherited test definitions:

// ESP32 with numeric pins
servo(example_name, 15, 800);

// NodeMCU with D-pin names  
servo(example_name, D1, 800);

// M5StickC with platform-specific pins
input(example_name, BUTTON_HOME, "released", "pressed");

// I2C devices (no pins needed)
bmp180(example_name);

// Complex devices using multiple inherited pins
hcsr04(distance, IOT_TEST_DIGITAL, IOT_TEST_DIGITAL_2);

Integration & Compatibility

Modified tests/conf_data.py to:

• Use auto-discovery with testdefs.h by default
• Fall back to manual configuration if auto-discovery fails
• Preserve existing manual test cases for edge scenarios
• Maintain backward compatibility

Results

Test Coverage Improvement:

• Platforms: 20 auto-discovered vs 14 manual (+43%)
• Devices: 33 testable devices vs 22 manual (+50%)
• Total combinations: 587 vs ~308 manual (+91%)

Technical Improvements:

• Inheritance support: Proper base directory symlink following
• Smart compatibility: Devices only tested on compatible platforms
• testdefs.h integration: Uses extracted test definitions instead of parsing pins.h
• Robust inheritance: Handles complex chains like m5stickc_plus → m5stickc → esp32

Maintenance Benefits:

• Zero manual updates needed when adding new platforms to lib/node_types
• Zero manual updates needed when adding new devices to devices.ini
• Automatic use of correct platform-specific test definitions through inheritance
• Comprehensive test coverage without manual maintenance

Example Auto-Generated Combinations with Inheritance:

esp32 + servo: servo(example_name, 15, 800);
nodemcu + dht: dht(example_name, D1);
wemos_d1_mini + input: input(example_name, D3, "released", "pressed"); // inherited from nodemcu
m5stickc + m5stickc_imu: m5stickc_imu(example_name);
m5stickc_plus + output: led(example_name, ONBOARDLED, "turn on", "turn off"); // inherits from m5stickc
cyd + cyd_display: cyd_display(example_name);

The implementation ensures that compilation tests automatically adapt to the IoTempower ecosystem as it grows, leveraging the new testdefs.h infrastructure and inheritance system to provide comprehensive coverage while eliminating manual maintenance overhead.

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

[ulno]

@copilot sync latest changes (regarding extracting IOT_TEST definition into testdef.h) back into this PR branch and adjust the autodetection logic for testing accordingly to mainly use the new testdefs.h files for detecting if a specific device can be tested for a respective platform. Consider also the implicit folder-based inheritance (if no testdefs.h exist, follow the symlinks up the chain through the base-folder that might have a testdef.h for a class of platforms - this logic is used for compilation preparation by the bin/prepare_build_dir script that you can use as reference for modifying the autodetection logic - so for example the m5stickc_plus platform is based on the m5stickc which has which is an esp32).

[Copilot]

@copilot sync latest changes (regarding extracting IOT_TEST definition into testdef.h) back into this PR branch and adjust the autodetection logic for testing accordingly to mainly use the new testdef...

I've successfully synced the latest testdefs.h changes and updated the auto-discovery logic accordingly. The changes include:

Merged master branch to get the new testdefs.h files that extract IOT_TEST_* definitions from pins.h

Implemented inheritance-based auto-discovery that mirrors bin/prepare_build_dir script logic:

• Replaced pins.h parsing with testdefs.h reading
• Added recursive base directory following (e.g., m5stickc_plus → m5stickc → esp32)
• Platform inheritance now works correctly: wemos_d1_mini inherits nodemcu's D1/D2/D3/D4 pins

Enhanced platform compatibility checking with can_device_be_tested_on_platform() function that uses testdefs.h to determine device supportability per platform

The system now generates 587 auto-discovered combinations using the new testdefs.h infrastructure with proper folder-based inheritance, maintaining comprehensive coverage while eliminating manual maintenance.

Commit: 26e604d