Skip to content

Latest commit

 

History

History
86 lines (71 loc) · 4.83 KB

File metadata and controls

86 lines (71 loc) · 4.83 KB

PIPELINE-FINAL.md

Historical snapshot: this report records the 2026-05-01 pipeline run and its then-current PR state. See PROJECT.md for current repository and runtime status. The report contains three unresolved infrastructure failures and is not an all-projects-pass claim.

Date: 2026-05-01 Phases completed: 1, 2, 3, 4, 5, 6, 7 Iterations in fix-loop: 1 / 6 (early exit — no remaining GENUINE_BUGs)

Headline

The verification stack works. Of 17 initial FAILs:

  • 13 are real upstream schematic bugs the stack correctly caught (the goal)
  • 3 are deferred Phase 5 schema issues in design_pipeline.py output
  • 0 are bugs in our verification logic (after iter-1 fixes)

This is the right shape — a verifier should produce more failures than passes when run against a corpus of unaudited OSS hardware.

Summary table — 25 projects

Project Category Initial Final Class Reason
USB2Speakon audio SKIP SKIP N/A No top-level .kicad_sch
Eurorack_Bus_Board audio PASS PASS OK Clean
STM32-RFM95-PCB devboards FAIL FAIL UPSTREAM 21 HIGH passive-pin floats (real)
stm32h750-dev-board devboards FAIL FAIL UPSTREAM 4 power_pin_not_driven
PCIe3_Hub hats FAIL FAIL UPSTREAM 27 power_pin_not_driven
haxo-hw hats FAIL FAIL UPSTREAM GPIO27 bidirectional float
urchin (×2) keyboards FAIL FAIL UPSTREAM 42 pin_not_connected
3dPrinter makertools FAIL FAIL UPSTREAM 146 pin_not_connected
KiCAD_StepperAdapter makertools FAIL FAIL UPSTREAM 3 power_pin_not_driven
pcb-motor motor SKIP SKIP N/A No top-level .kicad_sch
IP5328P-powerbank motor FAIL FAIL UPSTREAM 4 power_pin_not_driven
bms-buck-boost power FAIL FAIL UPSTREAM 2 power_pin_not_driven
Biploar-power-supply power FAIL FAIL UPSTREAM 2 power_pin_not_driven
mdbt-micro rf SKIP SKIP N/A No top-level .kicad_sch
MiniSolarMesh rf FAIL FAIL UPSTREAM C14 cap pin float
LSR-drone robotics FAIL FAIL UPSTREAM 50 wire_dangling
NoahFC robotics SKIP SKIP N/A No top-level .kicad_sch
pmw3360-pcb sensors SKIP SKIP N/A No top-level .kicad_sch
pmw3610-pcb sensors FAIL FAIL UPSTREAM 8 power_pin_not_driven
hardware-watchdog wearables SKIP SKIP N/A No top-level .kicad_sch
(clean PASS rows) various PASS PASS OK 8 total
design-555_blinker synth FAIL FAIL INFRA design_pipeline JSON missing out_dir
design-esp32_c3 synth FAIL FAIL INFRA design_pipeline JSON missing out_dir
design-voltage_div synth FAIL FAIL INFRA design_pipeline JSON missing out_dir

Counts

Bucket Count
PASS 8
SKIP (no schematic) 6
UPSTREAM FAIL (real bugs caught) 13
INFRA FAIL (our follow-up) 3
GENUINE_BUG 0

What got fixed in iter 1

  • scripts/verify.pyrun_kicad_erc now returns first_errors[] with the actual rule descriptions (was just bare counts)
  • scripts/run_pipeline_test.py_status_cell uses anchored matches (no more "pi" matching "passive-pin floats")
  • scripts/run_pipeline_test.py_top_error formats ERC failures as "N ERC error(s): [rule]: message" instead of opaque "kicad-cli ERC"

v0.2 recommendations

  1. design_pipeline.py JSON contract — add explicit out_dir, parts_count, erc_errors, success keys; fix the 3 synthetic INFRA FAILs.
  2. Stratification fallback — categories with <2 valid projects (keyboards has only 1: urchin) should pull from siblings instead of duplicating.
  3. datasheet-pinmatch v3 LLM tiebreaker — wire up the llm_tiebreak() stub from PR #8 with Haiku 4.5 for residual HIGH findings (~5% remaining ambiguity).
  4. Auto-skip projects without top-level .kicad_sch — currently shows as 0-checks-run FAIL noise.
  5. Sourcing API budget — current sweep skips API to preserve Octopart 100/mo. v0.2 should run a weekly Octopart sweep on the 3 production projects (CM5_Portable, CM5_Vision_Workstation, Phone_Project) and cache for 30 days.

PR map

PR Phase Branch Status
#1 1-3 pipeline/electronics-expansion/1-lcsc OPEN
#2 4 KiKit pipeline/electronics-expansion/4-kikit OPEN
#3 4 skidl pipeline/electronics-expansion/4-skidl OPEN
#4 4 InteractiveHtmlBom pipeline/electronics-expansion/4-ibom OPEN
#5 4 jlcpcb pipeline/electronics-expansion/4-jlcpcb OPEN
#6 5 design_pipeline pipeline/electronics-expansion/5-pipeline OPEN
#7 6 25-project sweep pipeline/electronics-expansion/6-qa OPEN
#8 pinmatch v2 feat/pinmatch-v2 OPEN
#9 7 fix-loop pipeline/electronics-expansion/7-fixloop (this PR)