The conformance suite reports 362 passing, 48 skipped of 410 specs. The 48 are not engine failures — they fail to load:
YamlDotNet.Core.YamlException : Property 'parentId' not found on type 'BattleScribeSpec.Roster.StepDef'.
YamlDotNet.Core.YamlException : Property 'catalogues' not found on type 'BattleScribeSpec.Roster.ExpectedStateDef'.
The pinned revision of lib/battlescribe-spec (e40d06c) is internally inconsistent: its spec fixtures use properties that its own TestKit C# types do not define.
These were previously invisible
ConformanceTests.AllSpecs used to swallow load failures with a bare catch { continue; }, silently removing those specs from the run. The suite reported 351 specs, all passing, while 48 were being discarded. #322 changed this to report them as skipped, which is where the number comes from.
This matters beyond the count: any future change that breaks spec loading would have looked green. It already caught one — bumping YamlDotNet to 18.1.0 breaks SpecLoader entirely, taking all 362 passing specs to 0 (see the hold in #327).
Fixing it
The pin is 58 commits behind battlescribe-spec main. Moving to upstream head resolves the parse failures but is not a drop-in: the suite becomes 489 specs with 119 failures, i.e. genuine engine conformance work against newer spec expectations.
That is a real piece of work, not a submodule bump — worth scoping separately. Until then the pin stays and the 48 are honestly reported as skipped.
The conformance suite reports 362 passing, 48 skipped of 410 specs. The 48 are not engine failures — they fail to load:
The pinned revision of
lib/battlescribe-spec(e40d06c) is internally inconsistent: its spec fixtures use properties that its own TestKit C# types do not define.These were previously invisible
ConformanceTests.AllSpecsused to swallow load failures with a barecatch { continue; }, silently removing those specs from the run. The suite reported 351 specs, all passing, while 48 were being discarded. #322 changed this to report them as skipped, which is where the number comes from.This matters beyond the count: any future change that breaks spec loading would have looked green. It already caught one — bumping YamlDotNet to 18.1.0 breaks
SpecLoaderentirely, taking all 362 passing specs to 0 (see the hold in #327).Fixing it
The pin is 58 commits behind
battlescribe-specmain. Moving to upstream head resolves the parse failures but is not a drop-in: the suite becomes 489 specs with 119 failures, i.e. genuine engine conformance work against newer spec expectations.That is a real piece of work, not a submodule bump — worth scoping separately. Until then the pin stays and the 48 are honestly reported as skipped.