From PR #346 review (2026-07-02).
bfcl_v4_multi_turn_scorer.py:59 — score() untested. Untested branches: force_terminated early-fail, turn_count_mismatch, the checker_exception catch, valid/invalid bookkeeping, unweighted-mean category aggregation, and the empty-valid_subset_accs -> 0.0 branch. Can be tested by patching multi_turn_checker (same approach as the runner/execution-bridge tests). Also pin the string-typed subset_scores contract (f"{acc:.2f}").dataset_manager/predefined/bfcl_v4/__init__.py:180 — generate() selection logic untested. It defines the frozen ~995-sample MLPerf gate composition. Untested: unknown-category ValueError; multi-turn subset filtering + all-filtered empty-DataFrame warning; cached-parquet SHA re-verification; 0 < pct <= 100 validation; subset_floor full-take vs max(1, int(total*pct/100)); _resolve_subset_pct precedence; max_samples truncation. Testable with a synthetic frame monkeypatched past the load step. A regression here silently changes what the accuracy gate measures.
Deferred out of PR #346.
From PR #346 review (2026-07-02).
bfcl_v4_multi_turn_scorer.py:59—score()untested. Untested branches:force_terminatedearly-fail,turn_count_mismatch, thechecker_exceptioncatch, valid/invalid bookkeeping, unweighted-mean category aggregation, and the empty-valid_subset_accs-> 0.0 branch. Can be tested by patchingmulti_turn_checker(same approach as the runner/execution-bridge tests). Also pin the string-typedsubset_scorescontract (f"{acc:.2f}").dataset_manager/predefined/bfcl_v4/__init__.py:180—generate()selection logic untested. It defines the frozen ~995-sample MLPerf gate composition. Untested: unknown-categoryValueError; multi-turn subset filtering + all-filtered empty-DataFrame warning; cached-parquet SHA re-verification;0 < pct <= 100validation;subset_floorfull-take vsmax(1, int(total*pct/100));_resolve_subset_pctprecedence;max_samplestruncation. Testable with a synthetic frame monkeypatched past the load step. A regression here silently changes what the accuracy gate measures.Deferred out of PR #346.