CII wave 2: five more validated tasks (9/30) - #53
Merged
Conversation
All five pass scripts/validate_tasks.py --sandbox docker (gold=1.0, pre-patch=0.0, deterministic x3) with complexity-scaled budgets stamped: - oss-packaging-interpreter-tag-identifier (py, multi_file/easy): pypa/packaging #1351 — non-identifier interpreter tags silently accepted and mis-split. Deliberate easy anchor. - oss-echo-problem-details (go, multi_file/medium): labstack/echo #3062 — RFC 9457 Problem Details error handler. Vendored, go-1.26. - oss-zod-codepoint-length (ts, system/hard): zod #6441 — string lengths in code points across checks + compiled codegen path. - oss-clap-mangen-override-usage (rust, system/medium): clap #6467 — man-page SYNOPSIS ignores override_usage; cross-crate fix. Vendored, rust-2024. - oss-sqlglot-multi-table-ddl (py, system/hard): sqlglot #8229 — breaking AST change for multi-table ANALYZE/DROP across 6 files. Wave-2 dedup caught anyio#1228 and attrs#1592 already living in python-1. CANDIDATES.md updated with admissions + shortlist state. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second task wave for the VulcanBench Coding Intelligence Index — five new tasks, all through the full admission gate (gold=1.0, pre-patch=0.0, deterministic ×3 in Docker) with complexity-scaled budgets stamped and
stamp_task_budgets.py --checkclean across the suite.Suite is now 9/30 — languages Py 3 / TS 2 / Go 2 / Rust 2, complexity 6× multi_file + 3× system, with one deliberate easy anchor. Dedup sweep caught two shortlisted PRs already living in
python-1(anyio#1228, attrs#1592) before any work was wasted.Vendored-task integrity (the #52 lesson) is proven again: clean
git archivecheckouts of the clap and echo slices build offline (cargo --offlinechecksums intact under thetasks/*/*/repo/** -textrule).Test plan
vulcanbench run --suite cii --model mock:synthetic --sandbox local --max-concurrency 3— all 9 tasks, exit 0--checkclean🤖 Generated with Claude Code