chore(deps): Bump the actions group across 1 directory with 3 updates - #27
Closed
dependabot[bot] wants to merge 17 commits into
Closed
chore(deps): Bump the actions group across 1 directory with 3 updates#27dependabot[bot] wants to merge 17 commits into
dependabot[bot] wants to merge 17 commits into
Conversation
Adapters for Claude Code, Gemini CLI, Oxagen, Stella (+ mock for CI); held-out verification the agent can never author; normalized token and uniform cost accounting; matched-config enforcement with unmatched-model warnings; agent-error quarantine; Wilson/McNemar/seeded-bootstrap stats; per-run manifests, transcripts, diffs, and reproduce commands.
Repo moved to github.com/oxageninc/arena. Updates the package name, repository URL, README title + clone command, harness self-name in the run manifest and CLI banner, and the Stella link (now oxageninc/stella).
Adds harbor/ — a Python Harbor adapter (BaseInstalledAgent) that runs your coding agent through Harbor's containerized SWE-bench / Terminal-Bench verifier head-to-head against the agents Harbor already ships (Claude Code, Gemini, Codex, Cursor, Aider, ...), all scored by the same repo-native test suite. - ByoAgent: wire up your own agent from a TOML/JSON spec — no Python. - OxagenAgent / StellaAgent: specs for agents Harbor lacks; worked examples. - Spec-driven install (binary upload or script) + one-shot run + best-effort token/cost parsing with the same cache-aware normalization as the TS side. - A non-zero agent exit never aborts scoring — the verifier decides pass/fail. - Pure logic (spec/metrics/command) is Harbor-free and unit-tested; agent tests run against real Harbor 0.6.1 (39 tests green) and verify Harbor's factory loads each agent by import path. - Docs + run.sh + a CI job (pip install -e .; pytest). Main README + METHODOLOGY now point at this adapter as the scale-up path.
Harbor requires Python >=3.12, so the CI job used 3.11 and could not resolve harbor. Bump CI to 3.12, set requires-python accordingly, and pin the dep to the tested 0.6.x line.
Snapshot a run's per-agent metrics as a committed baseline, then fail CI when a later run regresses: - arena baseline save <runDir> [--agent A] -> arena-baseline.json - arena baseline show - arena gate <runDir> [--require-significant] [--allow-task-mismatch] [...] exits non-zero on regression. Checks per agent: resolve-rate drop (strict, or only when it clears 95% CI noise with --require-significant), median token/cost increase past thresholds, wall-clock (reported; enforced with --speed-increase). Refuses to compare across different task sets unless --allow-task-mismatch. Thresholds via flags or arena-gate.json. src/summary.ts factors the per-agent aggregation into one source of truth shared by report + baseline + gate. 14 new unit tests (51 total); example CI workflow + gate config under examples/. README + METHODOLOGY document it.
…, dependency review Quality: - Add Biome (lint + format), pinned devDep, as a CI gate (pnpm lint). Normalized formatting across src/test; removed dead code Biome caught (unused pricing local in orchestrator; unused test locals). Supply chain / security (standard for popular OSS repos): - .github/dependabot.yml — weekly npm + pip + github-actions updates. - dependency-review workflow — blocks PRs adding high-severity vulns. - OpenSSF Scorecard workflow — security-posture scoring to the Security tab. - SECURITY.md — private vulnerability reporting policy. - CODEOWNERS — routes review to the maintainer. - Least-privilege 'permissions: contents: read' on CI. typecheck + 51 tests + biome ci all green.
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 25.9.5 to 26.1.1. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) --- updated-dependencies: - dependency-name: "@types/node" dependency-version: 26.1.1 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Updates the requirements on [harbor](https://github.com/harbor-framework/harbor-cookbook) to permit the latest version. - [Commits](https://github.com/harbor-framework/harbor-cookbook/commits) --- updated-dependencies: - dependency-name: harbor dependency-version: 0.19.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Mac Anderson <mac@oxagen.sh>
Bumps [typescript](https://github.com/microsoft/TypeScript) from 5.9.3 to 7.0.2. - [Release notes](https://github.com/microsoft/TypeScript/releases) - [Commits](https://github.com/microsoft/TypeScript/commits) --- updated-dependencies: - dependency-name: typescript dependency-version: 7.0.2 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Mac Anderson <mac@oxagen.sh>
Bumps the actions group with 7 updates: | Package | From | To | | --- | --- | --- | | [actions/checkout](https://github.com/actions/checkout) | `4` | `7` | | [pnpm/action-setup](https://github.com/pnpm/action-setup) | `4` | `6` | | [actions/setup-node](https://github.com/actions/setup-node) | `4` | `6` | | [actions/setup-python](https://github.com/actions/setup-python) | `5` | `6` | | [actions/dependency-review-action](https://github.com/actions/dependency-review-action) | `4` | `5` | | [ossf/scorecard-action](https://github.com/ossf/scorecard-action) | `2.4.0` | `2.4.3` | | [github/codeql-action](https://github.com/github/codeql-action) | `3` | `4` | Updates `actions/checkout` from 4 to 7 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v7) Updates `pnpm/action-setup` from 4 to 6 - [Release notes](https://github.com/pnpm/action-setup/releases) - [Commits](pnpm/action-setup@v4...v6) Updates `actions/setup-node` from 4 to 6 - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v4...v6) Updates `actions/setup-python` from 5 to 6 - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5...v6) Updates `actions/dependency-review-action` from 4 to 5 - [Release notes](https://github.com/actions/dependency-review-action/releases) - [Commits](actions/dependency-review-action@v4...v5) Updates `ossf/scorecard-action` from 2.4.0 to 2.4.3 - [Release notes](https://github.com/ossf/scorecard-action/releases) - [Changelog](https://github.com/ossf/scorecard-action/blob/main/RELEASE.md) - [Commits](ossf/scorecard-action@v2.4.0...v2.4.3) Updates `github/codeql-action` from 3 to 4 - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: pnpm/action-setup dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-node dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-python dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/dependency-review-action dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: ossf/scorecard-action dependency-version: 2.4.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: github/codeql-action dependency-version: '4' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Mac Anderson <mac@oxagen.sh>
* chore(deps-dev): Bump vitest from 2.1.9 to 3.2.6 Bumps [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) from 2.1.9 to 3.2.6. - [Release notes](https://github.com/vitest-dev/vitest/releases) - [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md) - [Commits](https://github.com/vitest-dev/vitest/commits/v3.2.6/packages/vitest) --- updated-dependencies: - dependency-name: vitest dependency-version: 3.2.6 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> * fix * fix --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Mac Anderson <mac@oxagen.sh>
Replaces #18 — the Dependabot branch accumulated lockfile conflicts after the vitest-3.x and typescript-7 merges landed on main. This reapplies the same two patch bumps (biome 2.5.2->2.5.4, tsx 4.19.2->4.23.1) on a clean base with a freshly-regenerated lockfile.
…end (#20) Harness (TypeScript): - parse: brace-matched envelope scan so pretty-printed JSON (gemini-cli) parses; line-based JSONL scan kept as the first pass - adapters/base: kill the whole process group on timeout (agents spawn subprocesses that held the stdio pipes open past the cap); settle on "exit" with a drain grace; cache version (was two spawns per trial) - orchestrator: drop dead loadPricing call and module-level pricing cache (load once, pass down); timeout now takes precedence over a post-kill test pass; pass taskDir through AdapterRunArgs instead of the MockAdapter.currentTaskDir mutable static - workspace: async fs/child_process APIs (no event-loop blocking inside the async run loop); disable commit.gpgsign in seeded repos - report: aggregate via perAgentSummary (was duplicated inline math that contradicted summary.ts's single-source-of-truth contract); no more NaN cells when an agent has zero scored trials - cli: validate numeric flags instead of silently running with NaN - METHODOLOGY: document timeout precedence Harbor adapter (Python): - remove dead tomli fallback (package requires Python >= 3.12) - frozen dataclasses for specs and parsed metrics; validate metrics.kind - collections.abc.Mapping; class-level _agent_output/_spec defaults; tolerate float ARENA_TIMEOUT values - add ruff (lint config in pyproject, run in CI); fix all findings Branding + site: - black & white stadium mark (nested geometric stadium shapes), IBM Plex type, all-neutral palette; static site under web/ for arena.oxagen.sh - logo assets in assets/, logo in README
The arena and stella repositories moved from the oxageninc org to the macanderson account; point all badges, links, and package metadata at the new home.
…ch paper (#22) * fix: verification integrity, metric honesty, and CLI hardening Highest-severity first: - Timeouts kill the agent's whole process tree (POSIX process groups); trials can no longer hang on pipes held by orphaned grandchildren, and survivors can no longer tamper with held-out verification. - Zero-token (unparseable-envelope) trials are excluded from token/cost medians and deltas instead of dragging them toward zero past the gate. - claude-code preserves full model ids (no floating "sonnet" alias), so version pinning, matchedModels, and pricing lookups hold. - Gemini normalization counts thoughts (reasoning) and tool tokens. - Pretty-printed JSON envelopes parse (TS + Harbor Python mirror). - Per-trial error containment + incremental results.json; wall clock measures spawn-to-exit only; git-diff failure is "unknown", never "empty"; report table shares perAgentSummary with the gate (no NaN). - CLI: -o short flag (as documented), strict numeric flags, unknown verify ids error, duplicate agent specs rejected, typo'd gate thresholds error instead of silently disabling the check. - Cost with unpriced cache writes is null, never guessed. - Harbor: float-tolerant ARENA_TIMEOUT (never silently unbounded), loud empty-{budget} failure, metrics.kind validated at load. * feat(site): arena.oxagen.sh landing page + Agent Benchmark Protocol draft Static site (site/) for arena.oxagen.sh: landing page with the matched-model scoreboard, and the ABP design spec. Spec source lives at docs/agent-benchmark-protocol.md: four interchange schemas (task, run, trace, verdict), the live GitHub-issue task pipeline with golden traces, smallest-normalized-diff scoring, conformance levels, and roadmap. * docs: The State of Agent Benchmarking (paper) + site rendering Research paper surveying the July 2026 agent-benchmark landscape through the engine-vs-model lens: 28 cited references, gap analysis of four capabilities (matched-model isolation, efficiency grading, statistical CI gating, live tasks with golden traces), COI disclosed. Markdown source in docs/, rendered at arena.oxagen.sh/paper/. README links the site, paper, and spec. * docs: architecture map (module graph, trial data flow, contracts)
Bumps the actions group with 3 updates in the / directory: [actions/setup-node](https://github.com/actions/setup-node), [actions/setup-python](https://github.com/actions/setup-python) and [ossf/scorecard-action](https://github.com/ossf/scorecard-action). Updates `actions/setup-node` from 6 to 7 - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v6...v7) Updates `actions/setup-python` from 6 to 7 - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v6...v7) Updates `ossf/scorecard-action` from 2.4.3 to 2.4.4 - [Release notes](https://github.com/ossf/scorecard-action/releases) - [Changelog](https://github.com/ossf/scorecard-action/blob/main/RELEASE.md) - [Commits](ossf/scorecard-action@v2.4.3...v2.4.4) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-python dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: ossf/scorecard-action dependency-version: 2.4.4 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com>
dependabot
Bot
added
dependencies
dependabot
Bot
added
dependencies
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @dependabot[bot], you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
macanderson
force-pushed
the
dependabot/github_actions/actions-a09b3de486
branch
from
July 30, 2026 21:57
337427c to
688de70
Compare
Contributor
Author
|
This pull request was built based on a group rule. Closing it will not ignore any of these versions in future pull requests. To ignore these dependencies, configure ignore rules in dependabot.yml |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps the actions group with 3 updates in the / directory: actions/setup-node, actions/setup-python and ossf/scorecard-action.
Updates
actions/setup-nodefrom 6 to 7Release notes
Sourced from actions/setup-node's releases.
... (truncated)
Commits
8207627Migrate to ESM and upgrade dependencies (#1574)04be95cAdd cache-primary-key and cache-matched-key as outputs (#1577)7c2c68ddocs: Update caching recommendations to mitigate cache poisoning risks (#1567)6a61c03Merge pull request #1569 from jasongin/update-actions-cache-5.1.030eb73bResolve high-severity audit issues4e1a87aUpdate dist360237fStrict equality4f8aac5Bump@actions/cacheto 5.1.0, log cache write deniedf4a67bbOnly usemirrorTokeningetManifestif it's provided (#1548)0355742Remove dummy NODE_AUTH_TOKEN export (#1558)Updates
actions/setup-pythonfrom 6 to 7Release notes
Sourced from actions/setup-python's releases.
... (truncated)
Commits
5fda3b9Pin SHA commits and update docs with latest versions (#1338)4ab7e95Merge pull request #1337 from actions/philip-gai/bump-actions-cache-6-2-00f3a009Remove the pip-install input (#1336)f8cf429Migrate to ESM and upgrade dependencies (#1330)54baeeaValidate and retry manifest fetch to prevent silent failures (#1332)c709277Annotation code fix (#1335)6849080remove EOL Python versions and Bumps numpy text fixture (#1333)0903b46Bump certifi from 2020.6.20 to 2024.7.4 in /tests/data (#1328)Updates
ossf/scorecard-actionfrom 2.4.3 to 2.4.4Release notes
Sourced from ossf/scorecard-action's releases.
Commits
2d11466Bump action tag for v2.4.4 release (#1688)1bd3285🌱 Bump the docker-images group across 1 directory with 2 updates (#1...913edce🌱 Bump github.com/containerd/containerd from 1.7.32 to 1.7.33 (#1671)0957b8f🌱 Bump golang.org/x/net from 0.56.0 to 0.57.0 (#1680)f0061eb🌱 Bump google.golang.org/grpc from 1.81.1 to 1.82.1 (#1687)20ee732🌱 Bump github.com/sigstore/cosign/v2 from 2.6.3 to 2.6.4 (#1685)9f295ef🌱 Bump the github-actions group with 6 updates (#1686)69bf556🌱 Bump github.com/sigstore/sigstore-go from 1.1.4 to 1.2.0 (#1681)94e8b96🌱 Bump github.com/sigstore/rekor from 1.5.0 to 1.5.2 (#1673)c7a1b37🌱 Bump github.com/sigstore/fulcio from 1.8.5 to 1.8.6 (#1675)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot show <dependency name> ignore conditionswill show all of the ignore conditions of the specified dependency@dependabot ignore <dependency name> major versionwill close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself)@dependabot ignore <dependency name> minor versionwill close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself)@dependabot ignore <dependency name>will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself)@dependabot unignore <dependency name>will remove all of the ignore conditions of the specified dependency@dependabot unignore <dependency name> <ignore condition>will remove the ignore condition of the specified dependency and ignore conditions