Skip to content

Commit e1a7e4b

Browse files
haasonsaashaasonsaas
committed
Add agent authorship attribution convention
1 parent 0944b07 commit e1a7e4b

8 files changed

Lines changed: 365 additions & 1 deletion

‎.github/pull_request_template.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ Commands and evidence:
3232
## Agent / Automation Notes
3333

3434
- [ ] This PR was agent-authored or agent-assisted
35+
- [ ] Maestro-authored commits include the required authorship trailers
3536
- [ ] I checked live GitHub state before publishing
3637
- [ ] Review feedback and failing checks have been rechecked
3738
- [ ] No local secrets, generated scratch artifacts, or unrelated worktree changes are included

‎.github/scripts/classify-agent-authorship.rb‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
#!/usr/bin/env ruby
2+
# frozen_string_literal: true
3+
4+
require "json"
5+
require "optparse"
6+
7+
options = {
8+
github_output: nil,
9+
}
10+
11+
OptionParser.new do |parser|
12+
parser.on("--github-output PATH", "Append key=value outputs for GitHub Actions") do |path|
13+
options[:github_output] = path
14+
end
15+
end.parse!
16+
17+
input = ARGF.read
18+
19+
messages = input.each_line.map do |line|
20+
next if line.strip.empty?
21+
22+
parsed = JSON.parse(line)
23+
if parsed.is_a?(Hash)
24+
parsed.dig("commit", "message") || parsed["message"]
25+
end
26+
end.compact
27+
28+
required_patterns = {
29+
"co_author" => /^Co-Authored-By:\s*Maestro <maestro@evalops\.dev>\s*$/i,
30+
"version" => /^Maestro-Version:\s*\S.+$/i,
31+
"prompt_id" => /^Maestro-Prompt-Id:\s*\S.+$/i,
32+
"approvals_id" => /^Maestro-Approvals-Id:\s*\S.+$/i,
33+
}
34+
35+
marker_pattern = /
36+
^Co-Authored-By:\s*Maestro\s+<maestro@evalops\.dev>\s*$ |
37+
^Maestro-(?:Version|Prompt-Id|Approvals-Id):
38+
/ix
39+
40+
agent_commits = 0
41+
human_commits = 0
42+
incomplete_commits = 0
43+
44+
messages.each do |message|
45+
has_marker = message.lines.any? { |line| line.match?(marker_pattern) }
46+
47+
unless has_marker
48+
human_commits += 1
49+
next
50+
end
51+
52+
agent_commits += 1
53+
missing_required = required_patterns.values.any? do |pattern|
54+
message.lines.none? { |line| line.match?(pattern) }
55+
end
56+
incomplete_commits += 1 if missing_required
57+
end
58+
59+
label =
60+
if agent_commits.positive? && human_commits.positive?
61+
"mixed-authorship"
62+
elsif agent_commits.positive?
63+
"agent-authored"
64+
else
65+
"human-authored"
66+
end
67+
68+
outputs = {
69+
"label" => label,
70+
"total_commits" => messages.length,
71+
"agent_commits" => agent_commits,
72+
"human_commits" => human_commits,
73+
"incomplete_agent_commits" => incomplete_commits,
74+
}
75+
76+
outputs.each { |key, value| puts "#{key}=#{value}" }
77+
78+
if options[:github_output]
79+
File.open(options[:github_output], "a") do |file|
80+
outputs.each { |key, value| file.puts("#{key}=#{value}") }
81+
end
82+
end

‎.github/workflow-templates/agent-authorship-labels.properties.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"name": "Agent authorship labels",
3+
"description": "Apply agent-authored, human-authored, or mixed-authorship labels to pull requests based on Maestro commit trailers.",
4+
"iconName": "octicon tag",
5+
"categories": [
6+
"Automation",
7+
"Code review"
8+
]
9+
}

‎.github/workflow-templates/agent-authorship-labels.yml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
name: Agent authorship labels
2+
3+
on:
4+
pull_request_target:
5+
types: [opened, synchronize, reopened, ready_for_review, edited]
6+
7+
permissions:
8+
contents: read
9+
pull-requests: read
10+
issues: write
11+
12+
jobs:
13+
label:
14+
uses: evalops/.github/.github/workflows/agent-authorship-label.yml@main

‎.github/workflows/agent-authorship-label.yml‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
name: agent-authorship-label
2+
3+
on:
4+
workflow_call:
5+
inputs:
6+
fail_on_incomplete:
7+
description: "Fail when a Maestro-marked commit is missing one or more required Maestro trailers"
8+
required: false
9+
type: boolean
10+
default: false
11+
12+
permissions:
13+
contents: read
14+
pull-requests: read
15+
issues: write
16+
17+
jobs:
18+
label:
19+
runs-on: ubuntu-latest
20+
steps:
21+
- name: Checkout org workflow helpers
22+
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
23+
with:
24+
repository: evalops/.github
25+
ref: main
26+
path: org-defaults
27+
28+
- name: Resolve pull request
29+
id: pr
30+
shell: bash
31+
run: |
32+
set -euo pipefail
33+
34+
number="$(jq -r '.pull_request.number // empty' "${GITHUB_EVENT_PATH}")"
35+
if [ -z "${number}" ]; then
36+
echo "::error::agent-authorship-label must run from a pull_request or pull_request_target event."
37+
exit 1
38+
fi
39+
40+
echo "number=${number}" >> "${GITHUB_OUTPUT}"
41+
42+
- name: Fetch pull request commits
43+
shell: bash
44+
env:
45+
GH_TOKEN: ${{ github.token }}
46+
PR_NUMBER: ${{ steps.pr.outputs.number }}
47+
run: |
48+
set -euo pipefail
49+
gh api --paginate "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/commits" \
50+
--jq '.[] | {sha: .sha, message: .commit.message}' > commits.jsonl
51+
52+
- name: Classify authorship
53+
id: classify
54+
shell: bash
55+
run: |
56+
set -euo pipefail
57+
ruby org-defaults/.github/scripts/classify-agent-authorship.rb \
58+
--github-output "${GITHUB_OUTPUT}" \
59+
commits.jsonl
60+
61+
- name: Ensure authorship labels exist
62+
shell: bash
63+
env:
64+
GH_TOKEN: ${{ github.token }}
65+
run: |
66+
set -euo pipefail
67+
68+
ensure_label() {
69+
local name="$1"
70+
local color="$2"
71+
local description="$3"
72+
73+
if gh api "repos/${GITHUB_REPOSITORY}/labels/${name}" >/dev/null 2>&1; then
74+
gh api --method PATCH "repos/${GITHUB_REPOSITORY}/labels/${name}" \
75+
-f color="${color}" \
76+
-f description="${description}" >/dev/null
77+
else
78+
gh api --method POST "repos/${GITHUB_REPOSITORY}/labels" \
79+
-f name="${name}" \
80+
-f color="${color}" \
81+
-f description="${description}" >/dev/null
82+
fi
83+
}
84+
85+
ensure_label "agent-authored" "6f42c1" "All PR commits carry Maestro authorship trailers"
86+
ensure_label "human-authored" "0e8a16" "No PR commits carry Maestro authorship trailers"
87+
ensure_label "mixed-authorship" "fbca04" "Some PR commits carry Maestro authorship trailers"
88+
89+
- name: Apply authorship label
90+
shell: bash
91+
env:
92+
GH_TOKEN: ${{ github.token }}
93+
PR_NUMBER: ${{ steps.pr.outputs.number }}
94+
AUTHORSHIP_LABEL: ${{ steps.classify.outputs.label }}
95+
run: |
96+
set -euo pipefail
97+
98+
for label in agent-authored human-authored mixed-authorship; do
99+
gh api --method DELETE \
100+
"repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/labels/${label}" >/dev/null 2>&1 || true
101+
done
102+
103+
gh api --method POST "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/labels" \
104+
-f "labels[]=${AUTHORSHIP_LABEL}" >/dev/null
105+
106+
echo "Applied ${AUTHORSHIP_LABEL} to #${PR_NUMBER}."
107+
108+
- name: Check required Maestro trailers
109+
if: ${{ inputs.fail_on_incomplete && steps.classify.outputs.incomplete_agent_commits != '0' }}
110+
shell: bash
111+
env:
112+
INCOMPLETE_AGENT_COMMITS: ${{ steps.classify.outputs.incomplete_agent_commits }}
113+
run: |
114+
echo "::error::${INCOMPLETE_AGENT_COMMITS} Maestro-marked commit(s) are missing required authorship trailers."
115+
exit 1

‎.github/workflows/codex-rails-check.yml‎

Lines changed: 39 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,12 @@ on:
66
- "AGENTS.md"
77
- "**/AGENTS.md"
88
- ".agents/skills/**"
9+
- ".github/scripts/**"
910
- ".github/ISSUE_TEMPLATE/**"
1011
- ".github/pull_request_template.md"
11-
- ".github/workflows/codex-rails-check.yml"
12+
- ".github/workflows/**"
13+
- ".github/workflow-templates/**"
14+
- "profile/**"
1215
workflow_call:
1316
inputs:
1417
require_agents:
@@ -38,6 +41,41 @@ jobs:
3841
fi
3942
ruby -e 'require "yaml"; ARGV.each { |f| YAML.load_file(f); puts "ok #{f}" }' "${files[@]}"
4043
44+
- name: Validate workflow YAML
45+
shell: bash
46+
run: |
47+
set -euo pipefail
48+
shopt -s nullglob
49+
files=(.github/workflows/*.yml .github/workflows/*.yaml .github/workflow-templates/*.yml .github/workflow-templates/*.yaml)
50+
if [ "${#files[@]}" -eq 0 ]; then
51+
echo "No workflow YAML files found."
52+
exit 0
53+
fi
54+
ruby -e 'require "yaml"; ARGV.each { |f| YAML.load_file(f); puts "ok #{f}" }' "${files[@]}"
55+
56+
- name: Validate workflow template metadata
57+
shell: bash
58+
run: |
59+
set -euo pipefail
60+
shopt -s nullglob
61+
files=(.github/workflow-templates/*.properties.json)
62+
if [ "${#files[@]}" -eq 0 ]; then
63+
echo "No workflow template metadata files found."
64+
exit 0
65+
fi
66+
ruby -e '
67+
require "json"
68+
ARGV.each do |f|
69+
data = JSON.parse(File.read(f))
70+
icon = data["iconName"].to_s
71+
if !icon.empty? && !icon.match?(/\Aocticon [a-z0-9-]+\z/) && !File.exist?(".github/workflow-templates/#{icon}.svg")
72+
warn "#{f}: iconName must be an octicon reference like \"octicon tag\" or a local SVG basename"
73+
exit 1
74+
end
75+
puts "ok #{f}"
76+
end
77+
' "${files[@]}"
78+
4179
- name: Check AGENTS.md files
4280
shell: bash
4381
env:

‎profile/AGENT_AUTHORSHIP.md‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# Agent Authorship Attribution
2+
3+
EvalOps uses agent-written code in the same systems that sell audit, approvals,
4+
and governance. Our own repositories should therefore answer a basic operating
5+
question: which production changes were written by an agent, under which human's
6+
direction, and through which approval chain?
7+
8+
This convention makes agent authorship git-native, visible in GitHub, and ready
9+
for audit-service indexing.
10+
11+
## Commit Trailers
12+
13+
Every Maestro-authored commit must include these trailers:
14+
15+
```text
16+
Co-Authored-By: Maestro <maestro@evalops.dev>
17+
Maestro-Version: <maestro-version> / <model-identifier>
18+
Maestro-Prompt-Id: <prompt-registry-id>
19+
Maestro-Approvals-Id: <approvals-service-request-id>
20+
```
21+
22+
Use one trailer block per commit. If a human materially edits agent output before
23+
commit, keep the human as the git author and keep the Maestro trailers so the
24+
chain remains visible.
25+
26+
### Field Rules
27+
28+
| Trailer | Required | Purpose |
29+
|---|---:|---|
30+
| `Co-Authored-By` | Yes | Lets GitHub render Maestro as a co-author and gives git-native provenance. |
31+
| `Maestro-Version` | Yes | Records the Maestro build and model identifier used for the change. |
32+
| `Maestro-Prompt-Id` | Yes | Links the commit to the prompt registry entry that shaped the work. |
33+
| `Maestro-Approvals-Id` | Yes | Links the commit to the approvals request that authorized the change. |
34+
35+
If an identifier is not available, do not invent one. Use the best durable
36+
identifier the producing system has and file a follow-up against that system.
37+
38+
## Pull Request Labels
39+
40+
The reusable workflow in this repository applies exactly one authorship label to
41+
each PR:
42+
43+
| Label | Meaning |
44+
|---|---|
45+
| `agent-authored` | Every commit in the PR carries Maestro authorship metadata. |
46+
| `human-authored` | No commit in the PR carries Maestro authorship metadata. |
47+
| `mixed-authorship` | Some commits carry Maestro metadata and some do not. |
48+
49+
The labels are a GitHub UI affordance. The commit trailers remain the source of
50+
truth because they travel with the git history.
51+
52+
## Reusable Workflow
53+
54+
Adopt the org workflow from the GitHub Actions template picker, or add this file
55+
to a repository as `.github/workflows/agent-authorship-labels.yml`:
56+
57+
```yaml
58+
name: Agent authorship labels
59+
60+
on:
61+
pull_request_target:
62+
types: [opened, synchronize, reopened, ready_for_review, edited]
63+
64+
permissions:
65+
contents: read
66+
pull-requests: read
67+
issues: write
68+
69+
jobs:
70+
label:
71+
uses: evalops/.github/.github/workflows/agent-authorship-label.yml@main
72+
```
73+
74+
The workflow creates the three labels if they are missing, removes stale
75+
authorship labels, and applies the label that matches the current PR commit set.
76+
77+
## Audit Indexing
78+
79+
Audit ingestion should parse trailers from every commit merged to protected
80+
branches and index at least:
81+
82+
- commit SHA
83+
- git author and committer
84+
- `Maestro-Version`
85+
- `Maestro-Prompt-Id`
86+
- `Maestro-Approvals-Id`
87+
- merged PR number and repository
88+
89+
The target product query is:
90+
91+
```text
92+
For this production line, show the commit, Maestro version, prompt, approvals
93+
request, human author, and merge PR that produced it.
94+
```
95+
96+
## Backfill
97+
98+
Do not rewrite old commit history to add trailers. For pre-convention PRs, use
99+
best-effort labels only when evidence is clear. If evidence is heuristic, prefer
100+
a separate `agent-authored-pre-convention` follow-up instead of weakening the
101+
meaning of the three current labels.

0 commit comments

Comments
 (0)