Skip to content

Add agent authorship attribution convention - #11

Merged
haasonsaas merged 2 commits into
mainfrom
jonathan/agent-authorship-convention
Apr 28, 2026
Merged

Add agent authorship attribution convention#11
haasonsaas merged 2 commits into
mainfrom
jonathan/agent-authorship-convention

Conversation

@haasonsaas

Copy link
Copy Markdown
Contributor

Summary

Adds the .github-owned pieces for evalops/.github#8:

  • document the Maestro authorship trailer convention in profile/AGENT_AUTHORSHIP.md
  • add a reusable workflow that labels PRs as agent-authored, human-authored, or mixed-authorship from commit trailers
  • add an org workflow template for downstream adoption
  • extend the org rails check to validate workflow YAML/template metadata
  • add the authorship trailer reminder to the org PR template

Test Plan

  • ruby -c .github/scripts/classify-agent-authorship.rb
  • classifier smoke cases for agent, human, mixed, and incomplete Maestro trailer commits
  • ruby -e 'require "yaml"; ARGV.each { |f| YAML.load_file(f); puts "ok #{f}" }' .github/ISSUE_TEMPLATE/*.yml .github/workflows/*.yml .github/workflow-templates/*.yml\n- ruby -e 'require "json"; ARGV.each { |f| JSON.parse(File.read(f)); puts "ok #{f}" }' .github/workflow-templates/*.properties.json\n- actionlint .github/workflows/*.yml .github/workflow-templates/*.yml\n- git diff --check\n\n## Follow-Up\n\nThis does not close evalops/.github#8 by itself; Maestro emission, platform audit indexing, org rollout, and the demo surface still need their owner-repo changes.

@cursor

cursor Bot commented Apr 28, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Introduces a pull_request_target workflow that uses the GitHub token to manage labels, so misconfiguration could affect PR labeling/permissions. Changes are otherwise confined to org docs and CI guardrails.

Overview
Defines a Maestro agent authorship convention (required git commit trailers) and documents it in profile/AGENT_AUTHORSHIP.md, with profile/README.md linking to the new guidance.

Adds a reusable GitHub Actions workflow (.github/workflows/agent-authorship-label.yml) plus a workflow template (.github/workflow-templates/agent-authorship-labels.yml + metadata) that classifies PR commits via .github/scripts/classify-agent-authorship.rb, ensures agent-authored/human-authored/mixed-authorship labels exist, and applies exactly one label to each PR (optionally failing on incomplete trailers).

Updates the PR template with a checkbox reminding authors to include the required Maestro trailers, and expands codex-rails-check to validate workflow YAML and workflow-template metadata when .github/** or profile/** changes land.

Reviewed by Cursor Bugbot for commit e1a7e4b. Bugbot is set up for automated code reviews on this repo. Configure here.

@haasonsaas
haasonsaas force-pushed the jonathan/agent-authorship-convention branch from b0651d3 to e1a7e4b Compare April 28, 2026 15:37

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Marker and required co-author regexes accept different whitespace
    • Updated the required Co-Authored-By regex to use the same flexible whitespace matching as the marker regex.
Preview (e8aa3c27c2)
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
--- a/.github/pull_request_template.md
+++ b/.github/pull_request_template.md
@@ -32,6 +32,7 @@
 ## Agent / Automation Notes
 
 - [ ] This PR was agent-authored or agent-assisted
+- [ ] Maestro-authored commits include the required authorship trailers
 - [ ] I checked live GitHub state before publishing
 - [ ] Review feedback and failing checks have been rechecked
 - [ ] No local secrets, generated scratch artifacts, or unrelated worktree changes are included

diff --git a/.github/scripts/classify-agent-authorship.rb b/.github/scripts/classify-agent-authorship.rb
new file mode 100644
--- /dev/null
+++ b/.github/scripts/classify-agent-authorship.rb
@@ -1,0 +1,82 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "json"
+require "optparse"
+
+options = {
+  github_output: nil,
+}
+
+OptionParser.new do |parser|
+  parser.on("--github-output PATH", "Append key=value outputs for GitHub Actions") do |path|
+    options[:github_output] = path
+  end
+end.parse!
+
+input = ARGF.read
+
+messages = input.each_line.map do |line|
+  next if line.strip.empty?
+
+  parsed = JSON.parse(line)
+  if parsed.is_a?(Hash)
+    parsed.dig("commit", "message") || parsed["message"]
+  end
+end.compact
+
+required_patterns = {
+  "co_author" => /^Co-Authored-By:\s*Maestro\s+<maestro@evalops\.dev>\s*$/i,
+  "version" => /^Maestro-Version:\s*\S.+$/i,
+  "prompt_id" => /^Maestro-Prompt-Id:\s*\S.+$/i,
+  "approvals_id" => /^Maestro-Approvals-Id:\s*\S.+$/i,
+}
+
+marker_pattern = /
+  ^Co-Authored-By:\s*Maestro\s+<maestro@evalops\.dev>\s*$ |
+  ^Maestro-(?:Version|Prompt-Id|Approvals-Id):
+/ix
+
+agent_commits = 0
+human_commits = 0
+incomplete_commits = 0
+
+messages.each do |message|
+  has_marker = message.lines.any? { |line| line.match?(marker_pattern) }
+
+  unless has_marker
+    human_commits += 1
+    next
+  end
+
+  agent_commits += 1
+  missing_required = required_patterns.values.any? do |pattern|
+    message.lines.none? { |line| line.match?(pattern) }
+  end
+  incomplete_commits += 1 if missing_required
+end
+
+label =
+  if agent_commits.positive? && human_commits.positive?
+    "mixed-authorship"
+  elsif agent_commits.positive?
+    "agent-authored"
+  else
+    "human-authored"
+  end
+
+outputs = {
+  "label" => label,
+  "total_commits" => messages.length,
+  "agent_commits" => agent_commits,
+  "human_commits" => human_commits,
+  "incomplete_agent_commits" => incomplete_commits,
+}
+
+outputs.each { |key, value| puts "#{key}=#{value}" }
+
+if options[:github_output]
+  File.open(options[:github_output], "a") do |file|
+    outputs.each { |key, value| file.puts("#{key}=#{value}") }
+  end
+end

diff --git a/.github/workflow-templates/agent-authorship-labels.properties.json b/.github/workflow-templates/agent-authorship-labels.properties.json
new file mode 100644
--- /dev/null
+++ b/.github/workflow-templates/agent-authorship-labels.properties.json
@@ -1,0 +1,9 @@
+{
+  "name": "Agent authorship labels",
+  "description": "Apply agent-authored, human-authored, or mixed-authorship labels to pull requests based on Maestro commit trailers.",
+  "iconName": "octicon tag",
+  "categories": [
+    "Automation",
+    "Code review"
+  ]
+}

diff --git a/.github/workflow-templates/agent-authorship-labels.yml b/.github/workflow-templates/agent-authorship-labels.yml
new file mode 100644
--- /dev/null
+++ b/.github/workflow-templates/agent-authorship-labels.yml
@@ -1,0 +1,14 @@
+name: Agent authorship labels
+
+on:
+  pull_request_target:
+    types: [opened, synchronize, reopened, ready_for_review, edited]
+
+permissions:
+  contents: read
+  pull-requests: read
+  issues: write
+
+jobs:
+  label:
+    uses: evalops/.github/.github/workflows/agent-authorship-label.yml@main

diff --git a/.github/workflows/agent-authorship-label.yml b/.github/workflows/agent-authorship-label.yml
new file mode 100644
--- /dev/null
+++ b/.github/workflows/agent-authorship-label.yml
@@ -1,0 +1,115 @@
+name: agent-authorship-label
+
+on:
+  workflow_call:
+    inputs:
+      fail_on_incomplete:
+        description: "Fail when a Maestro-marked commit is missing one or more required Maestro trailers"
+        required: false
+        type: boolean
+        default: false
+
+permissions:
+  contents: read
+  pull-requests: read
+  issues: write
+
+jobs:
+  label:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout org workflow helpers
+        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
+        with:
+          repository: evalops/.github
+          ref: main
+          path: org-defaults
+
+      - name: Resolve pull request
+        id: pr
+        shell: bash
+        run: |
+          set -euo pipefail
+
+          number="$(jq -r '.pull_request.number // empty' "${GITHUB_EVENT_PATH}")"
+          if [ -z "${number}" ]; then
+            echo "::error::agent-authorship-label must run from a pull_request or pull_request_target event."
+            exit 1
+          fi
+
+          echo "number=${number}" >> "${GITHUB_OUTPUT}"
+
+      - name: Fetch pull request commits
+        shell: bash
+        env:
+          GH_TOKEN: ${{ github.token }}
+          PR_NUMBER: ${{ steps.pr.outputs.number }}
+        run: |
+          set -euo pipefail
+          gh api --paginate "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/commits" \
+            --jq '.[] | {sha: .sha, message: .commit.message}' > commits.jsonl
+
+      - name: Classify authorship
+        id: classify
+        shell: bash
+        run: |
+          set -euo pipefail
+          ruby org-defaults/.github/scripts/classify-agent-authorship.rb \
+            --github-output "${GITHUB_OUTPUT}" \
+            commits.jsonl
+
+      - name: Ensure authorship labels exist
+        shell: bash
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          set -euo pipefail
+
+          ensure_label() {
+            local name="$1"
+            local color="$2"
+            local description="$3"
+
+            if gh api "repos/${GITHUB_REPOSITORY}/labels/${name}" >/dev/null 2>&1; then
+              gh api --method PATCH "repos/${GITHUB_REPOSITORY}/labels/${name}" \
+                -f color="${color}" \
+                -f description="${description}" >/dev/null
+            else
+              gh api --method POST "repos/${GITHUB_REPOSITORY}/labels" \
+                -f name="${name}" \
+                -f color="${color}" \
+                -f description="${description}" >/dev/null
+            fi
+          }
+
+          ensure_label "agent-authored" "6f42c1" "All PR commits carry Maestro authorship trailers"
+          ensure_label "human-authored" "0e8a16" "No PR commits carry Maestro authorship trailers"
+          ensure_label "mixed-authorship" "fbca04" "Some PR commits carry Maestro authorship trailers"
+
+      - name: Apply authorship label
+        shell: bash
+        env:
+          GH_TOKEN: ${{ github.token }}
+          PR_NUMBER: ${{ steps.pr.outputs.number }}
+          AUTHORSHIP_LABEL: ${{ steps.classify.outputs.label }}
+        run: |
+          set -euo pipefail
+
+          for label in agent-authored human-authored mixed-authorship; do
+            gh api --method DELETE \
+              "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/labels/${label}" >/dev/null 2>&1 || true
+          done
+
+          gh api --method POST "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/labels" \
+            -f "labels[]=${AUTHORSHIP_LABEL}" >/dev/null
+
+          echo "Applied ${AUTHORSHIP_LABEL} to #${PR_NUMBER}."
+
+      - name: Check required Maestro trailers
+        if: ${{ inputs.fail_on_incomplete && steps.classify.outputs.incomplete_agent_commits != '0' }}
+        shell: bash
+        env:
+          INCOMPLETE_AGENT_COMMITS: ${{ steps.classify.outputs.incomplete_agent_commits }}
+        run: |
+          echo "::error::${INCOMPLETE_AGENT_COMMITS} Maestro-marked commit(s) are missing required authorship trailers."
+          exit 1

diff --git a/.github/workflows/codex-rails-check.yml b/.github/workflows/codex-rails-check.yml
--- a/.github/workflows/codex-rails-check.yml
+++ b/.github/workflows/codex-rails-check.yml
@@ -6,9 +6,12 @@
       - "AGENTS.md"
       - "**/AGENTS.md"
       - ".agents/skills/**"
+      - ".github/scripts/**"
       - ".github/ISSUE_TEMPLATE/**"
       - ".github/pull_request_template.md"
-      - ".github/workflows/codex-rails-check.yml"
+      - ".github/workflows/**"
+      - ".github/workflow-templates/**"
+      - "profile/**"
   workflow_call:
     inputs:
       require_agents:
@@ -38,6 +41,41 @@
           fi
           ruby -e 'require "yaml"; ARGV.each { |f| YAML.load_file(f); puts "ok #{f}" }' "${files[@]}"
 
+      - name: Validate workflow YAML
+        shell: bash
+        run: |
+          set -euo pipefail
+          shopt -s nullglob
+          files=(.github/workflows/*.yml .github/workflows/*.yaml .github/workflow-templates/*.yml .github/workflow-templates/*.yaml)
+          if [ "${#files[@]}" -eq 0 ]; then
+            echo "No workflow YAML files found."
+            exit 0
+          fi
+          ruby -e 'require "yaml"; ARGV.each { |f| YAML.load_file(f); puts "ok #{f}" }' "${files[@]}"
+
+      - name: Validate workflow template metadata
+        shell: bash
+        run: |
+          set -euo pipefail
+          shopt -s nullglob
+          files=(.github/workflow-templates/*.properties.json)
+          if [ "${#files[@]}" -eq 0 ]; then
+            echo "No workflow template metadata files found."
+            exit 0
+          fi
+          ruby -e '
+            require "json"
+            ARGV.each do |f|
+              data = JSON.parse(File.read(f))
+              icon = data["iconName"].to_s
+              if !icon.empty? && !icon.match?(/\Aocticon [a-z0-9-]+\z/) && !File.exist?(".github/workflow-templates/#{icon}.svg")
+                warn "#{f}: iconName must be an octicon reference like \"octicon tag\" or a local SVG basename"
+                exit 1
+              end
+              puts "ok #{f}"
+            end
+          ' "${files[@]}"
+
       - name: Check AGENTS.md files
         shell: bash
         env:

diff --git a/profile/AGENT_AUTHORSHIP.md b/profile/AGENT_AUTHORSHIP.md
new file mode 100644
--- /dev/null
+++ b/profile/AGENT_AUTHORSHIP.md
@@ -1,0 +1,101 @@
+# Agent Authorship Attribution
+
+EvalOps uses agent-written code in the same systems that sell audit, approvals,
+and governance. Our own repositories should therefore answer a basic operating
+question: which production changes were written by an agent, under which human's
+direction, and through which approval chain?
+
+This convention makes agent authorship git-native, visible in GitHub, and ready
+for audit-service indexing.
+
+## Commit Trailers
+
+Every Maestro-authored commit must include these trailers:
+
+```text
+Co-Authored-By: Maestro <maestro@evalops.dev>
+Maestro-Version: <maestro-version> / <model-identifier>
+Maestro-Prompt-Id: <prompt-registry-id>
+Maestro-Approvals-Id: <approvals-service-request-id>
+```
+
+Use one trailer block per commit. If a human materially edits agent output before
+commit, keep the human as the git author and keep the Maestro trailers so the
+chain remains visible.
+
+### Field Rules
+
+| Trailer | Required | Purpose |
+|---|---:|---|
+| `Co-Authored-By` | Yes | Lets GitHub render Maestro as a co-author and gives git-native provenance. |
+| `Maestro-Version` | Yes | Records the Maestro build and model identifier used for the change. |
+| `Maestro-Prompt-Id` | Yes | Links the commit to the prompt registry entry that shaped the work. |
+| `Maestro-Approvals-Id` | Yes | Links the commit to the approvals request that authorized the change. |
+
+If an identifier is not available, do not invent one. Use the best durable
+identifier the producing system has and file a follow-up against that system.
+
+## Pull Request Labels
+
+The reusable workflow in this repository applies exactly one authorship label to
+each PR:
+
+| Label | Meaning |
+|---|---|
+| `agent-authored` | Every commit in the PR carries Maestro authorship metadata. |
+| `human-authored` | No commit in the PR carries Maestro authorship metadata. |
+| `mixed-authorship` | Some commits carry Maestro metadata and some do not. |
+
+The labels are a GitHub UI affordance. The commit trailers remain the source of
+truth because they travel with the git history.
+
+## Reusable Workflow
+
+Adopt the org workflow from the GitHub Actions template picker, or add this file
+to a repository as `.github/workflows/agent-authorship-labels.yml`:
+
+```yaml
+name: Agent authorship labels
+
+on:
+  pull_request_target:
+    types: [opened, synchronize, reopened, ready_for_review, edited]
+
+permissions:
+  contents: read
+  pull-requests: read
+  issues: write
+
+jobs:
+  label:
+    uses: evalops/.github/.github/workflows/agent-authorship-label.yml@main
+```
+
+The workflow creates the three labels if they are missing, removes stale
+authorship labels, and applies the label that matches the current PR commit set.
+
+## Audit Indexing
+
+Audit ingestion should parse trailers from every commit merged to protected
+branches and index at least:
+
+- commit SHA
+- git author and committer
+- `Maestro-Version`
+- `Maestro-Prompt-Id`
+- `Maestro-Approvals-Id`
+- merged PR number and repository
+
+The target product query is:
+
+```text
+For this production line, show the commit, Maestro version, prompt, approvals
+request, human author, and merge PR that produced it.
+```
+
+## Backfill
+
+Do not rewrite old commit history to add trailers. For pre-convention PRs, use
+best-effort labels only when evidence is clear. If evidence is heuristic, prefer
+a separate `agent-authored-pre-convention` follow-up instead of weakening the
+meaning of the three current labels.

diff --git a/profile/README.md b/profile/README.md
--- a/profile/README.md
+++ b/profile/README.md
@@ -2,6 +2,10 @@
 
 The organizational operating system for AI agent workforces — evaluation, governance, and observability for shipping accountable AI.
 
+## Operating Conventions
+
+- [Agent authorship attribution](AGENT_AUTHORSHIP.md) — git trailers, PR labels, and audit indexing for Maestro-authored code.
+
 ## Platform Services
 
 Discover repos by topic:

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit e1a7e4b. Configure here.

Comment thread .github/scripts/classify-agent-authorship.rb Outdated
@haasonsaas
haasonsaas merged commit 4ce399d into main Apr 28, 2026
3 checks passed
@haasonsaas
haasonsaas deleted the jonathan/agent-authorship-convention branch April 28, 2026 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Convention: agent authorship attribution on commits, PRs, and audit trail

2 participants