diff --git a/PLAN.md b/PLAN.md index 7ff2c1c..b27c4ed 100644 --- a/PLAN.md +++ b/PLAN.md @@ -337,7 +337,7 @@ code-review platform. Production work includes: ## Non-goals -- Ranking humans on a public leaderboard +- Treating the evidence-linked `contributor × subsystem` leaderboard as a global ability, employment or performance ranking - Automatically deciding whose fault an incident was - Letting high-history contributors bypass baseline controls - Treating vector similarity as causal evidence diff --git a/README.md b/README.md index b1b5bfe..7eb1d8d 100644 --- a/README.md +++ b/README.md @@ -41,11 +41,11 @@ PR ships → canary fails in production → operator confirms the causal linkage The reviewer is deliberately disposable. Kill it mid-demo and its successor boots with **zero prior messages**, reads the contract from MongoDB, and blocks a PR that its predecessor would have waved through. That behavioral delta — not a longer prompt — is the whole claim. -**How it frames people.** The Elo is on the *change*, not the author. PR-Elo rates how much proven evidence a pull request must carry to enter a subsystem — a rating that moves when a canary proves something, never a rating of an engineer. +**How it frames people.** Enforcement stays on the *change*: no contributor score can waive a required check. The live demo also shows a transparent `contributor × subsystem` review-confidence score, computed from linked evidence events rather than inferred ability. It answers “how much proof has this contributor recently carried in this area?”, not “how good is this engineer?” | Rule | What it means | |---|---| -| 🎯 **The change is what's rated** | Output is a *review-depth requirement* on a PR entering a subsystem. Standing is scoped `contributor × subsystem`, read as history — never published as a global developer score. | +| 🎯 **The change is still gated** | Output is a *review-depth requirement* on a PR entering a subsystem. Contributor history can increase scrutiny but never remove baseline controls. | | ❓ **Unknown by default** | States are `unknown · building · proven · proof-debt`. No evidence ≠ trusted, and ≠ suspect. | | 🏛️ **Organizational, not personal** | A confirmed payments failure guards *every* future payments PR, regardless of author. | | ↩️ **Recovery is always open** | Proof debt is paid down by executed evidence; contracts unlock. | @@ -147,7 +147,7 @@ Embeddings run **locally** (`Xenova/all-MiniLM-L6-v2`, 384-dim) — no embedding | Surface | Path | What it is | |---|---|---| | **Canonical demo** | `receipts/public/` | The operator console (numbered controls) and the live stage screen. Backed by real MongoDB writes and driven by change streams over SSE. **This is what runs on stage.** | -| **Live GitHub proof** | `receipts/public/live.html` | A PR targeting an authorized fork triggers a signed webhook, becomes an idempotent MongoDB receipt, is reviewed by the disposable worker and receives a GitHub commit status plus review comment. | +| **Live GitHub proof** | `receipts/public/live.html` | A PR targeting an authorized fork triggers a signed webhook, becomes an idempotent MongoDB receipt, is reviewed by the disposable worker and receives a GitHub commit status plus review comment. The same page recomputes an evidence-linked contributor leaderboard from MongoDB. | | **Product vision** | `receipts/web/` | React 19 + TypeScript strict + Vite 8 + Tailwind v4. Routes: `/` Courtroom (live stream), `/contributor/:id` Dossier, `/review/:id` Case File, with a control-condition compare at `/review/rev-512`. Dark/light, WCAG AA tokens, `prefers-reduced-motion` parity. Design spec: [`receipts/web/DESIGN.md`](receipts/web/DESIGN.md) — *"a microfiche reader in a dark evidence room."* | **Honest boundary:** the React UI runs on **fixture replay** and is not yet wired to the MongoDB API — it renders the product this becomes, not live data. The mechanics claims in this README are all proven by the `public/` demo and the source referenced above. @@ -191,6 +191,8 @@ Forward events only to `POST /webhooks/github` through a one-way relay such as S The webhook acknowledges only after its delivery ID is durably recorded. The worker fetches the real GitHub patch, maps changed paths to a subsystem, lets MongoDB contracts select scrutiny and posts the result as `PR-Elo / persistent review`. Replayed delivery IDs and repeated publications are deduplicated in MongoDB; startup recovery retries both interrupted deliveries and reviewed receipts whose GitHub publication did not finish. +The live leaderboard starts each `contributor × subsystem` scope at 100. Missing required evidence is −5, review concerns are −3, a public auto-revert is −6, a corrective re-land is +4 and verified evidence is +3. The MongoDB contributor rows are a disclosed Aug 6–13 public sample; the `gorajing` row is calculated from append-only verdict events on actual signed-webhook receipts in the demo fork. A repaired PR keeps its earlier failure event before adding its recovery event. Every movement links to its public PR or commit. This is a demo review-readiness signal, not an employment or performance metric. +
Optional checks & the product-vision UI diff --git a/receipts/public/live.html b/receipts/public/live.html index 6ff9fe1..3142fce 100644 --- a/receipts/public/live.html +++ b/receipts/public/live.html @@ -16,6 +16,9 @@ .memory{margin-top:16px;padding:17px;border:1px dashed #625c48;background:#1c1912}.memory h3{margin:8px 0 7px;font:600 20px Georgia,serif}.memory p{color:#c9c4b4;margin:0;line-height:1.5}.memory .source{margin-top:11px;font:11px/1.5 ui-monospace,monospace;color:var(--muted)} .timeline{list-style:none;padding:0;margin:14px 0 0}.timeline li{display:grid;grid-template-columns:12px 1fr;gap:12px;padding:0 0 19px;color:var(--muted);line-height:1.4}.timeline li:before{content:"";width:9px;height:9px;border-radius:50%;background:var(--line);margin-top:5px;box-shadow:0 0 0 5px #ffffff05}.timeline li.done:before{background:var(--green)}.timeline b{color:var(--ink);display:block;font-size:13px}.timeline span{font:11px ui-monospace,monospace} .empty{color:var(--muted);padding:46px 0;text-align:center;font:15px/1.6 Georgia,serif}.foot{margin-top:16px;display:flex;justify-content:space-between;gap:16px;color:var(--muted);font:11px ui-monospace,monospace}.btn{display:inline-flex;margin-top:16px;padding:10px 13px;border:1px solid var(--line);text-decoration:none;font:700 11px ui-monospace,monospace;text-transform:uppercase;letter-spacing:.1em} + .board{margin-top:16px}.board-head{display:flex;align-items:end;justify-content:space-between;gap:24px}.board-head h2{font-size:clamp(26px,3vw,38px)}.board-copy{max-width:570px;margin:8px 0 0;color:var(--muted);font:13px/1.5 ui-monospace,monospace}.legend{display:flex;flex-wrap:wrap;justify-content:flex-end;gap:7px}.legend span,.band{border:1px solid var(--line);padding:6px 8px;color:var(--muted);font:700 10px/1 ui-monospace,monospace;text-transform:uppercase;letter-spacing:.06em}.legend .negative{color:var(--red)}.legend .positive{color:var(--green)} + .leaderboard{list-style:none;padding:0;margin:20px 0 0;border-top:1px solid var(--line)}.leader-row{display:grid;grid-template-columns:40px minmax(180px,1.1fr) minmax(160px,.9fr) 110px minmax(210px,1.2fr);gap:14px;align-items:center;min-height:78px;border-bottom:1px solid var(--line)}.leader-row.current{background:linear-gradient(90deg,#f0a92f13,transparent);box-shadow:inset 3px 0 var(--amber)}.rank{color:#6f6b60;font:600 22px Georgia,serif;text-align:center}.who a{font:700 15px ui-monospace,monospace;text-decoration:none}.who small,.scope small,.event small{display:block;margin-top:6px;color:var(--muted);font:10px/1.35 ui-monospace,monospace}.you{display:inline-block;margin-left:7px;padding:3px 5px;background:var(--amber);color:#18130b;font:900 9px/1 ui-monospace,monospace;letter-spacing:.08em;vertical-align:2px}.scope{font:13px/1.3 Georgia,serif}.score{text-align:right}.score strong{display:block;font:600 33px/1 Georgia,serif}.movement{font:800 11px ui-monospace,monospace}.movement.down{color:var(--red)}.movement.up{color:var(--green)}.band{display:inline-block;margin-top:5px;padding:4px 6px}.band.elevated{color:var(--amber)}.band.maximum{color:var(--red)}.event{font:12px/1.35 Georgia,serif}.event a{text-decoration-color:#706b5d;text-underline-offset:3px}.transition{color:var(--muted);font:10px ui-monospace,monospace}.board-empty{padding:25px 0;color:var(--muted);font:13px ui-monospace,monospace} + @media(max-width:900px){.leader-row{grid-template-columns:32px 1fr 100px}.scope{grid-column:2}.event{grid-column:2/4;padding-bottom:14px}.board-head{align-items:start;flex-direction:column}.legend{justify-content:flex-start}} @media(max-width:800px){.pipeline{grid-template-columns:1fr 1fr}.grid{grid-template-columns:1fr}header{display:block}.live{margin-top:16px}} @@ -32,6 +35,10 @@
Open a pull request against gorajing/mongo.
The live case will appear here without pressing a button.
+
+
Evidence ledger · Aug 6–13 + live fork

Contributor review confidence

Evidence-readiness within one subsystem, not an employee-performance rating. Every movement links to the PR, revert or corrective re-land that caused it.

+
  1. Loading evidence-linked scores…
+
Not updated yetThe reviewer can die. The lesson remains.
diff --git a/receipts/src/github.js b/receipts/src/github.js index 49d5f93..42206e5 100644 --- a/receipts/src/github.js +++ b/receipts/src/github.js @@ -269,7 +269,7 @@ function reviewComment(receipt, result, key, targetUrl) { `**Why this scrutiny:** ${memory}`, ...(unmet ? ['', unmet] : []), '', - `Reviewer: \`${receipt.review.by}\` · Mongo receipt: \`#${receipt.prNum}\` · Contributor credibility unchanged`, + `Reviewer: \`${receipt.review.by}\` · Mongo receipt: \`#${receipt.prNum}\` · Subsystem review-readiness event recorded`, ...(targetUrl ? [`[Open the live review receipt](${targetUrl})`] : []), ].join('\n'); } diff --git a/receipts/src/leaderboard.js b/receipts/src/leaderboard.js new file mode 100644 index 0000000..ab1164e --- /dev/null +++ b/receipts/src/leaderboard.js @@ -0,0 +1,113 @@ +const BASELINE = 100; + +const DELTAS = Object.freeze({ + 'missing-evidence': -5, + 'review-concerns': -3, + 'auto-revert': -6, + 'corrective-reland': 4, + 'verified-evidence': 3, +}); + +function dateValue(value) { + const time = new Date(value).getTime(); + return Number.isFinite(time) ? time : 0; +} + +export function scoreEvents(events, baseline = BASELINE) { + let score = baseline; + const scored = [...events] + .sort((left, right) => dateValue(left.at) - dateValue(right.at)) + .map((event) => { + const delta = DELTAS[event.kind] ?? 0; + const from = score; + score += delta; + return { ...event, delta, from, to: score }; + }); + return { score, trend: [baseline, ...scored.map((event) => event.to)], events: scored }; +} + +function eventKind(status) { + if (status === 'blocked') return 'missing-evidence'; + if (status === 'concerns') return 'review-concerns'; + if (status === 'approved') return 'verified-evidence'; + return null; +} + +export function reviewEventForReceipt(receipt, status = receipt.status, at = null) { + const kind = eventKind(status); + if (!kind) return null; + const number = receipt.github?.number ?? receipt.prNum; + return { + kind, + status, + at: at ?? receipt.github?.publication?.publishedAt ?? receipt.ts, + label: `PR #${number} ${status}`, + url: receipt.github?.htmlUrl ?? null, + source: receipt.github?.repository ?? 'MongoDB receipt', + }; +} + +function receiptEvents(receipt) { + if (!receipt.reviewEvents?.length) { + const current = reviewEventForReceipt(receipt); + return current ? [current] : []; + } + return receipt.reviewEvents + .map((event) => { + const generated = reviewEventForReceipt(receipt, event.status, event.at); + return generated ? { ...generated, ...event } : null; + }) + .filter(Boolean); +} + +function reviewBand(score) { + if (score >= 100) return { key: 'normal', label: 'normal review' }; + if (score >= 90) return { key: 'elevated', label: 'elevated review' }; + return { key: 'maximum', label: 'maximum review' }; +} + +export function buildContributorLeaderboard({ profiles, receipts }) { + const byScope = new Map( + profiles.map((profile) => [`${profile.handle}\u0000${profile.subsystem}`, { ...profile }]), + ); + + for (const receipt of receipts) { + if (!receipt.author || !receipt.subsystem || !receipt.github) continue; + const key = `${receipt.author}\u0000${receipt.subsystem}`; + if (!byScope.has(key)) { + byScope.set(key, { + handle: receipt.author, + repo: receipt.github.repository, + subsystem: receipt.subsystem, + isCurrentUser: receipt.author === 'gorajing', + events: [], + }); + } + } + + const rows = []; + for (const profile of byScope.values()) { + const liveEvents = receipts + .filter((receipt) => receipt.author === profile.handle && receipt.subsystem === profile.subsystem) + .flatMap(receiptEvents); + const scored = scoreEvents([...(profile.events ?? []), ...liveEvents]); + rows.push({ + handle: profile.handle, + repo: profile.repo, + subsystem: profile.subsystem, + isCurrentUser: Boolean(profile.isCurrentUser), + profileUrl: profile.profileUrl ?? `https://github.com/${profile.handle}`, + basis: profile.basis ?? 'Live PR-Elo receipts', + ...scored, + movement: scored.score - BASELINE, + band: reviewBand(scored.score), + }); + } + + return rows.sort((left, right) => right.score - left.score || left.handle.localeCompare(right.handle)); +} + +export const leaderboardScoring = { + baseline: BASELINE, + deltas: DELTAS, +}; diff --git a/receipts/src/ops.js b/receipts/src/ops.js index 11ac2be..7536d27 100644 --- a/receipts/src/ops.js +++ b/receipts/src/ops.js @@ -7,8 +7,9 @@ import { client } from './db.js'; import { embed } from './embed.js'; import { - contributors, receipts, incidents, contracts, nextSeq, VECTOR_INDEX, + contributors, receipts, incidents, contracts, leaderboardProfiles, nextSeq, VECTOR_INDEX, } from './schema.js'; +import { buildContributorLeaderboard, leaderboardScoring } from './leaderboard.js'; export async function submitPR({ author, subsystem, changeType = 'feature', title, code, fnName = null, @@ -203,3 +204,29 @@ export async function standing() { } return rows; } + +/** + * Public, evidence-linked review readiness beside the live fork contributor. + * Profiles provide bounded public history; live GitHub receipts provide the + * score movements that judges can watch happen in this session. + */ +export async function contributorLeaderboard() { + const [profiles, githubReceipts] = await Promise.all([ + leaderboardProfiles().find({ active: true }).toArray(), + receipts().find( + { 'github.repository': { $type: 'string' } }, + { + projection: { + prNum: 1, author: 1, subsystem: 1, status: 1, ts: 1, + reviewEvents: 1, + 'github.number': 1, 'github.repository': 1, 'github.htmlUrl': 1, + 'github.publication.publishedAt': 1, + }, + }, + ).toArray(), + ]); + return { + rows: buildContributorLeaderboard({ profiles, receipts: githubReceipts }), + scoring: leaderboardScoring, + }; +} diff --git a/receipts/src/reviewer.js b/receipts/src/reviewer.js index b5f137c..a105000 100644 --- a/receipts/src/reviewer.js +++ b/receipts/src/reviewer.js @@ -10,6 +10,7 @@ import { initEmbedder } from './embed.js'; import { setupSchema, waitForIndex, receipts, contracts, reviewerStatus, nextSeq, } from './schema.js'; +import { reviewEventForReceipt } from './leaderboard.js'; import { similarIncidents, runCanary } from './ops.js'; import { standardReview, deepReview } from './llm.js'; @@ -35,7 +36,13 @@ export function makeReviewer(generation) { `${c.requirement} required before this ${pr.subsystem} change can merge.`, unmet: unmet.map((u) => ({ incidentNum: u.incidentNum, evidenceKey: u.evidenceKey, requirement: u.requirement })), }; - await receipts().updateOne({ prNum: pr.prNum }, { $set: { status: 'blocked', review } }); + await receipts().updateOne( + { prNum: pr.prNum }, + { + $set: { status: 'blocked', review }, + $push: { reviewEvents: reviewEventForReceipt(pr, 'blocked', new Date()) }, + }, + ); return review; } @@ -70,7 +77,13 @@ export function makeReviewer(generation) { } const review = { by, msgCount, mode: escalate ? 'deep' : 'standard', ...result }; const status = review.verdict === 'approve' ? 'approved' : 'concerns'; - await receipts().updateOne({ prNum: pr.prNum }, { $set: { status, review } }); + await receipts().updateOne( + { prNum: pr.prNum }, + { + $set: { status, review }, + $push: { reviewEvents: reviewEventForReceipt(pr, status, new Date()) }, + }, + ); return review; } diff --git a/receipts/src/schema.js b/receipts/src/schema.js index 78d2718..8993a0c 100644 --- a/receipts/src/schema.js +++ b/receipts/src/schema.js @@ -10,7 +10,7 @@ * contributors { _id: agentName, kind, createdAt } * pr_receipts { prNum, author, subsystem, changeType, title, code, fnName, * evidence[], canaryCases[], onFail, status, outcome, - * review, canary, ts } + * review, reviewEvents[], canary, ts } * status: submitted | approved | concerns | blocked | merged * outcome: pending | clean | failed * incidents { num, prNum, author, subsystem, description, failingCase, @@ -23,6 +23,8 @@ * status, receiptPrNum, receivedAt, processedAt } * github_publications { _id: publicationKey, repository, prNum, headSha, * status, statusUrl, commentUrl, publishedAt } + * leaderboard_profiles { _id: handle+subsystem, handle, repo, subsystem, + * basis, events[], active, isCurrentUser } */ import { client, db } from './db.js'; @@ -37,6 +39,7 @@ export const counters = () => db().collection('counters'); export const reviewerStatus = () => db().collection('reviewer_status'); export const githubDeliveries = () => db().collection('github_webhook_deliveries'); export const githubPublications = () => db().collection('github_publications'); +export const leaderboardProfiles = () => db().collection('leaderboard_profiles'); export async function nextSeq(name) { const r = await counters().findOneAndUpdate( @@ -68,6 +71,7 @@ export async function setupSchema() { ); await githubDeliveries().createIndex({ status: 1, receivedAt: 1 }); await githubPublications().createIndex({ status: 1, createdAt: 1 }); + await leaderboardProfiles().createIndex({ active: 1, handle: 1 }); const existing = await incidents().listSearchIndexes().toArray().catch(() => []); if (!existing.some((i) => i.name === VECTOR_INDEX)) { diff --git a/receipts/src/seed-github-demo.js b/receipts/src/seed-github-demo.js index 863d490..57c8bab 100644 --- a/receipts/src/seed-github-demo.js +++ b/receipts/src/seed-github-demo.js @@ -5,7 +5,10 @@ */ import { client } from './db.js'; import { embed, initEmbedder } from './embed.js'; -import { contracts, incidents, setupSchema, waitForIndex } from './schema.js'; +import { reviewEventForReceipt } from './leaderboard.js'; +import { + contracts, incidents, leaderboardProfiles, receipts, setupSchema, waitForIndex, +} from './schema.js'; const INCIDENT = { num: 132850, @@ -25,6 +28,102 @@ const INCIDENT = { ], }; +const LEADERBOARD_PROFILES = [ + { + _id: 'dmoody256:build/package-test', + handle: 'dmoody256', + repo: 'mongodb/mongo', + subsystem: 'build/package-test', + basis: '9 public commits during Aug 6–13; no scored failure event in this bounded sample', + profileUrl: 'https://github.com/dmoody256', + events: [], + }, + { + _id: 'sleaux:build/resmoke', + handle: 'sleaux', + repo: 'mongodb/mongo', + subsystem: 'build/resmoke', + basis: '9 public commits during Aug 6–13; no scored failure event in this bounded sample', + profileUrl: 'https://github.com/sleaux', + events: [], + }, + { + _id: 'nzolnierzmdb:query/named-enum', + handle: 'nzolnierzmdb', + repo: 'mongodb/mongo', + subsystem: 'query/named-enum', + basis: 'Public SERVER-132850 landing → auto-revert → corrective re-land', + profileUrl: 'https://github.com/nzolnierzmdb', + events: [ + { + kind: 'auto-revert', + label: 'SERVER-132850 auto-reverted', + at: '2026-08-06T20:38:07Z', + url: 'https://github.com/mongodb/mongo/commit/f552a7a32f39d5ad237dda68b3770f9fd65fdfd8', + source: 'mongodb/mongo', + }, + { + kind: 'corrective-reland', + label: 'SERVER-132850 corrective re-land', + at: '2026-08-07T14:25:06Z', + url: 'https://github.com/mongodb/mongo/commit/639953dff87d640ff58d63536ca7f6edd985debf', + source: 'mongodb/mongo', + }, + ], + }, + { + _id: 'elise-yz:replication/hash', + handle: 'elise-yz', + repo: 'mongodb/mongo', + subsystem: 'replication/hash', + basis: 'Public SERVER-132629 landing → auto-revert → corrective re-land', + profileUrl: 'https://github.com/elise-yz', + events: [ + { + kind: 'auto-revert', + label: 'SERVER-132629 auto-reverted', + at: '2026-08-07T09:26:20Z', + url: 'https://github.com/mongodb/mongo/commit/eff8cff0a62a78ab3d02643e8ec6b7f7ba5f7611', + source: 'mongodb/mongo', + }, + { + kind: 'corrective-reland', + label: 'SERVER-132629 corrective re-land', + at: '2026-08-07T19:00:10Z', + url: 'https://github.com/mongodb/mongo/commit/7d3eef2dfe353af4604b6cc3efdf61cd888816f9', + source: 'mongodb/mongo', + }, + ], + }, + { + _id: 'igpraznik:sharding/index-builds', + handle: 'igpraznik', + repo: 'mongodb/mongo', + subsystem: 'sharding/index-builds', + basis: 'Public SERVER-132822 landing → auto-revert; no re-land found in the Aug 6–13 window', + profileUrl: 'https://github.com/igpraznik', + events: [ + { + kind: 'auto-revert', + label: 'SERVER-132822 auto-reverted', + at: '2026-08-13T10:14:08Z', + url: 'https://github.com/mongodb/mongo/commit/9e0f6747a4106bbd89d766fa6ebf703d213e1f9b', + source: 'mongodb/mongo', + }, + ], + }, + { + _id: 'gorajing:query/named-enum', + handle: 'gorajing', + repo: 'gorajing/mongo', + subsystem: 'query/named-enum', + basis: 'Live signed-webhook PR-Elo receipts from the demo fork', + profileUrl: 'https://github.com/gorajing', + isCurrentUser: true, + events: [], + }, +]; + async function main() { await setupSchema(); await initEmbedder(); @@ -58,8 +157,32 @@ async function main() { }, { upsert: true }, ); + for (const profile of LEADERBOARD_PROFILES) { + const { _id, ...fields } = profile; + await leaderboardProfiles().updateOne( + { _id }, + { $set: { ...fields, active: true, updatedAt: new Date() } }, + { upsert: true }, + ); + } + + // Backfill the two already-demonstrated fork PRs once. Future reviewer + // transitions append immutable entries as their mutable receipt status moves. + const legacyReceipts = await receipts().find({ + 'github.repository': { $type: 'string' }, + status: { $in: ['blocked', 'concerns', 'approved'] }, + reviewEvents: { $exists: false }, + }).toArray(); + for (const receipt of legacyReceipts) { + const event = reviewEventForReceipt(receipt); + if (!event) continue; + await receipts().updateOne( + { _id: receipt._id, reviewEvents: { $exists: false } }, + { $set: { reviewEvents: [event] } }, + ); + } - console.log('GitHub demo memory ready: query/named-enum requires msvc-preprocessor evidence.'); + console.log('GitHub demo memory + evidence-linked contributor leaderboard ready.'); await client.close(); } diff --git a/receipts/src/server.js b/receipts/src/server.js index fe30649..c156660 100644 --- a/receipts/src/server.js +++ b/receipts/src/server.js @@ -9,9 +9,11 @@ import { client, db } from './db.js'; import { initEmbedder } from './embed.js'; import { setupSchema, waitForIndex, receipts, incidents, contracts, reviewerStatus, - githubDeliveries, githubPublications, + githubDeliveries, githubPublications, leaderboardProfiles, } from './schema.js'; -import { submitPR, shipPR, confirmIncident, addEvidence, standing } from './ops.js'; +import { + submitPR, shipPR, confirmIncident, addEvidence, contributorLeaderboard, standing, +} from './ops.js'; import { processGitHubDelivery, processPendingGitHubDeliveries, @@ -123,13 +125,14 @@ app.get('/api/live', api(async () => { { 'github.repository': { $exists: true } }, { sort: { ts: -1 }, projection: { canaryCases: 0 } }, ); - const [publication, contract, incident, reviewer] = await Promise.all([ + const [publication, contract, incident, reviewer, leaderboard] = await Promise.all([ receipt ? githubPublications().findOne({ receiptPrNum: receipt.prNum }, { sort: { createdAt: -1 } }) : null, contracts().findOne({ demoKey: 'named-enum-msvc' }), incidents().findOne({ num: 132850 }, { projection: { embedding: 0 } }), reviewerStatus().findOne({ _id: 'reviewer' }), + contributorLeaderboard(), ]); const reviewerAlive = Boolean( reviewer?.alive && Date.now() - new Date(reviewer.lastSeen).getTime() < 6000, @@ -142,6 +145,7 @@ app.get('/api/live', api(async () => { contract, incident, reviewer: reviewer ? { ...reviewer, alive: reviewerAlive } : null, + leaderboard, }; })); @@ -198,6 +202,9 @@ async function main() { watchSafe('github_publications', githubPublications, (evt) => { if (evt.fullDocument) push('publication', evt.fullDocument); }); + watchSafe('leaderboard_profiles', leaderboardProfiles, (evt) => { + if (evt.fullDocument) push('leaderboard', evt.fullDocument); + }); // Reviewer liveness: heartbeat doc older than 6s (or alive:false) reads as // dead. This is what makes killing the process visible on the big screen. diff --git a/receipts/test/leaderboard.test.js b/receipts/test/leaderboard.test.js new file mode 100644 index 0000000..79df6f7 --- /dev/null +++ b/receipts/test/leaderboard.test.js @@ -0,0 +1,100 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; + +import { buildContributorLeaderboard, scoreEvents } from '../src/leaderboard.js'; + +test('scoreEvents exposes every transition from the 100 baseline', () => { + const scored = scoreEvents([ + { kind: 'missing-evidence', label: 'PR #1 blocked', at: '2026-08-13T23:50:41Z' }, + { kind: 'missing-evidence', label: 'PR #2 blocked', at: '2026-08-14T01:15:53Z' }, + ]); + + assert.equal(scored.score, 90); + assert.deepEqual(scored.trend, [100, 95, 90]); + assert.deepEqual(scored.events.map(({ from, to, delta }) => ({ from, to, delta })), [ + { from: 100, to: 95, delta: -5 }, + { from: 95, to: 90, delta: -5 }, + ]); +}); + +test('leaderboard combines public MongoDB evidence with live fork receipts', () => { + const profiles = [ + { + handle: 'nzolnierzmdb', + repo: 'mongodb/mongo', + subsystem: 'query/named-enum', + events: [ + { kind: 'auto-revert', label: 'SERVER-132850 auto-reverted', at: '2026-08-06T20:35:00Z' }, + { kind: 'corrective-reland', label: 'SERVER-132850 re-landed', at: '2026-08-07T14:25:06Z' }, + ], + }, + { + handle: 'gorajing', + repo: 'gorajing/mongo', + subsystem: 'query/named-enum', + isCurrentUser: true, + events: [], + }, + ]; + const receipts = [ + { + prNum: 1, + author: 'gorajing', + subsystem: 'query/named-enum', + status: 'blocked', + ts: new Date('2026-08-13T23:50:41Z'), + github: { number: 1, htmlUrl: 'https://github.com/gorajing/mongo/pull/1' }, + }, + { + prNum: 2, + author: 'gorajing', + subsystem: 'query/named-enum', + status: 'blocked', + ts: new Date('2026-08-14T01:15:53Z'), + github: { number: 2, htmlUrl: 'https://github.com/gorajing/mongo/pull/2' }, + }, + ]; + + const rows = buildContributorLeaderboard({ profiles, receipts }); + assert.deepEqual(rows.map(({ handle, score }) => ({ handle, score })), [ + { handle: 'nzolnierzmdb', score: 98 }, + { handle: 'gorajing', score: 90 }, + ]); + assert.equal(rows[1].events[1].url, 'https://github.com/gorajing/mongo/pull/2'); + assert.equal(rows[1].isCurrentUser, true); +}); + +test('a recovered PR preserves its blocked event before adding verified evidence', () => { + const rows = buildContributorLeaderboard({ + profiles: [{ + handle: 'gorajing', + repo: 'gorajing/mongo', + subsystem: 'query/named-enum', + isCurrentUser: true, + events: [], + }], + receipts: [{ + prNum: 3, + author: 'gorajing', + subsystem: 'query/named-enum', + status: 'approved', + ts: new Date('2026-08-14T02:00:00Z'), + github: { + number: 3, + repository: 'gorajing/mongo', + htmlUrl: 'https://github.com/gorajing/mongo/pull/3', + }, + reviewEvents: [ + { kind: 'missing-evidence', status: 'blocked', at: new Date('2026-08-14T02:00:01Z') }, + { kind: 'verified-evidence', status: 'approved', at: new Date('2026-08-14T02:04:00Z') }, + ], + }], + }); + + assert.equal(rows[0].score, 98); + assert.deepEqual(rows[0].trend, [100, 95, 98]); + assert.deepEqual(rows[0].events.map((event) => event.label), [ + 'PR #3 blocked', + 'PR #3 approved', + ]); +});