Skip to content

Add structured telemetry extraction from OCR result for longitudinal quality analysis #2676

Description

@acoliver

Problem

Each OCR run produces rich structured data (result JSON with category/severity/path per finding), but our workflow does not extract or persist this data in a queryable form. To make data-driven decisions about OCR quality, concurrency, publication routing, and rule.json prompt tuning, we need structured telemetry across runs.

Currently the only data we have is ad-hoc manual analysis of individual run artifacts. There is no longitudinal view.

Proposed solution

Add a telemetry extraction step that runs after each OCR review and produces a structured summary:

Telemetry record

For each OCR run, emit a JSON object with:

{
  "run_id": "...",
  "pr_number": ...,
  "sha": "...",
  "ocr_version": "1.7.9",
  "concurrency": 2,
  "wall_clock_seconds": ...,
  "files_previewed": ...,
  "files_reviewed": ...,  // derived from result
  "total_findings": ...,
  "findings_by_category": { "bug": N, "maintainability": N, ... },
  "findings_by_severity": { "high": N, "medium": N, "low": N },
  "findings_by_category_x_severity": { "bug/high": N, ... },
  "inline_posted": ...,
  "already_resolved": ...,  // count of findings already addressed
  "infrastructure_failure": false,
  "policy_failure": false,
  "file_read_failures": [...],
  "token_usage": ...  // if available
}

Storage

Two options (can be implemented independently):

Option A: GitHub Actions artifacts

  • Upload the telemetry JSON as a named artifact on every run.
  • Simple, no infrastructure, but requires manual download for analysis.

Option B: GitHub Actions step summary

  • Render a compact markdown summary in the workflow run page (using $GITHUB_STEP_SUMMARY).
  • Immediate visibility in the Actions UI without downloading artifacts.

Dashboard

Once telemetry records exist, a simple script can aggregate them across runs to answer:

  • Average finding count per run (and trend).
  • Most common categories/severities (and trend).
  • Average wall-clock time per run by concurrency setting.
  • File-read failure rate.
  • Inline comment volume over time.

This data directly informs #2672 (publication routing policy tuning) and #2676 (rule.json prompt iteration).

Acceptance criteria

  • Each OCR run emits a structured telemetry JSON.
  • Telemetry includes finding counts by category, severity, and category x severity.
  • Telemetry includes wall-clock time, file counts, and failure flags.
  • Telemetry is uploaded as an artifact.
  • A markdown summary is rendered in the Actions step summary.
  • An aggregation script exists to produce longitudinal statistics across runs.

Relationship to other issues

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions