Skip to content

Add web app layer: FastAPI backend, job queue and per-frame coordinate CSV export - #2

Open
dylangriffinshub wants to merge 4 commits into
DesusLove:mainfrom
dylangriffinshub:upstream/tracking-pipeline
Open

Add web app layer: FastAPI backend, job queue and per-frame coordinate CSV export#2
dylangriffinshub wants to merge 4 commits into
DesusLove:mainfrom
dylangriffinshub:upstream/tracking-pipeline

Conversation

@dylangriffinshub

Copy link
Copy Markdown

Adds a deployable web app on top of the existing CLI, plus the per-frame coordinate export the pipeline did not have.

The CLI (examples/soccer/main.py) is untouched. Everything new lives in a separate pitchvision/ package.

Pipeline (pitchvision/pipeline.py)

  • Extracts the tracking flow into a reusable run_pipeline() that a server can call, rather than a generator tied to a CLI and local file paths.
  • Adds a per-frame CSV export: one row per tracked object with frame, timestamp_s, tracker_id, class, team, centre point, bbox and confidence.
  • Caches team assignment per tracker id. A player's shirt does not change, so each track is classified once instead of on every frame. This is the main speedup: SigLIP embedding was previously repeated for every detection on every frame.
  • PipelineConfig exposes max_seconds, frame_stride, imgsz, device and torch_threads so the same code runs on a small CPU box or a GPU.

Web app (pitchvision/api.py, pitchvision/static/)

  • FastAPI service with a SQLite-backed job queue and a single background worker, so concurrent uploads serialise instead of thrashing the machine.
  • Upload page with progress polling; OpenAPI docs at /api/docs.
  • Jobs interrupted by a restart are marked failed on startup rather than left hanging in processing.

Bug worth fixing regardless of this PR

TeamClassifier requires sentencepiece and protobuf (SigLIP's processor pulls them in). Neither is in examples/soccer/requirements.txt, so on a clean install constructing the classifier raises ImportError.

That failure mode is quiet and damaging: if the exception is caught broadly, clustering is skipped and every player is labelled team 0, so the run looks successful while producing wrong data. This PR surfaces the reason via PipelineResult.team_error instead of failing silently. You may want the requirements.txt fix upstream on its own.

Verified

On a 6s broadcast clip: 432 / 426 team split, referees detected separately, ball tracked, 956 CSV rows across 50 processed frames. Sample input, annotated output, stills and CSV are committed in docs/samples/.

Performance

Measured on an Intel i3-8100B (4 cores, no GPU): 1.20 s/frame at imgsz=640, 2.67 s/frame at 960. CPU inference is the bottleneck, which is why the defaults cap clip length and stride. On CUDA, PipelineConfig(device='cuda') with frame_stride=1 is the only change needed.

Known issue, pre-existing

Goalkeepers are assigned by proximity to team centroids. Goalkeeper kit matches neither outfield team, so they can be miscoloured. Behaviour inherited from resolve_goalkeepers_team_id, not introduced here.

Notes

  • .gitignore now excludes var/ (runtime state) and the downloaded weights.
  • Four <video> tags in the README were replaced: they point at Git LFS files not present in the repo, and GitHub does not render relative <video src> anyway, so they showed as broken. They now reference the committed samples.
  • MIT licence and upstream attribution to roboflow/sports retained.

Happy to split this up or rework any part of it.

dylangriffinshub and others added 4 commits July 25, 2026 16:00
Wraps the existing detection/tracking code into a service that runs at
pitchvision.hopto.org, so footage can be uploaded and analysed in a browser.

Pipeline (pitchvision/pipeline.py):
- Extracts the tracking flow out of the CLI into run_pipeline(), callable
  from a worker: video in, annotated MP4 + coordinate CSV out.
- Adds the per-frame CSV export that was missing: one row per tracked
  object with tracker id, class, team, centre point, bbox and confidence.
- Caches team assignment per tracker id, so each player is classified once
  instead of on every frame.
- Surfaces team-clustering failures via PipelineResult.team_error. A broad
  except was previously swallowing a missing-dependency ImportError, which
  silently labelled every player team 0 while appearing to succeed.

API and UI (pitchvision/api.py, static/index.html):
- FastAPI service with a SQLite-backed job queue and a single background
  worker, so concurrent uploads serialise rather than thrashing the CPU.
- Upload page with progress polling, sample output, pipeline walkthrough
  and a feature breakdown; OpenAPI docs at /api/docs.
- Interrupted jobs are marked failed on startup rather than left hanging.

Docs and samples:
- docs/samples/ holds committed input and output from a real run (clip,
  annotated clip, stills, CSV) as evidence of what the pipeline produces.
- docs/deployment.md records the host setup, the fragile Intel-Mac
  dependency pins, and the stale Colima bind mount that makes caddy
  reload silently no-op.
- docs/benchmark-stepout.md is a written competitive review used to shape
  positioning. No third-party assets are vendored; all media here is our
  own output.
- .gitignore now tracks docs/samples/ while excluding var/ runtime state
  and the large model weights fetched by setup.sh.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Icons and metadata:
- SVG favicon plus 32px, 180px apple-touch and 512px PNG fallbacks, drawn
  as a pitch with the app's own team/referee colours.
- site.webmanifest, theme-color, canonical URL, description, Open Graph
  and Twitter card tags using the real sample frame as the preview image,
  and SoftwareApplication structured data.

Benchmark section:
- Replaces the plain text table with a to-scale turnaround chart and
  side-by-side capability columns, so the comparison reads at a glance.
- Every figure is attributed and linked to the StepOut page it came from
  so a reader can verify it, with an explicit note that StepOut is a
  separate, unaffiliated company.
- States where this project is behind (event tagging, heatmaps, player
  identity) alongside where it leads, and cites StepOut's published scale
  figures for context. No accuracy percentage is claimed for PitchVision,
  since it has not been scored against labelled ground truth.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
var/weights/ held ~415MB of downloaded model weights but matched none of
the enumerated var/* rules, so it showed as untracked and was one `git add
-A` away from entering history. Ignoring the directory outright removes
that class of mistake: anything new under var/ is machine-local by
definition.

Also ignores .venv/ and documents why the tracked files in
examples/soccer/data/ are LFS stubs that the real downloads must not
overwrite.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Prepares this branch as a neutral upstream contribution:

- Drops the personal deployment host from the README (live demo section,
  curl examples now use localhost, roadmap entry) and from the page
  metadata (canonical URL, absolute og:/twitter: image URLs, author tags,
  structured data).
- Removes the competitor comparison section from the landing page and
  deletes docs/benchmark-stepout.md. That is positioning for a specific
  deployment, not something upstream should carry.
- Rewrites docs/deployment.md as general guidance: architecture, tuning
  via PipelineConfig, the measured CPU baseline, and the dependency
  constraints, with no reference to one particular server.
- Removes the now-unused chart/comparison CSS.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant