diff --git a/docs/development/plans/jabs-hub-backed-projects-plan.md b/docs/development/plans/jabs-hub-backed-projects-plan.md new file mode 100644 index 00000000..c30ab3ad --- /dev/null +++ b/docs/development/plans/jabs-hub-backed-projects-plan.md @@ -0,0 +1,747 @@ +# Plan: Hub-Backed JABS Projects — Client Integration (jabs-behavior-classifier) + +- **Status:** Draft / planning +- **Author:** Glen Beane (with Claude Code) +- **Repository:** `jabs-behavior-classifier` (this repo — the PySide6 desktop GUI / Python client). +- **Companion doc:** the Hub-side (Go) design lives in the `jabs-hub` repo at + `docs/specs/0006-jabs-project-backend-design.md`. The REST API, the PostgreSQL data model, and the + annotation-document schema (Appendix A.1) are the **shared contract** — keep them in sync. +- **Scope:** the client half of Hub-backed JABS projects under a **cloud-native** model: the desktop + GUI becomes a caching client that pulls videos, pose files, annotations, and project metadata from + JABS Hub / Google Cloud Storage into a local cache, works against the cache (frame-accurate + labeling, feature extraction, training, offline), and pushes changes back. + +--- + +## 1. Overview — JABS Hub-backed projects + +A **Hub-backed JABS project is a cloud project**: its videos, pose files, behavior annotations, +and project metadata all live in **JABS Hub** (PostgreSQL + Google Cloud Storage, at +`github.com/TheJacksonLaboratory/jabs-hub`). The desktop GUI is a **caching client** — it pulls +what it needs from Hub / GCS into a local **cache directory**, works against the cache, and pushes +changes back. There is **no hybrid**: a Hub-backed project never has authoritative local-only +media. Local-only (non-Hub) projects are unchanged. + +- **Videos live in a shared library.** Videos — external/uploaded files *or* JABS-2.0 device + recordings — are uploaded into a Hub **video library**, decoupled from any recording session. A + project references library videos **many-to-many**: one stored copy can back multiple projects. +- **Source of truth = cloud; local = cache.** One uniform pull → cache → push model for every + *source* artifact (video, pose, annotations, metadata). **Derived** artifacts (features, + predictions, classifiers) are recomputed locally from the cached pose and stay local (§4.9). +- **Offline-first.** The cache makes a project fully usable offline; edits queue and sync on + reconnect. +- **Primary driver.** Replace the "zip up `jabs/annotations/` and email it" label-sharing workflow + with automatic, versioned sync (§2.3). Because the exact pose file also comes from Hub, the + identity-alignment footgun (labels are keyed by identity, which the pose file defines) is + **eliminated**, not merely flagged. + +**This is a two-part plan**, split across the two repositories it touches: + +- **Client integration** — *this document* (`jabs-behavior-classifier`): OIDC auth, the Hub + client, the caching layer (media resolver + annotation cache), sync engine, opt-in/open flows, + and playback of Hub-hosted media. +- **Hub backend** — `jabs-hub` repo, `docs/specs/0006-jabs-project-backend-design.md`: the video + library + decoupled upload, the `projects` / `project_videos` (many-to-many) / `annotations` + data model, the REST API, membership/authorization, and media object storage. + +### Phasing (cloud-native, library-first build order) + +| Phase | Title | Delivers | +|------|-------|----------| +| **0** | Foundations | OIDC auth in the client + `HubClient`; the client cache/abstraction seams (annotation store + media resolver) landed as a pure refactor with local-only behavior unchanged. (Hub: auth on new routes, base scaffolding.) | +| **1** | Video library + media | Client: a Hub-aware media resolver + local media cache + lazy hydration + playback prefetch — download/cache to **open and play** a Hub project's videos. (Uploading into the library is a web-UI / `jabs-cli` concern.) (Hub: `videos` table, decoupled upload, list/search, content-addressed storage, signed download.) Deliverable: the GUI opens a Hub project and plays its videos. | +| **2** | Hub-backed projects | Client: **open** a cloud project (created in the web UI) referencing library videos; the local project dir is a cache; project-settings sync. (Hub: `projects` + `project_members` + `project_videos` join + metadata.) | +| **3** | Annotations (label sharing) | Client: annotation cache + sync engine + offline outbox + conflict handling; pose consistency guaranteed. (Hub: `annotations` + history + optimistic-concurrency contract + behavior index.) Delivers the label-sharing payoff. | +| **Post-MVP** | Collaboration + ML | Web preview playback + prediction-track overlay (library browse itself is part of the initial web UI), classifier registry/model cards, real-time collaboration, normalized fine-grained labels. | + +Phase 0 lands the seams first so they are independently reviewable and de-risk the rest (mirrors +the 0001 Parquet feature-cache implementation plan, which extracted the I/O boundary +before changing behavior). + +**Relationship to prior work:** this supersedes the older *"JABS Hub"* Word design doc (the +device/recording/processing platform already exists) **and** an earlier iteration of this plan that +migrated annotations first and left media local. The cloud-native model was chosen deliberately +over that piecemeal approach: the video library needs cloud media anyway, and full cloud storage +makes the label-sharing use case correct (shared pose ⇒ aligned identities) rather than merely +warned. + +--- + +## 2. Goals, Non-Goals, and Motivating Use Cases + +### 2.1 Goals + +1. **Cloud-native Hub-backed projects.** A Hub-backed project stores its video, pose, annotations, + and metadata in Hub; the GUI caches locally and stays fully usable offline. Local-only projects + are unaffected. +2. **Video library as the media home.** The GUI pulls library videos (and pose) down for + playback/labeling/training. Uploading into the library is a web-UI / `jabs-cli` task, not a + desktop-GUI flow. +3. **Automatic annotation sync** with version history, attribution, and conflict handling — + replacing the manual zip hand-off. +4. **Uniform caching model.** One pull/cache/push mechanism for every source artifact; derived + artifacts recomputed locally from the cached pose. +5. **Single source of authority.** Hub is authoritative online; the client reconciles the cache + against it, resolving the "which copy is current" ambiguity. + +### 2.2 Non-Goals + +- Replacing the desktop GUI with a web labeling client. The GUI stays the frame-accurate labeling + surface; the JABS Hub web UI (to be developed) creates/manages projects and the video library but + does not do frame-accurate labeling. +- **Creating or managing Hub projects in the desktop GUI** (create a project, add library videos, + share) — that happens in the JABS Hub web UI. The desktop GUI **opens** an existing Hub project by + its identifier (§4.7). +- Storing **derived** artifacts (features/predictions/classifiers) in Hub for the MVP — they are + recomputed locally from the cached pose (§4.9). An **optional cloud cache for features** (the + expensive one) and a classifier registry for predictions/classifiers are post-MVP (§10). +- Real-time collaborative cursors / live presence (post-MVP). +- True frame streaming of remote video (playback is download-to-cache + prefetch; §4.6). +- **Fine-grained permissions** (roles, per-video/per-behavior rights, group- or lab-level access). + The MVP is authenticated access + per-project membership, enforced by Hub — see §2.5 for the full + in-scope / out-of-scope breakdown, and D24. + +### 2.3 Motivating use cases + +1. **Asymmetric label sharing (primary driver; needs Phase 3 annotation sync *and* project sharing, D19).** A behavior expert labels + videos; a colleague trains/evaluates classifiers from those labels. Today the expert zips + `/jabs/annotations/` and sends it — manual, overwrites local work, no history, and no + guarantee both sides use the *same* pose file. In a cloud project both users open the same Hub + project; labels sync automatically with history and attribution, and because the **pose file is + pulled from Hub too**, the collaborator's labels always align to the correct identities. No zip, + no overwrite, no pose-mismatch footgun. +2. **Create a project in the web UI, open it in the GUI.** In the JABS Hub web UI a user creates a + project and adds videos to it from the library (JABS recordings or uploaded external files), then + opens it in the desktop GUI (§4.7) — the media hydrates on demand. +3. **Work from any machine / HPC** with no manual file copying — the cache hydrates from Hub. +4. **Going offline, or onto low bandwidth.** Before a flight, a field trip, or a move to a slow or + metered link, a user hydrates the **whole** project in one deliberate action — every video, pinned + pose file, and annotation document — instead of relying on lazy per-video downloads that would + fail once the connection is gone, and pins it so cache eviction leaves it alone (§4.6, D22). + +### 2.4 Unobtrusiveness for non-Hub users (hard requirement) + +Many JABS users do not have access to (or do not use) JABS Hub. Hub integration MUST be invisible +to them: + +- **No startup cost.** The app launches and every local-project workflow behaves exactly as today — + no network calls, no auth prompts, no "sign in to Hub" nags, no added latency. Hub code paths are + **lazy-imported** and never execute until the user invokes a Hub action. +- **One unobtrusive entry point.** A single **File → "Open Project from JABS Hub"** menu item + (§4.7); no toolbars, banners, or modal prompts intrude on local use. Local project open/create + never touches Hub. +- **Opt-in auth.** OIDC login is triggered only by a Hub action, never at startup; credentials live + in the OS keyring, never in project files. +- **Graceful absence.** If the Hub client dependency or connectivity is unavailable, only Hub + actions are affected — local projects are untouched, with a clear message rather than a crash. + +A regression test asserts that launching the app and opening a local project make **zero** Hub +imports or network calls (§7). + +### 2.5 Permissions and authorization — what is in and out of scope + +Authorization is **enforced by Hub**, not by the client (Hub doc D5, recorded in the `jabs-hub` +repo as **ADR-0014**), but the *goals* it has to meet belong in scope here so they are not left +implicit. Users are **not** assumed to be uniformly trustworthy with write access to everything. + +**In scope (MVP)** + +- **Authenticated access only.** Every Hub request carries an OIDC access token (§4.5); there is no + anonymous, shared-secret, or unauthenticated read path. Without a token the client sees no projects. +- **Per-project membership, enforced server-side on every request.** A project has an owner and a + membership set (`project_members`); Hub authorizes each call against the caller's membership. Per + D19 projects are **personal (owner-only)** in the first cut, so the MVP membership set is normally + one user — but the check is a membership check from day one, so sharing is an added row, not a + redesign. +- **Membership gates projects, not the video library.** A library video exists before and + independently of any project (ADR-0012), so project membership is not a well-defined question for + library media: **ADR-0014 Decision 3 makes the library readable *and writable* by any authenticated + Hub caller.** For a JABS user that means any colleague with Hub access can list, download, and add + to the library — including the videos a project pins — while **projects and annotations**, the + scientific work product, are what membership actually protects. This deliberately narrows ADR-0001 + §4 ("the API checks project membership before generating any URL") to *project* media. Do not + present library media in the GUI as though it were private to the project that references it. This + is the MVP posture, **not the end state** — group-scoped library visibility is a known future + requirement (below), so treat the open catalogue as temporary and keep browse paths behind a + filtered query. +- **The client is not a trust boundary.** The GUI only *reflects* permissions (e.g. greying out + labeling for a project it may not write); it never grants them. A patched client, a hand-edited + `hub.json`, or a guessed project ID gains nothing — every read and write is re-authorized by Hub. +- **Possessing a project identifier is not access.** Pasting a project link (§4.7) only names the + project; the caller must still be a member. Links are shareable *because* they confer no rights. +- **Short-lived, per-request media credentials.** Signed GCS URLs are minted per authorized request + and expire; a cached URL is not a durable capability, and copying a cache directory to another + machine does not carry Hub access with it. +- **Attribution comes from the token.** Annotation history records the identity Hub derives from the + access token, not the client-supplied `labeler` field (Appendix A.1), which stays a display value. + +**Out of scope (MVP) — named so they are deliberate omissions, not oversights** + +- **Roles and fine-grained permissions.** No viewer/labeler/owner distinction, no per-behavior or + per-video permissions: a project member reads and writes everything in that project. A role model + is post-MVP (§10); when it lands, the client's only job is to honor a `permissions` field on the + project manifest and disable the corresponding UI. +- **Lab-, group-, or organization-level access control**, including any mapping from JAX directory + groups. Membership is per project and explicit. +- **Group-scoped library visibility — out of scope for the MVP, but a known future requirement, not a + hypothetical.** Library videos will eventually need to be readable only to specified groups + (collaborator data, embargoed studies), so the lab-wide catalogue above is a starting posture rather + than the end state. Enforcement is Hub-side (ADR-0014 Decision 3 today, its OQ2 for the ACL model); + the client's job when it lands is to **honor** a visibility/permissions field on the manifest and + the library listing, not to enforce one — which is why the client must not build UI that assumes an + open catalogue (e.g. "browse everything" affordances with no filtered-query path behind them). + **Revocation cannot be retroactive for a cached project:** the cache holds video and pose bytes on + disk (§4.2), so withdrawing a group's access later does not reach bytes already pulled. That is + inherent to offline-first, and it means group scoping has to be applied at upload/link time rather + than relied on as a later withdrawal. +- **Locking as a permission concept.** Concurrent writers are handled by optimistic concurrency + (D2/§4.8), not by access control; behavior/video-level locking is post-MVP (§10). +- **Protecting the local cache.** Cached video, pose, and annotation files are ordinary files under + normal filesystem permissions — no client-side encryption at rest, and no attempt to stop a local + user from reading a cache they have filesystem access to. Consequence to document for users: do not + place a Hub project cache on shared storage whose readers are not all project members. Tokens are + the exception — OS keyring, never in project files (§2.4). +- **Audit and compliance reporting.** Annotation version history provides attribution and rollback, + not a tamper-evident audit log. + +--- + +## 3. Current State — client seams we build on + +### 3.1 Video/pose resolution seams (now core, not Phase 2) + +- Video↔pose pairing is name-derived: `NAME.mp4` ↔ `NAME_pose_est_v{N}.h5` (`_POSE_SUFFIX_RE`, + `packages/jabs-core/.../utilities.py:69`; `get_pose_path`, + `src/jabs/pose_estimation/__init__.py:40`). +- **Single video path resolver:** `VideoManager.video_path(video_file)` → + `Path(video_dir, video_file)` (`src/jabs/project/video_manager.py:251`). **This is the seam a + Hub-aware resolver overrides** to download-on-demand into the cache. +- **Single video open point:** `VideoReader.__init__` → `cv2.VideoCapture(str(path))` + (`src/jabs/video_reader/video_reader.py:19`). Path-only; needs random-access seeking → the + client must **download the full file to the cache before opening** (no true streaming). +- **`video_dir` / `pose_dir` are already decoupled** from the project directory (`ProjectPaths`, + `src/jabs/project/project_paths.py:18`; `Project.__init__` accepts them, `project.py:189`). This + is exactly the plumbing to point at a cache root. +- **Pose loading needs the full local file** (hashed end-to-end via `hash_file`, + `utilities.py:39`; h5py random access). No lazy/partial read → download-to-cache. +- **Feature/prediction caches key on name + validate on pose hash** (blake2b), not on path + (`src/jabs/feature_extraction/features.py:178`; `packages/jabs-io/.../feature_cache/base.py:71`; + `src/jabs/project/prediction_manager.py:167`). **Consequence:** a byte-identical pose pulled from + Hub keeps derived caches valid — the basis for keeping derived artifacts local (§4.9). +- **Project open touches every video/pose file** (`_validate_pose_files`, `video_manager.py:238`; + scan workers, `parallel_workers.py:175`) — must become **manifest-driven + lazy** so opening a + cloud project does not download everything up front (§4.6). + +### 3.2 Annotation seams + +- **Writes funnel through one method:** `Project.save_annotations(annotations, pose)` + (`src/jabs/project/project.py:618`) — atomic temp-file `replace()` into + `jabs/annotations/