Skip to content

Export complete datasets as verified Ultralytics detection and pose archives - #103

Merged
alexschmander merged 13 commits into
feat/review-missing-object-evidencefrom
feat/ultralytics-dataset-export
Sep 7, 2026
Merged

Export complete datasets as verified Ultralytics detection and pose archives#103
alexschmander merged 13 commits into
feat/review-missing-object-evidencefrom
feat/ultralytics-dataset-export

Conversation

@alexschmander

@alexschmander alexschmander commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

Current-head CI: all selected jobs and the required Testing check passed. Successful run. This supersedes earlier awaiting-CI statements below; independent acceptance remains pending.

Behavior or contract

Closes #54. Export complete ground truth and unchanged original images as verified Ultralytics detection or pose ZIPs, with documented round-trip preservation and loss. Native identities and workflow history are not restored on re-import.

Train, validation, and test are independently selectable and selected by default. The separate fallback assigns images without split provenance. Empty workflow selection disables preflight without a warning. Reload restores only active or Ready captures; finished-job diagnostics remain in explicit history views. Background status polls preserve form controls and layout.

Image-count and size quotas are optional and unlimited by default. Explicit operator quotas, concurrency controls, and retention remain supported. Capture avoids a full index rehash for every image and avoids durable syncs for disposable spool files; final source checks and archive publication durability remain intact.

Base: #101, feat/review-missing-object-evidence at 9405867924d038c164a617f881e9c0b5733a173b. Current head: d7583c72241e59565b60bc2df872c4d82b0626b4. Full stack: main → #98#101#103.

Production ownership path

Domain owns profiles, mappings, coverage, splits and pose bounds. Repository capture replays event cuts under image locks. Storage owns original validation, private capture, ZIP verification, guarded publication, cancellation, recovery and retention. API owns authorization and streamed delivery. Shared UI and client DTOs own selection and job interaction. Import owns the explicit acknowledged absent-pose policy.

Acceptance criteria and evidence

Criterion Evidence
Both profiles and content-preserving round trips Domain/storage encoding tests, production re-import and pinned reader evidence
Omit incomplete work without false negatives Coverage, capture, split-filter and effective-review tests
Original bytes, paths, source consistency and atomic publication Hash, format, archive, mutation, publication and recovery tests
Authorized lifecycle and download API role/CSRF/session tests and storage service tests
Selection, polling, history, retry and failure cleanup Focused UI suite and terminal capture-limit cleanup regression

Regression protection

Capture tests cover unselected splits before event reads, default-unlimited limits, streamed metadata, configuration checks at publication, and failure cleanup without downloadable artifacts. UI tests cover real selection commands, stable polling, stale-response rejection, explicit history inspection and terminal-job retry.

Verification

The focused export UI suite passed 16 tests on the export-history update. The later empty-selection warning change received focused diff review. The final parent corrections passed their targeted production/UI regressions in #98 and #101. Broad current-head verification runs in GitHub CI after push.

Earlier combined canonical verification, both production round trips, and the pinned offline Ultralytics reader passed on recorded earlier heads. Those results are historical evidence and are not substituted for current-head CI or independent acceptance.

Visual or browser evidence

Earlier export inspection covered all six native viewports, a real Chromium preflight/start/download flow, and a delayed 3.5-second poll without form displacement. High-DPR evidence was limited to screenshots; no 200% interaction claim is made for that run. Fresh native/browser/external-reader passes were not repeated for the current parent corrections.

Documentation

Current contracts are in docs/export.md, docs/assignment.md, docs/api.md, docs/configuration.md, docs/operations.md, and docs/ui-ownership.md. Configuration examples document optional quotas.

Risks and omitted checks

Export requires the documented Linux filesystem primitives and one server per datasets root. Unlimited quotas do not imply unlimited physical disk or memory. Published ZIPs require extraction before import. Large-dataset speedup has not been benchmarked; the exact legacy metadata-quota failure was not reproduced conclusively. Follow-up independent acceptance remains pending.

Worktree integrity

  • Unrelated worktree changes were preserved and excluded.
  • No credentials, runtime datasets, generated distributions, or sensitive diagnostics are committed.

Handoff status

  • Implementation evidence is assembled; current-head CI passed.
  • Required Testing succeeded on the current head.
  • Existing author assignments and reviewer requests are preserved.
  • Ready for review after current-head CI and follow-up independent acceptance.

Latest stack update carries #98 self-submitted review reopening and bounded conflict diagnostics. GPT-5.6-Luna Max passed focused storage/API regressions and verified the final active stack ancestry. Broad verification awaits GitHub CI on the current head.

CI follow-up: the prior exact-head run passed all selected jobs except Clippy and its dependent Testing aggregate. Fixed test-module ordering and an unnecessary integer cast in #98; these changes carry through the stack. Fresh hosted CI is required for the updated heads.

GPT-5.6-Luna Max verified the two lint fixes with cargo clippy --locked -p labello-storage --all-targets --all-features -- -D warnings (passed), final formatting/diff checks, exact tested-file parity, clean active worktrees, and main → #98#101#103 ancestry.

@alexschmander alexschmander self-assigned this Sep 5, 2026
@alexschmander
alexschmander marked this pull request as ready for review September 5, 2026 18:50
@alexschmander
alexschmander marked this pull request as draft September 5, 2026 20:26
@alexschmander
alexschmander marked this pull request as ready for review September 5, 2026 20:55
@alexschmander
alexschmander changed the base branch from feat/previous-review-revision to feat/review-missing-object-evidence September 6, 2026 07:38
@alexschmander
alexschmander force-pushed the feat/ultralytics-dataset-export branch 2 times, most recently from 9079c68 to 0d64158 Compare September 7, 2026 09:30
@alexschmander
alexschmander force-pushed the feat/ultralytics-dataset-export branch from 0d64158 to bdb9c96 Compare September 7, 2026 09:42
@alexschmander
alexschmander force-pushed the feat/ultralytics-dataset-export branch 2 times, most recently from edaf192 to 9b3898d Compare September 7, 2026 10:32
@alexschmander
alexschmander force-pushed the feat/ultralytics-dataset-export branch from 59879ba to 8381693 Compare September 7, 2026 11:04
@alexschmander
alexschmander force-pushed the feat/ultralytics-dataset-export branch 2 times, most recently from 3b18168 to a57b693 Compare September 7, 2026 12:08
@alexschmander
alexschmander force-pushed the feat/ultralytics-dataset-export branch from a57b693 to d7583c7 Compare September 7, 2026 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Export Ultralytics detection and pose datasets with explicit round-trip guarantees

1 participant