Skip to content

Support importing into or merging with an existing dataset #50

Description

@alexschmander

Problem

Explicit ground-truth import can only create a new dataset; operators cannot safely merge supported imported content into an existing dataset.

Current behavior

Import builds/verifies a complete dataset and publishes it atomically with no replacement, which deliberately avoids complex identity, config, and audit-history merges.

Desired behavior

Define and implement a merge/import workflow that preserves no-replace safety, image identity, configuration compatibility, and reconstructable provenance.

Acceptance criteria

  • Define matching/conflict policy for dataset/task/class/image/annotation identities and duplicate paths/content.
  • Plan and preview all mutations before an authorized commit.
  • Represent imported changes as replayable events with exact source/profile provenance and coverage semantics.
  • Use a durable, resumable transaction with rollback/no partial publication.
  • Cover active assignments, concurrent changes, schema/config mismatch, duplicate images, cancellation/restart, statistics, and snapshots.

Scope

Merge supported explicit ground-truth import profiles into an existing dataset; no predictions, segmentation, or remote acquisition unless separately supported.

Implementation notes

Preserve BLAKE3 image identity and never implement by copying files into managed paths.

Tracking source

docs/tracking/feature-requests.md:40

Import into or merge with an existing dataset without weakening no-replace publication and audit-history guarantees.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: storageFilesystem persistence, repositories, and durable jobsenhancementNew feature or requestreliabilityRecovery, durability, failure handling, and operational robustnessworkflow: importGround-truth import lifecycle

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions