End-to-end staging: IngestSpec.with_uploaded_sources

[underchemist]

Parent

#56

What to build

Tie parsing and staging together so a user can take a spec that references local source data, upload it, and ingest — in one flow.

IngestSpec.with_uploaded_sources(client, source_file_sets=[...]):

• Accepts a PrescientClient or IngestClient (staging uses the STS upload session).
• For each named source file set: resolve its location's local path and upload the files its pattern selects (via the staging helper from Pattern-filtered staging upload helper #60), into that location's destination prefix.
• Compute one uuid4 destination prefix per distinct location (s3://{upload_bucket}/{prefix}/{uuid4}/), so multiple sets referencing the same location share a destination and that location's path is rewritten once.
• Return a NEW, immutable IngestSpec whose staged locations are now s3:// (the original instance is untouched). A file matched by two staged sets that share a location is uploaded each time (idempotent overwrite).
• Render per-set progress using the upload callback.
• Raise if a named set or its location is missing, or the location is already s3://.

The full path is then: IngestSpec.from_file(...).with_uploaded_sources(...) → IngestResource.create(...) → wait/start. Requires OAuth/IDP auth (upload uses STS assume-role-with-web-identity; not supported in API-key mode).

Acceptance criteria

• Staging uploads each named set's pattern-matched files under a per-location uuid4 prefix and rewrites that location's path to it
• Two sets sharing a location share one prefix; the location path is rewritten exactly once
• Returns a new s3-only IngestSpec; the original instance is unchanged (immutability)
• A missing set/location, or an already-s3:// location, raises a clear error
• End-to-end path works: from_file → with_uploaded_sources → create
• Tests cover per-location prefixing, shared-location behavior, immutability, and error cases (uuid4 patched for determinism)

Blocked by

• IngestSpec: build an ingestion from a parsed spec #59
• Pattern-filtered staging upload helper #60