Test Dev experience with Ingest

[michaelconnor00]

The latest RC has the Ingest client and Ingest resource, see release 0.4.3rc.2. This is packaged from the develop branch and is up to date with main.

This issue is to use the new ingest features with the sandbox env to evaluate the developer experience and provide feedback or suggestions on improvements. See the docs/ingest.ipynb for usage example

The use case of the Ingest feature here are:

• A Python Notebook or script user wants to ingest data from filesystem or their own public bucket (private buckets require a K8s secret to exist)
• A user creates a cron script to run on a machine with an api key to run ingest on some frequency.

Note: The ingest api is available on the Tailnet (can be slow), or use K8s port forwarding.

[michaelconnor00]

Also, I added better logging support, and haven't tested it. There is now a debug=True|False, and a logfile=Path(). They are documented, but run an ingest with debug=True, logfile=... and post the log here. If the log is a mess, we will want to clean it up.

[underchemist]

Slow response time of requests made through Python to API

Requests made through python http client (ingest client uses requests) appear to hang unless an explicit timeout is set for the request, in which case the request only receives a response after the client timeout time has exceeded. This request may still be successful e.g. ingest_client.check()

Workaround

Port forward ingest api and override prescient_ingestion_endpoint_url

Logging

2026-06-24 09:48:52,511 DEBUG prescient_sdk: poll status=Status.INGESTING
2026-06-24 09:48:57,512 DEBUG prescient_sdk: _request method=GET path=http://localhost:8000/v1/ingestion/129
2026-06-24 09:48:58,700 DEBUG prescient_sdk: _response method=GET path=http://localhost:8000/v1/ingestion/129 status=200
2026-06-24 09:48:58,700 DEBUG prescient_sdk: poll status=Status.INGESTING
2026-06-24 09:49:03,701 DEBUG prescient_sdk: _request method=GET path=http://localhost:8000/v1/ingestion/129
2026-06-24 09:49:05,093 DEBUG prescient_sdk: _response method=GET path=http://localhost:8000/v1/ingestion/129 status=200
2026-06-24 09:49:05,094 DEBUG prescient_sdk: poll status=Status.INGESTING
2026-06-24 09:49:10,094 DEBUG prescient_sdk: _request method=GET path=http://localhost:8000/v1/ingestion/129
2026-06-24 09:49:11,341 DEBUG prescient_sdk: _response method=GET path=http://localhost:8000/v1/ingestion/129 status=200
2026-06-24 09:49:11,341 DEBUG prescient_sdk: poll status=Status.INGESTING
2026-06-24 09:49:16,342 DEBUG prescient_sdk: _request method=GET path=http://localhost:8000/v1/ingestion/129
2026-06-24 09:49:17,689 DEBUG prescient_sdk: _response method=GET path=http://localhost:8000/v1/ingestion/129 status=200
2026-06-24 09:49:17,689 DEBUG prescient_sdk: poll status=Status.INGESTING

No comment, this is just an example of the logging output

IngestionResource repr doesn't work as expected in terminal

From the usage example

ing = IngestionResource.from_id(client, id=42)
ing.refresh()
ing  # rich HTML summary in Jupyter; ANSI table in a terminal

I observe that I get the default object repr instead of __rich__, using ipython repl

Naming

Decide between usage of Ingest and Ingestion e.g. IngestClient vs IngestionResource

API misc

these are notes to create as issues in prescient-ingestion

• STAC item validation fails due to no collection property in item body (required if "rel": "collection" link exists) but the items in pgstac do contain a collection property and link rel type.
• STAC collection not validated https://github.com/sparkgeo/prescient-ingestion/issues/394
• ingest spec https://github.com/sparkgeo/prescient-ingestion/blob/main/docs/Specification.md#locations says that local filesystem paths are supported but

locations:
  source_location:
    path: "/mnt/extradrive/enhanced_small"
  target_location:
    path: "s3://earth.prescient.sandbox/dnd-demo"
  property_location:
    path: "/home/ysebastien/dev/prescient-sdk"
source_files:
  enhanced_files:
    location: source_location
    pattern: ".*.tif$"

Something like this does not work

[michaelconnor00]

I didn't notice Ingest vs Ingestion. I think Ingest is preferred. Can you fix it on the develop branch?

[underchemist]

@michaelconnor00 yes I will update develop.

One more question for

prescient-sdk/prescient_sdk/config.py

Line 46 in d5a2e1e

prescient_ingest_endpoint_url: Optional[str] = Field(

is it actually intended to be optional? I assume not, and am going to revert it from Optional[str] to str