[alanshaw]

An accepted blob now waits a configurable time, one hour by default, before Piri computes its CommP and adds it to the PDP pipeline. A blob deleted inside that window costs no CommP and no on-chain root. The blob is retrievable throughout and nothing on the wire changes.

Why one hour

Published traces of file deletion put most deletions of fresh data in the first minutes and flatten within about an hour; the 90th percentile of age at deletion in the Yahoo HDFS traces is 22 to 75 minutes. An hour saves nearly everything a delay can. Backup retention deletes, weeks out, are beyond any delay. The cost is an hour added to time to first proof.

How

• The task engine that drives the PDP pipeline now selects blobs for the CommP task by the time they were accepted. No migration is needed, and a change to the setting (pdp.aggregation.commp.delay, dynamic) applies to blobs already waiting; 0 releases them.
• Adding a blob to the pipeline no longer spawns a CommP task immediately, unless the delay is 0. The task runner picks the blob up on its next trigger once the delay has passed.
• No new code for a removal (a /blob/release invocation) of a blob inside the delay: the existing removal sweep drops it from the pipeline and releases its bytes.

Telemetry

Gauges for blobs waiting out the delay and for blobs past it that no task has picked up yet, plus a counter and an age histogram for blobs removed before aggregation, split by whether their CommP had run. The histogram is the node's own age-at-deletion distribution, which is what should tune the default. Docs and an example alert are included; the Grafana panels and alert rule are fil-forge/infra-central#291.

Not in scope: /pdp/info still computes CommP on demand, which would bypass the delay for a caller polling it. Nothing calls it today; the handler carries a note.

Tests pass, with the Postgres-backed ones run locally with the macOS skip lifted. Footnote: the duration validator used for config values now rejects negative durations.

🤖 Generated with Claude Code

[alanshaw]

Why the default delay is one hour

The delay exists to avoid computing a CommP and adding an on-chain root for a blob that is deleted soon after upload. Its value is set by how deletion risk falls with a blob's age. No public trace covers an S3-style object store, so the default comes from the closest published deletion-age distributions. All of them put most deletions of fresh data in the first minutes and flatten within about an hour.

Study	Workload	Age at deletion
Abad, Big Data Storage Workload Characterization (UIUC thesis, from IISWC 2012)	Two Yahoo Hadoop clusters, 6-month traces	Median 117 s (PROD) and 239 s (R&D); 90th percentile 22 min and 75 min (Table 4.7)
Leung et al., USENIX ATC 2008	NetApp CIFS, corporate and engineering, 3 months	Of files eventually deleted, 56% and 39% die within 100 ms; only 19% and 7% live past 24 h
Ousterhout et al., 1985; Baker et al., 1991; Vogels, 1999	BSD, Sprite and Windows NT desktops	65 to 80% of new files deleted within seconds to 3 minutes
Muralidhar et al., f4, OSDI 2014	Facebook photos, videos, attachments	Not quantified by age; "most deletes are for young BLOBs", delete rate low once content is a month old; over 80% of deletes hit content under 3 months

The Hadoop trace is the closest in shape to an object store, and its 90th percentile sits at about an hour. Past that each extra hour saves little: the CIFS data shows a full day only moves coverage from roughly 80% to 90% of deletions. Backup retention deletes, weeks or months out, are beyond any delay. The cost of an hour is an hour added to time to first proof, which the pipeline already spends waiting for an aggregate to fill and for the next proving period.

AWS answers the same cost differently: S3 Standard-IA bills a 30-day minimum, Glacier Instant Retrieval 90 days and Deep Archive 180 days. A minimum billing duration is a pricing decision, so it is not part of this change.

The default is a starting point. The age histogram this change adds (piri_pdp_pipeline_cancelled_age_seconds) is the node's own age-at-deletion distribution, and should replace the literature once staging has run long enough.

[Copilot]

🟡 Changes recommended

Pending releases can still incur hashing, disabled delays can be lost during installation, and cancellation metrics can overstate avoided work.

3 open issues

• omitempty resets explicitly disabled delay to the default · New
• Pending removals can incorrectly trigger CommP hashing · New
• Cancellation metrics misreport hashing when CommP mapping persists · New

What changed in this PR

Adds a configurable, one-hour default delay before accepted blobs enter CommP processing, allowing early removals to avoid unnecessary work.

Changes:

• Adds dynamic delay configuration and time-based task selection.
• Adds backlog and cancellation telemetry.
• Documents operation and tests delayed processing.

File	Description
pkg/​ucanhandlers/​pdp/​info.go	Documents the on-demand bypass.
pkg/​pdp/​service/​telemetry.go	Defines cancellation metrics.
pkg/​pdp/​service/​service.go	Initializes cancellation metrics.
pkg/​pdp/​service/​piece_remove.go	Records cancellation stage and age.
pkg/​pdp/​service/​piece_remove_test.go	Tests held-blob removal and metrics.
pkg/​pdp/​pipeline/​task_settle.go	Defers CommP handoff when configured.
pkg/​pdp/​pipeline/​task_settle_test.go	Tests delayed handoff.
pkg/​pdp/​pipeline/​task_commp.go	Claims due blobs in bounded batches.
pkg/​pdp/​pipeline/​fxmodule.go	Registers policy and telemetry.
pkg/​pdp/​pipeline/​commp_metrics.go	Exposes held and overdue backlog gauges.
pkg/​pdp/​pipeline/​commp_delay_test.go	Tests delay, claims, and backlog.
pkg/​pdp/​pipeline/​calculator.go	Suppresses immediate tasks during delay.
pkg/​pdp/​pipeline/​calculator_test.go	Adds delay-aware test helpers.
pkg/​pdp/​aggregation/​commp/​policy/​policy.go	Binds policy to dynamic configuration.
pkg/​pdp/​aggregation/​commp/​policy/​policy_test.go	Tests policy updates and validation.
pkg/​pdp/​aggregation/​commp/​commp.go	Defines delay policy and limits.
pkg/​fx/​app/​common.go	Supplies CommP configuration.
pkg/​config/​pdp.go	Adds delay configuration and validation.
pkg/​config/​dynamic/​schema.go	Rejects negative durations.
pkg/​config/​dynamic/​schema_test.go	Tests zero and negative durations.
pkg/​config/​defaults.go	Registers the delay key and default.
pkg/​config/​app/​pdp.go	Carries delay into application configuration.
docs/​content/​operator-guide/​monitoring.md	Documents metrics and an alert.
docs/​content/​configuration/​pdp/​aggregation/​index.md	Describes delayed pipeline entry.
docs/​content/​configuration/​pdp/​aggregation/​commp.md	Documents delay configuration.
docs/​content/​configuration/​index.md	Lists the dynamic delay setting.
docs/​content/​concepts/​telemetry.md	Catalogs pipeline metrics.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Resolve the telemetry/documentation inaccuracies and ensure due-row claiming and backlog metrics cannot misreport or stall work.

1 open issue

• Exclude pending removals from the backlog alert query · New

3 resolved since last review

• Cancellation metrics misreport hashing when CommP mapping persists
• Pending removals can incorrectly trigger CommP hashing
• omitempty resets explicitly disabled delay to the default

_{Review effort: Lite}

[Copilot]

🔵 Needs a closer look

Documentation still needs to qualify the /pdp/info on-demand CommP exception; final human review is warranted.

No open issues 🎉

1 resolved since last review

• Exclude pending removals from the backlog alert query

_{Review effort: Lite}