fix(sign): fetch existing Rekor entries by default - #1709
Open
lodekeeper wants to merge 1 commit into
Open
Conversation
🦋 Changeset detectedLatest commit: 58ad28d The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Signed-off-by: lodekeeper <lodekeeper@users.noreply.github.com>
lodekeeper
force-pushed
the
fix/rekor-fetch-on-conflict-default
branch
from
July 28, 2026 17:09
7f557a6 to
58ad28d
Compare
Author
|
Fixed the DCO-only blocker by amending the commit with the required |
|
@bdehamer curious what your thoughts are on this one. There were a few issues across a bunch of repos but wondering if there was a reason this defaulted to |
Author
|
Some author context on the motivating bug (deferring to @bdehamer on the reason for the original We hit this as retry-after-success |
wemeetagain
pushed a commit
to ChainSafe/lodestar
that referenced
this pull request
Aug 5, 2026
## Current recommendation
As of 2026-07-28 19:2x UTC, the latest unstable `publish.yml` runs have
green npm-publish jobs again. I do **not** recommend merging this local
`pnpm patch` while the current signal is green.
Keep this PR open as a ready emergency downstream patch and as the
Lodestar pointer to the upstream sigstore-js fix. Merge it only if the
Rekor 409 provenance failure recurs before the upstream fix is
released/consumed, or if an RC/stable release is blocked by this exact
retry-after-success failure mode.
Close/drop this PR once sigstore-js ships the upstream default change
and npm/libnpmpublish/lerna-lite consume a version that enables the
Rekor conflict recovery path without a Lodestar-local patch.
## Problem
The `Publish` workflow intermittently failed during dev publishes with:
```text
lerna-lite ERR! TLOG_CREATE_ENTRY_ERROR error creating tlog entry - (409) an equivalent entry already exists in the transparency log with UUID ...
```
The failure was observed across multiple unstable publishes on
2026-07-23 and 2026-07-24. Later unstable publishes are green again,
which points to a transient Rekor/signing-path trigger rather than a
deterministic Lodestar publishing regression.
## Root cause
npm provenance uploads a signed entry to the Sigstore Rekor transparency
log via `@sigstore/sign` (pulled in through `lerna-lite -> libnpmpublish
-> sigstore`). The observed failure matches a **retry-after-success**
path:
1. Rekor creates the entry server-side.
2. The client times out or retries after a transient failure.
3. The retry submits the same equivalent entry.
4. Rekor correctly returns `409 an equivalent entry already exists`.
5. `@sigstore/sign` treats that 409 as fatal because the default Rekor
witness config has `fetchOnConflict: false`.
`@sigstore/sign` already has the needed recovery behavior: on 409 it can
fetch the existing entry and continue. The issue is that the high-level
default disables it.
## Emergency downstream patch
This PR uses `pnpm patch sigstore` to flip the Rekor witness default
used by the Lodestar publish stack:
```diff
new RekorWitness({
rekorBaseURL: options.rekorURL,
- fetchOnConflict: false,
+ fetchOnConflict: true,
retry: options.retry ?? DEFAULT_RETRY,
timeout: options.timeout ?? DEFAULT_TIMEOUT,
})
```
On a 409 the client fetches the already-created Rekor entry and the
publish proceeds. Provenance is preserved; the client uses the entry
Rekor already recorded.
## Upstream
Opened upstream sigstore-js tracking and fix:
- Issue: sigstore/sigstore-js#1708
- PR: sigstore/sigstore-js#1709
Similar downstream reports:
- SocialGouv/code-du-travail-numerique#7419
- apify/apify-shared-js#649
## Alternatives considered
- **Bump `lerna-lite`/`sigstore`** -- no effect yet; `fetchOnConflict:
false` is still the default in currently consumed versions.
- **Disable provenance on dev** -- drops supply-chain provenance;
rejected.
- **Re-run the job** -- can work for dev publishes when a later commit
changes the tarball/version, but is not a robust stable/RC release
strategy if the same package/version is wedged after Rekor already
recorded the entry.
- **Increase timeout** -- may reduce timeout-triggered retries, but does
not make the Rekor create-entry operation idempotent across
5xx/429/retry-after-success cases.
## Notes
- First `patchedDependencies` entry in the repo.
- Treat this as a hold/emergency patch unless the failure recurs or a
release publish is blocked before upstream is available.
🤖 Generated with AI assistance
---------
Co-authored-by: lodekeeper <lodekeeper@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Matthew Keil <github@mail.matthewkeil.com>
Co-authored-by: matthewkeil <me@matthewkeil.com>
Merged
30 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TLogClientto fetching existing Rekor entries on 409 conflictsfetchOnConflict: falsebehavior unchangedContext
This fixes a retry-after-success failure mode seen by npm provenance publishers. If Rekor commits a create-entry request but the client retries after a timeout or transient retryable failure, the retry receives
409 an equivalent entry already exists. The client already has recovery logic for that condition, but the default wasfetchOnConflict: false, so high-level callers such as npm provenance publishing treated the benign duplicate as a fatalTLOG_CREATE_ENTRY_ERROR.Related downstream reports:
Verification
npm run buildnpm test --workspace @sigstore/sign -- witness/tlog/client.test.tsnpm run lint:check -- --quietGenerated with AI assistance.