Skip to content

Default retry middleware replays non-idempotent POSTs on 429/5xx (can create duplicate API tokens / billed jobs) #1042

Description

@sneg55

Default retry middleware replays non-idempotent POSTs, so a transient 5xx can create duplicate API tokens or billed jobs

Context: API Makeathon participant. Found while reviewing the client middleware; independently confirmed by a second review pass.

Retry support is a default feature. The client wraps RetryTransientMiddleware in a ConditionalMiddleware whose only gate is whether the request body can be cloned:

// kittycad/src/lib.rs:210-232 (wasm copy at 269-293)
let retry_policy = ExponentialBackoff::builder().build_with_max_retries(3);
...
ConditionalMiddleware::new(
    RetryTransientMiddleware::new_with_policy(retry_policy),
    |req: &reqwest::Request| req.try_clone().is_some(),
)

retry is enabled in kittycad/Cargo.toml by default. Cloneability only means the request can be replayed; it does not mean the operation is idempotent. Generated JSON/form POSTs are cloneable, and reqwest-retry's default strategy retries 429 and 5xx regardless of HTTP method.

Concrete failure

client.api_tokens().create_for_user(...) or an ML text-to-cad POST hits a gateway 502 after the server has already accepted and started (and billed) the operation. The middleware re-sends the same POST up to 3 more times, so one user action creates up to 4 API tokens or 4 billed generations. A 429 (rate limited or out of credits) also triggers immediate re-POSTs, worsening the limit.

Verify

Point the client at a local server that returns 500 twice then 200 for a POST endpoint and count received requests: it is 3 for one call. With the retry feature disabled it is 1.

Suggested fix

Add an HTTP-method check to the conditional so only idempotent methods (GET/HEAD/PUT/DELETE) are retried, or make POST retry opt-in per endpoint.

Environment

Reviewed against the current main of KittyCAD/kittycad.rs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions