Allow setting custom upload limits

[klauspost]

Adds to client configuration:

// UploadLimits overrides the upload limits the client enforces before sending
// a request. The defaults are the limits Amazon S3 imposes; only raise them
// when the remote endpoint is known to accept the larger values.
//
// A zero field means "use the default", so the zero UploadLimits behaves
// exactly like Amazon S3 — except for MaxSinglePutObjectSize, whose default is
// deliberately not enforced by PutObject. See that field.
//
// New rejects limits it cannot derive a part layout from. No field may be
// negative; the remaining bounds are noted on each field.
type UploadLimits struct {
	// MinPartSize is the smallest size allowed for a part that is not the
	// last part of a multipart upload. Defaults to 5 MiB. May not exceed
	// MaxPartSize.
	MinPartSize int64

	// MaxPartSize is the largest size allowed for a single part.
	// Defaults to 5 GiB. MaxPartSize * MaxPartsCount must fit in an int64.
	MaxPartSize int64

	// MaxPartsCount is the maximum number of parts in a single multipart
	// upload. Together with MaxPartSize this caps the object size the client
	// is willing to upload. Defaults to 10000, and may not exceed 2^53
	// because the part layout is computed in float64.
	MaxPartsCount int64

	// MaxSinglePutObjectSize is the largest object the remote accepts in a
	// single PUT. Defaults to 5 GiB.
	//
	// Unlike the other fields, the 5 GiB default is not enforced by PutObject:
	// MinIO and AIStor accept single PUTs far above Amazon's limit, so gating on
	// the default would refuse uploads that work today. Setting it explicitly
	// does enforce it — PutObject then rejects a larger object outright when
	// PutObjectOptions.DisableMultipart is set, and otherwise sends it as a
	// multipart upload.
	//
	// The resolved value, default included, always bounds the single PUT that
	// PutObject falls back to when a multipart upload fails with AccessDenied.
	// Core.PutObject never checks it.
	MaxSinglePutObjectSize int64
}

Summary by CodeRabbit

• New Features

  • Added configurable upload limits for part sizes, part counts, single-request uploads, and maximum object size.
  • Custom limits can be set when creating a client, with S3-compatible defaults applied automatically.
  • Multipart uploads and object composition now honor configured limits.

• Bug Fixes

  • Improved upload-size validation and part-size calculations.
  • Prevented incomplete unknown-length uploads from being stored as truncated objects.
  • Oversized uploads now use multipart uploads when supported.

• Tests

  • Added coverage for customized, invalid, and boundary upload-limit configurations.

[coderabbitai]

📝 Walkthrough

Walkthrough

The client now supports configurable upload limits through Options.UploadLimits. Upload and compose paths resolve these limits per client. Part sizing, object-size validation, multipart checks, single-PUT routing, and unknown-length stream handling now use the resolved limits.

Changes

Configurable upload limits

Layer / File(s)	Summary
Upload limits contract and client wiring constants.go, upload-limits.go, api.go, upload-limits_test.go	UploadLimits defines configurable limits, default resolution, maximum object size calculation, and validation. Options accepts overrides, and Client stores validated limits.
Limit-aware part sizing and stream safety api-put-object-common.go, api-put-object-multipart.go, api-put-object-streaming.go, api-put-object.go, api-append-object.go, api-error-response.go, upload-limits_test.go	Part sizing and upload validation use client-specific limits. Single-PUT routing uses the configured endpoint limit. Unknown-length multipart paths reject unread data after the final allowed part.
ComposeObject limit enforcement api-compose-object.go, api-compose-object_test.go, validate_uploadpartcopy_checksum_test.go	ComposeObject uses client limits for source counts, part sizes, object sizes, part counting, and even splits. Tests validate copy ranges and rejection cases.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: 🟡 Moderate · up to 33061

The PR adds configurable upload limits, but the current implementation can reject valid uploads, generate multipart parts that violate configured limits, allow appended objects to exceed those limits, and exceed MaxPartSize in an integer-rounding edge case. These correctness issues should be fixed before merging.

Sequence Diagram(s)

sequenceDiagram
  participant Options
  participant privateNew
  participant Client
  participant UploadLimits
  participant PutObject
  Options->>privateNew: provide UploadLimits
  privateNew->>UploadLimits: validate limits
  privateNew->>Client: store validated limits
  PutObject->>Client: resolve object and part limits
  Client->>UploadLimits: read configured or default values
  UploadLimits-->>Client: return validation and part-size results

Loading

Poem

A rabbit checks each limit,
Part sizes hop through guarded gates,
Streams reveal stray bytes,
Compose ranges line up neatly,
Tests keep the burrow bright.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 90.63% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 32 functions across 13 files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding configurable upload limits to the client.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

✅ Action performed

Comments resolved and changes approved.

[poornas]

Suggested change

	// Upload limits enforced before sending a request. Always read through
	// its accessors, which resolve zero fields to the S3 defaults.
	// Upload limits enforced before sending a request. Read through its
	// accessors, which resolve zero fields to the S3 defaults — except
	// MaxSinglePutObjectSize on the PutObject path, where an unset field
	// deliberately means "no limit" (see api-put-object.go).

[poornas]

LGTM, minor comment nit