Skip to content

perf: prefetch next ListObjects page while consumer iterates current page #218

@harshavardhana

Description

@harshavardhana

Problem

`ListObjectsResult::operator++()` fetches the next page only after the caller has consumed the current page. See `src/client.cc:89-127` (Populate) and `include/miniocpp/client.h:54-66` (iterator). For a 1M-object bucket at 1000/page, that's 1000 sequential round-trips with the consumer blocked between each.

Suggested approach

When the response arrives with `IsTruncated=true`, kick off the next page fetch on a background thread / via the async surface (#217). `ListObjectsResult` holds the prefetched response and serves it without a round-trip when the iterator crosses the page boundary.

Internal-only change, no API impact for sync callers. Memory: one extra page in flight.

Impact

Medium. Roughly halves wall-clock time on large bucket listings when the per-page network round-trip is comparable to the per-page consumer work.

Roadmap

Tier 2 item from the C++ modernization audit. Related: #217 (async API surface).

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions