Skip to content

Add memory usage benchmarks for pack generation #11

Description

@ggueret

Summary

The streaming pack refactoring (#6, PR #10) reduced memory from ~3x repo size to ~1 object at a time, but this gain is not measured by existing benchmarks (which only track time).

We need benchmarks that measure peak memory allocation during pack generation to:

  • Quantify the improvement from the streaming refactoring
  • Detect memory regressions in future changes
  • Establish a baseline for v0.2 further optimizations

Solution

Add a memory-tracking benchmark using jemalloc stats or a custom allocator wrapper that records peak allocation. Measure peak RSS for small/medium/large fixtures during generate_pack.

Optionally integrate with the CI bench workflow to track memory alongside time on gh-pages.

Sessions

  1. Implement memory benchmark -- custom allocator or jemalloc stats to measure peak allocation during pack generation
  2. Compare before/after streaming -- checkout pre-streaming commit, measure, compare with current
  3. CI integration -- if feasible, add memory data points to the gh-pages dashboard

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions