Skip to content

Add TieredAllocator for capacity-bounded on-chip/spill allocation#32

Draft
fpedd wants to merge 1 commit into
mainfrom
tiered-allocator
Draft

Add TieredAllocator for capacity-bounded on-chip/spill allocation#32
fpedd wants to merge 1 commit into
mainfrom
tiered-allocator

Conversation

@fpedd

@fpedd fpedd commented Jul 10, 2026

Copy link
Copy Markdown
Owner

The existing allocators all minimize a single unbounded pool. They cannot
answer the question a hardware team actually asks: given a fixed on-chip
memory, which buffers live there and which spill off-chip? TieredAllocator
fills that gap.

It models the two tiers within one address space: the fast memory occupies
offsets [0, capacity), every buffer that first-fit can place under that
ceiling stays on-chip, and the rest are packed contiguously above capacity
(the spill region). A buffer is on-chip iff its height (offset + size) is at
most capacity, so the result is an ordinary valid allocation and works with
validate_allocation, plot_allocation, and the benchmark harness unchanged.

TieredConfig exposes capacity (bytes; None means unbounded, the default) and
order, the priority buffers get for the fast tier (default largest size
first, keeping the bytes most expensive to spill on-chip). With capacity=None
the allocator reduces to first-fit in order, so it is constructible with no
arguments and usable directly by registry name "tiered_allocator".

The existing allocators all minimize a single unbounded pool. They cannot
answer the question a hardware team actually asks: given a fixed on-chip
memory, which buffers live there and which spill off-chip? TieredAllocator
fills that gap.

It models the two tiers within one address space: the fast memory occupies
offsets [0, capacity), every buffer that first-fit can place under that
ceiling stays on-chip, and the rest are packed contiguously above capacity
(the spill region). A buffer is on-chip iff its height (offset + size) is at
most capacity, so the result is an ordinary valid allocation and works with
validate_allocation, plot_allocation, and the benchmark harness unchanged.

TieredConfig exposes capacity (bytes; None means unbounded, the default) and
order, the priority buffers get for the fast tier (default largest size
first, keeping the bytes most expensive to spill on-chip). With capacity=None
the allocator reduces to first-fit in order, so it is constructible with no
arguments and usable directly by registry name "tiered_allocator".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant