Skip to content

prepacked_cache: fix byte-accounting for evicted float matrices#374

Open
EylonKrause wants to merge 1 commit into
google:masterfrom
EylonKrause:fix/prepacked-cache-float-sums-accounting
Open

prepacked_cache: fix byte-accounting for evicted float matrices#374
EylonKrause wants to merge 1 commit into
google:masterfrom
EylonKrause:fix/prepacked-cache-float-sums-accounting

Conversation

@EylonKrause

Copy link
Copy Markdown

Problem

PrepackedCache tracks total cached bytes in buffers_bytes_ to enforce max_buffers_bytes_. Insertion and eviction account for the sums buffer asymmetrically for floating-point matrices, so the running total drifts.

Root cause

AllocateBuffers (prepacked_cache.cc:29-39) allocates and counts a sums buffer only for non-floating-point matrices — a float matrix adds just DataBytes to buffers_bytes_ (Get, line 100). But EjectOne (line 124) unconditionally subtracted DataBytes + SumsBytes.

A float matrix's sums_type still reports a nonzero element size: it is set to Type::Create<SumsType>() with SumsType = Scalar for floating-point scalars (create_trmul_params.h:62-69), i.e. float (size 4). So SumsBytes = layout.cols * sums_type.size > 0 (mat.h:436-439) even though no sums buffer was allocated. Each float eviction therefore over-subtracts cols * 4, letting buffers_bytes_ drift below true usage (possibly negative) so the cache silently overshoots its cap.

Fix

Subtract SumsBytes only for non-floating-point matrices, mirroring AllocateBuffers.

Verification

Pure accounting logic; reproduced with a standalone model of insert/evict (float matrix, data=1000, cols*4=256):

inserted (added to buffers_bytes_): 1000
after evict  buggy buffers_bytes_ = -256  (should be 0)  -> DRIFT/WRONG
after evict  fixed buffers_bytes_ = 0     (should be 0)  -> OK

The existing prepacked_cache_test.cc does not catch this: its dummy float PEMat leaves sums_type at the default (size 0), and no test evicts a float matrix. Happy to add a regression test that evicts a float matrix.

AllocateBuffers only allocates and counts a `sums` buffer for
non-floating-point matrices; float matrices never allocate one, so a
float insertion only adds DataBytes to buffers_bytes_. EjectOne, however,
unconditionally subtracted DataBytes + SumsBytes. A float matrix's
sums_type still reports a nonzero element size, so SumsBytes is nonzero
and each float eviction over-subtracted it, letting buffers_bytes_ drift
below true usage (possibly negative) so the cache silently overshoots
max_buffers_bytes_. Subtract SumsBytes only for non-floating-point
matrices, mirroring AllocateBuffers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant