Skip to content

Loader.Load allocates a batchRequest + channel before the cache check #122

Description

@zkrebbekx

Loader.Load allocates the batchRequest (and its done channel) at the very top of the function, before the cache lookup:

func (l *Loader[K, V]) Load(originalContext context.Context, key K) Thunk[V] {
	ctx, finish := l.tracer.TraceLoad(originalContext, key)
	req := &batchRequest[K, V]{
		key:  key,
		done: make(chan struct{}),
	}
	...
	if v, ok := l.cache.Get(ctx, key); ok {
		...
		return v   // req is never used on this path
	}

req is only used on a cache miss — it's sent to the batcher (l.curBatcher.input <- req) and read by the returned thunk. On a cache hit it's allocated and immediately discarded, so every cached Load pays for a struct + channel allocation it never uses. In read-heavy workloads where the same keys are loaded repeatedly, this is pure waste and shows up prominently in allocation profiles.

Moving the allocation below the cache-hit early-return eliminates it for all cache hits.

Benchmark (Load on a primed/cached key, -benchmem):

allocs/op B/op ns/op
before 3 160 45.7
after 1 16 20.5

The remaining allocation on the hit path is the no-op finish closure returned by NoopTracer.TraceLoad (a fresh func(Thunk[V]){} per call, which isn't free inside a generic function) — happy to file that separately if of interest.

The full package test suite passes with the change. Fix proposed in #123.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions