Skip to content

Prefetch research: batch UID resolution in blocks resolveuid and linkintegrity (catalog-query N+1) #4356

Description

@jensens

Part of the research plan for #4350 (see anchor comment there for the full decomposition).

This track is related but partly orthogonal to ZODB prefetch: blocks resolveuid and linkintegrity resolve UIDs one at a time, so the N+1 here is catalog queries (one searchResults(UID=...) per UID), not just object loads. Batching the catalog query is the first-order fix; ZODB prefetch only matters where full objects are loaded afterwards.

Findings (plone.restapi 10.0.1, scan 2026-07-06)

# Call site Pattern
D1 serializer/blocks.py:80 + serializer/utils.py:26 resolve_uid → one catalog query + brain per UID in block data (per field, per block, recursive)
D2 services/linkintegrity/get.py:32-33 uuidToObject(uid) per uid in loop — catalog query + traversal each

For Volto sites, D1 runs on every content GET with blocks — listing/teaser/grid-heavy pages resolve dozens of UIDs sequentially.

Tasks

  • D1: two-phase serialization — scan the (recursive) block data for all UIDs first, one catalog query UID={'query': uids}, build a uid→brain map, then substitute. Cache per request.
  • D2: same shape — collect uids, single batched catalog query, then resolve.
  • Measure whether a subsequent object-load phase remains (e.g. image scale generation touching the target) — if yes, chain a prefetch on the resulting brains per the result-set strategy sub-issue.
  • Implementation PRs belong on plone.restapi; research and numbers tracked here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions