Skip to content

Implement content-based equality to prevent redundant updates and storage bloat #216

Description

@erral

Problem

NamedFile and NamedBlobFile objects currently lack an __eq__ implementation, falling back to Python's default identity comparison. In the context of plone.restapi (or any code that replaces field values), this causes a major performance and storage regression:

When an image is PATCHed with the exact same bytes, the deserializer creates a new object instance. Because new_instance != old_instance, Plone perceives a change, triggers a ZODB write, fires an ObjectModifiedEvent, re-indexes the catalog, and invalidates all existing image scales.

Impact

In high-frequency update scenarios (like automated content imports), this leads to:

  1. Massive Data.fs growth due to redundant transaction history.
  2. Blob storage bloat from unnecessary scale regeneration.
  3. Increased CPU load from redundant indexing and image processing.

Proposed Solution

  1. Implement __eq__ in NamedFile and NamedBlobFile comparing contentType, filename, and a content checksum.
  2. Introduce a private _hash attribute (SHA-256) calculated during _setData using chunked reading to save memory.
  3. Update the internal _modified timestamp ONLY if the content hash actually changes.
  4. ZODB Safety: For legacy objects lacking a _hash, the comparison should compute it in memory but MUST NOT save it back to the object during a read operation (avoiding "write-on-read").

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions