Skip to content

chore: compute artifact digests by reading files in chunks - #1134

Open
lengau with Copilot wants to merge 3 commits into
mainfrom
copilot/compute-artifact-digests-by-chunks
Open

chore: compute artifact digests by reading files in chunks#1134
lengau with Copilot wants to merge 3 commits into
mainfrom
copilot/compute-artifact-digests-by-chunks

Conversation

Copilot AI commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Hashes.from_path was loading entire artifact files into memory via path.read_bytes() before computing SHA1/SHA256 digests — a problem for large artifacts.

Changes

  • craft_application/models/manifest.py: Replaced path.read_bytes() with incremental 64KB chunk reads, feeding each chunk to both hash objects simultaneously.
# Before
read_bytes = path.read_bytes()
sha1=hashlib.sha1(read_bytes).hexdigest(),
sha256=hashlib.sha256(read_bytes).hexdigest(),

# After
with path.open("rb") as f:
    while chunk := f.read(65536):
        sha1.update(chunk)
        sha256.update(chunk)

This matches the existing chunked-read pattern already used in remote/utils.py. Once Python 3.10 support is dropped, this can be replaced with hashlib.file_digest().

Copilot AI requested review from Copilot and removed request for Copilot July 15, 2026 17:53
Copilot AI linked an issue Jul 15, 2026 that may be closed by this pull request
Copilot AI requested review from Copilot and removed request for Copilot July 15, 2026 17:55
Copilot AI changed the title [WIP] Read file by chunks for artifact digests Compute artifact digests by reading files in chunks Jul 15, 2026
Copilot AI requested a review from lengau July 15, 2026 17:56
@lengau lengau changed the title Compute artifact digests by reading files in chunks perf: compute artifact digests by reading files in chunks Jul 15, 2026
@lengau lengau changed the title perf: compute artifact digests by reading files in chunks chore: compute artifact digests by reading files in chunks Jul 15, 2026
@lengau
lengau marked this pull request as ready for review July 15, 2026 22:44
@lengau
lengau requested review from Copilot July 15, 2026 22:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates artifact digest computation to avoid loading entire files into memory, improving behavior for large artifacts.

Changes:

  • Reworked Hashes.from_path() to stream file contents in 64KB chunks instead of path.read_bytes()
  • Updated digest computation to reuse hashlib objects via incremental update() calls

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compute artifact digests by chunks

3 participants