Skip to content

Parallelize checkpoint archive construction for large dirty worktrees #1

Description

@jbellis

Problem

Importing or checkpointing a session snapshots each repository serially. On a dirty brokkbench session import, Git snapshot/archive construction remained CPU-bound for minutes before native resume could start.

This is a worst-case input, but it makes archive creation dominate session import and close latency.

Observation

The importer should continue to use git ls-files --others --exclude-standard: repository-local ignore rules select valid inputs. Hel should not blanket-skip arbitrary directory types. The follow-up is about throughput after Git has selected the input set.

Desired direction

Parallelize independent archive work where safe (repository collection, payload digest/compression, and possibly archive writing), while preserving:

  • atomic archive creation and verification;
  • bounded memory and disk use;
  • deterministic manifests;
  • the current rejection of unsafe or unsupported file types; and
  • no writes to the user worktree.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions