MILAB-6721: stop double-counting input size in the analyze memory formula

[PaulNewling]

What

The analyze memory request counted the input size twice. baseMemGiB — 64 GiB, or 110/192 for contig/cell and MiTool presets — is a total-memory value for the whole analysis, but the formula added a data term on top of it:

before:  mem = clamp(baseMemGiB + 4 x size("reads"), baseMemGiB, 256 GiB)
after:   mem = clamp(4 x size("reads"),              baseMemGiB, 256 GiB)

No max() is needed in its place: .between(base, 256) already supplies base as the lower bound. That bound was dead code until now, because an additive term can never fall below the floor it was added to.

Effect

reads	floor	before	after
31.75 GiB	110 GiB	237 GiB	127 GiB
32 GiB	64 GiB	192 GiB	128 GiB
empty	110 GiB	110 GiB	110 GiB
96 GiB	64 GiB	256 GiB (cap)	256 GiB (cap)

Large samples request materially less. Small samples are unchanged — they were floor-bound before and stay floor-bound. Lowering the floors themselves is deliberately out of scope; that needs separate agreement on whether 64/110/192 can safely come down.

The 256 GiB ceiling is untouched, as is the "Advanced Settings" memory override, which takes a separate perProcessMemGB branch this change never reaches.

Structure

Two commits, kept separate on purpose:

1. Extract the arithmetic into mem-formula.lib.tengo behind a characterization test pinning today's values. Computes identical results.
2. Flip the behaviour. This commit's test diff is the whole record of what moved — read it alone and you see 237 become 127.

The formula previously sat inside self.body(func(inputs) {...}) where no test could reach it. The extraction is the seam that makes it assertable.

hash_override UUID bump

mixcr-analyze.tpl.tengo carries a pinned //tengo:hash_override, bumped here to a fresh UUID. This forces a one-time recompute of results cached against the old hash.

The previous value was shared byte-for-byte with mixcr-amplicon-alignment, whose body differs substantially — and two templates pinning one UUID are interchangeable as far as the backend is concerned. Both now pin distinct fresh values; see the companion PR in that repo.

Testing

pnpm exec pl-tengo test passes TestBaseMemGiBSelection and TestMemFormula. Each was confirmed failing before its fix, so neither is vacuous — the flip commit's test failed with expect: 136365211648, got 254476812288 until the lib changed. pnpm run build:dev-no-software and pnpm exec pl-tengo check are clean; the compiled lib carries no plus and the compiled template carries the new UUID.

No live run yet — this is unit- and build-level verification only. A meaningful live check needs an input the backend has never computed, or a previously-failed one. Memory and CPU go into metaExtra and are excluded from a resource's canonical ID, specifically so dedup reuses results across differing grants, so re-running an already-successful sample cannot surface a new grant however correct the change is.

Two pre-existing issues in this repo, neither introduced here and both out of scope:

• pnpm run test is unusable as a gate — it chains a vitest suite needing a live backend, which fails with ECONNREFUSED 127.0.0.1:6345.
• pnpm exec changeset status fails on yaml.safeLoad is removed in js-yaml 4 from read-yaml-file@1.1.0, reproducible with this branch's changeset removed. The changeset was validated with the repo's own @changesets/parse instead.

Greptile Summary

This PR extracts and corrects the MiXCR analyze memory formula so input size is counted once while preserving preset floors, the 256 GiB ceiling, and explicit memory overrides. It also changes the analyze template hash to invalidate cached results and adds characterization coverage.

• baseMemGiB: The total-memory floor selected from pipeline steps—64 GiB normally, 110 GiB for contig/cell assembly, and 192 GiB for MiTool. It is extracted into mem-formula.lib.tengo and used as the clamp’s lower bound instead of an additive term.
• memFormula: The resource formula used to request analyze RAM. It changes from clamp(base + 4 × reads size, base, 256 GiB) to clamp(4 × reads size, base, 256 GiB).
• size("reads"): The stored compressed size of FASTQ inputs tagged as reads. It remains multiplied by four but is no longer combined additively with the memory floor.
• staticFallback: The memory used when the backend cannot evaluate the dynamic formula. It remains the selected baseMemGiB.
• perProcessMemGB: The Advanced Settings memory override. Its separate branch remains unchanged and bypasses the formula.
• hash_override: The analyze template’s pinned cache identity. It receives a new UUID to force one-time recomputation rather than reusing results associated with the previous formula.
• Tengo test target: The workflow package’s test script now runs pl-tengo test before Vitest, covering the extracted floor selection and memory arithmetic.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
workflow/src/mem-formula.lib.tengo	Extracts preset floor selection and implements the non-additive, bounded analyze memory formula with the original static fallback.
workflow/src/mem-formula.test.tengo	Covers all floor-selection branches, representative input sizes, ceiling behavior, and the static fallback.
workflow/src/mixcr-analyze.tpl.tengo	Delegates analyze memory sizing to the extracted library and changes the template hash while preserving the explicit override path.
workflow/package.json	Adds Tengo unit tests to the workflow package’s standard test command.
.changeset/analyze-mem-no-double-count.md	Documents the corrected formula, expected resource reductions, preserved bounds and override, and cache recomputation.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Analyze request] --> B{perProcessMemGB defined?}
    B -- Yes --> C[Use explicit GiB override]
    B -- No --> D[Inspect preset pipeline steps]
    D --> E[Select floor: 64, 110, or 192 GiB]
    E --> F[Compute 4 × stored reads size]
    F --> G[Clamp between selected floor and 256 GiB]
    G --> H[Submit CPU and RAM resources]

Loading

_{Reviews (2): Last reviewed commit: "MILAB-6721: clarify memory formula comme..." | Re-trigger Greptile}

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[PaulNewling]

@greptileai

[PaulNewling]

Merging after review from @julenmendieta .
Discussions in August 5th chat thread.