Skip to content

Commit 6cd591e

Browse files
thomasahleclaude
andcommitted
Foreach grouping: isomorphic elementwise chains as torch._foreach_* calls
New emission-level planning pass _plan_foreach (FOREACH_GROUPING flag, plan/suppress idiom like SDPA/addmm): families of shape-isomorphic MapNode/LinearNode/pure-Hadamard-EinsumNode chains — the 53 per-parameter AdamW update chains — key by a shape-erased structural signature (op + scalar params / ordered weights / broadcast-multiply steps, operand roles recursive) and emit at the family anchor as one OUT-OF-PLACE torch._foreach_* call per chain step, tuple-unpacked onto the members' pre-assigned t-names so output collection, the regex DLE and EMIT_DEL liveness keep working textually. A verify-and-demote fixpoint guarantees deferral never moves a producer past a consumer. Scalars enter as python floats; identity/0-dim alignment patterns are rank-erased so 2-D weight chains and 1-D bias chains share a family. Emission mirrors the singleton ATen ops exactly (mul.Scalar then add, never add(alpha=)), so the flag A/B is bit-identical. Status: correct and partial — 27 grouped calls on the nano step (sqrt 54->30, reciprocal 62->38); the remaining singletons demote through cross-family anchor inversions and sub-minimum families (census in the session log). Window-based family scheduling to recover them is filed. Implementation largely by a background agent against the workflow-judge design; finished and gated here. Suite 884 passed; demo trains to 1.000; pyright clean on this file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 8a37b4c commit 6cd591e

1 file changed

Lines changed: 467 additions & 45 deletions

File tree

0 commit comments

Comments
 (0)