Migrate flow to V4 interpreter (eval4) + soldeer + reusable workflows

[thedavidmeister]

Goal

Bring flow onto the current rain stack so it tracks latest and can use the upstream reusable CI/deploy workflows: migrate the interpreter integration V2 → V4 (eval4), de-submodule to soldeer, and convert workflows to the rainix reusables. flow is currently the org's straggler — submodules, OZ 4.x, the deprecated V2 interpreter API, and a vestigial deploy (manual-sol-artifacts.yaml runs rainix-sol-artifacts, which requires a script/Deploy.sol flow doesn't have → it would fail on dispatch).

Why now

• The dead magic-nix-cache already breaks flow's deploy workflow; the proper fix is the reusable, which needs soldeer + a real Deploy.sol.
• flow is pinned to a removed/deprecated interpreter API: LibEncodedDispatch no longer exists in rain.interpreter.interface, and the flat V2 interfaces (IInterpreterV2/StoreV2, IExpressionDeployerV3, IInterpreterCallerV2) are now under deprecated/. flow on V2 may already be incompatible with current deployed interpreters.

Upstream reference (security semantics — DO NOT invent)

Follow RaindexV6 / LibRaindex (the money-moving reference already on V4). The V4 model is eval-time, no deploy-time integrity check (the IExpressionDeployer model is gone):

(StackItem[] memory stack, bytes32[] memory writes) = evaluable.interpreter.eval4(
    EvalV4({
        store: evaluable.store,
        namespace: DEFAULT_STATE_NAMESPACE.qualifyNamespace(address(this)),
        bytecode: evaluable.bytecode,           // raw bytecode, not a deployed expression address
        sourceIndex: SourceIndexV2.wrap(FLOW_ENTRYPOINT),
        context: LibContext.build(context, signedContexts),  // bytes32[][]
        inputs: new StackItem[](0),
        stateOverlay: new bytes32[](0)
    })
);
if (writes.length > 0) evaluable.store.set(namespace, writes);

Work — interpreter V2 → V4

1. Config EvaluableConfigV3{deployer,bytecode,constants} → EvaluableV4{interpreter,store,bytecode} (no deployer/constants; bytecode carries constants).
2. EvaluableV2{interpreter,store,expression} → EvaluableV4; use V4 LibEvaluable hashing for registeredFlows.
3. _flowStack: replace eval2(...encodedDispatch...) with the eval4(EvalV4{...}) block above.
4. Stack/writes type uint256[] → StackItem[](bytes32) + uint256[] → bytes32[] — ripples through LibFlow.stackToFlow, the RAIN_FLOW_SENTINEL decode, and the transfer encoding. (Biggest piece.)
5. Drop imports: LibEncodedDispatch, IExpressionDeployerV3, IInterpreterV2/StoreV2, IInterpreterCallerV2, EvaluableConfigV3. Add: IInterpreterV4, IInterpreterStoreV3, IInterpreterCallerV4 (EvaluableV4/EvalV4/SourceIndexV2), V4 LibContext/LibEvaluable, StackItem.
6. IFlowV5 → IFlowV6 (breaking): flow() takes EvaluableV4; callerContext uint256[]→bytes32[]; stackToFlow() takes StackItem[]. Bump dependent flow interfaces.
7. Security semantics: drop the init-time deployExpression2 integrity check (UnsupportedFlowInputs/InsufficientFlowOutputs); enforce flow validity at eval-time per upstream (require stack.length >= MIN_FLOW_SENTINELS).
8. Rewrite all ~10 test files (they build EvaluableConfigV3 + deploy expressions; Flow.time.t.sol's LibStackGeneration.generateFlowStack helper, etc.).

Work — de-submodule to soldeer (remappings-only, no src edits)

foundry.toml [dependencies] + [soldeer] recursive_deps=false + remappings; remove lib/ + .gitmodules. Deps: forge-std 1.16.1, OZ (5.6.1 to track latest — note OZ 4→5 is an extra code axis: security/→utils/ for ReentrancyGuard, ECDSAUpgradeable removed, Multicall*; OR pin OZ 4.9.6 to defer), rain-factory 0.1.1, rain-interpreter-interface 0.1.0, rain-math-fixedpoint 0.2.0, rain-solmem 0.1.3 + transitive. Per-file remaps for the deprecated interfaces (interface/IInterpreterCallerV2.sol → interface/deprecated/v1/..., IInterpreterStoreV2 → deprecated/v2/...).

Work — reusable workflows (the original ask)

• rainix.yaml (CI) → rainlanguage/rainix/.github/workflows/rainix-sol.yaml (secrets: inherit). [its inline nix was already modernised in deploy: always sign with PRIVATE_KEY (never PRIVATE_KEY_DEV) #473]
• Add script/Deploy.sol (deterministic Zoltu deploy of the Flow impl) + BuildPointers.sol + src/generated/Flow.pointers.sol. No legacy-redeploy concern (deploy was vestigial).
• manual-sol-artifacts.yaml → reusable rainix-manual-sol-artifacts.yaml (DEPLOYMENT_SUITE).
• Add git-clean.yaml → reusable rainix-copy-artifacts.yaml.

Notes

• Breaking change to a financial contract → needs design review + heavy testing.
• Will conflict with the ~22 open PRs (touches Flow.sol + tests) — sequence after the queue drains or coordinate rebases.
• Scoping done 2026-06-06; local WIP foundry.toml/remappings were drafted in /tmp (not pushed).

[thedavidmeister]

File-level implementation spec (scoped from current code + upstream RaindexV6/LibRaindex)

src/lib/LibFlow.sol

• stackToFlow(Pointer, Pointer) is pointer-based → UNCHANGED (reads raw 32-byte words; the uint256→StackItem change doesn't reach it).
• flow(FlowTransferV1, IInterpreterStoreV2, uint256[] kvs) → flow(FlowTransferV1, IInterpreterStoreV3, bytes32[] kvs); interpreterStore.set(DEFAULT_STATE_NAMESPACE, kvs) (V3 set takes bytes32[]).
• Imports: IInterpreterStoreV2→IInterpreterStoreV3; DEFAULT_STATE_NAMESPACE from the V4 namespace location.

src/interface/IFlowV6.sol (new; bump from V5)

• flow(EvaluableV4 evaluable, bytes32[] callerContext, SignedContextV1[] signedContexts) (was EvaluableV2 + uint256[]).
• stackToFlow(StackItem[] stack) (was uint256[]).
• import EvaluableV4 from rain.interpreter.interface/.../IInterpreterCallerV4.sol; keep FlowTransferV1/RAIN_FLOW_SENTINEL/MIN_FLOW_SENTINELS from the flow interface chain.

src/concrete/Flow.sol

• Contract decl: IInterpreterCallerV2→IInterpreterCallerV4, IFlowV5→IFlowV6. using LibEvaluable for EvaluableV4.
• Drop imports: LibEncodedDispatch, IInterpreterV2/StoreV2, EvaluableConfigV3, IInterpreterCallerV2, EvaluableV2. Add: IInterpreterV4, IInterpreterStoreV3, IInterpreterCallerV4(EvaluableV4,EvalV4,SourceIndexV2), StackItem, V4 LibContext, V4 LibEvaluable, LibNamespace, LibBytes32Matrix.
• initialize(bytes): decode EvaluableV4[] (config IS the evaluable; no deploy step).
• flowInit: remove deployer.deployExpression2(...) + the io integrity check (UnsupportedFlowInputs/InsufficientFlowOutputs). Just registeredFlows[evaluable.hash()] = FLOW_IS_REGISTERED per config + emit. (Upstream RaindexV6 does no init integrity check; eval-time sentinel consumption enforces structure, and stack.length >= MIN_FLOW_SENTINELS is the eval-time guard.)
• _flowStack:

  bytes32[][] memory context = LibContext.build(LibBytes32Matrix.matrixFrom(callerContext), signedContexts);
  emit ContextV2(msg.sender, context);
  // ... registered check via evaluable.hash() ...
  (StackItem[] memory stack, bytes32[] memory kvs) = evaluable.interpreter.eval4(
      EvalV4({
          store: evaluable.store,
          namespace: LibNamespace.qualifyNamespace(DEFAULT_STATE_NAMESPACE, address(this)),
          bytecode: evaluable.bytecode,
          sourceIndex: FLOW_ENTRYPOINT,
          context: context,
          inputs: new StackItem[](0),
          stateOverlay: new bytes32[](0)
      })
  );
  uint256[] memory stackU; assembly ("memory-safe") { stackU := stack }   // StackItem[] and uint256[] share layout
  return (stackU.dataPointer(), stackU.endPointer(), kvs);

• FLOW_MAX_OUTPUTS + LibEncodedDispatch go away (V4 has no encoded dispatch). Context event → ContextV2.

foundry.toml remapping fix (OZ 4.9.6 has dirs at ROOT, no contracts/)

openzeppelin-contracts/contracts/=dependencies/@openzeppelin-contracts-4.9.6/
openzeppelin-contracts-upgradeable/contracts/=dependencies/@openzeppelin-contracts-upgradeable-4.9.6/
openzeppelin/=dependencies/@openzeppelin-contracts-upgradeable-4.9.6/

Per-file remaps for deprecated interfaces: rain.interpreter.interface/interface/IInterpreterCallerV2.sol=dependencies/rain-interpreter-interface-0.1.0/src/interface/deprecated/v1/IInterpreterCallerV2.sol (+ IInterpreterV2,IExpressionDeployerV3 → deprecated/v1/; IInterpreterStoreV2 → deprecated/v2/) — only needed by the deprecated V2 imports, which the V4 rewrite deletes, so most of these vanish once Flow.sol is on V4.

Open: OZ 4.9.6 (kept here to minimise churn) vs 5.6.1 ("track latest"). OZ 4→5 (security/→utils/, ECDSAUpgradeable removed, Multicall*) is a separate axis — recommend V4 first (this), OZ 5 follow-up.

Verify loop = CI on the draft PR.

[thedavidmeister]

Progress (commit pushed to #475)

• Flow.sol migrated to V4: EvaluableV4, eval4(EvalV4{...}), deployer + LibEncodedDispatch + io integrity check removed (eval-time validation per RaindexV6), StackItem stack via assembly cast to uint256[] for the pointer-based LibFlow decode.
• New IFlowV6; LibFlow on IInterpreterStoreV3 + bytes32[] kvs.
• Other flow interface files' deprecated V2 imports rewritten to soldeer's relocated interface/deprecated/v1|v2/ paths (soldeer-generated remappings.txt does not honor foundry.toml per-file remaps, so rewriting the imports was the reliable fix).

Remaining build-fix loop (transitive soldeer deps, recursive_deps=false)

Build now resolves flow's own imports; failing on transitive deps to declare in foundry.toml [dependencies]:

• rain-lib-hash (0.1.0) — LibHashNoAlloc.sol.
• OZ version split: rain deps (rain-factory/etc.) import @openzeppelin-contracts-5.6.1/... (Clones, SignatureChecker, MessageHashUtils) while flow's own code uses OZ 4.9.6 paths. Install both OZ 4.9.6 (flow) + 5.6.1 (rain deps) — distinct import roots so they coexist. (Or take the "track latest" path and migrate flow's own code to OZ 5.x, dropping 4.9.6.)
• Expect a few more rain transitive deps (rain-math-float/binary, rain-metadata, rain-lib-typecast, rain-sol-codegen, rain-intorastring, rain-lib-memkv) — add as the build surfaces them.

After src compiles

Rewrite ~10 test files (build EvaluableConfigV3/deploy expressions → EvaluableV4); add script/Deploy.sol + pointers; convert workflows to reusables.

[thedavidmeister]

Decision: track latest OZ — flow goes to OZ 5.6.1 (committed to #475)

foundry.toml now declares OZ 5.6.1 (single OZ version, shared with the rain deps). flow's own code must move 4→5:

flow OZ import-path rewrites (4.x → 5.x)

• openzeppelin-contracts-upgradeable/contracts/security/ReentrancyGuardUpgradeable.sol → .../utils/ReentrancyGuardUpgradeable.sol (security/→utils/).
• openzeppelin/utils/cryptography/ECDSAUpgradeable.sol → OZ5 removed the upgradeable ECDSA → use openzeppelin-contracts/contracts/utils/cryptography/ECDSA.sol (non-upgradeable library) and update the call site (recover/tryRecover signatures changed in 5.x).
• MulticallUpgradeable, ERC721HolderUpgradeable, ERC1155HolderUpgradeable paths unchanged (utils/ / token/.../utils/), but see init below.
• Remappings already point openzeppelin-contracts(-upgradeable)/contracts/ → the 5.6.1 packages.

flow code changes (OZ5 breaking)

• flowInit calls __ERC721Holder_init(), __ERC1155Holder_init(), __Multicall_init(), __ReentrancyGuard_init(). In OZ5 the stateless holders dropped their __X_init() — drop __ERC721Holder_init()/__ERC1155Holder_init() (and verify __Multicall_init() still exists; __ReentrancyGuard_init() remains). Keep them inside initializer/onlyInitializing.
• Any ECDSAUpgradeable.recover(...) → ECDSA.recover(...) (5.x returns/reverts differently; check SignatureChecker if used).

Then finish the build-iterate loop (recursive_deps=false)

Add transitive deps as the build surfaces them (rain-lib-hash/math-float/metadata/lib-typecast already added; expect rain-math-binary, rain-sol-codegen, rain-intorastring, rain-lib-memkv, solady?). Then the ~10 test rewrites + script/Deploy.sol/pointers + reusable rainix-sol/rainix-manual-sol-artifacts/rainix-copy-artifacts.

[thedavidmeister]

✅ src migrated — compiles on V4 + OZ5 + soldeer (pushed to #475)

• Interpreter V4 (eval4/EvaluableV4/StackItem), new IFlowV6, LibFlow on IInterpreterStoreV3/bytes32[].
• OZ 5.6.1: flow's holders + ECDSA + ReentrancyGuard now from the non-upgradeable @openzeppelin-contracts-5.6.1 package (the OZ5-upgradeable soldeer pkg only ships MulticallUpgradeable); dropped the stateless __ERC721Holder_init/__ERC1155Holder_init/__ReentrancyGuard_init; @openzeppelin/contracts/ bridge remap added.
• soldeer de-submodule complete (clean reinstall; deps incl rain-lib-hash, rain-math-float).
• Every remaining compile error is in test/, not src.

Remaining = test rewrite (~10 files) + deploy/workflows

Test files still on the V2 model — apply the same migration as src:

• EvaluableV2 → EvaluableV4 (import from IInterpreterCallerV4); the tests build expressions via a deployer mock + EvaluableConfigV3/deployExpression2 → now construct EvaluableV4{interpreter, store, bytecode} directly.
• LibEncodedDispatch.encode2(...) / .eval2 → the eval4(EvalV4{...}) model (interpreter mock must implement eval4); LibStackGeneration.generateFlowStack helper returns StackItem[]/uint256[] consistently.
• uint256[] stacks/kvs → StackItem[]/bytes32[].
• OZ5 in tests: ECDSAUpgradeable→ECDSA, and ECDSA.toEthSignedMessageHash → MessageHashUtils.toEthSignedMessageHash (OZ5 moved it). SignContextLib.sol import already swapped; the toEthSignedMessageHash call still needs MessageHashUtils.
• InterpreterMockTest/FlowTest abstracts: update the mock interpreter to IInterpreterV4.eval4.

Then: script/Deploy.sol (deterministic Zoltu of the Flow impl) + BuildPointers/pointers; reusable rainix-sol / rainix-manual-sol-artifacts / rainix-copy-artifacts; REUSE headers + slither config. CI on #475 is the verify loop.