hermiq.workload-step has never executed through the flow engine — unit-tested only, no integration coverage

[rubenvdlinde]

Found while completing the flow-model migration (ConductionNL/hydra#529).

The gap

hermiq contributes two node types to OpenRegister's flow engine. Their coverage is asymmetric:

node type	unit	executed through the engine
hermiq.agent-step	✅	✅ openregister/tests/newman/openregister-flow-engine.postman_collection.json
hermiq.workload-step	✅ tests/Unit/Flow/HermiqWorkloadNodeTest.php	❌ never

Searched: no integration test, no e2e spec, not in the newman flow-engine collection. Its only non-source references are the ExApp README, a docker-compose file, a deploy script, StageDispatchService.php, and an openspec proposal.

So "hermiq contributes working nodes to OR's engine" is measured for agent-step and asserted for workload-step. Given that hydra's dispatch and applier flows both depend on it, that is the weaker half carrying the more consequential work.

Why it was skipped, and why this is not a simple copy of the agent-step case

agent-step is testable in the newman suite because it is an API call with a parseable answer. The suite authors a flow, runs it with sync: true, and asserts on the result.

workload-step is a different shape. Its contract from lib/Flow/HermiqWorkloadNode.php:

required : repo · ref · command (array)
optional : output (default "stage") · owner · credentialId · pushCredentialId
           timeoutMs · toolRepo · toolRef · push · scope

It clones a repository and executes a command inside the ExApp sidecar. That needs a reachable repo, a running sidecar, and a credential path — real side effects rather than a request/response pair. That is presumably why it was left, and it is a legitimate reason, not an oversight.

What a first test could look like

The cheapest honest coverage is not a full stage run. Two smaller assertions each prove something that is currently unproven:

1. The node resolves and is dispatched at all. A flow with a hermiq.workload-step whose repo/ref point at something trivially reachable, asserting the run reaches that transition and produces an item under the configured output key (default stage). Even a failing workload proves the engine resolved the type, dispatched it, and routed the result — which is exactly what has never been demonstrated.
2. The required-field guard fires. validate() throws UnexpectedValueException on a missing repo, ref or command. A flow omitting one should fail the run rather than pass through silently. This needs no sidecar at all and is worth having on its own.

⚠️ Assert on itemsIn/itemsOut per transition, not on the run's status. A flow reporting completed with itemsOut: 0 everywhere is a pass-through, and that is precisely how the whole hydra pipeline looked healthy while nine of twelve flows had never run. The per-transition counts are the only thing that distinguishes work from a heartbeat.

⚠️ sync: true on the run endpoint is what makes any of this testable. Async needs a worker, and every dev stack has backgroundjobs_mode=cron with nothing calling cron.php, so queue-and-poll times out against a perfectly healthy engine.

Context

This is the last unproven piece of the hermiq→OpenRegister node contribution. The rest is now verified: both types are registered in FlowNodeRegistry, all 12 node types hydra's flows reference resolve against the 22 registered, and the engine demonstrably executes (openregister#... — Hydra lock reaper runs every 10 minutes and its per-transition log shows types resolving and items moving).

[rubenvdlinde]

Correction to the coverage table

The table records hermiq.agent-step as ✅ executed through the engine via openregister/tests/newman/openregister-flow-engine.postman_collection.json. That is true on an instance with hermiq and Ollama. It is not true in CI, and CI is where the claim gets its authority.

The case is guarded:

const guard = c.requires === 'agent'
    ? ["if (!pm.collectionVariables.get('agent')) {",
       "    console.log('SKIP ${c.key}: no agent fixture (hermiq/ollama absent) — the AI case did not run')",
       "    pm.collectionVariables.set('${v}_skipped', '1')", '}']
    : null

and the agent variable comes from a fixture request that POSTs to /apps/openregister/api/objects/hermiq/agent — closing with:

pm.test('agent fixture resolved (or hermiq absent)', () => pm.expect(true).to.be.true)

openregister's .github/workflows/code-quality.yml never installs hermiq (no additional-apps entry; grep -rn hermiq .github/workflows/*.yml returns only an unrelated comment). So the POST fails, agent stays empty, the case skips, and the fixture's own assertion is true.

case_mailbox_summary_ai_skipped was set and never read — the only trace was a console.log in a green run.

So the honest table is:

node type	unit	executed through the engine, in CI
hermiq.agent-step	✅	❌ skipped every run (needs hermiq + Ollama)
hermiq.workload-step	✅	❌ never

This does not weaken the issue — it strengthens it. The asymmetry it describes is real, but the baseline is lower than recorded: neither node has been demonstrated through the engine in CI, so "hermiq contributes working nodes to OR's engine" is asserted for both.

Landed toward this

ConductionNL/openregister#2444 adds a 98 — coverage folder to that collection which (a) puts COVERAGE: N of M flow-engine cases executed in a test title so it reaches the run summary, and (b) fails when a case skips without declaring a prerequisite. It deliberately does not fail today's skip, which is declared — it stops the next one being invisible.

That does not close this issue. The two assertions proposed in the description are still the work, and option 2 (the required-field guard firing through the engine) is still the one that needs no sidecar.

⚠️ Worth checking before writing it: whether validateConfig() runs at flow SAVE time. If it does, a flow omitting repo cannot be persisted, and the guard has to be proven at authoring rather than at run — a different test from the one the description sketches.

[rubenvdlinde]

hermiq.workload-step has now executed through the engine

Ran it live on the development instance, using option 2 from this issue's own description — the required-field guard, which needs no sidecar.

A three-node flow, openregister.trigger-manual → hermiq.workload-step → openregister.end, with the workload config deliberately omitting repo/ref/command. Saved with HTTP 201 (the node type survived the round-trip), then run with sync: true.

The per-step rows — not the run status:

node_id | node_type                    | status    | error
--------+------------------------------+-----------+---------------------------------
t1      | openregister.trigger-manual  | completed |
w1      | hermiq.workload-step         | failed    | A workload step needs a repository.

The run's own status was stopped, which on its own says nothing — this issue's warning about itemsIn/itemsOut is the same point, and the step rows are what carry it.

What this proves

• The engine resolves hermiq.workload-step from the registry (also confirmed independently: GET /api/flow/node-catalog returns 22 types and lists it).
• The engine dispatches into it — the node ran.
• Its required-field guard fires through the engine, with the node's own message, not a generic one.
• The failure propagates: the run stopped and openregister.end never executed. It did not pass through silently, which was the specific risk.

So the table in the description can move workload-step from "executed through the engine: ❌ never" to the guard path is proven.

What it does not prove

The clone-and-execute path. That still needs a reachable repository, a running sidecar and a credential — exactly as this issue said, and the reason it was skipped originally. Assertion 1 from the description remains open.

Correction to the other half of the table

agent-step is recorded as ✅ engine-executed via openregister's newman flow-engine collection. That case is guarded on an agent fixture created by POSTing to /apps/openregister/api/objects/hermiq/agent, and openregister's CI never installs hermiq — so the fixture fails, the guard skips the case, and the fixture's own assertion is pm.expect(true).to.be.true. It has been skipped in every CI run.

ConductionNL/openregister#2444 (merged) now prints COVERAGE: N of M flow-engine cases executed in a test title and fails any case that skips without declaring a prerequisite — so the next invisible skip is not invisible.

Worth knowing for whoever writes the permanent test

FlowNodePreflight is advisory, not enforced at save: it is a controller endpoint for inspecting a document you are not writing. That is why the invalid flow above could be persisted at all, and it is what makes this test shape possible — the guard is a RUN-time assertion, not an authoring-time one.