Release: merge development into beta

appinfo/info.xml

@@ -23,7 +23,7 @@
 - 📋 Pas bedrijfsregels toe op endpoint-verkeer en houd een audit trail per object bij
 
     ]]></description>
-	<version>0.3.13-unstable.20260831053109</version>
+	<version>0.3.16-unstable.20260831214826</version>
 	<licence>EUPL-1.2</licence>
 	<author mail="info@conduction.nl" homepage="https://www.conduction.nl/">Conduction</author>
 	<namespace>Integriq</namespace>
@@ -229,6 +229,17 @@
 			     Last in the block, and post-migration only: it removes rows and
 			     creates nothing, so no step depends on it, and a fresh install
 			     has nothing to remove. -->
+			<!-- retire-integriq-flow-schema task 2: writes the OpenRegister
+			     nodes/edges graph onto every legacy flow object, IN PLACE and
+			     ADDITIVELY (steps stays as the rollback shape, the two engines
+			     dual-run through the migration window). Idempotent: an object
+			     already carrying nodes is skipped, never overwritten; a flow
+			     the translator refuses is logged and keeps running on
+			     FlowRunnerService. Placed after InitializeRegister so the flow
+			     schema (with its new nodes/edges properties) exists. The occ
+			     command integriq:flow:steps-to-graph re-runs or rolls back the
+			     same migration on demand. -->
+			<step>OCA\Integriq\Repair\MigrateFlowStepsToGraph</step>
 			<step>OCA\Integriq\Repair\RemoveRetiredCronJobs</step>
 			<!-- Written but never registered, so it had never run once: a class
 			     that exists is not a class that runs. It moves openconnector
@@ -374,6 +385,13 @@
 		     schema or method of its own and an object trigger refuses a partial
 		     scope. WRITES NOTHING; the emitted document ships disabled. -->
 		<command>OCA\Integriq\Command\RuleToFlow</command>
+		<!-- retire-integriq-flow-schema task 2: drives the steps-to-graph flow
+		     migration by hand. Dry run by default; the apply flag writes the
+		     graph in place, the rollback flag removes it again (steps is kept
+		     through both directions, so rollback is always possible).
+		     (XML comments must not contain a double hyphen, so the flags are
+		     not written with their leading dashes here.) -->
+		<command>OCA\Integriq\Command\FlowStepsToGraph</command>
 	</commands>
 
 	<settings>

docs/features/README.md

@@ -13,6 +13,7 @@ Integriq is an API gateway and integration hub for Nextcloud. It brings enterpri
 | [Reliability](reliability.md) | Retry policy, per-Source circuit breaker, sync dead letters | Implemented |
 | [Rules](rules.md) | Authentication, file handling, locking, and audit trail rules | Implemented |
 | [Jobs](jobs.md) | Cron-based scheduled task execution | Implemented |
+| [Flow nodes](flow-nodes.md) | Contributed step types for OpenRegister's flow engine | Implemented |
 | [Events & Webhooks](events.md) | CloudEvents emission, subscription, and consumer processing | Implemented |
 | [Logging & Monitoring](logging.md) | Call logs, sync logs, and Prometheus metrics | Implemented |
 | [Configuration Management](configuration-management.md) | Import/export, configuration groups, slug-based references | Implemented |

docs/features/flow-nodes.md

@@ -0,0 +1,99 @@
+# Flow nodes
+
+## Overview
+
+OpenRegister runs the fleet's one flow engine. Integriq does not run its own graphs. It contributes step types, so a flow can do what Integriq is good at: call an API, run a synchronization, apply a mapping, ask a person, emit an event.
+
+You build the flow in OpenRegister's flow editor. The Integriq steps appear in the palette when both apps are enabled.
+
+| Node | What the step does |
+|------|--------------------|
+| `openconnector.source-call` | Make one governed API call per item through a configured Source |
+| `openconnector.synchronization-run` | Run a configured Synchronization and hand each synchronised object onward |
+| `openconnector.source-paginate` | Fetch one page of objects from a Source |
+| `openconnector.apply-mapping` | Apply a configured Mapping to every item |
+| `openconnector.contract` / `contract-commit` / `contract-sweep` | The decomposed synchronization's contract steps |
+| `openconnector.fetch-file` | Fetch a file referenced by an item |
+| `openconnector.approval-request` | Pause the run until someone approves or rejects |
+| `openconnector.event-emit` | Emit a CloudEvent for every item |
+
+## Call an API from a flow
+
+Add a `source-call` step. Pick a Source, give it a path and a method:
+
+```json
+{
+  "id": "step-apply-label",
+  "type": "openconnector.source-call",
+  "config": {
+    "source": "demo-forge-api",
+    "endpoint": "/issues/{{issue.number}}/labels",
+    "method": "POST",
+    "body": { "labels": ["{{triage.proposedLabel}}"] },
+    "output": "labelResult"
+  }
+}
+```
+
+The step runs once per item. `{{dotted.path}}` placeholders resolve from each item's record, and the response lands under the key you name in `output`. The call goes through `CallService`, so the Source's enablement, host guard, rate limits and call logging all apply unchanged.
+
+## Why there is no raw-URL node
+
+You cannot type a URL into a flow step. The step names a Source, and the endpoint is a path inside that Source's location. An absolute URL, a `//host` path or a `../` escape is rejected before any request goes out.
+
+This is the whole security model, not a missing convenience. A Source is where an administrator decides which hosts may be called, how often, and with which credential. A URL field in a flow document would hand that decision to every flow author and turn the editor into a request forger. If a host is worth calling, give it a Source first.
+
+Credentials follow the same line. A step has no token field. Authentication comes from the Source's `credentialRef`, resolved by the credential broker at call time. No secret ever sits in a flow document.
+
+## Why an unattributed run fails closed
+
+Every call runs as the flow run's owner, read from the run context. When no owner resolves, the step refuses and raises. There is no fallback to an admin, to the Source's creator, or to nobody.
+
+An anonymous authenticated outbound call is the failure we refuse to ship. A loud error names the gap; a silent fallback hides it behind someone else's identity.
+
+## Ask a person: the approval step
+
+`openconnector.approval-request` parks the run and creates a pending approval request. The approvers see it on the Pending approvals page and in their shared task list, like every other Integriq approval.
+
+```json
+{
+  "id": "approve-publish",
+  "type": "openconnector.approval-request",
+  "config": {
+    "question": "Publish this dataset?",
+    "approverGroup": "data-stewards",
+    "ttlSeconds": 86400
+  }
+}
+```
+
+- **Approved.** The run resumes. The decision, the approver and the comment land on every item under `approval`, so a later step can route on them.
+- **Rejected.** By default the run continues and your reject edge reads `approval.decision`. Set `failOnReject: true` when a no should fail the run.
+- **Expired.** The run fails. An approval nobody answered never counts as answered.
+
+An answer wakes the run immediately. If that wake-up is ever lost, the step re-checks the approval request itself on its next heartbeat, so a decision is never stranded.
+
+## Emit an event
+
+`openconnector.event-emit` sends one CloudEvent per item through the existing event pipeline. Name a `type` and a `source`, and subscriptions pick it up exactly as they would for any other Integriq event.
+
+## Migrate old step-list flows
+
+Flows built in Integriq's earlier step-list editor still exist as ordered `steps[]`. One command translates them onto the engine's graph shape:
+
+```bash
+occ integriq:flow:steps-to-graph            # dry run: reports what would happen
+occ integriq:flow:steps-to-graph --apply    # writes nodes/edges onto each flow
+```
+
+The migration is additive and repeatable. `steps` stays on the object, a flow that already carries `nodes` is skipped, and a flow the translator cannot express faithfully is refused with the reasons listed. The same pass also runs automatically on upgrade.
+
+Changed your mind? Roll it back:
+
+```bash
+occ integriq:flow:steps-to-graph --rollback --apply
+```
+
+## Next steps
+
+Create a [Source](sources.md) for the API you want to call, then open OpenRegister's flow editor and add a `source-call` step against it.

eslint.config.mjs

@@ -144,11 +144,44 @@ export default [
 			'no-console': 'off',
 			'n/no-process-exit': 'off',
 			'n/hashbang': 'off',
-			// `_` / `__` as a deliberate throwaway binding — `catch (_)`, a
-			// discarded destructuring slot. Narrow on purpose: the pattern matches
-			// UNDERSCORES ONLY, so a real name that happens to start with `_` is
-			// still reported. v9 drives plain `.js` through the CORE rule (the
-			// `@typescript-eslint` swap is per-file-type), so it is set here.
+			// Tests import devDependencies by definition; this rule is about what
+			// ships in the published package, which tests/ never does.
+			'n/no-unpublished-import': 'off',
+			// 🔴 COMMENTS ONLY, and the exception is load-bearing. Four
+			// api-direct specs document the `testIgnore` glob that excludes them
+			// from the gate-19 UI run, and a block comment cannot contain the
+			// literal `**` + `/` because that closes it at `*/`. The files carry a
+			// ZERO-WIDTH SPACE (U+200B) between the two to break the sequence.
+			//
+			// Deleting the character to satisfy the rule would terminate the
+			// comment early and break the file. In CODE an invisible character is
+			// a genuine hazard — a look-alike inside an identifier or a string —
+			// and the rule still catches that. In a comment it cannot change
+			// behaviour.
+			'no-irregular-whitespace': ['error', { skipComments: true }],
+		},
+	},
+
+	{
+		// `_` / `__` as a deliberate throwaway binding — `catch (_)`, a discarded
+		// destructuring slot. Narrow on purpose: the pattern matches UNDERSCORES
+		// ONLY, so a real name that happens to start with `_` is still reported.
+		//
+		// 🔴 `.js` / `.mjs` ONLY, NOT `.ts`. The CORE rule is not TypeScript-aware:
+		// applied to a `.ts` file it reads the parameter NAMES inside a function
+		// TYPE as bindings and reports them unused. Measured on humaniq —
+		//
+		//   t?: (app: string, key: string) => string
+		//
+		// produced four `no-unused-vars` errors for `app` and `key`, which are
+		// documentation, not variables. The same mis-scoping made every unused
+		// `catch (e)` in a `.ts` spec report TWICE, once per rule.
+		//
+		// v9 already turns the core rule off for `.ts` and drives
+		// `@typescript-eslint/no-unused-vars` instead; naming `.ts` here switched
+		// it back on. TypeScript files are handled by the block below.
+		files: ['tests/**/*.js', 'tests/**/*.mjs'],
+		rules: {
 			'no-unused-vars': [
 				'error',
 				{
@@ -165,12 +198,117 @@ export default [
 					ignoreRestSiblings: true,
 				},
 			],
-			// Tests import devDependencies by definition; this rule is about what
-			// ships in the published package, which tests/ never does.
-			'n/no-unpublished-import': 'off',
 		},
 	},
 
+	{
+		// The TypeScript half of the block above. Same intent, same patterns, on
+		// the rule that actually understands the language: it knows a name inside
+		// a function type is not a binding, so type annotations stay quiet while a
+		// genuinely dead local is still reported.
+		files: ['tests/**/*.ts', 'tests/**/*.tsx'],
+		rules: {
+			'@typescript-eslint/no-unused-vars': [
+				'error',
+				{
+					varsIgnorePattern: '^_+$',
+					caughtErrors: 'all',
+					caughtErrorsIgnorePattern: '^_+$',
+					argsIgnorePattern: '^_',
+					ignoreRestSiblings: true,
+				},
+			],
+		},
+	},
+
+	{
+		// 🔴 Node-side CLI tooling under `scripts/`, which is COMMONJS. Flat
+		// config defaults every `.js` to ESM with browser-ish globals, so without
+		// this block eslint reports the CommonJS wrapper itself as undefined
+		// identifiers. Measured on this app: 52 of the 233 errors under
+		// `tests/` + `scripts/` were `no-undef`, ALL of them in `scripts/`, and
+		// all five names were the environment rather than a typo — `process` 23,
+		// `require` 20, `__dirname` 6, `__filename` 2, `module` 1.
+		//
+		// This is describing the environment, not relaxing a rule, and it is the
+		// same argument the test-globals block below makes: declaring them keeps
+		// `no-undef` able to do its real job, which is catching a genuinely
+		// misspelled identifier. Suppressing the rule instead would bury that.
+		//
+		// `no-console` is off because printing its report is what a CLI checker
+		// is FOR.
+		//
+		// 🔴 NO `n/*` ENTRIES HERE, DELIBERATELY. `eslint-plugin-n` is NOT
+		// registered for these files under eslint 10 + @nextcloud/eslint-config
+		// 9, so `'n/no-process-exit': 'off'` would be dead config that reads as
+		// if it were doing something. Measured both ways on this app: 0 `n/`
+		// findings with the entries and 0 without.
+		//
+		// What DID report was the opposite — four `scripts/*.js` carried
+		// `/* eslint-disable n/no-process-exit */` and `/* eslint-disable
+		// n/shebang */` left over from the eslintrc era, and an inline disable
+		// naming an unregistered plugin is itself an error ("Definition for rule
+		// 'n/shebang' was not found"). Those 8 comments are removed; do not add
+		// `n/*` rules back to replace them.
+		//
+		// ⚠️ `.js` and `.cjs` ONLY. A `scripts/*.mjs` is genuinely ESM and must
+		// keep the default `sourceType`, or `import` stops parsing there.
+		files: ['scripts/**/*.js', 'scripts/**/*.cjs'],
+		languageOptions: {
+			sourceType: 'commonjs',
+			globals: {
+				require: 'readonly',
+				module: 'writable',
+				exports: 'writable',
+				process: 'readonly',
+				__dirname: 'readonly',
+				__filename: 'readonly',
+				console: 'readonly',
+				Buffer: 'readonly',
+				global: 'readonly',
+				URL: 'readonly',
+				TextEncoder: 'readonly',
+				TextDecoder: 'readonly',
+			},
+		},
+		rules: {
+			'no-console': 'off',
+		},
+	},
+
+	{
+		// The ESM half of the block above. A `scripts/*.mjs` is genuinely a module
+		// and must keep the default `sourceType`, so it gets Node's globals but
+		// none of the CommonJS wrapper. Measured: `process` reported undefined 2x
+		// in hermiq's generate-opengemeenten-icons.mjs and 4x in openregister's
+		// l10n/runtime-check.mjs, which the `.js`/`.cjs` block deliberately does
+		// not match.
+		files: ['scripts/**/*.mjs', 'tests/**/*.mjs'],
+		languageOptions: {
+			globals: {
+				process: 'readonly',
+				console: 'readonly',
+				Buffer: 'readonly',
+				global: 'readonly',
+				URL: 'readonly',
+				TextEncoder: 'readonly',
+				TextDecoder: 'readonly',
+			},
+		},
+		rules: {
+			'no-console': 'off',
+		},
+	},
+
+	{
+		// eslint must not try to PARSE a shell script. `tests/e2e/seed.test.sh`
+		// matches the `**/*.test.*` glob some presets use, and eslint then reads
+		// it as JavaScript and reports "Parsing error: Unexpected character" —
+		// a finding about a file it should never have opened.
+		ignores: ['**/*.sh', '**/*.bash'],
+	},
+
+
 	// eslint-config-prettier LAST OF THE PRESETS, and it has to be: it only turns
 	// rules OFF, and what it turns off is everything prettier owns — including the
 	// `@stylistic/*` family v9 introduces (`indent`, `quotes`, `semi`).