feat: enforce workflow control, approval, correlation, and provenance invariants

[MKlolbullen]

Implements workflow design rules 6–9

6 — Control evidence must not become branch input

• Reject typed nodes that have control-only incoming edges and no data edge.
• Preserve the valid adaptive pattern: one real artifact/data edge plus any number of control-only evidence edges.

7 — Intrusive work must sit behind an explicit approval boundary

• Nodes marked intrusive must declare an approval key.
• Validation requires an upstream gate with requires_approval: true and the same key.
• Approval gates themselves must use an explicit approval key.

8 — Correlate candidates before reporting

• Correlation groups retain every contributing observation and source while selecting a representative record.
• Finding/evidence merge nodes create correlation sidecars directly from their raw parent outputs, before normalized merged data proceeds downstream.
• Report sinks produce a structured correlation report with candidate findings and verified evidence kept as separate lifecycle stages.

9 — Preserve raw evidence and provenance

• Every recorded node result gets a provenance sidecar containing SHA-256, size, line count, format, exact input paths, exact output paths, and source-node identity where known.
• Raw files are never replaced or deleted by correlation.
• Correlation/report artifacts are written under analysis/ and linked through node metadata.

Tests

• Reject control-only typed consumers.
• Accept data + control adaptive branches.
• Reject unguarded or mismatched intrusive approval paths.
• Accept matching approval-gate ancestry.
• Verify correlation retains two raw scanner observations rather than collapsing provenance.
• Verify merge provenance hashes both exact input files.
• Verify final correlation reports separate candidates from evidence and retain both verification observations.

This change does not add scanners or broaden active-testing behavior; it strengthens workflow semantics, evidence integrity, and reporting correctness.

Summary by Sourcery

Enforce workflow control and approval invariants while preserving raw evidence, provenance, and lifecycle-aware correlation reporting.

New Features:

• Add structured correlation groups and reports that retain all raw observations and distinguish candidate findings from verification evidence.
• Generate per-node provenance sidecars and analysis-linked correlation artifacts for recorded workflow results.

Bug Fixes:

• Prevent correlation from discarding contributing observations or replacing raw evidence.
• Reject typed consumers that receive only control evidence and enforce explicit approval boundaries for intrusive nodes.

Enhancements:

• Require approval keys on approval gates and intrusive nodes, with matching approval-gate ancestry validated before execution.
• Record hashes, sizes, line counts, formats, exact paths, and source-node identities for input and output artifacts.

Tests:

• Add coverage for control/data edge validation, approval-boundary enforcement, complete correlation provenance, merge input hashing, and separated candidate/evidence reporting.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a7899baf-d0a5-4d0d-bda5-10b6a0c9ae70

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[sourcery-ai]

Reviewer's Guide

Enforces new workflow semantics around control-only edges, intrusive work approval boundaries, correlation groups, and byte-level provenance, with supporting tests and reporting sidecars.

Sequence diagram for provenance and correlation recording

sequenceDiagram
    participant Parent as Parent outputs
    participant Merge as Merge node
    participant Sink as Report sink
    participant Files as Analysis files

    Parent->>Merge: RecordNodeOutput
    Merge->>Files: persistMergeCorrelation
    Files-->>Merge: Correlation groups with raw observations
    Merge->>Files: persistNodeProvenance
    Parent->>Sink: Normalized inputs
    Sink->>Files: persistCorrelationReport
    Files-->>Sink: Candidates and evidence separated
    Sink->>Files: persistNodeProvenance

Loading

Flow diagram for workflow policy validation

flowchart TD
    Incoming["Incoming edges"] --> Classify["Classify data and control edges"]
    Classify --> TypedCheck["Typed consumer has data edge"]
    TypedCheck --> ApprovalCheck["Validate intrusive approval boundary"]
    ApprovalCheck --> Valid["Workflow accepted"]
    TypedCheck --> RejectControl["Reject control-only typed input"]
    ApprovalCheck --> RejectApproval["Reject missing or mismatched approval gate"]

Loading

File-Level Changes

Change	Details	Files
Enforce validation rules for control-only inputs and intrusive approval boundaries in DAG semantics.	Add approval-key requirement for nodes with RequiresApproval policy. Detect and reject nodes that have typed inputs but only control edges and no data edges. Introduce validateApprovalBoundaries and hasApprovalGateAncestor to require intrusive nodes to sit behind matching approval gates. Wire approval-boundary validation into DAG.Validate() and keep topological ordering logic intact.	internal/graph/semantics.go internal/graph/policy_validation_test.go
Redesign correlation to use structured groups that retain all raw observations and drive both merge-time correlation and sink-level reports.	Replace flat CorrelateRecords grouping with CorrelateGroups that keeps representative record, all observations, and sorted sources. Keep CorrelateRecords as a compatibility wrapper returning only representatives. Add correlation sidecar persistence at merge time, including artifact typing heuristics and metadata links. Add correlation report generation at sink time, separating candidate findings from evidence and hashing the report artifact. Add helpers for reading correlation groups and determining correlation input artifact type from node metadata.	internal/pipeline/correlation.go internal/pipeline/dataflow.go internal/pipeline/provenance_correlation_test.go
Introduce node-level provenance sidecars that describe exact input/output artifacts with hashes and metadata, linked from node outputs.	Define ArtifactRef and NodeProvenance structures for byte-level evidence tracking. Implement persistNodeProvenance to compute SHA-256, size, line count, format, and source-node identity for inputs/outputs and write analysis sidecars. Link provenance files and timestamps into NodeOutput metadata without modifying raw artifacts. Add shared helpers for artifact hashing, source node resolution, and deduplicated/sorted string handling.	internal/pipeline/provenance.go internal/pipeline/dataflow.go internal/pipeline/provenance_correlation_test.go

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.