diff --git a/.github/actions/pipe-diff/README.md b/.github/actions/pipe-diff/README.md new file mode 100644 index 000000000..edb45a6f3 --- /dev/null +++ b/.github/actions/pipe-diff/README.md @@ -0,0 +1,128 @@ +# RocketRide Pipe Diff action + +A composite GitHub Action that posts a **semantic** diff of changed `.pipe` +pipeline files as a single, sticky pull-request comment. + +Raw JSON diffs of `.pipe` files are dominated by canvas coordinate churn — the +per-node `ui` block and the top-level `viewport` — which buries the changes that +actually matter. This action wraps the local [`rocketride diff`](../../../packages/client-python) +CLI subcommand to surface only what changed: nodes added/removed, provider +changes, config field changes, and edge (wiring) additions/removals. + +The action runs **entirely on the runner**. Like the `rocketride diff` command it +wraps, it never contacts the RocketRide engine or the network — it only reads +files and git objects and compares parsed JSON. It therefore needs no +`--uri`/`--apikey`/`--token` credentials of any kind. + +## Usage + +Add a workflow that checks out the repo and runs this action on pull requests: + +```yaml +name: Pipe diff +on: + pull_request: + +# Reading the repo, and writing the sticky comment. +permissions: + contents: read + pull-requests: write + +jobs: + pipe-diff: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + - uses: ./.github/actions/pipe-diff +``` + +`actions/checkout` with its default `fetch-depth: 1` is enough — the action +fetches the PR base commit itself before diffing. + +### With inputs + +```yaml + - uses: ./.github/actions/pipe-diff + with: + files: 'pipelines/**/*.pipe' # restrict which .pipe files are considered + python-version: '3.12' + cli-version: '==1.4.0' # pin a specific published rocketride release + comment: 'true' # set 'false' to skip commenting + include-layout: 'false' # set 'true' to also report ui/viewport churn +``` + +## Inputs + +| Input | Default | Description | +| ---------------- | ------------- | ----------- | +| `files` | `**/*.pipe` | Git pathspec glob selecting the `.pipe` files to consider. Git's `:(glob)` magic is applied automatically, so `**` matches across directories. Only files that actually changed versus the PR base are diffed. | +| `python-version` | `3.12` | Python version passed to `actions/setup-python`. RocketRide requires Python >= 3.10. | +| `cli-version` | `` (empty) | pip version specifier appended to the `rocketride` requirement, e.g. `==1.4.0` or `>=1.4,<2`. Empty installs the latest published release. The run **fails fast** with a clear message if the installed CLI lacks the `diff` subcommand. | +| `comment` | `true` | When `true`, post or update one sticky PR comment. Set to `false` to compute the diff (job log + outputs) without commenting; that mode needs no `pull-requests: write` permission. | +| `include-layout` | `false` | When `true`, treat canvas layout churn (per-node `ui` blocks and the top-level `viewport`) as meaningful and enumerate it. When `false` (default) a pure-layout change is reported as `no semantic changes (layout only)`. | + +## Outputs + +| Output | Description | +| ---------------------- | ----------- | +| `changed-files` | Number of changed `.pipe` files considered (added or modified vs the PR base). | +| `has-semantic-changes` | `true` when at least one changed `.pipe` file has semantic changes, else `false`. | +| `comment-body-file` | Path to the generated Markdown comment body (useful for debugging or reuse). | + +## Permissions + +The sticky-comment step needs `pull-requests: write` (comments are issue +comments on the PR). `contents: read` is required to check out and diff the +repository. If you set `comment: false`, only `contents: read` is needed. + +The action authenticates the GitHub REST calls with the workflow's built-in +`GITHUB_TOKEN`; it never prints the token. + +## Behavior + +- **Sticky comment.** The action maintains exactly one comment per PR, found and + updated via a hidden HTML marker (``). Re-runs + update that comment in place instead of stacking new ones. +- **Layout noise is hidden by default.** A change that only moves nodes on the + canvas produces exit code 0 from the CLI and is summarized as + `No semantic changes (layout only).`. Pass `include-layout: true` to enumerate + the `ui`/`viewport` deltas instead. +- **Version changes are always reported** (they are semantic-ish), regardless of + `include-layout`. +- **New files.** A `.pipe` file added in the PR (absent in the base) is diffed + against an empty pipeline, so every node and edge shows up as added. +- **Deleted files.** Removed `.pipe` files are noted in the comment (there is no + working-tree file to diff). +- **Failure semantics.** The action does **not** fail merely because semantic + changes exist — it is informational. It **does** fail (and emits an `error` + annotation) when a changed `.pipe` file cannot be parsed or diffed, so a broken + pipeline in a PR is surfaced as a red check. + +## Local equivalent + +Everything the action does per file is one local CLI call — no server, no token: + +```bash +# Diff a working-tree .pipe file against the PR base commit, as Markdown: +rocketride diff --git "$(git merge-base origin/main HEAD)" pipelines/my.pipe --markdown + +# Or diff two files directly: +rocketride diff old.pipe new.pipe + +# Add --include-layout to surface ui/viewport churn; --json for machine output. +``` + +Exit codes match the action's per-file logic: `0` = no semantic changes, +`1` = semantic changes, `2` = usage error / unreadable / unparseable file. + +## Pinned dependencies + +Third-party actions are pinned by full commit SHA (with a version comment), +matching the convention in [`.github/workflows`](../../workflows): + +- `actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065` (v5) +- `actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea` (v7) + +## License + +MIT — see the repository [`LICENSE`](../../../LICENSE). diff --git a/.github/actions/pipe-diff/action.yml b/.github/actions/pipe-diff/action.yml new file mode 100644 index 000000000..76404b972 --- /dev/null +++ b/.github/actions/pipe-diff/action.yml @@ -0,0 +1,330 @@ +# MIT License +# +# Copyright (c) 2026 Aparavi Software AG +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +# Composite action that posts a semantic diff of changed .pipe pipeline files as +# a sticky pull-request comment. It wraps the local `rocketride diff` subcommand +# (Python SDK/CLI), which compares parsed .pipe JSON entirely on the runner and +# never contacts the RocketRide engine or the network. Raw JSON diffs of .pipe +# files are dominated by canvas coordinate churn (the per-node "ui" block and the +# top-level "viewport"); this surfaces what actually changed — nodes, edges, and +# config — so reviewers see signal instead of layout noise. +name: RocketRide Pipe Diff +description: >- + Comment a semantic diff of changed .pipe pipeline files on a pull request, + hiding canvas-layout noise. Runs entirely on the runner; never contacts the + RocketRide engine or the network. +author: RocketRide, Inc. + +# Branding for the GitHub Marketplace / action listing. +branding: + icon: git-pull-request + color: purple + +inputs: + files: + description: >- + Git pathspec glob (":(glob)" magic is applied automatically) selecting the + .pipe files to consider. The action diffs only those that actually changed + versus the PR base. + required: false + default: '**/*.pipe' + python-version: + description: Python version passed to actions/setup-python (RocketRide requires >= 3.10). + required: false + default: '3.12' + cli-version: + description: >- + pip version specifier appended to the "rocketride" requirement, e.g. + "==1.4.0" or ">=1.4,<2". Leave empty to install the latest published + release. The run fails fast with a clear message if the installed CLI does + not provide the "diff" subcommand. + required: false + default: '' + comment: + description: >- + When "true" (default), post or update ONE sticky PR comment with the diff. + Set to "false" to compute the diff and write a job summary without + commenting (requires no pull-requests:write permission). + required: false + default: 'true' + include-layout: + description: >- + When "true", treat canvas layout churn (per-node "ui" blocks and the + top-level "viewport") as meaningful and enumerate it. Default "false" hides + it and reports a pure-layout change as "no semantic changes (layout only)". + required: false + default: 'false' + +outputs: + changed-files: + description: Number of changed .pipe files considered (added or modified vs the PR base). + value: ${{ steps.diff.outputs.changed-files }} + has-semantic-changes: + description: '"true" when at least one changed .pipe file has semantic changes, else "false".' + value: ${{ steps.diff.outputs.has-semantic-changes }} + comment-body-file: + description: Path to the generated Markdown comment body (useful for debugging or reuse). + value: ${{ steps.diff.outputs.comment-body-file }} + +runs: + using: composite + steps: + - name: Set up Python + uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 + with: + python-version: ${{ inputs.python-version }} + + - name: Install RocketRide CLI + shell: bash + env: + CLI_VERSION: ${{ inputs.cli-version }} + run: | + set -euo pipefail + # Build the pip requirement. CLI_VERSION, when set, is a PEP 508 version + # specifier appended to the distribution name (e.g. "==1.4.0", ">=1.4"). + if [ -n "${CLI_VERSION}" ]; then + requirement="rocketride${CLI_VERSION}" + else + requirement="rocketride" + fi + echo "Installing ${requirement}" + python -m pip install --disable-pip-version-check --quiet "${requirement}" + + - name: Verify the CLI provides 'diff' + shell: bash + run: | + set -euo pipefail + if ! command -v rocketride >/dev/null 2>&1; then + echo "::error::'rocketride' CLI is not on PATH after installation." + exit 1 + fi + # `rocketride diff --help` exits 0 only when the subcommand exists; + # argparse exits non-zero ("invalid choice: 'diff'") on older releases. + if ! rocketride diff --help >/dev/null 2>&1; then + installed="$(python -c 'import importlib.metadata as m; print(m.version("rocketride"))' 2>/dev/null || echo unknown)" + echo "::error::The installed 'rocketride' CLI (version ${installed}) does not provide the 'diff' subcommand. Pin a release that includes it via the action's 'cli-version' input once 'rocketride diff' is published." + exit 1 + fi + echo "rocketride diff is available." + + - name: Compute semantic pipe diffs + id: diff + shell: bash + env: + FILES_GLOB: ${{ inputs.files }} + INCLUDE_LAYOUT: ${{ inputs.include-layout }} + BASE_SHA: ${{ github.event.pull_request.base.sha }} + run: | + # Backticks in the single-quoted printf formats below are LITERAL + # Markdown code-span characters, not command substitutions; single + # quotes are required to keep them literal (SC2016 is a false positive). + # shellcheck disable=SC2016 + set -euo pipefail + + marker='' + body_file="${RUNNER_TEMP:-/tmp}/rocketride-pipe-diff-comment.md" + : > "${body_file}" + + # Emit step outputs in one place so every early return stays consistent. + emit_outputs() { + { + echo "changed-files=${1}" + echo "has-semantic-changes=${2}" + echo "pipe-files-changed=${3}" + echo "errored=${4}" + echo "comment-body-file=${body_file}" + } >> "${GITHUB_OUTPUT}" + } + + # A pull_request event is required: the base commit is what we diff + # against. Outside that context there is nothing to compare, so skip. + if [ -z "${BASE_SHA}" ]; then + echo "::warning::No pull_request base SHA in context; the pipe-diff action only runs on pull_request events. Skipping." + printf '%s\n' "${marker}" > "${body_file}" + emit_outputs 0 false false false + exit 0 + fi + + # actions/checkout uses fetch-depth 1 by default, so the base commit is + # usually absent. Fetch just that object. GitHub allows fetching an + # arbitrary reachable SHA directly. + if ! git cat-file -e "${BASE_SHA}^{commit}" 2>/dev/null; then + echo "Fetching PR base commit ${BASE_SHA}" + git fetch --no-tags --depth=1 origin "${BASE_SHA}" 2>/dev/null \ + || git fetch --no-tags origin "${BASE_SHA}" 2>/dev/null \ + || true + fi + if ! git cat-file -e "${BASE_SHA}^{commit}" 2>/dev/null; then + echo "::error::Could not fetch the PR base commit ${BASE_SHA}. Ensure actions/checkout ran before this action and left credentials available (the default)." + exit 1 + fi + + # Collect changed .pipe files (added or modified; deletions handled + # separately). NUL-delimited to survive unusual path characters. + changed_files=() + while IFS= read -r -d '' path; do + changed_files+=("${path}") + done < <(git diff -z --name-only --diff-filter=d "${BASE_SHA}" HEAD -- ":(glob)${FILES_GLOB}") + + # Deleted .pipe files: reported as a note (no working-tree file to diff). + deleted_files=() + while IFS= read -r -d '' path; do + deleted_files+=("${path}") + done < <(git diff -z --name-only --diff-filter=D "${BASE_SHA}" HEAD -- ":(glob)${FILES_GLOB}") + + changed_count="${#changed_files[@]}" + deleted_count="${#deleted_files[@]}" + + # Nothing to report: emit an empty (marker-only) body. The comment step + # will refresh an existing sticky comment but will NOT create a new one. + if [ "${changed_count}" -eq 0 ] && [ "${deleted_count}" -eq 0 ]; then + echo "No changed .pipe files versus the base." + { + printf '%s\n' "${marker}" + printf '## RocketRide pipeline diff\n\n' + printf 'No `.pipe` files changed in this pull request.\n' + } > "${body_file}" + emit_outputs 0 false false false + exit 0 + fi + + include_layout_flag="" + if [ "${INCLUDE_LAYOUT}" = "true" ]; then + include_layout_flag="--include-layout" + fi + + sections_file="${RUNNER_TEMP:-/tmp}/rocketride-pipe-diff-sections.md" + : > "${sections_file}" + + any_semantic=false + any_error=false + err_file="${RUNNER_TEMP:-/tmp}/rocketride-pipe-diff-stderr.txt" + + # The "${arr[@]+...}" guard expands to nothing for an empty array, so the + # loop is a no-op under `set -u` on every bash version (a bare + # "${arr[@]}" of an empty array errors on bash < 4.4). + for f in ${changed_files[@]+"${changed_files[@]}"}; do + printf '### `%s`\n\n' "${f}" >> "${sections_file}" + + # Build the command as an always-non-empty array so "${cmd[@]}" is + # safe under `set -u` on every bash version (an empty-array expansion + # errors on bash < 4.4). + cmd=(rocketride diff --git "${BASE_SHA}" "${f}" --markdown) + if [ -n "${include_layout_flag}" ]; then + cmd+=("${include_layout_flag}") + fi + + set +e + out="$("${cmd[@]}" 2>"${err_file}")" + rc=$? + set -e + + case "${rc}" in + 0) + # Exit 0 = no semantic changes. Any textual change the git diff + # picked up was pure layout/formatting noise. + printf 'No semantic changes (layout only).\n\n' >> "${sections_file}" + ;; + 1) + any_semantic=true + printf '%s\n\n' "${out}" >> "${sections_file}" + ;; + *) + any_error=true + err_msg="$(cat "${err_file}")" + echo "::error file=${f}::rocketride diff failed for ${f}: ${err_msg}" + { + printf 'Could not diff this file (exit %s):\n\n' "${rc}" + printf '```\n%s\n```\n\n' "${err_msg}" + } >> "${sections_file}" + ;; + esac + done + + for f in ${deleted_files[@]+"${deleted_files[@]}"}; do + printf '### `%s`\n\n' "${f}" >> "${sections_file}" + printf 'Pipeline file removed in this pull request.\n\n' >> "${sections_file}" + done + + # Top summary line. + if [ "${any_error}" = "true" ]; then + summary='One or more pipeline files could not be diffed. See details below.' + elif [ "${any_semantic}" = "true" ]; then + summary='Semantic pipeline changes detected. See details below.' + else + summary='No semantic changes (layout only).' + fi + + { + printf '%s\n' "${marker}" + printf '## RocketRide pipeline diff\n\n' + printf '%s\n\n' "${summary}" + cat "${sections_file}" + printf -- '---\n' + printf 'Semantic diff of `.pipe` files versus the PR base. Canvas layout (per-node `ui` and the top-level `viewport`) is ignored by default. Generated by the RocketRide pipe-diff action.\n' + } > "${body_file}" + + emit_outputs "${changed_count}" "${any_semantic}" true "${any_error}" + + - name: Post or update sticky PR comment + if: ${{ inputs.comment == 'true' && github.event_name == 'pull_request' }} + uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7 + env: + BODY_FILE: ${{ steps.diff.outputs.comment-body-file }} + PIPE_FILES_CHANGED: ${{ steps.diff.outputs.pipe-files-changed }} + with: + script: | + const fs = require('fs'); + const marker = ''; + const body = fs.readFileSync(process.env.BODY_FILE, 'utf8'); + const pipeFilesChanged = process.env.PIPE_FILES_CHANGED === 'true'; + const { owner, repo } = context.repo; + const issue_number = context.issue.number; + + if (!issue_number) { + core.info('No pull-request number in context; skipping comment.'); + return; + } + + // Find our one sticky comment by its hidden marker. + const comments = await github.paginate(github.rest.issues.listComments, { + owner, repo, issue_number, per_page: 100, + }); + const existing = comments.find((c) => c.body && c.body.includes(marker)); + + if (existing) { + await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body }); + core.info(`Updated sticky pipe-diff comment (${existing.id}).`); + } else if (pipeFilesChanged) { + await github.rest.issues.createComment({ owner, repo, issue_number, body }); + core.info('Created sticky pipe-diff comment.'); + } else { + core.info('No .pipe changes and no existing comment; nothing to post.'); + } + + - name: Fail if any pipe file could not be diffed + if: ${{ steps.diff.outputs.errored == 'true' }} + shell: bash + run: | + set -euo pipefail + echo "::error::One or more changed .pipe files could not be parsed or diffed. See the annotations and the PR comment above." + exit 1 diff --git a/docs/README-python-client.md b/docs/README-python-client.md index 37f79bf7a..04f7f6f8b 100644 --- a/docs/README-python-client.md +++ b/docs/README-python-client.md @@ -555,9 +555,101 @@ rocketride stop --token # Terminate a running task rocketride list # List all active tasks rocketride events ALL --token # Stream task events rocketride rrext_store get_all_projects # List stored projects +rocketride diff old.pipe new.pipe # Semantic diff of two .pipe files (local; no server) ``` -All commands accept `--uri` and `--apikey` flags, or read from environment variables. +All commands **except `diff`** accept `--uri` and `--apikey` flags, or read from environment variables. `diff` runs entirely locally and takes no connection flags — see below. + +### Semantic pipeline diff (`rocketride diff`) + +A raw `git diff` of a `.pipe` file is dominated by **canvas coordinate churn**: every node has a `ui` block whose `position.x`/`position.y` change whenever you nudge it, plus a top-level `viewport` that shifts when you pan or zoom. Those lines say nothing about behavior, yet they bury the changes that do — a swapped LLM provider, a re-tuned chunk size, a rewired retrieval step. + +`rocketride diff` computes a **semantic** diff instead. It matches components by `id` and reports what changed across three axes — **Nodes** (added / removed / re-provisioned), **Edges** (the wiring between components, from each node's `input[]` data lanes and `control[]` agent-orchestration lanes), and **Config** (per-field changes as readable dotted paths like `config.default.strlen`) — while collapsing all layout churn into a single line. + +> **Local only — no engine, no auth.** Unlike every other subcommand, `diff` never connects to an engine or the network. It reads files (or a git ref) and compares parsed JSON on your machine, so it takes **none** of the `--uri` / `--apikey` / `--token` arguments the other commands share. + +```bash +# Compare two files (old, then new) +rocketride diff old.pipe new.pipe + +# Compare a working-tree file against a git ref (uses `git show :`) +rocketride diff --git HEAD rag.pipe +rocketride diff --git main rag.pipe +``` + +| Flag | Description | +| --- | --- | +| ` ` | The two files to compare (positional). Omit one and pass `--git` instead. | +| `--git ` | Diff the working-tree `FILE` against `` via `git show :`. If the file is absent in ``, everything is reported as added. | +| `--include-layout` | Enumerate the layout churn (each node's `ui` block and the `viewport`) that is hidden by default, as `ui.*` field changes. | +| `--json` | Emit a single JSON document to stdout (mutually exclusive with `--markdown`). | +| `--markdown` | Emit compact, PR-comment-friendly Markdown to stdout (mutually exclusive with `--json`). | +| `--exit-zero` | Always exit `0` on a successful run, even when changes are found (non-gating). | + +**Exit codes** make it easy to gate a CI job: + +| Code | Meaning | +| --- | --- | +| `0` | No semantic changes (or any successful run when `--exit-zero` is set). | +| `1` | Semantic changes were found. | +| `2` | Usage error, or an unreadable / unparseable file or bad git ref. | + +Layout (the `ui` blocks and `viewport`) is ignored by default — a pure canvas move exits `0`. Version changes (the top-level `version` field) are always reported and always count as a change. + +**Three output modes.** The default is grouped, colored text (color auto-disables when piped or when `NO_COLOR` is set), with `+` added, `-` removed, `~` changed: + +```text +Pipeline diff: 1 node changed, layout changed + +Config + chunker_1 + ~ config.default.strlen: 512 -> 1024 + +Layout: changed (ui/viewport) +``` + +`--json` emits one stable, sorted document with `nodes`, `edges`, and a `summary` block: + +```json +{ + "edges": { "added": [], "removed": [] }, + "nodes": { + "added": [], + "changed": [ + { + "config_changes": [ + { "kind": "changed", "new": 1024, "old": 512, "path": "config.default.strlen" } + ], + "id": "chunker_1", + "provider_change": null + } + ], + "removed": [] + }, + "summary": { + "config_changes": 1, "edges_added": 0, "edges_removed": 0, + "has_semantic_changes": true, "layout_changed": true, + "nodes_added": 0, "nodes_changed": 1, "nodes_removed": 0, + "provider_changes": 0, "version_change": null + } +} +``` + +`--markdown` emits a compact PR-comment-friendly report (`|` and backticks safely escaped) suitable for posting from a GitHub Action: + +```markdown +**Pipeline diff:** 1 node changed, layout changed + +**Config** + +| Node | Field | Change | +| --- | --- | --- | +| `chunker_1` | `config.default.strlen` | `512` → `1024` | + +_Layout (ui/viewport) changed._ +``` + +**Review pipelines like code in CI.** Because `diff` is local and exits non-zero on change, it drops straight into a pull-request check: a GitHub Actions job runs `rocketride diff --git origin/ --markdown` and posts the summary as a PR comment. This is the review half of the "pipelines as code" loop — **validate** a `.pipe` to confirm it is well-formed before it runs (via [`client.validate()`](#services-validation-and-ping)), **evaluate** it to measure output quality, and `diff` it to see exactly what changed between revisions. See the [side-by-side example and PR-comment recipe](https://github.com/rocketride-org/rocketride-server/blob/develop/examples/pipe-diff-example.md). ## Configuration diff --git a/examples/pipe-diff-example.md b/examples/pipe-diff-example.md new file mode 100644 index 000000000..df249e5c2 --- /dev/null +++ b/examples/pipe-diff-example.md @@ -0,0 +1,259 @@ +# Semantic pipeline diff: raw JSON vs. `rocketride diff` + +A `.pipe` file is JSON, so `git diff` "works" on it — but the output is mostly +noise. Every component carries a `ui` block whose `position.x` / `position.y` +change whenever you nudge a node on the canvas, and the top-level `viewport` +shifts when you pan or zoom. None of that changes what the pipeline *does*, yet it +dominates the diff and buries the changes that matter. + +`rocketride diff` is a **semantic** diff for `.pipe` files. It compares the parts +that affect behavior — nodes, wiring, and config — and collapses all canvas churn +into a single line. This page shows the exact same change through both lenses. + +## The change + +Someone upgraded a RAG pipeline in one commit: + +- **Swapped the LLM** from OpenAI to Anthropic (`llm_openai` → `llm_anthropic`, + with the matching profile/key config). +- **Tightened the prompt** — added a "cite your sources" instruction. +- **Inserted a reranker** (`reranker_cohere`) between retrieval and the prompt, so + the `documents` lane now flows `qdrant_1 → reranker_1 → prompt_1` instead of + `qdrant_1 → prompt_1`. +- ...and, incidentally, **tidied the canvas** — nudged every node a few pixels and + zoomed out a little. + +Three real changes. One cosmetic one. Here is what each tool shows. + +## What a raw `git diff` shows + +Of the **69** added/removed lines in the raw diff, **33 are pure canvas +coordinates** — nearly half the diff is noise, and it's interleaved with the real +changes so a reviewer has to hunt for them. Excerpt (repetitive `x`/`y` hunks +elided): + +```diff +@@ -11,8 +11,8 @@ + }, + "ui": { + "position": { +- "x": 20, +- "y": 500 ++ "x": 32, ++ "y": 512 + }, + "measured": { + +@@ ... 5 more identical x/y position hunks, one per node ... @@ + +@@ -80,18 +80,45 @@ + } + }, + { ++ "id": "reranker_1", ++ "provider": "reranker_cohere", ++ "config": { ++ "profile": "rerank-3", ++ "parameters": {}, ++ "topN": 5 ++ }, ++ "input": [ ++ { "lane": "documents", "from": "qdrant_1" } ++ ], ++ "ui": { ++ "position": { "x": 690, "y": 380 }, ++ "measured": { "width": 150, "height": 66 }, ++ "nodeType": "default", ++ "formDataValid": true ++ } ++ }, ++ { + "id": "prompt_1", + "provider": "prompt", + "config": { + "instructions": [ +- "Answer using only the retrieved context." ++ "Answer using only the retrieved context. Cite the source document for every claim." + ], + "parameters": {} + }, + "input": [ + { + "lane": "documents", +- "from": "qdrant_1" ++ "from": "reranker_1" + }, + +@@ -113,11 +140,11 @@ + { + "id": "llm_1", +- "provider": "llm_openai", ++ "provider": "llm_anthropic", + "config": { +- "profile": "openai-4o", +- "openai-4o": { +- "apikey": "${ROCKETRIDE_OPENAI_KEY}" ++ "profile": "claude", ++ "claude": { ++ "apikey": "${ROCKETRIDE_ANTHROPIC_KEY}" + }, + "parameters": {} + }, + +@@ ... 3 more x/y position hunks ... @@ + +@@ -168,9 +195,9 @@ + "project_id": "1327e7c0-8479-4ab7-a319-c4dc944daeb5", + "viewport": { +- "x": 0, +- "y": 0, +- "zoom": 1 ++ "x": -120, ++ "y": -40, ++ "zoom": 0.85 + }, + "version": 1 + } +``` + +To review this, you scroll past six coordinate hunks and a viewport change to find +the three lines that actually matter — and it's on you to notice that the +`prompt_1` input silently switched from `qdrant_1` to `reranker_1`. + +## What `rocketride diff` shows + +```bash +rocketride diff old.pipe new.pipe +``` + +```text +Pipeline diff: 1 node added, 2 nodes changed, 2 edges added, 1 edge removed, layout changed + +Nodes + + reranker_1 (reranker_cohere) + ~ llm_1 provider: llm_openai -> llm_anthropic + +Edges + + qdrant_1 --documents--> reranker_1 + + reranker_1 --documents--> prompt_1 + - qdrant_1 --documents--> prompt_1 + +Config + llm_1 + + config.claude = {"apikey": "${ROCKETRIDE_ANTHROPIC_KEY}"} + - config.openai-4o = {"apikey": "${ROCKETRIDE_OPENAI_KEY}"} + ~ config.profile: openai-4o -> claude + prompt_1 + ~ config.instructions[0]: Answer using only the retrieved context. -> Answer using only the retrieved context. Cite the source document for every claim. + +Layout: changed (ui/viewport) +``` + +Every real change is called out — including the rewire, shown explicitly as a +removed edge (`qdrant_1 --documents--> prompt_1`) plus two added edges through the +new reranker — and the entire canvas cleanup is one honest `Layout: changed` line. +The command exits `1` because there are semantic changes; a pure canvas move would +exit `0`. (Pass `--include-layout` if you *do* want the individual `ui.*` +coordinates enumerated.) + +## For a PR comment: `--markdown` + +`--markdown` renders the same diff as a compact, comment-safe report — the form +the CI job below posts on the pull request: + +```markdown +## Pipeline changes in rag.pipe + +**Pipeline diff:** 1 node added, 2 nodes changed, 2 edges added, 1 edge removed, layout changed + +**Nodes** +- + `reranker_1` (`reranker_cohere`) +- ~ `llm_1` provider: `llm_openai` → `llm_anthropic` + +**Edges** +- + `qdrant_1` --`documents`--> `reranker_1` +- + `reranker_1` --`documents`--> `prompt_1` +- - `qdrant_1` --`documents`--> `prompt_1` + +**Config** + +| Node | Field | Change | +| --- | --- | --- | +| `llm_1` | `config.claude` | + `{'apikey': '${ROCKETRIDE_ANTHROPIC_KEY}'}` | +| `llm_1` | `config.openai-4o` | - `{'apikey': '${ROCKETRIDE_OPENAI_KEY}'}` | +| `llm_1` | `config.profile` | `openai-4o` → `claude` | +| `prompt_1` | `config.instructions[0]` | `Answer using only the retrieved context.` → `Answer using only the retrieved context. Cite the source document for every claim.` | + +_Layout (ui/viewport) changed._ +``` + +There's also `--json`, a single stable, sorted document (`nodes`, `edges`, and a +`summary` block) for feeding other tooling. See the +[CLI reference](../docs/README-python-client.md#semantic-pipeline-diff-rocketride-diff) +for every flag and exit code. + +## Use it in CI (the pipe-diff GitHub Action) + +Because `diff` is local, network-free, and exits non-zero on change, it drops +straight into a pull-request check. The supported integration is the bundled +[`pipe-diff` composite action](../.github/actions/pipe-diff/), which finds **every** +changed `.pipe`, diffs each against the PR base, and posts one **sticky PR comment** +so reviewers see the semantic change instead of the coordinate noise: + +```yaml +# .github/workflows/pipe-diff.yml +name: Pipeline diff +on: pull_request +permissions: + contents: read + pull-requests: write +jobs: + diff: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 # default fetch-depth: 1 is fine; the action fetches the base itself + - uses: ./.github/actions/pipe-diff +``` + +The action installs the CLI, resolves the PR base commit (fetching it under the +default shallow checkout), and updates a single comment in place on re-runs. Its +[README](../.github/actions/pipe-diff/README.md) documents the `files`, +`cli-version`, `comment`, and `include-layout` inputs. + +If you would rather not vendor the composite action, the same result can be +assembled inline for a single file — check out with `fetch-depth: 0` so the base +branch is present for `--git`, then diff and post the Markdown yourself: + +```yaml +# Alternative: inline equivalent of the composite action, for a single file. + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # need the base branch for `git show` + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + - run: pip install rocketride + - name: Semantic diff of the pipeline + run: | + rocketride diff --git "origin/${{ github.base_ref }}" rag.pipe \ + --markdown --exit-zero | tee pipe-diff.md + - name: Comment on the PR + uses: marocchino/sticky-pull-request-comment@v2 + with: + path: pipe-diff.md +``` + +`--exit-zero` keeps the comment step from being skipped when changes exist. Drop +`--exit-zero` (and the sticky-comment step) if you'd rather the job **fail** the +check whenever a pipeline changes and require an explicit review. + +## Why this matters + +This is the review half of treating **pipelines as code**: **validate** a `.pipe` +to confirm it is well-formed before it ever runs, **evaluate** it to measure output +quality, and **diff** it to see exactly what changed between revisions — the same +lint / test / review loop teams already run on their source, now applied to their +AI pipelines. A reviewer approving "OpenAI → Anthropic, added a reranker, tightened +the prompt" is reviewing the pipeline; a reviewer squinting at `"x": 900` → `"x": +1120` is reviewing the canvas. `rocketride diff` makes sure they only ever have to +do the first. diff --git a/packages/client-python/src/rocketride/cli/commands/__init__.py b/packages/client-python/src/rocketride/cli/commands/__init__.py index 7edf47528..a663143b9 100644 --- a/packages/client-python/src/rocketride/cli/commands/__init__.py +++ b/packages/client-python/src/rocketride/cli/commands/__init__.py @@ -34,6 +34,7 @@ EventsCommand: Monitor real-time pipeline events ListCommand: List all active tasks StoreCommand: Project and template storage operations + DiffCommand: Semantic diff of two local .pipe pipeline files (no server) """ from .start import StartCommand @@ -43,6 +44,7 @@ from .events import EventsCommand from .list import ListCommand from .store import StoreCommand +from .diff import DiffCommand __all__ = [ 'StartCommand', @@ -52,4 +54,5 @@ 'EventsCommand', 'ListCommand', 'StoreCommand', + 'DiffCommand', ] diff --git a/packages/client-python/src/rocketride/cli/commands/diff.py b/packages/client-python/src/rocketride/cli/commands/diff.py new file mode 100644 index 000000000..2337a071f --- /dev/null +++ b/packages/client-python/src/rocketride/cli/commands/diff.py @@ -0,0 +1,263 @@ +# MIT License +# +# Copyright (c) 2026 Aparavi Software AG +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +""" +RocketRide CLI Semantic Pipeline Diff Command Implementation. + +This module provides the DiffCommand class for the ``rocketride diff`` command, +which produces a *semantic* diff of two ``.pipe`` pipeline files. Raw JSON diffs +of ``.pipe`` files are dominated by canvas coordinate churn (the per-component +``ui`` block and the top-level ``viewport``); this command hides that noise and +surfaces what actually changed: nodes added/removed, provider changes, config +field changes, and edge (wiring) additions/removals. + +Unlike every other RocketRide subcommand, ``diff`` is a purely local operation. +It reads files (or a git ref) and compares parsed JSON entirely on the client; +it never connects to the engine or the network. Consequently it takes none of +the ``--uri``/``--apikey``/``--token`` connection arguments the other commands +share -- this is a deliberate and documented difference. + +Usage: + rocketride diff + rocketride diff --git + +Flags: + --include-layout Include layout churn (component ``ui`` blocks and the + top-level ``viewport``) that is ignored by default. Version + changes are always reported regardless of this flag. + --json Emit a single JSON document to stdout. + --markdown Emit compact, PR-comment-friendly Markdown to stdout. + --exit-zero Force exit code 0 on success, even when changes are present + (useful for non-gating, informational runs). + +Exit codes: + 0 No semantic changes (or --exit-zero on any successful run). + 1 Semantic changes were found. + 2 Usage error, or an unreadable/unparseable file / bad git ref. + +Components: + DiffCommand: Main command implementation for semantic ``.pipe`` diffing +""" + +import json +import os +import sys +from typing import TYPE_CHECKING, Any, Dict, List, Optional + +from .base import BaseCommand + +# The pipediff engine is the semantic core of this command. These are the pinned +# public names from the rocketride.pipediff package (Implementer A's modules). +from ...pipediff import ( + PipeDiffError, + diff_pipes, + load_pipe, + resolve_git_ref, +) +from ...pipediff.reporters import render_human, render_json, render_markdown + +if TYPE_CHECKING: + from ..main import RocketRideClient + + +# An empty pipeline used as the "old" side when --git names a ref in which the +# file does not yet exist: everything in the working-tree file is then reported +# as newly added. +_EMPTY_PIPE: Dict[str, Any] = {'components': []} + + +def _should_use_color(stream: Any = None) -> bool: + """ + Decide whether ANSI color should be used for human-readable output. + + Color is suppressed when the ``NO_COLOR`` environment variable is present + (per the informal https://no-color.org convention, regardless of its value) + or when the target stream is not an interactive terminal. + + Args: + stream: Stream that will receive the output (defaults to ``sys.stdout``). + + Returns: + True when color escapes are appropriate, False otherwise. + """ + if stream is None: + stream = sys.stdout + if 'NO_COLOR' in os.environ: + return False + isatty = getattr(stream, 'isatty', None) + return bool(isatty()) if callable(isatty) else False + + +class DiffCommand(BaseCommand): + """ + Command implementation for the local, semantic ``.pipe`` diff. + + Load two pipeline files (or one working-tree file against a git ref), compute + a semantic diff that ignores canvas layout noise, and render the result as + colored text, JSON, or Markdown. This command performs no network I/O and + requires no authentication. + + Example: + ```python + command = DiffCommand(cli, args) + exit_code = await command.execute() + ``` + + Key Features: + - Two-file and ``--git `` comparison modes + - Layout-noise suppression with an opt-in ``--include-layout`` override + - Human / JSON / Markdown reporters, selected by flag + - Standardized exit codes (0 unchanged, 1 changed, 2 error) + - Pure stdout for report output; all errors are written to stderr + """ + + def __init__(self, cli, args): + """ + Initialize DiffCommand with CLI context and parsed arguments. + + Args: + cli: CLI instance (used only for shared plumbing; no connection is + established by this command). + args: Parsed command line arguments (paths, --git, --include-layout, + --json, --markdown, --exit-zero). + """ + super().__init__(cli, args) + + def _fail(self, message: str) -> int: + """ + Report a usage/processing error on stderr and return exit code 2. + + Errors are always written to stderr so that ``--json`` and ``--markdown`` + output on stdout stays pure and machine-parseable. + + Args: + message: Human-readable error description. + + Returns: + The integer exit code ``2``. + """ + print(f'Error: {message}', file=sys.stderr) + return 2 + + def _resolve_inputs(self) -> Optional[tuple]: + """ + Validate arguments and load the (old, new) pipeline objects. + + Returns: + A ``(old, new)`` tuple of parsed pipeline dicts on success, or + ``None`` when validation or loading failed (an error has already been + printed to stderr by the caller path via raised exceptions). + + Raises: + PipeDiffError: Propagated from load_pipe / resolve_git_ref for + unreadable, unparseable, or structurally invalid inputs, or for a + bad git ref. The caller converts these into exit code 2. + ValueError: For argument-usage problems (wrong number of paths for the + selected mode). The caller converts these into exit code 2. + """ + paths: List[str] = list(getattr(self.args, 'paths', None) or []) + git_ref: Optional[str] = getattr(self.args, 'git', None) + + if git_ref: + if len(paths) != 1: + raise ValueError('--git requires exactly one FILE to compare against the ref') + file_path = paths[0] + new_obj = load_pipe(file_path) + old_obj = resolve_git_ref(git_ref, file_path) + if old_obj is None: + # File absent in the ref: treat everything as newly added. + old_obj = _EMPTY_PIPE + return old_obj, new_obj + + if len(paths) != 2: + raise ValueError('exactly two files are required: rocketride diff ') + + old_obj = load_pipe(paths[0]) + new_obj = load_pipe(paths[1]) + return old_obj, new_obj + + def _render(self, diff: Any) -> str: + """ + Render the diff using the reporter selected by the command flags. + + ``--json`` takes precedence over ``--markdown`` when both are somehow set; + argparse normally makes them mutually exclusive. With no format flag the + colored human report is produced (color auto-detected from stdout). + + Args: + diff: The PipeDiff produced by the engine. + + Returns: + The fully rendered report string for printing to stdout. + """ + if getattr(self.args, 'json', False): + return json.dumps(render_json(diff), indent=2, ensure_ascii=False, sort_keys=True) + if getattr(self.args, 'markdown', False): + return render_markdown(diff) + return render_human(diff, use_color=_should_use_color(sys.stdout)) + + async def execute(self, client: 'RocketRideClient' = None) -> int: + """ + Execute the semantic pipe diff and return the appropriate exit code. + + This command is fully local: the ``client`` argument is accepted only to + match the common command interface and is never used. + + Args: + client: Unused. Present for signature compatibility with other + commands dispatched by the CLI. + + Returns: + Exit code per the command contract: + - 0 when there are no semantic changes, or when ``--exit-zero`` + was passed and the run otherwise succeeded. + - 1 when semantic changes were found. + - 2 on a usage error or an unreadable/unparseable input. + + Process Flow: + 1. Validate arguments and load the old/new pipeline objects. + 2. Compute the semantic diff (respecting --include-layout). + 3. Render with the selected reporter and print to stdout. + 4. Map the diff outcome to an exit code (honoring --exit-zero). + """ + include_layout = bool(getattr(self.args, 'include_layout', False)) + + try: + resolved = self._resolve_inputs() + except ValueError as exc: + return self._fail(str(exc)) + except PipeDiffError as exc: + return self._fail(str(exc)) + + old_obj, new_obj = resolved + + try: + diff = diff_pipes(old_obj, new_obj, include_layout=include_layout) + except PipeDiffError as exc: + return self._fail(str(exc)) + + # Report output goes to stdout only; nothing above this point wrote there. + print(self._render(diff)) + + if getattr(self.args, 'exit_zero', False): + return 0 + return 1 if diff.has_semantic_changes else 0 diff --git a/packages/client-python/src/rocketride/cli/main.py b/packages/client-python/src/rocketride/cli/main.py index daff72ed6..35866cde2 100644 --- a/packages/client-python/src/rocketride/cli/main.py +++ b/packages/client-python/src/rocketride/cli/main.py @@ -67,6 +67,7 @@ from .commands.events import EventsCommand from .commands.list import ListCommand from .commands.store import StoreCommand +from .commands.diff import DiffCommand try: # Try importing from installed package first @@ -508,6 +509,58 @@ def add_common_args(subparser): ) stat_parser.add_argument('path', help='File or directory path') + # Diff command - semantic diff of two .pipe pipeline files. + # + # This command is intentionally *local only*: it never contacts the engine + # or the network, so it does NOT take the shared --uri/--apikey/--token + # connection arguments (add_common_args) that every other command uses. + diff_parser = subparsers.add_parser( + 'diff', + help='Semantic diff of two .pipe pipeline files (local; no server)', + description=( + 'Compare two RocketRide .pipe pipeline files semantically, surfacing node, ' + 'edge, and config changes while ignoring canvas layout noise (the per-node ' + '"ui" block and top-level "viewport"). This command runs entirely locally ' + 'and never connects to the engine or network, so it takes no ' + '--uri/--apikey/--token arguments.' + ), + ) + diff_parser.add_argument( + 'paths', + nargs='*', + metavar='FILE', + help='Two pipe files to compare (old new), or a single FILE when using --git', + ) + diff_parser.add_argument( + '--git', + metavar='REF', + help='Diff the working-tree FILE against this git ref (via "git show REF:FILE")', + ) + diff_parser.add_argument( + '--include-layout', + action='store_true', + help='Include layout churn (per-node "ui" blocks and top-level "viewport") ignored by default', + ) + + # --json and --markdown select mutually exclusive output formats. + diff_format_group = diff_parser.add_mutually_exclusive_group() + diff_format_group.add_argument( + '--json', + action='store_true', + help='Emit the diff as a single JSON document to stdout', + ) + diff_format_group.add_argument( + '--markdown', + action='store_true', + help='Emit the diff as compact, PR-comment-friendly Markdown to stdout', + ) + + diff_parser.add_argument( + '--exit-zero', + action='store_true', + help='Always exit 0 on a successful run, even when changes are found (non-gating)', + ) + return parser async def run(self) -> int: @@ -537,6 +590,13 @@ async def run(self) -> int: parser.print_help() return 1 + # Diff command is fully local: no server connection, no auth, no client. + # Dispatch it here, before any connection setup, since it does not carry + # the shared --uri/--apikey/--token arguments the other commands rely on. + if self.args.command == 'diff': + self.command = DiffCommand(self, self.args) + return await self.command.execute() + # Validate we have something for apikey if not self.args.apikey: self.args.apikey = '' diff --git a/packages/client-python/src/rocketride/pipediff/__init__.py b/packages/client-python/src/rocketride/pipediff/__init__.py new file mode 100644 index 000000000..567b1c594 --- /dev/null +++ b/packages/client-python/src/rocketride/pipediff/__init__.py @@ -0,0 +1,67 @@ +# MIT License +# +# Copyright (c) 2026 Aparavi Software AG +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +""" +Semantic diff for RocketRide ``.pipe`` pipeline files. + +``rocketride diff`` surfaces what actually changed between two pipeline +definitions — added/removed/reconfigured nodes and rewired edges — instead of the +raw JSON churn dominated by canvas coordinates. This package is the pure, +network-free implementation behind the CLI subcommand and the PR-comment GitHub +Action. + +Public API: + Data model: + NodeChange, FieldChange, EdgeChange, PipeDiff + Engine: + load_pipe, diff_pipes, deep_diff_config, PipeDiffError + Git resolution: + resolve_git_ref + Reporters (rendering; provided alongside this module): + render_human, render_json, render_markdown +""" + +from .engine import PipeDiffError, deep_diff_config, diff_pipes, load_pipe +from .gitref import resolve_git_ref +from .model import EdgeChange, FieldChange, NodeChange, PipeDiff + +__all__ = [ + 'NodeChange', + 'FieldChange', + 'EdgeChange', + 'PipeDiff', + 'PipeDiffError', + 'load_pipe', + 'diff_pipes', + 'deep_diff_config', + 'resolve_git_ref', +] + +# Reporters live in the sibling ``reporters`` module (implemented in parallel). +# Import them opportunistically so the public surface is complete once they land, +# without making this package unimportable while they are still in progress. +try: + from .reporters import render_human, render_json, render_markdown # noqa: F401 +except ImportError: + pass +else: + __all__ += ['render_human', 'render_json', 'render_markdown'] diff --git a/packages/client-python/src/rocketride/pipediff/engine.py b/packages/client-python/src/rocketride/pipediff/engine.py new file mode 100644 index 000000000..e05b55bd1 --- /dev/null +++ b/packages/client-python/src/rocketride/pipediff/engine.py @@ -0,0 +1,449 @@ +# MIT License +# +# Copyright (c) 2026 Aparavi Software AG +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +""" +The semantic diff engine for RocketRide ``.pipe`` pipeline files. + +This module is the pure, network-free core of ``rocketride diff``. It loads and +validates pipeline JSON, then computes the semantic difference between two +pipelines: added/removed/re-provisioned/reconfigured nodes, added/removed edges, +version changes, and a coarse layout-changed flag. Nothing here touches the +RocketRide engine, a server, or the network — it is deliberately restricted to +local file and in-memory JSON work. + +Semantic model (see :mod:`rocketride.pipediff.model` for the shapes): + - **Nodes** are matched by their ``id``. Provider (node-type) changes and + deep config changes are reported independently. + - **Config** is deep-diffed into readable dotted paths (``config.a.b[0]``), + handling nested dictionaries and lists. + - **Edges** are the set of directed wires reconstructed from every + component's ``input[]`` (data lanes) and ``control[]`` (agent orchestration + lanes); added and removed wires are reported as a set difference. + - **Layout** — each component's ``ui`` block and the top-level ``viewport`` + are ignored by default and summarised by ``layout_changed``. When + ``include_layout`` is set, ``ui`` differences are additionally enumerated + as ``ui.*`` field changes on the affected node. + +Functions: + load_pipe: Read and validate a pipe from a path or an already-parsed dict. + deep_diff_config: Deep-diff two config dicts into a list of ``FieldChange``. + diff_pipes: Compute the full ``PipeDiff`` between two pipeline dicts. + +Exceptions: + PipeDiffError: Raised for unreadable, unparseable, or structurally invalid + pipelines, with human-readable context about the offending source. +""" + +from __future__ import annotations + +import json +import os +from typing import Any + +from .model import EdgeChange, FieldChange, NodeChange, PipeDiff + + +class PipeDiffError(Exception): + """ + Raised when a ``.pipe`` file cannot be read, parsed, or validated. + + The message always carries enough context (the source path or ```` plus + the specific problem) for the CLI to print an actionable usage error and exit + with code ``2``. It is used both by :func:`load_pipe` (file/JSON/shape errors) + and by :func:`rocketride.pipediff.gitref.resolve_git_ref` (git/parse errors). + """ + + +def load_pipe(path_or_obj: str | os.PathLike[str] | dict) -> dict: + """ + Load and validate a pipeline definition from a path or a parsed object. + + Accepts either a filesystem path (``str`` or ``os.PathLike``) pointing at a + ``.pipe`` JSON file, or an already-parsed ``dict`` (for example, the object + returned by resolving a git ref). In both cases the result is validated to be + a JSON object containing a ``components`` list of id-bearing objects. + + Args: + path_or_obj: A path to a ``.pipe`` file, or a parsed pipeline ``dict``. + + Returns: + The validated pipeline as a ``dict`` (the same object when a dict is + passed in; a freshly parsed object when a path is passed in). + + Raises: + PipeDiffError: If the file cannot be read, the JSON cannot be parsed, the + top level is not an object, ``components`` is missing or not a list, + or any component is not an object with a non-empty string ``id``. + """ + if isinstance(path_or_obj, dict): + return _validate_pipe(path_or_obj, '') + + if isinstance(path_or_obj, (str, os.PathLike)): + source = os.fspath(path_or_obj) + try: + with open(source, encoding='utf-8') as handle: + text = handle.read() + except FileNotFoundError as exc: + raise PipeDiffError(f'Pipe file not found: {source}') from exc + except OSError as exc: + raise PipeDiffError(f'Could not read pipe file {source}: {exc}') from exc + try: + obj = json.loads(text) + except json.JSONDecodeError as exc: + raise PipeDiffError(f'Invalid JSON in {source}: {exc}') from exc + return _validate_pipe(obj, source) + + raise PipeDiffError(f'load_pipe expects a path or dict, got {type(path_or_obj).__name__}') + + +def _validate_pipe(obj: Any, source: str) -> dict: + """ + Validate that ``obj`` is a well-formed pipeline object and return it. + + Args: + obj: The candidate pipeline (any JSON-decoded value). + source: A label for error messages (a path, or ``""``). + + Returns: + The validated pipeline ``dict`` (``obj`` unchanged). + + Raises: + PipeDiffError: If ``obj`` is not an object, lacks a ``components`` list, + or contains a component that is not an id-bearing object. + """ + if not isinstance(obj, dict): + raise PipeDiffError(f'{source}: pipe must be a JSON object, got {type(obj).__name__}') + components = obj.get('components') + if not isinstance(components, list): + raise PipeDiffError(f"{source}: pipe is missing a 'components' list") + for index, component in enumerate(components): + if not isinstance(component, dict): + raise PipeDiffError(f'{source}: component at index {index} is not an object') + component_id = component.get('id') + if not isinstance(component_id, str) or not component_id: + raise PipeDiffError(f"{source}: component at index {index} is missing a string 'id'") + return obj + + +def deep_diff_config(old: dict | None, new: dict | None) -> list[FieldChange]: + """ + Deep-diff two component config dictionaries into dotted-path field changes. + + Recursively compares nested dictionaries and lists, producing one + :class:`FieldChange` per differing leaf. Dict keys extend the path with + ``.key``; list elements extend it with ``[index]``. List differences are + reported index-by-index, with trailing extra elements reported as added or + removed. Every path is prefixed with ``config`` so it reads as a fully + qualified path (for example ``config.instructions[0]`` or + ``config.default.strlen``). + + Args: + old: The previous config dict (``None`` is treated as an empty dict). + new: The new config dict (``None`` is treated as an empty dict). + + Returns: + The list of field changes, ordered deterministically by key/index. Empty + when the two configs are equal. + """ + return _diff_value(old if old is not None else {}, new if new is not None else {}, 'config') + + +def diff_pipes(old: dict, new: dict, *, include_layout: bool = False) -> PipeDiff: + """ + Compute the semantic difference between two pipeline dictionaries. + + Nodes are matched by ``id``. For each matched node the provider and config + are compared independently; unmatched nodes become added/removed changes. + Edges are compared as a set of directed ``(from, lane, to)`` triples drawn + from every component's ``input[]`` and ``control[]`` wiring. The top-level + ``version`` change is always reported. Canvas layout (each ``ui`` block and + the ``viewport``) is summarised by ``layout_changed`` and, when + ``include_layout`` is set, enumerated as ``ui.*`` field changes per node. + + Args: + old: The previous pipeline (already loaded/validated via ``load_pipe``). + new: The new pipeline (already loaded/validated via ``load_pipe``). + include_layout: When ``True``, fold each node's ``ui`` differences into + its field changes (paths prefixed ``ui``) so canvas churn is opted + back into the reported changes. The ``layout_changed`` flag is + computed either way. + + Returns: + A :class:`PipeDiff` describing every node, edge, version, and layout + difference between ``old`` and ``new``. + """ + old_components = _components_by_id(old) + new_components = _components_by_id(new) + old_ids = set(old_components) + new_ids = set(new_components) + + node_changes: list[NodeChange] = [] + + for component_id in sorted(new_ids - old_ids): + node_changes.append( + NodeChange( + id=component_id, + kind='added', + provider_new=new_components[component_id].get('provider'), + ) + ) + for component_id in sorted(old_ids - new_ids): + node_changes.append( + NodeChange( + id=component_id, + kind='removed', + provider_old=old_components[component_id].get('provider'), + ) + ) + for component_id in sorted(old_ids & new_ids): + old_component = old_components[component_id] + new_component = new_components[component_id] + + provider_old = old_component.get('provider') + provider_new = new_component.get('provider') + if provider_old != provider_new: + node_changes.append( + NodeChange( + id=component_id, + kind='provider', + provider_old=provider_old, + provider_new=provider_new, + ) + ) + + field_changes = deep_diff_config(old_component.get('config'), new_component.get('config')) + if include_layout: + field_changes = field_changes + _diff_value( + old_component.get('ui') or {}, new_component.get('ui') or {}, 'ui' + ) + if field_changes: + node_changes.append(NodeChange(id=component_id, kind='config', field_changes=field_changes)) + + old_edges = _extract_edges(old) + new_edges = _extract_edges(new) + edge_changes: list[EdgeChange] = [] + for edge in sorted(new_edges - old_edges, key=_edge_sort_key): + edge_changes.append(EdgeChange(from_id=edge[0], lane=edge[1], to_id=edge[2], kind='added')) + for edge in sorted(old_edges - new_edges, key=_edge_sort_key): + edge_changes.append(EdgeChange(from_id=edge[0], lane=edge[1], to_id=edge[2], kind='removed')) + + version_change: tuple[Any, Any] | None = None + old_version = old.get('version') + new_version = new.get('version') + if old_version != new_version: + version_change = (old_version, new_version) + + layout_changed = _layout_changed(old, new, old_components, new_components, old_ids & new_ids) + + return PipeDiff( + node_changes=node_changes, + edge_changes=edge_changes, + version_change=version_change, + layout_changed=layout_changed, + ) + + +# --------------------------------------------------------------------------- +# Internal helpers +# --------------------------------------------------------------------------- + + +def _components_by_id(pipe: dict) -> dict[str, dict]: + """ + Index a pipeline's components by their ``id``. + + Non-object components and components without an ``id`` are skipped defensively + (``load_pipe`` normally rejects those upstream). If duplicate ids appear, the + last one wins. + + Args: + pipe: A pipeline dict. + + Returns: + A mapping of component id to the component object. + """ + indexed: dict[str, dict] = {} + for component in pipe.get('components', []) or []: + if isinstance(component, dict) and isinstance(component.get('id'), str): + indexed[component['id']] = component + return indexed + + +def _extract_edges(pipe: dict) -> set[tuple[Any, Any, Any]]: + """ + Reconstruct the set of directed wires in a pipeline. + + Each component's ``input[]`` entries become data edges + ``(input.from, input.lane, component.id)`` and its ``control[]`` entries + become orchestration edges ``(control.from, control.classType, component.id)``. + Control edges are included because agent workflows express their + agent-to-llm/tool/memory wiring exclusively through ``control[]``; ignoring + them would make the diff blind to agent reconfiguration. + + Args: + pipe: A pipeline dict. + + Returns: + A set of ``(from, lane, to)`` triples. + """ + edges: set[tuple[Any, Any, Any]] = set() + for component in pipe.get('components', []) or []: + if not isinstance(component, dict): + continue + to_id = component.get('id') + for wire in component.get('input') or []: + if isinstance(wire, dict): + edges.add((wire.get('from'), wire.get('lane'), to_id)) + for wire in component.get('control') or []: + if isinstance(wire, dict): + edges.add((wire.get('from'), wire.get('classType'), to_id)) + return edges + + +def _edge_sort_key(edge: tuple[Any, Any, Any]) -> tuple[str, str, str]: + """ + Provide a total, deterministic ordering for edge triples. + + Coerces each element to ``str`` so triples containing ``None`` (malformed + wires missing a ``from``/``lane``) still sort without raising. + + Args: + edge: A ``(from, lane, to)`` triple. + + Returns: + A ``(str, str, str)`` sort key. + """ + return (str(edge[0]), str(edge[1]), str(edge[2])) + + +def _layout_changed( + old: dict, + new: dict, + old_components: dict[str, dict], + new_components: dict[str, dict], + common_ids: set[str], +) -> bool: + """ + Determine whether any pure-layout data differs between two pipelines. + + Layout is the top-level ``viewport`` plus each retained component's ``ui`` + block. Added/removed nodes are excluded: their appearance/disappearance is + already a semantic change, so counting their ``ui`` here would be redundant + noise. This flag is a coarse hint and never gates the exit code by itself. + + Args: + old: The previous pipeline dict. + new: The new pipeline dict. + old_components: The old components indexed by id. + new_components: The new components indexed by id. + common_ids: Ids present in both pipelines. + + Returns: + ``True`` if the viewport or any retained node's ui block differs. + """ + if old.get('viewport') != new.get('viewport'): + return True + for component_id in common_ids: + if old_components[component_id].get('ui') != new_components[component_id].get('ui'): + return True + return False + + +def _diff_value(old: Any, new: Any, path: str) -> list[FieldChange]: + """ + Recursively diff two arbitrary JSON values at ``path``. + + Dispatches on type: two dicts are diffed key-wise, two lists index-wise, and + anything else is compared for equality. This is the shared engine behind both + the config diff and (when layout is opted in) the ui diff. + + Args: + old: The previous value. + new: The new value. + path: The dotted path accumulated so far for this position. + + Returns: + The field changes at or beneath ``path``. Empty when equal. + """ + if isinstance(old, dict) and isinstance(new, dict): + return _diff_mapping(old, new, path) + if isinstance(old, list) and isinstance(new, list): + return _diff_sequence(old, new, path) + if old != new: + return [FieldChange(path=path, kind='changed', old=old, new=new)] + return [] + + +def _diff_mapping(old: dict, new: dict, prefix: str) -> list[FieldChange]: + """ + Diff two dictionaries, extending ``prefix`` with ``.key`` per entry. + + Keys are visited in sorted order so output is deterministic regardless of the + input insertion order. + + Args: + old: The previous dict. + new: The new dict. + prefix: The dotted path of the dict itself. + + Returns: + The field changes for every added, removed, or changed key. + """ + changes: list[FieldChange] = [] + for key in sorted(set(old) | set(new), key=str): + child_path = f'{prefix}.{key}' + in_old = key in old + in_new = key in new + if in_old and not in_new: + changes.append(FieldChange(path=child_path, kind='removed', old=old[key], new=None)) + elif in_new and not in_old: + changes.append(FieldChange(path=child_path, kind='added', old=None, new=new[key])) + else: + changes.extend(_diff_value(old[key], new[key], child_path)) + return changes + + +def _diff_sequence(old: list, new: list, prefix: str) -> list[FieldChange]: + """ + Diff two lists positionally, extending ``prefix`` with ``[index]``. + + Elements at shared indices are diffed recursively (so nested dict/list + elements produce fine-grained paths). Trailing elements present on only one + side are reported as added or removed at their index. + + Args: + old: The previous list. + new: The new list. + prefix: The dotted path of the list itself. + + Returns: + The field changes across all indices. + """ + changes: list[FieldChange] = [] + shared = min(len(old), len(new)) + for index in range(shared): + changes.extend(_diff_value(old[index], new[index], f'{prefix}[{index}]')) + for index in range(shared, len(new)): + changes.append(FieldChange(path=f'{prefix}[{index}]', kind='added', old=None, new=new[index])) + for index in range(shared, len(old)): + changes.append(FieldChange(path=f'{prefix}[{index}]', kind='removed', old=old[index], new=None)) + return changes diff --git a/packages/client-python/src/rocketride/pipediff/gitref.py b/packages/client-python/src/rocketride/pipediff/gitref.py new file mode 100644 index 000000000..bdd9711b0 --- /dev/null +++ b/packages/client-python/src/rocketride/pipediff/gitref.py @@ -0,0 +1,159 @@ +# MIT License +# +# Copyright (c) 2026 Aparavi Software AG +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +""" +Git ref resolution for ``rocketride diff --git ``. + +This module reads a pipeline file's contents at a given git ref so the working +tree can be diffed against history (for example ``rocketride diff --git HEAD +pipeline.pipe``). It shells out to ``git show`` via :mod:`subprocess` with an +argument list — never a shell string — so pipeline paths and refs cannot inject +shell commands. + +Like the rest of :mod:`rocketride.pipediff`, this never contacts the RocketRide +engine or the network; ``git`` is the only external process invoked. + +Functions: + resolve_git_ref: Return the parsed pipeline at ``ref``, or ``None`` if the + file does not exist in that ref (treated by the caller as an all-added + pipeline). +""" + +from __future__ import annotations + +import json +import os +import subprocess + +from .engine import PipeDiffError, load_pipe + +# Upper bound so a wedged git process can never hang the CLI indefinitely. +_GIT_TIMEOUT_SECONDS = 30 + +# Substrings git uses when a ref is valid but the path is absent from it. These +# distinguish "file not in this ref" (return None) from a genuine failure such as +# an unknown revision (raise PipeDiffError). +_PATH_ABSENT_MARKERS = ( + 'does not exist in', + 'exists on disk, but not in', +) + + +def resolve_git_ref(ref: str, file_path: str) -> dict | None: + """ + Load a pipeline file's contents at a specific git ref. + + Resolves ``file_path`` to its repository-relative location and runs + ``git show :`` to retrieve the file as it existed at + ``ref``, then parses and validates the result. Used to diff a working-tree + ``.pipe`` file against an arbitrary commit, branch, or tag. + + Args: + ref: A git revision (commit sha, branch, tag, ``HEAD``, ``HEAD~1``, ...). + file_path: Path to the working-tree ``.pipe`` file (absolute or relative + to the current working directory). + + Returns: + The parsed and validated pipeline ``dict`` at ``ref``, or ``None`` if the + file does not exist in ``ref`` (the caller treats this as "everything is + new"). + + Raises: + PipeDiffError: If the path is not inside a git repository, the ref is + unknown, ``git`` is unavailable or times out, or the retrieved + contents are not a valid pipeline. + """ + # realpath (not just abspath) so the file resolves to the same physical path + # that ``git rev-parse --show-toplevel`` reports, keeping relpath consistent + # even when the repo is reached through a symlinked directory. + abs_path = os.path.realpath(file_path) + file_dir = os.path.dirname(abs_path) or '.' + + repo_root = _repository_root(file_dir, file_path) + rel_path = os.path.relpath(abs_path, repo_root).replace(os.sep, '/') + spec = f'{ref}:{rel_path}' + + result = _run_git(['-C', repo_root, 'show', spec], f'reading {spec}') + if result.returncode != 0: + stderr = result.stderr.strip() + if any(marker in stderr for marker in _PATH_ABSENT_MARKERS): + return None + raise PipeDiffError(f'git show {spec} failed: {stderr}') + + try: + obj = json.loads(result.stdout) + except json.JSONDecodeError as exc: + raise PipeDiffError(f'Invalid JSON in {spec}: {exc}') from exc + return load_pipe(obj) + + +def _repository_root(file_dir: str, file_path: str) -> str: + """ + Resolve the git repository root that contains ``file_dir``. + + Args: + file_dir: A directory inside the repository (the file's parent). + file_path: The original path, used only for error context. + + Returns: + The absolute path to the repository top level. + + Raises: + PipeDiffError: If ``file_dir`` is not inside a git repository (or git + otherwise fails to resolve the top level). + """ + result = _run_git( + ['-C', file_dir, 'rev-parse', '--show-toplevel'], + f'resolving repository for {file_path}', + ) + if result.returncode != 0: + raise PipeDiffError(f'Not a git repository (or unable to resolve it) for {file_path}: {result.stderr.strip()}') + return result.stdout.strip() + + +def _run_git(args: list[str], context: str) -> subprocess.CompletedProcess: + """ + Run a ``git`` subcommand with a fixed argument list and no shell. + + Args: + args: Arguments passed to ``git`` (already split; never a shell string). + context: A short description of the operation for error messages. + + Returns: + The completed process (callers inspect ``returncode``/``stdout``/ + ``stderr``). + + Raises: + PipeDiffError: If ``git`` is not found on ``PATH`` or the call times out. + """ + try: + return subprocess.run( + ['git', *args], + capture_output=True, + text=True, + encoding='utf-8', + timeout=_GIT_TIMEOUT_SECONDS, + ) + except FileNotFoundError as exc: + raise PipeDiffError('git executable not found; --git requires git on PATH') from exc + except subprocess.TimeoutExpired as exc: + raise PipeDiffError(f'git timed out while {context}') from exc diff --git a/packages/client-python/src/rocketride/pipediff/model.py b/packages/client-python/src/rocketride/pipediff/model.py new file mode 100644 index 000000000..ffa2446a1 --- /dev/null +++ b/packages/client-python/src/rocketride/pipediff/model.py @@ -0,0 +1,182 @@ +# MIT License +# +# Copyright (c) 2026 Aparavi Software AG +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +""" +Semantic diff data model for RocketRide ``.pipe`` pipeline files. + +This module defines the immutable-ish value objects that describe *what changed* +between two pipeline definitions, independent of how the change is rendered. The +engine (:mod:`rocketride.pipediff.engine`) produces these objects and the +reporters (:mod:`rocketride.pipediff.reporters`) consume them, so the shapes here +are the contract between the two halves of the ``rocketride diff`` feature. + +The model deliberately mirrors the three semantic axes of a pipeline: + - **Nodes** — the components on the canvas (``NodeChange``). + - **Edges** — the directed wiring between components (``EdgeChange``). + - **Config** — the per-node settings, expressed as dotted field paths + (``FieldChange``, nested inside a ``NodeChange``). + +Canvas layout (each component's ``ui`` block and the top-level ``viewport``) is +*not* enumerated field-by-field by default; it is summarised by +``PipeDiff.layout_changed`` so that coordinate churn never drowns out real +changes. + +Classes: + FieldChange: A single dotted-path change within a component's config/ui. + NodeChange: An added, removed, re-provisioned, or reconfigured component. + EdgeChange: An added or removed directed wire between two components. + PipeDiff: The complete semantic difference between two pipelines. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Any, Literal, Optional + +# Kind discriminators, kept as module-level aliases so the engine, reporters, +# and tests all refer to the same closed vocabularies. +FieldChangeKind = Literal['added', 'removed', 'changed'] +NodeChangeKind = Literal['added', 'removed', 'provider', 'config'] +EdgeChangeKind = Literal['added', 'removed'] + + +@dataclass(frozen=True) +class FieldChange: + """ + A single field-level change inside a component, keyed by a dotted path. + + Field changes are produced by the config deep-diff and (when layout is opted + in) the ui deep-diff. Paths are fully qualified and human-readable, e.g. + ``config.default.strlen``, ``config.instructions[0]``, or ``ui.position.x``. + List elements are addressed with ``[index]`` suffixes. + + Attributes: + path: Dotted path to the changed value, relative to the component + (prefixed with ``config`` or ``ui`` so it reads as an absolute path). + kind: ``"added"`` (present only in the new pipe), ``"removed"`` (present + only in the old pipe), or ``"changed"`` (present in both, differing). + old: The previous value, or ``None`` for an ``"added"`` change. + new: The new value, or ``None`` for a ``"removed"`` change. + """ + + path: str + kind: FieldChangeKind + old: Any = None + new: Any = None + + +@dataclass +class NodeChange: + """ + A change to a single pipeline component (node), identified by its ``id``. + + A component that exists on both sides may yield *two* ``NodeChange`` entries: + one with ``kind="provider"`` if its provider (node type) changed, and one + with ``kind="config"`` if its settings changed. This keeps ``kind`` a single + closed value while still expressing both facts. + + Attributes: + id: The component id (stable identity used to match nodes across pipes). + kind: The nature of the change: + - ``"added"``: node exists only in the new pipe (``provider_new`` set). + - ``"removed"``: node exists only in the old pipe (``provider_old`` set). + - ``"provider"``: node exists in both; its provider changed + (both ``provider_old`` and ``provider_new`` set). + - ``"config"``: node exists in both; one or more fields changed + (``field_changes`` non-empty). With layout opted in, ``field_changes`` + may include ``ui.*`` paths in addition to ``config.*`` paths. + provider_old: The component's provider before the change, when relevant. + provider_new: The component's provider after the change, when relevant. + field_changes: The dotted-path field changes for a ``"config"`` change. + """ + + id: str + kind: NodeChangeKind + provider_old: Optional[str] = None + provider_new: Optional[str] = None + field_changes: list[FieldChange] = field(default_factory=list) + + +@dataclass(frozen=True) +class EdgeChange: + """ + An added or removed directed wire between two components. + + Edges are reconstructed from each component's ``input[]`` lanes (data wiring) + and ``control[]`` entries (agent orchestration wiring). Both are directed + ``from`` a source component ``to`` the component that declares them, labelled + by a lane: the ``lane`` string for data edges, the ``classType`` string for + control edges (e.g. ``"llm"``, ``"tool"``, ``"memory"``). + + Attributes: + from_id: The source component id (the ``from`` value on the wire). + lane: The lane label (data ``lane`` or control ``classType``). + to_id: The destination component id (the component declaring the wire). + kind: ``"added"`` (only in the new pipe) or ``"removed"`` (only in the old). + """ + + from_id: str + lane: str + to_id: str + kind: EdgeChangeKind + + +@dataclass +class PipeDiff: + """ + The complete semantic difference between two ``.pipe`` pipelines. + + This is the single object handed to every reporter. It separates true + semantic change (nodes, edges, version) from cosmetic canvas churn + (``layout_changed``), which is the entire point of ``rocketride diff``. + + Attributes: + node_changes: All component-level changes, ordered deterministically + (added, then removed, then per-id provider/config changes). + edge_changes: All wiring changes (added edges first, then removed). + version_change: ``(old_version, new_version)`` when the top-level + ``version`` field differs, else ``None``. Always computed regardless + of layout options, since a version bump is semantically meaningful. + layout_changed: ``True`` when the top-level ``viewport`` or any retained + component's ``ui`` block differs. This is a coarse hint only; layout + details are enumerated as ``ui.*`` field changes only when the caller + opts layout in. + """ + + node_changes: list[NodeChange] = field(default_factory=list) + edge_changes: list[EdgeChange] = field(default_factory=list) + version_change: Optional[tuple[Any, Any]] = None + layout_changed: bool = False + + @property + def has_semantic_changes(self) -> bool: + """ + Whether this diff contains any change that should gate a non-zero exit. + + Returns ``True`` when there are node changes, edge changes, or a version + change. Pure layout churn (``layout_changed`` with no other change) does + **not** count as semantic, so a canvas-only move exits ``0``. + + Returns: + ``True`` if any semantic (non-layout) change is present. + """ + return bool(self.node_changes or self.edge_changes or self.version_change is not None) diff --git a/packages/client-python/src/rocketride/pipediff/reporters.py b/packages/client-python/src/rocketride/pipediff/reporters.py new file mode 100644 index 000000000..487d3ea6d --- /dev/null +++ b/packages/client-python/src/rocketride/pipediff/reporters.py @@ -0,0 +1,613 @@ +# MIT License +# +# Copyright (c) 2026 Aparavi Software AG +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +""" +Rendering utilities for semantic ``.pipe`` pipeline diffs. + +This module turns a :class:`~rocketride.pipediff.model.PipeDiff` (produced by the +pipediff engine) into three presentation formats used by the ``rocketride diff`` +command: + + - ``render_human`` : colored, section-grouped text for interactive terminals + - ``render_json`` : a single JSON-serializable document for tooling + - ``render_markdown`` : compact, PR-comment-friendly Markdown for the GitHub Action + +The renderers share a single normalization pass (``_organize``) so that every +format reports the same set of changes in the same deterministic order, +regardless of the order in which the engine emitted them. This determinism keeps +JSON output stable for snapshot tests and keeps PR comments from churning. + +Design notes: + - The renderers are intentionally *read-only* over the diff model: they never + construct model objects, so they depend on the model dataclasses for typing + only (guarded by ``TYPE_CHECKING``). This keeps the pipediff library free of + import-time coupling between the reporters and the engine, and lets the + module load even while sibling modules are still being written. + - ANSI color constants are defined locally rather than imported from + ``rocketride.cli.ui.colors``. The pipediff package is a library that the CLI + depends on; importing back into the CLI would invert that layering. The + constants below mirror the values in ``rocketride/cli/ui/colors.py``. + - Callers decide whether color is appropriate (TTY detection, ``NO_COLOR``) + and pass the result via ``use_color``; the renderer itself never inspects + the environment. + +Components: + render_human: Human-readable, optionally colored diff report + render_json: JSON-serializable diff document with a summary block + render_markdown: Compact Markdown diff suitable for embedding in PR comments +""" + +from __future__ import annotations + +import json +import re +from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple + +if TYPE_CHECKING: # pragma: no cover - typing-only imports + from .model import PipeDiff + + +# ANSI color codes. These mirror rocketride/cli/ui/colors.py; they are duplicated +# here so the pipediff library does not import from the CLI package (which would +# invert the dependency direction, since the CLI depends on pipediff). +_ANSI_RESET = '\033[0m' +_ANSI_RED = '\033[91m' +_ANSI_GREEN = '\033[92m' +_ANSI_YELLOW = '\033[93m' +_ANSI_GRAY = '\033[90m' + +# Change markers shared across all output formats. +_MARK_ADDED = '+' +_MARK_REMOVED = '-' +_MARK_CHANGED = '~' + +# Markdown marker glyphs (kept ASCII-safe and unambiguous in PR comments). +_MD_ADDED = '+' +_MD_REMOVED = '-' +_MD_CHANGED = '~' + + +class _Palette: + """ + Small helper that applies (or suppresses) ANSI color per change kind. + + When ``use_color`` is false every method returns its argument unchanged, so + the same rendering code path produces plain text for non-TTY output and for + ``NO_COLOR`` environments without branching at each call site. + """ + + def __init__(self, use_color: bool): + """Store whether color escapes should be emitted.""" + self._on = bool(use_color) + + def _wrap(self, code: str, text: str) -> str: + """Wrap ``text`` in an ANSI ``code``/reset pair when color is enabled.""" + if not self._on: + return text + return f'{code}{text}{_ANSI_RESET}' + + def added(self, text: str) -> str: + """Color ``text`` as an addition (green).""" + return self._wrap(_ANSI_GREEN, text) + + def removed(self, text: str) -> str: + """Color ``text`` as a removal (red).""" + return self._wrap(_ANSI_RED, text) + + def changed(self, text: str) -> str: + """Color ``text`` as a modification (yellow).""" + return self._wrap(_ANSI_YELLOW, text) + + def dim(self, text: str) -> str: + """Color ``text`` as secondary/dim (gray).""" + return self._wrap(_ANSI_GRAY, text) + + +# ========================================================================= +# NORMALIZATION +# ========================================================================= + + +def _provider_added(node_change: Any) -> Optional[str]: + """Return the provider to display for an added node.""" + return node_change.provider_new if node_change.provider_new is not None else node_change.provider_old + + +def _provider_removed(node_change: Any) -> Optional[str]: + """Return the provider to display for a removed node.""" + return node_change.provider_old if node_change.provider_old is not None else node_change.provider_new + + +def _organize(diff: 'PipeDiff') -> Dict[str, Any]: + """ + Collapse a :class:`PipeDiff` into a sorted, de-duplicated intermediate form. + + All three renderers consume this structure so that they agree on both the set + of reported changes and their ordering. Node changes are aggregated by id: a + node that has both a provider change and configuration changes (whether the + engine emitted them as one ``NodeChange`` or several) is merged into a single + ``changed`` entry. + + Args: + diff: The pipeline diff to normalize. + + Returns: + A dictionary with the following keys: + - ``nodes_added``: list of ``(id, provider)`` tuples, sorted by id + - ``nodes_removed``: list of ``(id, provider)`` tuples, sorted by id + - ``nodes_changed``: list of per-id dicts (provider change + config + field changes), sorted by id + - ``edges_added``: list of ``(from_id, lane, to_id)`` tuples, sorted + - ``edges_removed``: list of ``(from_id, lane, to_id)`` tuples, sorted + - ``version_change``: the ``(old, new)`` tuple or ``None`` + - ``layout_changed``: bool + - ``has_semantic_changes``: bool (delegates to the diff's property) + """ + nodes_added: List[Tuple[str, Optional[str]]] = [] + nodes_removed: List[Tuple[str, Optional[str]]] = [] + changed: Dict[str, Dict[str, Any]] = {} + + for node_change in diff.node_changes: + kind = node_change.kind + if kind == 'added': + nodes_added.append((node_change.id, _provider_added(node_change))) + elif kind == 'removed': + nodes_removed.append((node_change.id, _provider_removed(node_change))) + else: + # 'provider', 'config', or any future in-place change kind: fold into + # a single per-id entry keyed by node id. + entry = changed.setdefault( + node_change.id, + { + 'id': node_change.id, + 'provider_old': None, + 'provider_new': None, + 'has_provider_change': False, + 'config_changes': [], + }, + ) + if ( + node_change.provider_old is not None or node_change.provider_new is not None + ) and node_change.provider_old != node_change.provider_new: + entry['provider_old'] = node_change.provider_old + entry['provider_new'] = node_change.provider_new + entry['has_provider_change'] = True + for field_change in node_change.field_changes or []: + entry['config_changes'].append(field_change) + + # Sort config changes within each node for deterministic output. + nodes_changed: List[Dict[str, Any]] = [] + for entry in changed.values(): + entry['config_changes'] = sorted( + entry['config_changes'], + key=lambda fc: (fc.path, fc.kind), + ) + nodes_changed.append(entry) + + edges_added: List[Tuple[str, str, str]] = [] + edges_removed: List[Tuple[str, str, str]] = [] + for edge_change in diff.edge_changes: + triple = (edge_change.from_id, edge_change.lane, edge_change.to_id) + if edge_change.kind == 'added': + edges_added.append(triple) + elif edge_change.kind == 'removed': + edges_removed.append(triple) + + return { + 'nodes_added': sorted(nodes_added, key=lambda t: t[0]), + 'nodes_removed': sorted(nodes_removed, key=lambda t: t[0]), + 'nodes_changed': sorted(nodes_changed, key=lambda e: e['id']), + 'edges_added': sorted(edges_added), + 'edges_removed': sorted(edges_removed), + 'version_change': diff.version_change, + 'layout_changed': bool(diff.layout_changed), + 'has_semantic_changes': bool(diff.has_semantic_changes), + } + + +def _any_output(org: Dict[str, Any]) -> bool: + """Return True when there is anything at all worth printing.""" + return bool( + org['nodes_added'] + or org['nodes_removed'] + or org['nodes_changed'] + or org['edges_added'] + or org['edges_removed'] + or org['version_change'] + or org['layout_changed'] + ) + + +def _fmt_value(value: Any) -> str: + """ + Render a config value as a compact, readable string. + + Strings are returned bare; every other JSON type (numbers, booleans, ``None``, + dicts, lists) is serialized with ``json.dumps`` so the type stays unambiguous + (``null``/``true``/``false``) and nested structures render deterministically. + """ + if isinstance(value, str): + return value + return json.dumps(value, ensure_ascii=False, sort_keys=True) + + +def _summary_counts(org: Dict[str, Any]) -> Dict[str, int]: + """Compute the integer counts used by summary lines and the JSON summary.""" + config_changes = sum(len(entry['config_changes']) for entry in org['nodes_changed']) + provider_changes = sum(1 for entry in org['nodes_changed'] if entry['has_provider_change']) + return { + 'nodes_added': len(org['nodes_added']), + 'nodes_removed': len(org['nodes_removed']), + 'nodes_changed': len(org['nodes_changed']), + 'provider_changes': provider_changes, + 'config_changes': config_changes, + 'edges_added': len(org['edges_added']), + 'edges_removed': len(org['edges_removed']), + } + + +def _summary_phrase(org: Dict[str, Any]) -> str: + """ + Build a one-line human summary such as ``2 nodes added, 1 edge removed``. + + Only non-zero categories are included. Version and layout changes are appended + when present. When nothing changed the phrase is ``no semantic changes``. + """ + counts = _summary_counts(org) + parts: List[str] = [] + + def _plural(n: int, singular: str) -> str: + return f'{n} {singular}' if n == 1 else f'{n} {singular}s' + + if counts['nodes_added']: + parts.append(f'{_plural(counts["nodes_added"], "node")} added') + if counts['nodes_removed']: + parts.append(f'{_plural(counts["nodes_removed"], "node")} removed') + if counts['nodes_changed']: + parts.append(f'{_plural(counts["nodes_changed"], "node")} changed') + if counts['edges_added']: + parts.append(f'{_plural(counts["edges_added"], "edge")} added') + if counts['edges_removed']: + parts.append(f'{_plural(counts["edges_removed"], "edge")} removed') + + version_change = org['version_change'] + if version_change: + parts.append(f'version {_fmt_value(version_change[0])} → {_fmt_value(version_change[1])}') + + if org['layout_changed']: + parts.append('layout changed') + + if not parts: + return 'no semantic changes' + return ', '.join(parts) + + +# ========================================================================= +# HUMAN (TEXT) RENDERER +# ========================================================================= + + +def _human_field_line(field_change: Any, palette: _Palette) -> str: + """Render a single config FieldChange as an indented, marked text line.""" + path = field_change.path + kind = field_change.kind + if kind == 'added': + body = f'{_MARK_ADDED} {path} = {_fmt_value(field_change.new)}' + return ' ' + palette.added(body) + if kind == 'removed': + body = f'{_MARK_REMOVED} {path} = {_fmt_value(field_change.old)}' + return ' ' + palette.removed(body) + # changed + old = _fmt_value(field_change.old) + new = _fmt_value(field_change.new) + return ' ' + palette.changed(f'{_MARK_CHANGED} {path}: ') + f'{old} ' + palette.dim('->') + f' {new}' + + +def render_human(diff: 'PipeDiff', *, use_color: bool) -> str: + """ + Render a semantic pipe diff as grouped, optionally colored text. + + The report is organized into up to three sections — ``Nodes`` (additions, + removals, provider changes), ``Edges`` (added/removed wiring), and ``Config`` + (per-node field changes) — followed by standalone ``Version`` and ``Layout`` + lines when applicable. Change markers are ``+`` (added, green), ``-`` (removed, + red), and ``~`` (changed, yellow). Empty sections are omitted. + + Args: + diff: The pipeline diff to render. + use_color: When True, wrap markers and values in ANSI color escapes. The + caller is responsible for deciding whether color is appropriate (TTY + detection and ``NO_COLOR`` honoring live in the CLI); this function + only applies the choice. + + Returns: + A newline-joined string with no trailing newline. When there are no + changes at all the string is ``No semantic changes.``. + """ + org = _organize(diff) + palette = _Palette(use_color) + + if not _any_output(org): + return 'No semantic changes.' + + lines: List[str] = [] + lines.append(f'Pipeline diff: {_summary_phrase(org)}') + + # --- Nodes --- + if org['nodes_added'] or org['nodes_removed'] or any(e['has_provider_change'] for e in org['nodes_changed']): + lines.append('') + lines.append('Nodes') + for node_id, provider in org['nodes_added']: + lines.append(' ' + palette.added(f'{_MARK_ADDED} {node_id} ({provider})')) + for node_id, provider in org['nodes_removed']: + lines.append(' ' + palette.removed(f'{_MARK_REMOVED} {node_id} ({provider})')) + for entry in org['nodes_changed']: + if entry['has_provider_change']: + marker = palette.changed(f'{_MARK_CHANGED} {entry["id"]} provider: ') + lines.append( + ' ' + marker + f'{entry["provider_old"]} ' + palette.dim('->') + f' {entry["provider_new"]}' + ) + + # --- Edges --- + if org['edges_added'] or org['edges_removed']: + lines.append('') + lines.append('Edges') + for from_id, lane, to_id in org['edges_added']: + lines.append(' ' + palette.added(f'{_MARK_ADDED} {from_id} --{lane}--> {to_id}')) + for from_id, lane, to_id in org['edges_removed']: + lines.append(' ' + palette.removed(f'{_MARK_REMOVED} {from_id} --{lane}--> {to_id}')) + + # --- Config --- + config_nodes = [e for e in org['nodes_changed'] if e['config_changes']] + if config_nodes: + lines.append('') + lines.append('Config') + for entry in config_nodes: + lines.append(f' {entry["id"]}') + for field_change in entry['config_changes']: + lines.append(_human_field_line(field_change, palette)) + + # --- Version / Layout --- + if org['version_change']: + old = _fmt_value(org['version_change'][0]) + new = _fmt_value(org['version_change'][1]) + lines.append('') + lines.append('Version: ' + f'{old} ' + palette.dim('->') + f' {new}') + + if org['layout_changed']: + if not org['version_change']: + lines.append('') + lines.append(palette.dim('Layout: changed (ui/viewport)')) + + return '\n'.join(lines) + + +# ========================================================================= +# JSON RENDERER +# ========================================================================= + + +def _json_field_change(field_change: Any) -> Dict[str, Any]: + """Serialize a single FieldChange into a plain dict.""" + return { + 'path': field_change.path, + 'kind': field_change.kind, + 'old': field_change.old, + 'new': field_change.new, + } + + +def render_json(diff: 'PipeDiff') -> Dict[str, Any]: + """ + Render a semantic pipe diff as a single JSON-serializable document. + + The returned structure is stable and deterministically ordered: + + { + "nodes": {"added": [...], "removed": [...], "changed": [...]}, + "edges": {"added": [...], "removed": [...]}, + "summary": {...} + } + + Each added/removed node is ``{"id", "provider"}``. Each changed node is + ``{"id", "provider_change": {"old", "new"} | null, "config_changes": [...]}`` + where every config change is ``{"path", "kind", "old", "new"}``. Edges are + ``{"from", "lane", "to"}``. The ``summary`` block carries counts, the version + change (as ``[old, new]`` or ``null``), the layout flag, and the overall + ``has_semantic_changes`` boolean. + + Args: + diff: The pipeline diff to render. + + Returns: + A dict containing only JSON-serializable values. + """ + org = _organize(diff) + + nodes_added = [{'id': node_id, 'provider': provider} for node_id, provider in org['nodes_added']] + nodes_removed = [{'id': node_id, 'provider': provider} for node_id, provider in org['nodes_removed']] + + nodes_changed: List[Dict[str, Any]] = [] + for entry in org['nodes_changed']: + provider_change = None + if entry['has_provider_change']: + provider_change = {'old': entry['provider_old'], 'new': entry['provider_new']} + nodes_changed.append( + { + 'id': entry['id'], + 'provider_change': provider_change, + 'config_changes': [_json_field_change(fc) for fc in entry['config_changes']], + } + ) + + edges_added = [{'from': from_id, 'lane': lane, 'to': to_id} for from_id, lane, to_id in org['edges_added']] + edges_removed = [{'from': from_id, 'lane': lane, 'to': to_id} for from_id, lane, to_id in org['edges_removed']] + + counts = _summary_counts(org) + version_change = list(org['version_change']) if org['version_change'] else None + + summary = { + **counts, + 'version_change': version_change, + 'layout_changed': org['layout_changed'], + 'has_semantic_changes': org['has_semantic_changes'], + } + + return { + 'nodes': {'added': nodes_added, 'removed': nodes_removed, 'changed': nodes_changed}, + 'edges': {'added': edges_added, 'removed': edges_removed}, + 'summary': summary, + } + + +# ========================================================================= +# MARKDOWN RENDERER +# ========================================================================= + + +def _md_code(text: str) -> str: + """ + Wrap ``text`` in a Markdown code span, safely handling embedded backticks. + + Per CommonMark, a code span may use a run of N backticks as its delimiter as + long as the content contains no run of exactly N backticks; padding spaces are + stripped by the renderer. This picks a delimiter longer than the longest + internal backtick run so arbitrary values render literally. + + Line breaks are collapsed to spaces first. A CommonMark code span cannot span a + blank line, so a newline in an untrusted value (node id, provider, lane, config + value) would otherwise terminate the span and the enclosing bullet/table cell, + letting the trailing text render as real Markdown (headings, ``@`` mentions, + links) in the auto-posted PR comment. Neutralizing them here — the single choke + point every value passes through — covers bullets, table cells, and the version + line at once. + """ + text = str(text).replace('\r', ' ').replace('\n', ' ') + if '`' not in text: + return f'`{text}`' + longest = max(len(run) for run in re.findall(r'`+', text)) + fence = '`' * (longest + 1) + return f'{fence} {text} {fence}' + + +def _md_cell(text: str) -> str: + """ + Escape a string for use inside a Markdown table cell. + + GitHub's table parser splits on ``|`` even inside code spans, and literal + newlines break the row, so both are neutralized here after any code-span + formatting has been applied. + """ + return str(text).replace('\\', '\\\\').replace('|', '\\|').replace('\n', ' ') + + +def _md_field_change(field_change: Any) -> str: + """Render a config FieldChange as a compact Markdown 'change' fragment.""" + if field_change.kind == 'added': + return f'{_MD_ADDED} {_md_code(field_change.new)}' + if field_change.kind == 'removed': + return f'{_MD_REMOVED} {_md_code(field_change.old)}' + return f'{_md_code(field_change.old)} → {_md_code(field_change.new)}' + + +def render_markdown(diff: 'PipeDiff', *, title: Optional[str] = None) -> str: + """ + Render a semantic pipe diff as compact, PR-comment-friendly Markdown. + + The output leads with a bold one-line summary, then grouped sections: ``Nodes`` + and ``Edges`` as bullet lists and ``Config`` as a table (``Node | Field | + Change``). All node ids, providers, lanes, and values are wrapped in code + spans; table cells additionally escape ``|`` and newlines so untrusted config + values cannot break the table or the surrounding comment. Ordering is + deterministic, so re-running on unchanged input yields byte-identical output. + + Args: + diff: The pipeline diff to render. + title: Optional heading text. When provided it is emitted as an ``##`` + heading above the summary; otherwise no heading is emitted. + + Returns: + A Markdown string with no trailing newline. + """ + org = _organize(diff) + + lines: List[str] = [] + if title: + lines.append(f'## {title}') + lines.append('') + + lines.append(f'**Pipeline diff:** {_summary_phrase(org)}') + + if not _any_output(org): + return '\n'.join(lines) + + # --- Nodes --- + provider_changed = [e for e in org['nodes_changed'] if e['has_provider_change']] + if org['nodes_added'] or org['nodes_removed'] or provider_changed: + lines.append('') + lines.append('**Nodes**') + for node_id, provider in org['nodes_added']: + lines.append(f'- {_MD_ADDED} {_md_code(node_id)} ({_md_code(provider)})') + for node_id, provider in org['nodes_removed']: + lines.append(f'- {_MD_REMOVED} {_md_code(node_id)} ({_md_code(provider)})') + for entry in provider_changed: + lines.append( + f'- {_MD_CHANGED} {_md_code(entry["id"])} provider: ' + f'{_md_code(entry["provider_old"])} → {_md_code(entry["provider_new"])}' + ) + + # --- Edges --- + if org['edges_added'] or org['edges_removed']: + lines.append('') + lines.append('**Edges**') + for from_id, lane, to_id in org['edges_added']: + lines.append(f'- {_MD_ADDED} {_md_code(from_id)} --{_md_code(lane)}--> {_md_code(to_id)}') + for from_id, lane, to_id in org['edges_removed']: + lines.append(f'- {_MD_REMOVED} {_md_code(from_id)} --{_md_code(lane)}--> {_md_code(to_id)}') + + # --- Config (table) --- + config_nodes = [e for e in org['nodes_changed'] if e['config_changes']] + if config_nodes: + lines.append('') + lines.append('**Config**') + lines.append('') + lines.append('| Node | Field | Change |') + lines.append('| --- | --- | --- |') + for entry in config_nodes: + for field_change in entry['config_changes']: + node_cell = _md_cell(_md_code(entry['id'])) + field_cell = _md_cell(_md_code(field_change.path)) + change_cell = _md_cell(_md_field_change(field_change)) + lines.append(f'| {node_cell} | {field_cell} | {change_cell} |') + + # --- Version / Layout --- + if org['version_change']: + old = _md_code(_fmt_value(org['version_change'][0])) + new = _md_code(_fmt_value(org['version_change'][1])) + lines.append('') + lines.append(f'**Version:** {old} → {new}') + + if org['layout_changed']: + lines.append('') + lines.append('_Layout (ui/viewport) changed._') + + return '\n'.join(lines) diff --git a/packages/client-python/tests/test_diff_cli.py b/packages/client-python/tests/test_diff_cli.py new file mode 100644 index 000000000..5b7f58496 --- /dev/null +++ b/packages/client-python/tests/test_diff_cli.py @@ -0,0 +1,449 @@ +# MIT License +# +# Copyright (c) 2026 Aparavi Software AG +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +""" +Tests for the ``rocketride diff`` CLI subcommand (rocketride.cli.commands.diff). + +These exercise ``DiffCommand.execute`` end-to-end against real ``.pipe`` files on +disk, driving it through the same pinned engine and reporters the CLI uses in +production. The command is fully local (no server, no auth), so no client fixture +or network mock is needed; the only external dependency stubbed here is +``resolve_git_ref`` for ``--git`` mode, patched in the command's own module +namespace the same way the engine test suite stubs subprocess. + +Coverage mirrors the product contract: + - two-file happy path (semantic changes -> exit 1) + - identical files (-> exit 0, "No semantic changes.") + - ``--git`` mode with a mocked resolver (found ref, and file-absent -> all added) + - unreadable / unparseable / non-pipe input (-> exit 2, error on stderr) + - argument-usage errors (wrong file count for the mode -> exit 2) + - ``--json`` output purity (stdout is a single parseable document) + - ``--markdown`` output + - ``--include-layout`` toggling ui.* field lines and the exit code + - ``--exit-zero`` forcing 0 on changes but never masking an error + - argparse registration: 'diff' exists, takes no connection args, and makes + ``--json``/``--markdown`` mutually exclusive +""" + +import json +from types import SimpleNamespace + +import pytest + +from rocketride.cli.commands import diff as diff_module +from rocketride.cli.commands.diff import DiffCommand +from rocketride.cli.main import RocketRideCLI + + +# ========================================================================= +# Helpers / fixtures +# ========================================================================= + + +class _StubCLI: + """Minimal stand-in for the CLI context. + + ``DiffCommand`` stores the CLI on construction but never touches it during + ``execute`` (the command is fully local), so an attribute-free stub is enough + to satisfy ``BaseCommand.__init__``. + """ + + +def _diff_args( + paths=None, + *, + git=None, + include_layout=False, + json=False, # noqa: A002 - mirrors the argparse dest name exactly + markdown=False, + exit_zero=False, +): + """Build a parsed-args namespace shaped like the diff subparser's output.""" + return SimpleNamespace( + paths=list(paths or []), + git=git, + include_layout=include_layout, + json=json, + markdown=markdown, + exit_zero=exit_zero, + ) + + +def _write_pipe(tmp_path, name, obj): + """Write ``obj`` as JSON to ``name`` under ``tmp_path`` and return the path str.""" + path = tmp_path / name + path.write_text(json.dumps(obj), encoding='utf-8') + return str(path) + + +def _old_pipe(): + """A small but complete pipeline: webhook -> chat, with layout and version.""" + return { + 'version': 1, + 'viewport': {'x': 0, 'y': 0, 'zoom': 1}, + 'components': [ + { + 'id': 'a', + 'provider': 'webhook', + 'config': {'mode': 'Source'}, + 'ui': {'position': {'x': 0, 'y': 0}}, + }, + { + 'id': 'b', + 'provider': 'chat', + 'config': {'default': {'strlen': 512}}, + 'input': [{'lane': 'text', 'from': 'a'}], + 'ui': {'position': {'x': 10, 'y': 10}}, + }, + ], + } + + +def _new_pipe(): + """``_old_pipe`` with a semantic delta: node 'c' added, edge b->c, b.strlen changed.""" + pipe = _old_pipe() + pipe['components'][1]['config']['default']['strlen'] = 1024 + pipe['components'].append( + { + 'id': 'c', + 'provider': 'qdrant', + 'config': {}, + 'input': [{'lane': 'vec', 'from': 'b'}], + 'ui': {'position': {'x': 20, 'y': 20}}, + } + ) + return pipe + + +def _layout_only_new_pipe(): + """``_old_pipe`` with *only* canvas movement: node 'a' relocated, nothing else.""" + pipe = _old_pipe() + pipe['components'][0]['ui']['position'] = {'x': 400, 'y': 250} + return pipe + + +async def _run(args): + """Instantiate and execute a DiffCommand, returning its integer exit code.""" + command = DiffCommand(_StubCLI(), args) + return await command.execute() + + +# ========================================================================= +# Two-file mode +# ========================================================================= + + +@pytest.mark.asyncio +async def test_two_file_changes_exit_1(tmp_path, capsys): + old = _write_pipe(tmp_path, 'old.pipe', _old_pipe()) + new = _write_pipe(tmp_path, 'new.pipe', _new_pipe()) + + code = await _run(_diff_args([old, new])) + + out = capsys.readouterr().out + assert code == 1 + assert 'Nodes' in out + assert '+ c (qdrant)' in out + assert '+ b --vec--> c' in out + assert 'config.default.strlen' in out + + +@pytest.mark.asyncio +async def test_identical_files_exit_0(tmp_path, capsys): + same = _write_pipe(tmp_path, 'same.pipe', _old_pipe()) + + code = await _run(_diff_args([same, same])) + + captured = capsys.readouterr() + assert code == 0 + assert captured.out.strip() == 'No semantic changes.' + assert captured.err == '' + + +# ========================================================================= +# --git mode (resolver mocked) +# ========================================================================= + + +@pytest.mark.asyncio +async def test_git_mode_diffs_against_ref(tmp_path, capsys, monkeypatch): + new = _write_pipe(tmp_path, 'pipeline.pipe', _new_pipe()) + # Resolver returns the *old* pipe as it existed at the ref. + monkeypatch.setattr(diff_module, 'resolve_git_ref', lambda ref, path: _old_pipe()) + + code = await _run(_diff_args([new], git='HEAD')) + + out = capsys.readouterr().out + assert code == 1 + assert '+ c (qdrant)' in out + + +@pytest.mark.asyncio +async def test_git_mode_absent_file_is_all_added(tmp_path, capsys, monkeypatch): + new = _write_pipe(tmp_path, 'pipeline.pipe', _new_pipe()) + # None => file did not exist at the ref; everything is newly added. + monkeypatch.setattr(diff_module, 'resolve_git_ref', lambda ref, path: None) + + code = await _run(_diff_args([new], git='HEAD', json=True)) + + doc = json.loads(capsys.readouterr().out) + assert code == 1 + added_ids = {n['id'] for n in doc['nodes']['added']} + assert added_ids == {'a', 'b', 'c'} + assert doc['nodes']['removed'] == [] + + +@pytest.mark.asyncio +async def test_git_mode_requires_exactly_one_file(tmp_path, capsys): + a = _write_pipe(tmp_path, 'a.pipe', _old_pipe()) + b = _write_pipe(tmp_path, 'b.pipe', _new_pipe()) + + code = await _run(_diff_args([a, b], git='HEAD')) + + captured = capsys.readouterr() + assert code == 2 + assert captured.out == '' + assert '--git requires exactly one FILE' in captured.err + + +@pytest.mark.asyncio +async def test_git_resolver_error_exit_2(tmp_path, capsys, monkeypatch): + new = _write_pipe(tmp_path, 'pipeline.pipe', _new_pipe()) + + def _boom(ref, path): + raise diff_module.PipeDiffError('git show HEAD:pipeline.pipe failed: bad ref') + + monkeypatch.setattr(diff_module, 'resolve_git_ref', _boom) + + code = await _run(_diff_args([new], git='HEAD')) + + captured = capsys.readouterr() + assert code == 2 + assert captured.out == '' + assert 'Error:' in captured.err + assert 'bad ref' in captured.err + + +# ========================================================================= +# Error / usage handling (exit 2) +# ========================================================================= + + +@pytest.mark.asyncio +async def test_missing_file_exit_2(tmp_path, capsys): + existing = _write_pipe(tmp_path, 'old.pipe', _old_pipe()) + missing = str(tmp_path / 'does_not_exist.pipe') + + code = await _run(_diff_args([existing, missing])) + + captured = capsys.readouterr() + assert code == 2 + assert captured.out == '' + assert 'Error:' in captured.err + + +@pytest.mark.asyncio +async def test_invalid_json_exit_2(tmp_path, capsys): + old = _write_pipe(tmp_path, 'old.pipe', _old_pipe()) + bad = tmp_path / 'bad.pipe' + bad.write_text('{ this is not json', encoding='utf-8') + + code = await _run(_diff_args([old, str(bad)])) + + captured = capsys.readouterr() + assert code == 2 + assert captured.out == '' + assert 'Error:' in captured.err + + +@pytest.mark.asyncio +async def test_non_pipe_object_exit_2(tmp_path, capsys): + old = _write_pipe(tmp_path, 'old.pipe', _old_pipe()) + # Valid JSON object, but no 'components' list -> not a pipeline. + notpipe = _write_pipe(tmp_path, 'notpipe.pipe', {'hello': 'world'}) + + code = await _run(_diff_args([old, notpipe])) + + captured = capsys.readouterr() + assert code == 2 + assert captured.out == '' + assert 'Error:' in captured.err + + +@pytest.mark.asyncio +async def test_wrong_positional_count_exit_2(tmp_path, capsys): + only = _write_pipe(tmp_path, 'only.pipe', _old_pipe()) + + code = await _run(_diff_args([only])) # one file, no --git + + captured = capsys.readouterr() + assert code == 2 + assert captured.out == '' + assert 'two files are required' in captured.err + + +@pytest.mark.asyncio +async def test_no_files_exit_2(capsys): + code = await _run(_diff_args([])) + + captured = capsys.readouterr() + assert code == 2 + assert captured.out == '' + assert 'two files are required' in captured.err + + +# ========================================================================= +# Output-format flags +# ========================================================================= + + +@pytest.mark.asyncio +async def test_json_output_is_pure_and_parseable(tmp_path, capsys): + old = _write_pipe(tmp_path, 'old.pipe', _old_pipe()) + new = _write_pipe(tmp_path, 'new.pipe', _new_pipe()) + + code = await _run(_diff_args([old, new], json=True)) + + captured = capsys.readouterr() + assert code == 1 + # stdout must be exactly one JSON document, nothing else. + doc = json.loads(captured.out) + assert set(doc.keys()) == {'nodes', 'edges', 'summary'} + assert doc['summary']['has_semantic_changes'] is True + assert captured.err == '' + + +@pytest.mark.asyncio +async def test_markdown_output(tmp_path, capsys): + old = _write_pipe(tmp_path, 'old.pipe', _old_pipe()) + new = _write_pipe(tmp_path, 'new.pipe', _new_pipe()) + + code = await _run(_diff_args([old, new], markdown=True)) + + out = capsys.readouterr().out + assert code == 1 + assert '**Pipeline diff:**' in out + assert '**Nodes**' in out + assert '| Node | Field | Change |' in out + + +# ========================================================================= +# --include-layout +# ========================================================================= + + +@pytest.mark.asyncio +async def test_layout_only_hidden_by_default_exit_0(tmp_path, capsys): + old = _write_pipe(tmp_path, 'old.pipe', _old_pipe()) + new = _write_pipe(tmp_path, 'new.pipe', _layout_only_new_pipe()) + + code = await _run(_diff_args([old, new])) + + out = capsys.readouterr().out + # Pure canvas movement is non-semantic: exit 0, coarse layout hint, no ui.* lines. + assert code == 0 + assert 'Layout' in out + assert 'ui.position' not in out + + +@pytest.mark.asyncio +async def test_include_layout_surfaces_ui_fields_exit_1(tmp_path, capsys): + old = _write_pipe(tmp_path, 'old.pipe', _old_pipe()) + new = _write_pipe(tmp_path, 'new.pipe', _layout_only_new_pipe()) + + code = await _run(_diff_args([old, new], include_layout=True)) + + out = capsys.readouterr().out + # Opting layout in enumerates ui.* field changes and makes them count. + assert code == 1 + assert 'ui.position' in out + + +# ========================================================================= +# --exit-zero +# ========================================================================= + + +@pytest.mark.asyncio +async def test_exit_zero_forces_success_on_changes(tmp_path, capsys): + old = _write_pipe(tmp_path, 'old.pipe', _old_pipe()) + new = _write_pipe(tmp_path, 'new.pipe', _new_pipe()) + + code = await _run(_diff_args([old, new], exit_zero=True)) + + out = capsys.readouterr().out + # Changes are still reported, but the exit code is forced to 0 for non-gating use. + assert code == 0 + assert '+ c (qdrant)' in out + + +@pytest.mark.asyncio +async def test_exit_zero_does_not_mask_errors(tmp_path, capsys): + old = _write_pipe(tmp_path, 'old.pipe', _old_pipe()) + missing = str(tmp_path / 'nope.pipe') + + code = await _run(_diff_args([old, missing], exit_zero=True)) + + captured = capsys.readouterr() + # --exit-zero only forces 0 on a *successful* run; a load error is still exit 2. + assert code == 2 + assert captured.out == '' + assert 'Error:' in captured.err + + +# ========================================================================= +# argparse registration (parser-level, no execution) +# ========================================================================= + + +def test_parser_registers_diff_subcommand(): + parser = RocketRideCLI().setup_parser() + ns = parser.parse_args(['diff', 'old.pipe', 'new.pipe']) + assert ns.command == 'diff' + assert ns.paths == ['old.pipe', 'new.pipe'] + assert ns.git is None + assert ns.include_layout is False + assert ns.json is False + assert ns.markdown is False + assert ns.exit_zero is False + + +def test_parser_diff_takes_no_connection_args(): + parser = RocketRideCLI().setup_parser() + ns = parser.parse_args(['diff', 'old.pipe', 'new.pipe']) + # This command never touches the engine, so the shared connection args that + # every other subcommand carries must be absent from its namespace. + assert not hasattr(ns, 'apikey') + assert not hasattr(ns, 'uri') + assert not hasattr(ns, 'token') + + +def test_parser_json_and_markdown_are_mutually_exclusive(): + parser = RocketRideCLI().setup_parser() + with pytest.raises(SystemExit): + parser.parse_args(['diff', '--json', '--markdown', 'old.pipe', 'new.pipe']) + + +if __name__ == '__main__': + import sys + + sys.exit(pytest.main([__file__, '-v'])) diff --git a/packages/client-python/tests/test_pipediff_engine.py b/packages/client-python/tests/test_pipediff_engine.py new file mode 100644 index 000000000..0173e1caf --- /dev/null +++ b/packages/client-python/tests/test_pipediff_engine.py @@ -0,0 +1,426 @@ +# MIT License +# +# Copyright (c) 2026 Aparavi Software AG +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +"""Unit tests for the RocketRide pipe-diff engine, model, and config deep-diff.""" + +import copy +import json +from pathlib import Path + +import pytest + +from rocketride.pipediff import ( + EdgeChange, + PipeDiffError, + deep_diff_config, + diff_pipes, + load_pipe, +) + + +# --------------------------------------------------------------------------- +# Fixtures / builders +# --------------------------------------------------------------------------- + + +@pytest.fixture +def examples_dir(): + """Path to the repo's bundled ``examples/`` directory of ``.pipe`` files.""" + # tests/ -> client-python/ -> packages/ -> repo root -> examples/ + return Path(__file__).resolve().parents[3] / 'examples' + + +def _node(component_id, provider, config=None, inputs=None, control=None, ui=None): + """Build a single component dict, omitting empty optional blocks.""" + component = {'id': component_id, 'provider': provider, 'config': config or {}} + if inputs is not None: + component['input'] = inputs + if control is not None: + component['control'] = control + component['ui'] = ui if ui is not None else {'position': {'x': 0, 'y': 0}} + return component + + +def _pipe(components, version=1, viewport=None): + """Build a top-level pipe dict.""" + pipe = {'components': components, 'project_id': 'p', 'version': version} + if viewport is not None: + pipe['viewport'] = viewport + return pipe + + +def _sample_pipe(): + """A small but representative two-node RAG-ish pipeline.""" + return _pipe( + [ + _node('webhook_1', 'webhook', {'mode': 'Source', 'type': 'webhook'}), + _node( + 'parse_1', + 'parse', + {'profile': 'default', 'default': {'strlen': 512}}, + inputs=[{'lane': 'text', 'from': 'webhook_1'}], + ), + ] + ) + + +def _changes_by_kind(diff, kind): + return [change for change in diff.node_changes if change.kind == kind] + + +# --------------------------------------------------------------------------- +# Identity / no-change +# --------------------------------------------------------------------------- + + +def test_identical_pipes_produce_empty_diff(): + pipe = _sample_pipe() + diff = diff_pipes(pipe, copy.deepcopy(pipe)) + assert diff.node_changes == [] + assert diff.edge_changes == [] + assert diff.version_change is None + assert diff.layout_changed is False + assert diff.has_semantic_changes is False + + +# --------------------------------------------------------------------------- +# Nodes +# --------------------------------------------------------------------------- + + +def test_added_node_is_reported_with_provider(): + old = _sample_pipe() + new = copy.deepcopy(old) + new['components'].append(_node('llm_1', 'llm_openai', inputs=[{'lane': 'questions', 'from': 'parse_1'}])) + diff = diff_pipes(old, new) + + added = _changes_by_kind(diff, 'added') + assert [c.id for c in added] == ['llm_1'] + assert added[0].provider_new == 'llm_openai' + assert added[0].provider_old is None + assert diff.has_semantic_changes is True + + +def test_removed_node_is_reported_with_provider(): + old = _sample_pipe() + new = copy.deepcopy(old) + new['components'] = [c for c in new['components'] if c['id'] != 'parse_1'] + diff = diff_pipes(old, new) + + removed = _changes_by_kind(diff, 'removed') + assert [c.id for c in removed] == ['parse_1'] + assert removed[0].provider_old == 'parse' + assert removed[0].provider_new is None + + +def test_provider_change_is_reported_separately_from_config(): + old = _sample_pipe() + new = copy.deepcopy(old) + # Same id, different provider *and* a config change. + new['components'][1]['provider'] = 'parse_v2' + new['components'][1]['config']['default']['strlen'] = 1024 + diff = diff_pipes(old, new) + + provider_changes = _changes_by_kind(diff, 'provider') + assert len(provider_changes) == 1 + assert provider_changes[0].id == 'parse_1' + assert provider_changes[0].provider_old == 'parse' + assert provider_changes[0].provider_new == 'parse_v2' + + config_changes = _changes_by_kind(diff, 'config') + assert len(config_changes) == 1 + assert config_changes[0].id == 'parse_1' + paths = {fc.path for fc in config_changes[0].field_changes} + assert paths == {'config.default.strlen'} + + +# --------------------------------------------------------------------------- +# Config deep-diff +# --------------------------------------------------------------------------- + + +def test_deep_diff_config_handles_nested_dicts_added_removed_changed(): + old = {'profile': 'a', 'default': {'strlen': 512, 'gone': True}} + new = {'profile': 'b', 'default': {'strlen': 1024, 'added_key': 7}} + changes = {(fc.path, fc.kind): (fc.old, fc.new) for fc in deep_diff_config(old, new)} + + assert changes[('config.profile', 'changed')] == ('a', 'b') + assert changes[('config.default.strlen', 'changed')] == (512, 1024) + assert changes[('config.default.gone', 'removed')] == (True, None) + assert changes[('config.default.added_key', 'added')] == (None, 7) + + +def test_deep_diff_config_handles_list_element_change(): + old = {'instructions': ['Answer accurately.', 'Be terse.']} + new = {'instructions': ['Answer accurately and cite sources.', 'Be terse.']} + changes = deep_diff_config(old, new) + + assert len(changes) == 1 + change = changes[0] + assert change.path == 'config.instructions[0]' + assert change.kind == 'changed' + assert change.old == 'Answer accurately.' + assert change.new == 'Answer accurately and cite sources.' + + +def test_deep_diff_config_handles_list_growth_and_shrink(): + grow = deep_diff_config({'xs': [1]}, {'xs': [1, 2, 3]}) + assert {(c.path, c.kind, c.new) for c in grow} == { + ('config.xs[1]', 'added', 2), + ('config.xs[2]', 'added', 3), + } + + shrink = deep_diff_config({'xs': [1, 2, 3]}, {'xs': [1]}) + assert {(c.path, c.kind, c.old) for c in shrink} == { + ('config.xs[1]', 'removed', 2), + ('config.xs[2]', 'removed', 3), + } + + +def test_deep_diff_config_handles_list_of_dicts_index_paths(): + old = {'steps': [{'name': 'a', 'n': 1}]} + new = {'steps': [{'name': 'a', 'n': 2}]} + changes = deep_diff_config(old, new) + assert len(changes) == 1 + assert changes[0].path == 'config.steps[0].n' + assert (changes[0].old, changes[0].new) == (1, 2) + + +def test_deep_diff_config_equal_returns_empty(): + cfg = {'a': {'b': [1, 2, {'c': 3}]}} + assert deep_diff_config(cfg, copy.deepcopy(cfg)) == [] + + +def test_deep_diff_config_treats_none_as_empty(): + assert deep_diff_config(None, {'a': 1}) == [ + # single added key + *deep_diff_config({}, {'a': 1}) + ] + assert deep_diff_config(None, None) == [] + + +# --------------------------------------------------------------------------- +# Edges (input + control) +# --------------------------------------------------------------------------- + + +def test_edge_added_and_removed(): + old = _sample_pipe() + new = copy.deepcopy(old) + # Rewire parse_1 to read from a new lane/source, and add a fresh node+edge. + new['components'][1]['input'] = [{'lane': 'raw', 'from': 'webhook_1'}] + diff = diff_pipes(old, new) + + added = [e for e in diff.edge_changes if e.kind == 'added'] + removed = [e for e in diff.edge_changes if e.kind == 'removed'] + assert added == [EdgeChange(from_id='webhook_1', lane='raw', to_id='parse_1', kind='added')] + assert removed == [EdgeChange(from_id='webhook_1', lane='text', to_id='parse_1', kind='removed')] + + +def test_edge_rewire_shows_as_added_and_removed_pair(): + old = _pipe( + [ + _node('a', 'src'), + _node('b', 'src'), + _node('c', 'sink', inputs=[{'lane': 'x', 'from': 'a'}]), + ] + ) + new = copy.deepcopy(old) + # Rewire c's lane x from a -> b (same to/lane, different from). + new['components'][2]['input'] = [{'lane': 'x', 'from': 'b'}] + diff = diff_pipes(old, new) + + assert EdgeChange('b', 'x', 'c', 'added') in diff.edge_changes + assert EdgeChange('a', 'x', 'c', 'removed') in diff.edge_changes + assert len(diff.edge_changes) == 2 + + +def test_control_edges_are_diffed(): + old = _pipe( + [ + _node('agent_1', 'agent_rocketride'), + _node('llm_1', 'llm_openai', control=[{'classType': 'llm', 'from': 'agent_1'}]), + _node('tool_1', 'tool_python', control=[{'classType': 'tool', 'from': 'agent_1'}]), + ] + ) + new = copy.deepcopy(old) + # Detach the tool from the agent (remove its control wire). + new['components'][2].pop('control') + diff = diff_pipes(old, new) + + removed = [e for e in diff.edge_changes if e.kind == 'removed'] + assert removed == [EdgeChange(from_id='agent_1', lane='tool', to_id='tool_1', kind='removed')] + # The surviving llm control wire produces no change. + assert not [e for e in diff.edge_changes if e.kind == 'added'] + + +# --------------------------------------------------------------------------- +# Version +# --------------------------------------------------------------------------- + + +def test_version_change_is_reported_and_is_semantic(): + old = _sample_pipe() + new = copy.deepcopy(old) + new['version'] = 2 + diff = diff_pipes(old, new) + assert diff.version_change == (1, 2) + assert diff.has_semantic_changes is True + + +# --------------------------------------------------------------------------- +# Layout +# --------------------------------------------------------------------------- + + +def test_layout_only_change_is_not_semantic_by_default(): + old = _sample_pipe() + new = copy.deepcopy(old) + new['components'][1]['ui']['position'] = {'x': 999, 'y': 42} + diff = diff_pipes(old, new) + + assert diff.node_changes == [] + assert diff.edge_changes == [] + assert diff.layout_changed is True + assert diff.has_semantic_changes is False + + +def test_viewport_only_change_sets_layout_flag_but_is_not_semantic(): + old = _pipe([_node('a', 'src')], viewport={'x': 0, 'y': 0, 'zoom': 1}) + new = copy.deepcopy(old) + new['viewport'] = {'x': 10, 'y': 20, 'zoom': 2} + diff = diff_pipes(old, new) + + assert diff.layout_changed is True + assert diff.has_semantic_changes is False + + +def test_include_layout_surfaces_ui_field_changes(): + old = _sample_pipe() + new = copy.deepcopy(old) + new['components'][1]['ui']['position']['x'] = 999 + diff = diff_pipes(old, new, include_layout=True) + + config_changes = _changes_by_kind(diff, 'config') + assert len(config_changes) == 1 + paths = {fc.path for fc in config_changes[0].field_changes} + assert 'ui.position.x' in paths + # With layout opted in, the ui move now counts as a change. + assert diff.has_semantic_changes is True + # The flag is still set regardless of the include_layout mode. + assert diff.layout_changed is True + + +# --------------------------------------------------------------------------- +# Determinism +# --------------------------------------------------------------------------- + + +def test_node_change_ordering_is_deterministic(): + old = _pipe([_node('keep', 'src'), _node('zeta', 'src'), _node('alpha', 'src')]) + new = _pipe([_node('keep', 'src'), _node('beta', 'src'), _node('gamma', 'src')]) + diff = diff_pipes(old, new) + + added = [c.id for c in diff.node_changes if c.kind == 'added'] + removed = [c.id for c in diff.node_changes if c.kind == 'removed'] + assert added == ['beta', 'gamma'] # sorted + assert removed == ['alpha', 'zeta'] # sorted + + +# --------------------------------------------------------------------------- +# load_pipe validation +# --------------------------------------------------------------------------- + + +def test_load_pipe_from_path_roundtrip(tmp_path): + pipe = _sample_pipe() + path = tmp_path / 'pipeline.pipe' + path.write_text(json.dumps(pipe), encoding='utf-8') + assert load_pipe(str(path)) == pipe + + +def test_load_pipe_accepts_dict_passthrough(): + pipe = _sample_pipe() + assert load_pipe(pipe) is pipe + + +def test_load_pipe_missing_file_raises(): + with pytest.raises(PipeDiffError, match='not found'): + load_pipe('/nonexistent/path/to/file.pipe') + + +def test_load_pipe_malformed_json_raises(tmp_path): + path = tmp_path / 'bad.pipe' + path.write_text('{ not valid json', encoding='utf-8') + with pytest.raises(PipeDiffError, match='Invalid JSON'): + load_pipe(str(path)) + + +def test_load_pipe_non_object_raises(tmp_path): + path = tmp_path / 'list.pipe' + path.write_text('[1, 2, 3]', encoding='utf-8') + with pytest.raises(PipeDiffError, match='must be a JSON object'): + load_pipe(str(path)) + + +def test_load_pipe_missing_components_raises(): + with pytest.raises(PipeDiffError, match="missing a 'components' list"): + load_pipe({'project_id': 'p', 'version': 1}) + + +def test_load_pipe_components_not_a_list_raises(): + with pytest.raises(PipeDiffError, match="missing a 'components' list"): + load_pipe({'components': {'id': 'x'}}) + + +def test_load_pipe_component_missing_id_raises(): + with pytest.raises(PipeDiffError, match="index 0 is missing a string 'id'"): + load_pipe({'components': [{'provider': 'webhook'}]}) + + +def test_load_pipe_component_not_object_raises(): + with pytest.raises(PipeDiffError, match='index 0 is not an object'): + load_pipe({'components': ['not-a-dict']}) + + +def test_load_pipe_rejects_unsupported_type(): + with pytest.raises(PipeDiffError, match='expects a path or dict'): + load_pipe(12345) + + +# --------------------------------------------------------------------------- +# End-to-end against a real bundled example +# --------------------------------------------------------------------------- + + +def test_diff_of_real_example_against_itself_is_empty(examples_dir): + path = examples_dir / 'rag-pipeline.pipe' + if not path.exists(): + pytest.skip('bundled example not available') + pipe = load_pipe(str(path)) + diff = diff_pipes(pipe, copy.deepcopy(pipe)) + assert diff.has_semantic_changes is False + assert diff.layout_changed is False + + +if __name__ == '__main__': + raise SystemExit(pytest.main([__file__, '-v'])) diff --git a/packages/client-python/tests/test_pipediff_gitref.py b/packages/client-python/tests/test_pipediff_gitref.py new file mode 100644 index 000000000..b9033e21b --- /dev/null +++ b/packages/client-python/tests/test_pipediff_gitref.py @@ -0,0 +1,194 @@ +# MIT License +# +# Copyright (c) 2026 Aparavi Software AG +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +"""Tests for the git ref resolver used by ``rocketride diff --git``. + +Most cases stub ``subprocess.run`` so no real git or repository is needed; a +single, git-gated integration test exercises the real ``git show`` plumbing in +an isolated temporary repository. +""" + +import json +import shutil +import subprocess + +import pytest + +from rocketride.pipediff import PipeDiffError, gitref, resolve_git_ref + +SAMPLE_PIPE = { + 'components': [{'id': 'a', 'provider': 'src', 'config': {}}], + 'version': 1, +} + + +def _completed(returncode, stdout='', stderr=''): + """Build a stub ``CompletedProcess`` result.""" + return subprocess.CompletedProcess(args=['git'], returncode=returncode, stdout=stdout, stderr=stderr) + + +def _install_fake_git(monkeypatch, responses): + """Patch ``subprocess.run`` in gitref to reply per git subcommand. + + ``responses`` maps the subcommand ("rev-parse" or "show") to either a + ``CompletedProcess`` or an ``Exception`` instance (which is raised). + """ + + def fake_run(args, **_kwargs): + subcommand = args[3] # ['git', '-C', , , ...] + outcome = responses[subcommand] + if isinstance(outcome, BaseException): + raise outcome + return outcome + + monkeypatch.setattr(gitref.subprocess, 'run', fake_run) + + +# --------------------------------------------------------------------------- +# Mocked subprocess cases +# --------------------------------------------------------------------------- + + +def test_resolve_returns_parsed_pipe_when_found(monkeypatch): + _install_fake_git( + monkeypatch, + { + 'rev-parse': _completed(0, stdout='/repo\n'), + 'show': _completed(0, stdout=json.dumps(SAMPLE_PIPE)), + }, + ) + assert resolve_git_ref('HEAD', '/repo/pipeline.pipe') == SAMPLE_PIPE + + +@pytest.mark.parametrize( + 'stderr', + [ + "fatal: path 'pipeline.pipe' exists on disk, but not in 'HEAD'", + "fatal: path 'pipeline.pipe' does not exist in 'HEAD'", + ], +) +def test_resolve_returns_none_when_path_absent_in_ref(monkeypatch, stderr): + _install_fake_git( + monkeypatch, + { + 'rev-parse': _completed(0, stdout='/repo\n'), + 'show': _completed(128, stderr=stderr), + }, + ) + assert resolve_git_ref('HEAD', '/repo/pipeline.pipe') is None + + +def test_resolve_raises_on_unknown_ref(monkeypatch): + _install_fake_git( + monkeypatch, + { + 'rev-parse': _completed(0, stdout='/repo\n'), + 'show': _completed(128, stderr="fatal: invalid object name 'nope'."), + }, + ) + with pytest.raises(PipeDiffError, match='git show'): + resolve_git_ref('nope', '/repo/pipeline.pipe') + + +def test_resolve_raises_when_not_a_git_repo(monkeypatch): + _install_fake_git( + monkeypatch, + { + 'rev-parse': _completed(128, stderr='fatal: not a git repository (or any parent up to /)'), + }, + ) + with pytest.raises(PipeDiffError, match='Not a git repository'): + resolve_git_ref('HEAD', '/tmp/pipeline.pipe') + + +def test_resolve_raises_when_git_missing(monkeypatch): + _install_fake_git(monkeypatch, {'rev-parse': FileNotFoundError('git')}) + with pytest.raises(PipeDiffError, match='git executable not found'): + resolve_git_ref('HEAD', '/repo/pipeline.pipe') + + +def test_resolve_raises_on_timeout(monkeypatch): + _install_fake_git( + monkeypatch, + {'rev-parse': subprocess.TimeoutExpired(cmd='git', timeout=30)}, + ) + with pytest.raises(PipeDiffError, match='timed out'): + resolve_git_ref('HEAD', '/repo/pipeline.pipe') + + +def test_resolve_raises_on_invalid_json_content(monkeypatch): + _install_fake_git( + monkeypatch, + { + 'rev-parse': _completed(0, stdout='/repo\n'), + 'show': _completed(0, stdout='{ not json'), + }, + ) + with pytest.raises(PipeDiffError, match='Invalid JSON'): + resolve_git_ref('HEAD', '/repo/pipeline.pipe') + + +def test_resolve_raises_when_ref_content_is_not_a_pipe(monkeypatch): + _install_fake_git( + monkeypatch, + { + 'rev-parse': _completed(0, stdout='/repo\n'), + 'show': _completed(0, stdout=json.dumps({'version': 1})), + }, + ) + with pytest.raises(PipeDiffError, match="missing a 'components' list"): + resolve_git_ref('HEAD', '/repo/pipeline.pipe') + + +# --------------------------------------------------------------------------- +# Real git integration (isolated temp repo) +# --------------------------------------------------------------------------- + + +@pytest.mark.skipif(shutil.which('git') is None, reason='git not installed') +def test_resolve_git_ref_real_repository(tmp_path): + def git(*args): + result = subprocess.run(['git', *args], cwd=tmp_path, capture_output=True, text=True) + assert result.returncode == 0, result.stderr + return result + + git('init', '-q') + pipe_file = tmp_path / 'pipeline.pipe' + pipe_file.write_text(json.dumps(SAMPLE_PIPE), encoding='utf-8') + git('add', 'pipeline.pipe') + git('-c', 'user.email=t@example.com', '-c', 'user.name=Tester', 'commit', '-q', '-m', 'init') + + # The committed content is returned from HEAD. + assert resolve_git_ref('HEAD', str(pipe_file)) == SAMPLE_PIPE + + # Working-tree edits do not affect what HEAD reports. + pipe_file.write_text(json.dumps({**SAMPLE_PIPE, 'version': 2}), encoding='utf-8') + assert resolve_git_ref('HEAD', str(pipe_file))['version'] == 1 + + # A file never committed is absent from HEAD -> None (treated as all-added). + untracked = tmp_path / 'untracked.pipe' + untracked.write_text(json.dumps(SAMPLE_PIPE), encoding='utf-8') + assert resolve_git_ref('HEAD', str(untracked)) is None + + +if __name__ == '__main__': + raise SystemExit(pytest.main([__file__, '-v'])) diff --git a/packages/client-python/tests/test_pipediff_reporters.py b/packages/client-python/tests/test_pipediff_reporters.py new file mode 100644 index 000000000..2ea2be211 --- /dev/null +++ b/packages/client-python/tests/test_pipediff_reporters.py @@ -0,0 +1,367 @@ +# MIT License +# +# Copyright (c) 2026 Aparavi Software AG +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +""" +Tests for rocketride.pipediff.reporters. + +The reporters are read-only over the diff model and are duck-typed, so these +tests build fixtures from lightweight dataclasses that mirror the pinned shapes +of ``rocketride.pipediff.model`` (owner: Implementer A). The real model +dataclasses are structurally identical; exercising the reporters against these +mirrors runs the true render code without depending on the engine being present. + +Coverage: + - empty diff, layout-only diff, and a mixed diff for each renderer + - human color on/off (ANSI escape presence) + - JSON document shape and summary contents + - Markdown table escaping of ``|`` and backticks + - deterministic ordering independent of engine emission order +""" + +import json +import unittest +from dataclasses import dataclass, field +from typing import Any, List, Optional, Tuple + +from rocketride.pipediff.reporters import render_human, render_json, render_markdown + + +# ========================================================================= +# Fixtures mirroring the pinned rocketride.pipediff.model dataclass shapes. +# ========================================================================= + + +@dataclass +class FieldChange: + """Mirror of model.FieldChange.""" + + path: str + kind: str # 'added' | 'removed' | 'changed' + old: Any = None + new: Any = None + + +@dataclass +class NodeChange: + """Mirror of model.NodeChange.""" + + id: str + kind: str # 'added' | 'removed' | 'provider' | 'config' + provider_old: Optional[str] = None + provider_new: Optional[str] = None + field_changes: List[FieldChange] = field(default_factory=list) + + +@dataclass +class EdgeChange: + """Mirror of model.EdgeChange.""" + + from_id: str + lane: str + to_id: str + kind: str # 'added' | 'removed' + + +@dataclass +class PipeDiff: + """Mirror of model.PipeDiff, including has_semantic_changes.""" + + node_changes: List[NodeChange] = field(default_factory=list) + edge_changes: List[EdgeChange] = field(default_factory=list) + version_change: Optional[Tuple[Any, Any]] = None + layout_changed: bool = False + + @property + def has_semantic_changes(self) -> bool: + """Layout is non-semantic; version/nodes/edges are.""" + return bool(self.node_changes or self.edge_changes or self.version_change) + + +def _mixed_diff() -> PipeDiff: + """Build a representative diff touching every section.""" + return PipeDiff( + node_changes=[ + NodeChange(id='qdrant_3', kind='added', provider_new='qdrant'), + NodeChange(id='webhook_1', kind='removed', provider_old='webhook'), + NodeChange(id='chat_1', kind='provider', provider_old='chat', provider_new='chat_v2'), + NodeChange( + id='preprocessor_langchain_1', + kind='config', + field_changes=[ + FieldChange(path='config.default.strlen', kind='changed', old=512, new=1024), + FieldChange(path='config.parameters.top_p', kind='added', new=0.9), + FieldChange(path='config.mode', kind='removed', old='Source'), + ], + ), + ], + edge_changes=[ + EdgeChange(from_id='parse_1', lane='text', to_id='preprocessor_langchain_1', kind='added'), + EdgeChange(from_id='webhook_1', lane='tags', to_id='parse_1', kind='removed'), + ], + version_change=(3, 4), + layout_changed=False, + ) + + +class TestRenderHuman(unittest.TestCase): + def test_empty_diff_reports_no_changes(self) -> None: + self.assertEqual(render_human(PipeDiff(), use_color=False), 'No semantic changes.') + + def test_layout_only_is_not_no_changes(self) -> None: + out = render_human(PipeDiff(layout_changed=True), use_color=False) + self.assertNotEqual(out, 'No semantic changes.') + self.assertIn('Layout', out) + + def test_mixed_sections_present(self) -> None: + out = render_human(_mixed_diff(), use_color=False) + self.assertIn('Nodes', out) + self.assertIn('Edges', out) + self.assertIn('Config', out) + # Added / removed / provider-change / config markers. + self.assertIn('+ qdrant_3 (qdrant)', out) + self.assertIn('- webhook_1 (webhook)', out) + self.assertIn('chat_1 provider: chat -> chat_v2', out) + self.assertIn('+ parse_1 --text--> preprocessor_langchain_1', out) + self.assertIn('- webhook_1 --tags--> parse_1', out) + self.assertIn('~ config.default.strlen: 512 -> 1024', out) + self.assertIn('Version: 3 -> 4', out) + + def test_no_color_has_no_ansi(self) -> None: + out = render_human(_mixed_diff(), use_color=False) + self.assertNotIn('\033', out) + + def test_color_emits_ansi(self) -> None: + out = render_human(_mixed_diff(), use_color=True) + self.assertIn('\033[', out) + + def test_no_trailing_newline(self) -> None: + out = render_human(_mixed_diff(), use_color=False) + self.assertFalse(out.endswith('\n')) + + +class TestRenderJson(unittest.TestCase): + def test_empty_diff_shape(self) -> None: + doc = render_json(PipeDiff()) + self.assertEqual(set(doc.keys()), {'nodes', 'edges', 'summary'}) + self.assertEqual(set(doc['nodes'].keys()), {'added', 'removed', 'changed'}) + self.assertEqual(set(doc['edges'].keys()), {'added', 'removed'}) + self.assertEqual(doc['nodes']['added'], []) + self.assertEqual(doc['edges']['removed'], []) + self.assertFalse(doc['summary']['has_semantic_changes']) + self.assertIsNone(doc['summary']['version_change']) + self.assertFalse(doc['summary']['layout_changed']) + + def test_layout_only_summary(self) -> None: + doc = render_json(PipeDiff(layout_changed=True)) + self.assertTrue(doc['summary']['layout_changed']) + self.assertFalse(doc['summary']['has_semantic_changes']) + + def test_mixed_structure(self) -> None: + doc = render_json(_mixed_diff()) + self.assertEqual(doc['nodes']['added'], [{'id': 'qdrant_3', 'provider': 'qdrant'}]) + self.assertEqual(doc['nodes']['removed'], [{'id': 'webhook_1', 'provider': 'webhook'}]) + + # Two changed nodes: chat_1 (provider) and preprocessor (config). + changed_by_id = {entry['id']: entry for entry in doc['nodes']['changed']} + self.assertEqual(changed_by_id['chat_1']['provider_change'], {'old': 'chat', 'new': 'chat_v2'}) + self.assertEqual(changed_by_id['chat_1']['config_changes'], []) + + preprocessor = changed_by_id['preprocessor_langchain_1'] + self.assertIsNone(preprocessor['provider_change']) + # config_changes deterministically sorted by (path, kind). + paths = [fc['path'] for fc in preprocessor['config_changes']] + self.assertEqual(paths, sorted(paths)) + for fc in preprocessor['config_changes']: + self.assertEqual(set(fc.keys()), {'path', 'kind', 'old', 'new'}) + + self.assertEqual( + doc['edges']['added'], + [{'from': 'parse_1', 'lane': 'text', 'to': 'preprocessor_langchain_1'}], + ) + self.assertEqual( + doc['edges']['removed'], + [{'from': 'webhook_1', 'lane': 'tags', 'to': 'parse_1'}], + ) + + summary = doc['summary'] + self.assertEqual(summary['nodes_added'], 1) + self.assertEqual(summary['nodes_removed'], 1) + self.assertEqual(summary['nodes_changed'], 2) + self.assertEqual(summary['edges_added'], 1) + self.assertEqual(summary['edges_removed'], 1) + self.assertEqual(summary['config_changes'], 3) + self.assertEqual(summary['provider_changes'], 1) + self.assertEqual(summary['version_change'], [3, 4]) + self.assertTrue(summary['has_semantic_changes']) + + def test_provider_and_config_on_same_id_merge(self) -> None: + # Engine may emit provider + config as separate NodeChange records for the + # same id; they must collapse into a single 'changed' entry. + diff = PipeDiff( + node_changes=[ + NodeChange(id='n1', kind='provider', provider_old='a', provider_new='b'), + NodeChange( + id='n1', + kind='config', + field_changes=[FieldChange(path='config.x', kind='changed', old=1, new=2)], + ), + ] + ) + doc = render_json(diff) + self.assertEqual(len(doc['nodes']['changed']), 1) + entry = doc['nodes']['changed'][0] + self.assertEqual(entry['provider_change'], {'old': 'a', 'new': 'b'}) + self.assertEqual(len(entry['config_changes']), 1) + + def test_json_serializable(self) -> None: + # The document must round-trip through the json module unchanged. + doc = render_json(_mixed_diff()) + self.assertEqual(json.loads(json.dumps(doc)), doc) + + +class TestRenderMarkdown(unittest.TestCase): + def test_empty_diff_summary(self) -> None: + out = render_markdown(PipeDiff()) + self.assertIn('no semantic changes', out) + self.assertNotIn('**Nodes**', out) + self.assertNotIn('**Edges**', out) + + def test_title_heading(self) -> None: + out = render_markdown(PipeDiff(), title='pipeline.pipe') + self.assertTrue(out.startswith('## pipeline.pipe')) + + def test_mixed_sections(self) -> None: + out = render_markdown(_mixed_diff()) + self.assertIn('**Pipeline diff:**', out) + self.assertIn('**Nodes**', out) + self.assertIn('**Edges**', out) + self.assertIn('**Config**', out) + self.assertIn('| Node | Field | Change |', out) + self.assertIn('`qdrant_3`', out) + self.assertIn('**Version:**', out) + + def test_layout_only(self) -> None: + out = render_markdown(PipeDiff(layout_changed=True)) + self.assertIn('Layout', out) + + def test_newlines_in_untrusted_values_cannot_break_out(self) -> None: + # A .pipe file is untrusted PR content, and render_markdown output is + # dumped verbatim into an auto-posted sticky PR comment. A newline in a + # provider/lane/id/config value must not terminate its code span and let + # the trailing text render as real Markdown (headings, @-mentions, etc.). + payload = 'x\n\n## INJECTED HEADING\n\n@everyone please approve' + diff = PipeDiff( + node_changes=[ + NodeChange(id='n1', kind='added', provider_new=payload), + NodeChange( + id='n2\n\n### id-inject', + kind='config', + field_changes=[ + FieldChange(path='config.p\n\n### path-inject', kind='changed', old='a\r\nb', new=payload), + ], + ), + ], + edge_changes=[ + EdgeChange(from_id='a', lane='l\n\n### lane-inject', to_id='b', kind='added'), + ], + ) + out = render_markdown(diff, title='rag.pipe') + # The injected content must never break out of its code span onto its own + # line — that is what would make '## INJECTED HEADING' render as a real + # heading. Every injected marker must survive only inline inside a span, + # never at the start of a line, and no attacker-introduced blank line may + # appear before a heading. + self.assertNotIn('\n\n#', out) + for line in out.splitlines(): + self.assertFalse(line.lstrip().startswith('## INJECTED'), line) + self.assertFalse(line.lstrip().startswith('### '), line) + self.assertFalse(line.lstrip().startswith('@everyone'), line) + # Carriage returns are neutralized too (a lone \r can also break rows). + self.assertNotIn('\r', out) + + def test_table_escapes_pipe_and_backtick(self) -> None: + diff = PipeDiff( + node_changes=[ + NodeChange( + id='n1', + kind='config', + field_changes=[ + FieldChange(path='config.template', kind='changed', old='a|b', new='x`y'), + ], + ) + ] + ) + out = render_markdown(diff) + # The literal pipe from the value must be escaped inside the table cell. + self.assertIn('\\|', out) + self.assertNotIn('a|b', out) + # A backtick inside a value forces a longer code fence. + self.assertIn('``', out) + # Every table body row must have balanced, escaped columns (no stray bar). + table_rows = [ln for ln in out.splitlines() if ln.startswith('| ') and '---' not in ln] + for row in table_rows: + # Unescaped pipes only appear as the 4 column delimiters. + unescaped = row.replace('\\|', '') + self.assertEqual(unescaped.count('|'), 4, row) + + +class TestDeterminism(unittest.TestCase): + def _scrambled_diff(self) -> PipeDiff: + """Same logical changes as _mixed_diff but emitted in a different order.""" + return PipeDiff( + node_changes=[ + NodeChange( + id='preprocessor_langchain_1', + kind='config', + field_changes=[ + FieldChange(path='config.mode', kind='removed', old='Source'), + FieldChange(path='config.parameters.top_p', kind='added', new=0.9), + FieldChange(path='config.default.strlen', kind='changed', old=512, new=1024), + ], + ), + NodeChange(id='chat_1', kind='provider', provider_old='chat', provider_new='chat_v2'), + NodeChange(id='webhook_1', kind='removed', provider_old='webhook'), + NodeChange(id='qdrant_3', kind='added', provider_new='qdrant'), + ], + edge_changes=[ + EdgeChange(from_id='webhook_1', lane='tags', to_id='parse_1', kind='removed'), + EdgeChange(from_id='parse_1', lane='text', to_id='preprocessor_langchain_1', kind='added'), + ], + version_change=(3, 4), + layout_changed=False, + ) + + def test_human_is_order_independent(self) -> None: + self.assertEqual( + render_human(_mixed_diff(), use_color=False), + render_human(self._scrambled_diff(), use_color=False), + ) + + def test_json_is_order_independent(self) -> None: + self.assertEqual(render_json(_mixed_diff()), render_json(self._scrambled_diff())) + + def test_markdown_is_order_independent(self) -> None: + self.assertEqual(render_markdown(_mixed_diff()), render_markdown(self._scrambled_diff())) + + +if __name__ == '__main__': + unittest.main() diff --git a/packages/docs/content-static/cli.mdx b/packages/docs/content-static/cli.mdx index 2d5d24efc..04f8fc6f3 100644 --- a/packages/docs/content-static/cli.mdx +++ b/packages/docs/content-static/cli.mdx @@ -14,9 +14,9 @@ installing either package puts `rocketride` on your path. > **Python vs TypeScript CLI:** Both packages ship a `rocketride` command. The > core commands (`start`, `upload`, `status`, `stop`, `store`) are available in -> both, but flag names differ in a few places and the Python CLI includes two -> additional commands (`events`, `list`) not present in TypeScript. Differences -> are called out inline below. +> both, but flag names differ in a few places and the Python CLI includes three +> additional commands (`events`, `list`, `diff`) not present in TypeScript. +> Differences are called out inline below. ## Install @@ -117,8 +117,9 @@ rocketride stop --token ## Connecting -Every command accepts connection options, which also read from the environment -so you can set them once: +Every command _except_ [`diff`](#diff) — which runs entirely locally and never +contacts an engine — accepts connection options, which also read from the +environment so you can set them once: | Option | Env var | Default | Description | | ---------------- | ---------------------- | --------------------- | ---------------------------------------------------------------------- | @@ -141,6 +142,7 @@ connection is encrypted. | `events` | Stream all raw events from a running task. Python CLI only. | | `list` | List all active tasks. Python CLI only. | | `store` | File-store operations: `dir`, `type`, `write`, `rm`, `mkdir`, `stat`. | +| `diff` | Semantic diff of two `.pipe` files, ignoring canvas noise. Local; no server. Python CLI only. | ### start @@ -310,6 +312,204 @@ Sub-commands: | `mkdir ` | Create a directory in the store. | | `stat ` | Print metadata (size, type, modified time) for ``. | +### diff + +Use `diff` to see what actually changed between two versions of a `.pipe` file. +A raw `git diff` of a pipeline is dominated by **canvas coordinate churn** — every +time you nudge a node, its `ui.position` `x`/`y` values change, and the top-level +`viewport` shifts when you pan or zoom. Those lines say nothing about behavior, +yet they bury the changes that do: a swapped LLM provider, a re-tuned chunk size, +a rewired retrieval step. `diff` computes a **semantic** diff instead — it groups +changes into **Nodes** (added / removed / re-provisioned), **Edges** (the wiring +between components), and **Config** (per-field changes as readable dotted paths) — +and collapses all layout churn into a single line. + +> **Local only — no engine, no auth.** Unlike every other command, `diff` never +> connects to an engine or the network: it reads files (or a git ref) and compares +> parsed JSON entirely on your machine. It therefore takes **none** of the +> `--uri` / `--apikey` / `--token` connection options the other commands share. +> Available in the Python CLI only. + +```bash +# Compare two files on disk (old, then new) +rocketride diff old.pipe new.pipe + +# Compare a working-tree file against a git ref (uses `git show :`) +rocketride diff --git HEAD rag.pipe +rocketride diff --git main rag.pipe + +# Machine-readable output +rocketride diff old.pipe new.pipe --json +rocketride diff old.pipe new.pipe --markdown + +# Include the layout churn that is hidden by default +rocketride diff old.pipe new.pipe --include-layout +``` + +Key flags: + +| Flag | Description | +| --- | --- | +| ` ` | The two files to compare (positional). Omit one and pass `--git` instead. | +| `--git ` | Diff the single working-tree `FILE` against `` via `git show :`. If the file does not exist in ``, everything is reported as added. | +| `--include-layout` | Enumerate the layout churn (each node's `ui` block and the top-level `viewport`) that is hidden by default, as `ui.*` field changes. | +| `--json` | Emit a single JSON document to stdout (mutually exclusive with `--markdown`). | +| `--markdown` | Emit compact, PR-comment-friendly Markdown to stdout (mutually exclusive with `--json`). | +| `--exit-zero` | Always exit `0` on a successful run, even when changes are found. Use for informational, non-gating runs. | + +**Exit codes** make `diff` easy to gate a CI job on: + +| Code | Meaning | +| --- | --- | +| `0` | No semantic changes (or any successful run when `--exit-zero` is set). | +| `1` | Semantic changes were found. | +| `2` | Usage error, or an unreadable / unparseable file or bad git ref. | + +#### What counts as a change + +- **Nodes** are matched by `id`. A new or deleted `id` is an add/remove; an `id` + present on both sides with a different `provider` is a provider change. +- **Config** is deep-diffed into dotted paths — nested objects become + `config.default.strlen`, list items become `config.instructions[0]` — and each + leaf is reported as added, removed, or changed with its old → new value. +- **Edges** are the directed wires between components, reconstructed from every + component's `input[]` (data lanes) and `control[]` (agent orchestration lanes, + such as an agent's `llm`, `tool`, or `memory`). Rewiring a step shows up as a + removed edge plus an added edge. +- **Version** changes (the top-level `version` field) are always reported and + always count as a change — they are never hidden. +- **Layout** — each node's `ui` block and the `viewport` — is ignored by default + and summarized as a single `Layout: changed` line. A pure canvas move is **not** + a semantic change, so `diff` exits `0` on it. Pass `--include-layout` to see the + individual `ui.*` coordinates. + +#### Output modes + +The default **human** output is grouped and colored (color auto-disables when +piped or when `NO_COLOR` is set), with `+` added, `-` removed, and `~` changed: + +```text +Pipeline diff: 1 node changed, layout changed + +Config + chunker_1 + ~ config.default.strlen: 512 -> 1024 + +Layout: changed (ui/viewport) +``` + +`--json` emits one stable, sorted document — `nodes`, `edges`, and a `summary` +block with counts and the overall `has_semantic_changes` flag — ideal for feeding +another tool: + +```json +{ + "edges": { "added": [], "removed": [] }, + "nodes": { + "added": [], + "changed": [ + { + "config_changes": [ + { "kind": "changed", "new": 1024, "old": 512, "path": "config.default.strlen" } + ], + "id": "chunker_1", + "provider_change": null + } + ], + "removed": [] + }, + "summary": { + "config_changes": 1, + "edges_added": 0, + "edges_removed": 0, + "has_semantic_changes": true, + "layout_changed": true, + "nodes_added": 0, + "nodes_changed": 1, + "nodes_removed": 0, + "provider_changes": 0, + "version_change": null + } +} +``` + +`--markdown` emits a compact, PR-comment-friendly report — a one-line summary, +bullet lists for nodes and edges, and a table for config changes (with `|` and +backticks safely escaped so untrusted values can't break the comment): + +```markdown +**Pipeline diff:** 1 node changed, layout changed + +**Config** + +| Node | Field | Change | +| --- | --- | --- | +| `chunker_1` | `config.default.strlen` | `512` → `1024` | + +_Layout (ui/viewport) changed._ +``` + +#### Review pipelines like code in CI + +Because `diff` is local and exits non-zero on change, it drops straight into a +pull-request check. The supported way to wire it up is the bundled +[`pipe-diff` composite action](https://github.com/rocketride-org/rocketride-server/tree/develop/.github/actions/pipe-diff), +which finds every changed `.pipe`, diffs each against the PR base, and posts one +**sticky** comment so reviewers see "OpenAI → Anthropic, chunk size 512 → 1024, +retrieval rewired through a reranker" instead of a wall of coordinate noise: + +```yaml +# .github/workflows/pipe-diff.yml +name: Pipeline diff +on: pull_request +permissions: + contents: read + pull-requests: write +jobs: + diff: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 # default fetch-depth: 1 is fine; the action fetches the base itself + - uses: ./.github/actions/pipe-diff +``` + +The action installs the CLI, resolves the PR base commit, and maintains a single +sticky comment for you. See its +[README](https://github.com/rocketride-org/rocketride-server/blob/develop/.github/actions/pipe-diff/README.md) +for the `files`, `cli-version`, `comment`, and `include-layout` inputs. + +If you would rather not vendor the composite action, the same result can be +assembled inline — check out with `fetch-depth: 0` so the base branch is present +for `--git`, then diff and post the Markdown yourself: + +```yaml +# Alternative: inline equivalent of the composite action, for a single file. + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # need the base branch for `git show` + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + - run: pip install rocketride + - name: Semantic diff of the pipeline + run: | + rocketride diff --git "origin/${{ github.base_ref }}" rag.pipe \ + --markdown --exit-zero | tee pipe-diff.md + - name: Comment on the PR + uses: marocchino/sticky-pull-request-comment@v2 + with: + path: pipe-diff.md +``` + +This is the review half of the "pipelines as code" story: **validate** a `.pipe` +to confirm it is well-formed before it runs (via +[`client.validate()`](/develop/python)), **[evaluate](/evaluate/use-cases)** it to +measure output quality, and `diff` it to review exactly what changed between +revisions — the same lint / test / review loop teams already run on their source, +now applied to their pipelines. See the +[side-by-side example](https://github.com/rocketride-org/rocketride-server/blob/develop/examples/pipe-diff-example.md) +for the full before/after contrast and the PR-comment recipe. + ## Related - [TypeScript SDK](/develop/typescript) · [Python SDK](/develop/python): the