Repair build, fix execution engine, and add headless CLI + tests - #1
Conversation
The project no longer compiled and would panic at startup. This restores a
working build and adds substantial functionality on top.
Foundation / correctness
- Rewrite the tool-catalog loader to parse the current tools.yaml map schema
(cat/in/out/def/params) instead of the stale flat-list format that made
init() panic. Embed tools.yaml via go:embed so the binary is
self-contained and runs from any directory.
- Fix internal/tui/builder.go: add the missing selNode field, migrate to the
bubbletea v1.3.5 mouse API, replace the non-existent list.Separator and
lipgloss Unset() usages, and resolve pipe-type checks through the DAG.
- Add DAG helpers MaxLayer/RemoveFromLayer/InsertAtLayer used by the builder.
- Fix GetParallelNodes so all parallel nodes in a layer share one concurrent
group instead of being split into sequential steps.
Execution engine
- Unify placeholder handling: both {{input}}/{{domain}}/{{output}} and
$(target_file)/$(target)/$(output) are now supported (substituteArgs).
- Capture tool stdout into the output file when no output placeholder is
present, so stdout-streaming tools (e.g. "-o -") actually produce results.
- Merge and de-duplicate every tool's output in a layer before feeding the
next layer, instead of forwarding only the first tool's file.
Headless CLI
- cmd/termaid now supports run/preview/tools/validate/version/help
subcommands and still launches the TUI when invoked with no arguments,
making termaid scriptable and CI-friendly.
Quality
- Add unit tests for graph, catalog, and pipeline.
- Add a GitHub Actions CI workflow (gofmt/vet/test -race/build), a Makefile,
and a .gitignore.
- gofmt the whole tree; remove the accidental empty project.zip.
- Update README to document the CLI, placeholder styles, and stdout capture.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MQHqzzjxxDUvDePyPerast
Reviewer's GuideRestores a working build, modernizes the TUI builder and DAG execution semantics, and adds a headless CLI plus tests/CI by fixing catalog loading, dataflow, and placeholder handling while wiring the graph/pipeline layers together. Sequence diagram for headless CLI run executionsequenceDiagram
actor User
participant termaid_main
participant tui_RunHeadless as tui_RunHeadless
participant tui_LoadWorkflow as tui_LoadWorkflow
participant graph_DAG as graph_DAG
participant pipeline_Run as pipeline_Run
User->>termaid_main: termaid run -d domain -w workflow.json
termaid_main->>tui_RunHeadless: RunHeadless(ctx, path, domain, workdir, conc, writer)
tui_RunHeadless->>tui_LoadWorkflow: LoadWorkflow(path)
tui_LoadWorkflow->>graph_DAG: construct DAG from workflow
tui_LoadWorkflow-->>tui_RunHeadless: *graph.DAG
tui_RunHeadless->>pipeline_Run: Run(ctx, domain, workdir, cats, conc, statusCh)
pipeline_Run-->>tui_RunHeadless: Status events
tui_RunHeadless-->>termaid_main: error or nil
termaid_main-->>User: exit code and logs
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (22)
WalkthroughThe PR adds a command-line interface, headless workflow execution, embedded tool-catalog loading, pipeline output merging, DAG updates, TUI changes, build automation, CI checks, documentation, and formatting cleanup. ChangesTermaid functional stack
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant CLI
participant RunHeadless
participant Pipeline
participant StatusWriter
CLI->>RunHeadless: provide workflow, domain, workdir, and concurrency
RunHeadless->>Pipeline: load, validate, and execute workflow
Pipeline-->>StatusWriter: emit start, finish, and error statuses
RunHeadless-->>CLI: return execution result and completion summary
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Hey - I've found 2 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="internal/graph/dag_test.go" line_range="127-130" />
<code_context>
+ }
+}
+
+func TestToMermaidAndJSON(t *testing.T) {
+ g := NewDAG()
+ _ = g.AddNode("input", "subfinder-1", "subfinder", "-d {{domain}}", 1)
+
+ mmd := g.ToMermaid()
+ if !strings.HasPrefix(mmd, "graph LR") {
+ t.Fatalf("mermaid missing header: %q", mmd)
+ }
+
+ js := g.ToJSON()
+ if !strings.Contains(js, `"subfinder-1"`) || !strings.Contains(js, `"version": "2.0"`) {
+ t.Fatalf("json missing expected content: %s", js)
+ }
+}
</code_context>
<issue_to_address>
**suggestion (testing):** Add negative tests for ValidateMatrix to cover matrix inconsistency cases
Current tests only exercise happy-path behavior like ToMermaid/ToJSON and structural helpers. Please add tests that assert ValidateMatrix fails in specific cases, such as:
- Two nodes sharing the same Coordinate when at least one has Parallel == false, verifying the conflict error.
- A node present in g.Nodes but missing from g.Matrix at its (Layer, Position), verifying the "node not found in matrix" path.
These will ensure future DAG mutations keep the matrix and nodes in sync.
```suggestion
if !strings.Contains(js, `"subfinder-1"`) || !strings.Contains(js, `"version": "2.0"`) {
t.Fatalf("json missing expected content: %s", js)
}
}
func TestValidateMatrixRejectsNonParallelCoordinateConflicts(t *testing.T) {
g := NewDAG()
if err := g.AddNode("input", "a-1", "a", "", 1); err != nil {
t.Fatalf("unexpected error adding node a-1: %v", err)
}
if err := g.AddNode("a-1", "b-1", "b", "", 1); err != nil {
t.Fatalf("unexpected error adding node b-1: %v", err)
}
// Force both nodes onto the same coordinate, but make at least one non-parallel.
a := g.Nodes["a-1"]
b := g.Nodes["b-1"]
a.Layer, a.Position = 0, 0
b.Layer, b.Position = 0, 0
// Explicitly set parallel flags to exercise the conflict path.
a.Parallel = false
b.Parallel = true
g.Nodes["a-1"] = a
g.Nodes["b-1"] = b
// Manually construct a conflicting matrix cell.
if g.Matrix == nil {
g.Matrix = make(map[int]map[int][]string)
}
if g.Matrix[0] == nil {
g.Matrix[0] = make(map[int][]string)
}
g.Matrix[0][0] = []string{"a-1", "b-1"}
if err := g.ValidateMatrix(); err == nil {
t.Fatal("expected ValidateMatrix to fail for non-parallel nodes sharing a coordinate")
}
}
func TestValidateMatrixRejectsMissingNodeInMatrix(t *testing.T) {
g := NewDAG()
if err := g.AddNode("input", "a-1", "a", "", 1); err != nil {
t.Fatalf("unexpected error adding node a-1: %v", err)
}
// Sanity check: the initial DAG should be valid.
if err := g.ValidateMatrix(); err != nil {
t.Fatalf("expected initial matrix to be valid, got error: %v", err)
}
// Remove the node from the matrix at its (Layer, Position) but keep it in g.Nodes.
n := g.Nodes["a-1"]
layer, pos := n.Layer, n.Position
cell := g.Matrix[layer][pos]
var filtered []string
for _, id := range cell {
if id != "a-1" {
filtered = append(filtered, id)
}
}
g.Matrix[layer][pos] = filtered
if err := g.ValidateMatrix(); err == nil {
t.Fatal("expected ValidateMatrix to fail when node is missing from matrix at its coordinate")
}
}
```
</issue_to_address>
### Comment 2
<location path="internal/graph/dag_test.go" line_range="102-26" />
<code_context>
+ }
+}
+
+func TestInsertAtLayerRepositions(t *testing.T) {
+ g := NewDAG()
+ _ = g.AddNodeAtPosition("input", "a-1", "a", "", 1, 0, "", false)
+
+ g.RemoveFromLayer("a-1")
+ g.InsertAtLayer("a-1", 3, 2)
+
+ if n := g.Nodes["a-1"]; n.Layer != 3 || n.Position != 2 {
+ t.Fatalf("node position = (%d,%d), want (3,2)", n.Layer, n.Position)
+ }
+ if g.MaxLayer() != 3 {
+ t.Fatalf("MaxLayer = %d, want 3", g.MaxLayer())
+ }
+}
+
+func TestToMermaidAndJSON(t *testing.T) {
</code_context>
<issue_to_address>
**suggestion (testing):** Exercise MoveNode/CompactLayer and RemoveFromLayer on non-existent IDs to better cover DAG helpers
Please also add tests for the remaining DAG helpers:
- MoveNode and CompactLayer: move nodes across layers/positions, then compact a layer, and assert matrix membership, MaxX/MaxY, and ordering stay consistent.
- RemoveFromLayer with an unknown ID: verify it’s a no-op and does not panic.
This will better validate that the new layout-editing helpers keep the internal matrix consistent.
Suggested implementation:
```golang
func TestInsertAtLayerRepositions(t *testing.T) {
g := NewDAG()
_ = g.AddNodeAtPosition("input", "a-1", "a", "", 1, 0, "", false)
g.RemoveFromLayer("a-1")
g.InsertAtLayer("a-1", 3, 2)
if n := g.Nodes["a-1"]; n.Layer != 3 || n.Position != 2 {
t.Fatalf("node position = (%d,%d), want (3,2)", n.Layer, n.Position)
}
if g.MaxLayer() != 3 {
t.Fatalf("MaxLayer = %d, want 3", g.MaxLayer())
}
}
func TestMoveNodeAndCompactLayer(t *testing.T) {
g := NewDAG()
// layer 1: a-1 at pos 0, a-2 at pos 1
_ = g.AddNodeAtPosition("input", "a-1", "a", "", 1, 0, "", false)
_ = g.AddNodeAtPosition("input", "a-2", "a", "", 1, 1, "", false)
// layer 2: b-1 at pos 0
_ = g.AddNodeAtPosition("input", "b-1", "b", "", 2, 0, "", false)
// sanity: initial max dimensions
initialMaxX := g.MaxX()
initialMaxY := g.MaxY()
if initialMaxY != 2 {
t.Fatalf("initial MaxY = %d, want 2", initialMaxY)
}
// move a-2 from (1,1) -> (2,1)
g.MoveNode("a-2", 2, 1)
if n := g.Nodes["a-2"]; n.Layer != 2 || n.Position != 1 {
t.Fatalf("a-2 position = (%d,%d), want (2,1)", n.Layer, n.Position)
}
// compact layer 1 (should only contain a-1 at position 0 after compaction)
g.CompactLayer(1)
if n := g.Nodes["a-1"]; n.Layer != 1 || n.Position != 0 {
t.Fatalf("a-1 position after compact = (%d,%d), want (1,0)", n.Layer, n.Position)
}
// verify MaxX/MaxY stay consistent and non-decreasing in terms of capacity
if maxX, maxY := g.MaxX(), g.MaxY(); maxX < 0 || maxY < 1 {
t.Fatalf("unexpected MaxX/MaxY after moves/compaction: (%d,%d)", maxX, maxY)
}
// ensure moved node is still present in the layout matrix at its new coordinates
// (implementation relies on the internal matrix representation being indexed by [layer][position])
if nodeAtNewPos := g.matrix[g.Nodes["a-2"].Layer][g.Nodes["a-2"].Position]; nodeAtNewPos == nil || nodeAtNewPos.ID != "a-2" {
t.Fatalf("a-2 not found in layout matrix at its new position")
}
}
func TestRemoveFromLayerUnknownIDIsNoOp(t *testing.T) {
g := NewDAG()
_ = g.AddNodeAtPosition("input", "a-1", "a", "", 1, 0, "", false)
_ = g.AddNodeAtPosition("input", "b-1", "b", "", 2, 0, "", false)
beforeNodes := len(g.Nodes)
beforeMaxLayer := g.MaxLayer()
beforeMaxX := g.MaxX()
beforeMaxY := g.MaxY()
// removing an unknown ID should be a no-op and must not panic
g.RemoveFromLayer("non-existent-id")
if len(g.Nodes) != beforeNodes {
t.Fatalf("RemoveFromLayer(non-existent) changed node count: got %d, want %d", len(g.Nodes), beforeNodes)
}
if g.MaxLayer() != beforeMaxLayer {
t.Fatalf("RemoveFromLayer(non-existent) changed MaxLayer: got %d, want %d", g.MaxLayer(), beforeMaxLayer)
}
if g.MaxX() != beforeMaxX || g.MaxY() != beforeMaxY {
t.Fatalf("RemoveFromLayer(non-existent) changed MaxX/MaxY: got (%d,%d), want (%d,%d)",
g.MaxX(), g.MaxY(), beforeMaxX, beforeMaxY)
}
}
func TestToMermaidAndJSON(t *testing.T) {
```
The new tests assume:
1. `MoveNode(id string, layer, position int)` and `CompactLayer(layer int)` exist and do not return errors.
2. `MaxX()` and `MaxY()` are defined DAG helpers returning the maximum X (position) and Y (layer) extents.
3. The internal layout matrix is accessible as `g.matrix[layer][position]` and stores nodes with an `ID` field; if your actual matrix representation differs (e.g., different field name or structure), update the access in `TestMoveNodeAndCompactLayer` accordingly.
4. If `MoveNode`, `CompactLayer`, `MaxX`, or `MaxY` can return errors or have different signatures, adjust the test calls to handle those return values while preserving the assertions about node positions, matrix membership, and dimension consistency.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| if !strings.Contains(js, `"subfinder-1"`) || !strings.Contains(js, `"version": "2.0"`) { | ||
| t.Fatalf("json missing expected content: %s", js) | ||
| } | ||
| } |
There was a problem hiding this comment.
suggestion (testing): Add negative tests for ValidateMatrix to cover matrix inconsistency cases
Current tests only exercise happy-path behavior like ToMermaid/ToJSON and structural helpers. Please add tests that assert ValidateMatrix fails in specific cases, such as:
- Two nodes sharing the same Coordinate when at least one has Parallel == false, verifying the conflict error.
- A node present in g.Nodes but missing from g.Matrix at its (Layer, Position), verifying the "node not found in matrix" path.
These will ensure future DAG mutations keep the matrix and nodes in sync.
| if !strings.Contains(js, `"subfinder-1"`) || !strings.Contains(js, `"version": "2.0"`) { | |
| t.Fatalf("json missing expected content: %s", js) | |
| } | |
| } | |
| if !strings.Contains(js, `"subfinder-1"`) || !strings.Contains(js, `"version": "2.0"`) { | |
| t.Fatalf("json missing expected content: %s", js) | |
| } | |
| } | |
| func TestValidateMatrixRejectsNonParallelCoordinateConflicts(t *testing.T) { | |
| g := NewDAG() | |
| if err := g.AddNode("input", "a-1", "a", "", 1); err != nil { | |
| t.Fatalf("unexpected error adding node a-1: %v", err) | |
| } | |
| if err := g.AddNode("a-1", "b-1", "b", "", 1); err != nil { | |
| t.Fatalf("unexpected error adding node b-1: %v", err) | |
| } | |
| // Force both nodes onto the same coordinate, but make at least one non-parallel. | |
| a := g.Nodes["a-1"] | |
| b := g.Nodes["b-1"] | |
| a.Layer, a.Position = 0, 0 | |
| b.Layer, b.Position = 0, 0 | |
| // Explicitly set parallel flags to exercise the conflict path. | |
| a.Parallel = false | |
| b.Parallel = true | |
| g.Nodes["a-1"] = a | |
| g.Nodes["b-1"] = b | |
| // Manually construct a conflicting matrix cell. | |
| if g.Matrix == nil { | |
| g.Matrix = make(map[int]map[int][]string) | |
| } | |
| if g.Matrix[0] == nil { | |
| g.Matrix[0] = make(map[int][]string) | |
| } | |
| g.Matrix[0][0] = []string{"a-1", "b-1"} | |
| if err := g.ValidateMatrix(); err == nil { | |
| t.Fatal("expected ValidateMatrix to fail for non-parallel nodes sharing a coordinate") | |
| } | |
| } | |
| func TestValidateMatrixRejectsMissingNodeInMatrix(t *testing.T) { | |
| g := NewDAG() | |
| if err := g.AddNode("input", "a-1", "a", "", 1); err != nil { | |
| t.Fatalf("unexpected error adding node a-1: %v", err) | |
| } | |
| // Sanity check: the initial DAG should be valid. | |
| if err := g.ValidateMatrix(); err != nil { | |
| t.Fatalf("expected initial matrix to be valid, got error: %v", err) | |
| } | |
| // Remove the node from the matrix at its (Layer, Position) but keep it in g.Nodes. | |
| n := g.Nodes["a-1"] | |
| layer, pos := n.Layer, n.Position | |
| cell := g.Matrix[layer][pos] | |
| var filtered []string | |
| for _, id := range cell { | |
| if id != "a-1" { | |
| filtered = append(filtered, id) | |
| } | |
| } | |
| g.Matrix[layer][pos] = filtered | |
| if err := g.ValidateMatrix(); err == nil { | |
| t.Fatal("expected ValidateMatrix to fail when node is missing from matrix at its coordinate") | |
| } | |
| } |
| if kids := g.Nodes["input"].Children; len(kids) != 1 || kids[0] != "subfinder-1" { | ||
| t.Fatalf("input children = %v, want [subfinder-1]", kids) | ||
| } | ||
| } |
There was a problem hiding this comment.
suggestion (testing): Exercise MoveNode/CompactLayer and RemoveFromLayer on non-existent IDs to better cover DAG helpers
Please also add tests for the remaining DAG helpers:
- MoveNode and CompactLayer: move nodes across layers/positions, then compact a layer, and assert matrix membership, MaxX/MaxY, and ordering stay consistent.
- RemoveFromLayer with an unknown ID: verify it’s a no-op and does not panic.
This will better validate that the new layout-editing helpers keep the internal matrix consistent.
Suggested implementation:
func TestInsertAtLayerRepositions(t *testing.T) {
g := NewDAG()
_ = g.AddNodeAtPosition("input", "a-1", "a", "", 1, 0, "", false)
g.RemoveFromLayer("a-1")
g.InsertAtLayer("a-1", 3, 2)
if n := g.Nodes["a-1"]; n.Layer != 3 || n.Position != 2 {
t.Fatalf("node position = (%d,%d), want (3,2)", n.Layer, n.Position)
}
if g.MaxLayer() != 3 {
t.Fatalf("MaxLayer = %d, want 3", g.MaxLayer())
}
}
func TestMoveNodeAndCompactLayer(t *testing.T) {
g := NewDAG()
// layer 1: a-1 at pos 0, a-2 at pos 1
_ = g.AddNodeAtPosition("input", "a-1", "a", "", 1, 0, "", false)
_ = g.AddNodeAtPosition("input", "a-2", "a", "", 1, 1, "", false)
// layer 2: b-1 at pos 0
_ = g.AddNodeAtPosition("input", "b-1", "b", "", 2, 0, "", false)
// sanity: initial max dimensions
initialMaxX := g.MaxX()
initialMaxY := g.MaxY()
if initialMaxY != 2 {
t.Fatalf("initial MaxY = %d, want 2", initialMaxY)
}
// move a-2 from (1,1) -> (2,1)
g.MoveNode("a-2", 2, 1)
if n := g.Nodes["a-2"]; n.Layer != 2 || n.Position != 1 {
t.Fatalf("a-2 position = (%d,%d), want (2,1)", n.Layer, n.Position)
}
// compact layer 1 (should only contain a-1 at position 0 after compaction)
g.CompactLayer(1)
if n := g.Nodes["a-1"]; n.Layer != 1 || n.Position != 0 {
t.Fatalf("a-1 position after compact = (%d,%d), want (1,0)", n.Layer, n.Position)
}
// verify MaxX/MaxY stay consistent and non-decreasing in terms of capacity
if maxX, maxY := g.MaxX(), g.MaxY(); maxX < 0 || maxY < 1 {
t.Fatalf("unexpected MaxX/MaxY after moves/compaction: (%d,%d)", maxX, maxY)
}
// ensure moved node is still present in the layout matrix at its new coordinates
// (implementation relies on the internal matrix representation being indexed by [layer][position])
if nodeAtNewPos := g.matrix[g.Nodes["a-2"].Layer][g.Nodes["a-2"].Position]; nodeAtNewPos == nil || nodeAtNewPos.ID != "a-2" {
t.Fatalf("a-2 not found in layout matrix at its new position")
}
}
func TestRemoveFromLayerUnknownIDIsNoOp(t *testing.T) {
g := NewDAG()
_ = g.AddNodeAtPosition("input", "a-1", "a", "", 1, 0, "", false)
_ = g.AddNodeAtPosition("input", "b-1", "b", "", 2, 0, "", false)
beforeNodes := len(g.Nodes)
beforeMaxLayer := g.MaxLayer()
beforeMaxX := g.MaxX()
beforeMaxY := g.MaxY()
// removing an unknown ID should be a no-op and must not panic
g.RemoveFromLayer("non-existent-id")
if len(g.Nodes) != beforeNodes {
t.Fatalf("RemoveFromLayer(non-existent) changed node count: got %d, want %d", len(g.Nodes), beforeNodes)
}
if g.MaxLayer() != beforeMaxLayer {
t.Fatalf("RemoveFromLayer(non-existent) changed MaxLayer: got %d, want %d", g.MaxLayer(), beforeMaxLayer)
}
if g.MaxX() != beforeMaxX || g.MaxY() != beforeMaxY {
t.Fatalf("RemoveFromLayer(non-existent) changed MaxX/MaxY: got (%d,%d), want (%d,%d)",
g.MaxX(), g.MaxY(), beforeMaxX, beforeMaxY)
}
}
func TestToMermaidAndJSON(t *testing.T) {The new tests assume:
MoveNode(id string, layer, position int)andCompactLayer(layer int)exist and do not return errors.MaxX()andMaxY()are defined DAG helpers returning the maximum X (position) and Y (layer) extents.- The internal layout matrix is accessible as
g.matrix[layer][position]and stores nodes with anIDfield; if your actual matrix representation differs (e.g., different field name or structure), update the access inTestMoveNodeAndCompactLayeraccordingly. - If
MoveNode,CompactLayer,MaxX, orMaxYcan return errors or have different signatures, adjust the test calls to handle those return values while preserving the assertions about node positions, matrix membership, and dimension consistency.
The project no longer compiled and would panic at startup. This restores a
working build and adds substantial functionality on top.
Foundation / correctness
(cat/in/out/def/params) instead of the stale flat-list format that made
init() panic. Embed tools.yaml via go:embed so the binary is
self-contained and runs from any directory.
bubbletea v1.3.5 mouse API, replace the non-existent list.Separator and
lipgloss Unset() usages, and resolve pipe-type checks through the DAG.
group instead of being split into sequential steps.
Execution engine
present, so stdout-streaming tools (e.g. "-o -") actually produce results.
next layer, instead of forwarding only the first tool's file.
Headless CLI
subcommands and still launches the TUI when invoked with no arguments,
making termaid scriptable and CI-friendly.
Quality
and a .gitignore.
Co-Authored-By: Claude Fable 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01MQHqzzjxxDUvDePyPerast
Summary by Sourcery
Restore a reliable build and execution engine while making workflows scriptable through a self-contained headless CLI.
New Features:
run,preview,tools,validate,version, andhelpcommands while preserving the interactive TUI by default.Bug Fixes:
Enhancements:
Build:
CI:
Documentation:
Tests:
Summary by CodeRabbit
New Features
Documentation
Chores