Skip to content

fix(import): link Workflow-tool inner agents on CLI rescan (offline/headless runs)#235

Merged
hoangsonww merged 1 commit into
hoangsonww:masterfrom
omar-merhebi:fix/cli-import-links-workflow-subagents
Jul 17, 2026
Merged

fix(import): link Workflow-tool inner agents on CLI rescan (offline/headless runs)#235
hoangsonww merged 1 commit into
hoangsonww:masterfrom
omar-merhebi:fix/cli-import-links-workflow-subagents

Conversation

@omar-merhebi

@omar-merhebi omar-merhebi commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

A Claude Code Workflow-tool run (dynamic workflow / fleet of sub-agents) emits no hooks, so when it runs where hook events never reach the dashboard server — a headless claude -p run, a CI job, or an HPC/cluster compute node — its per-run journal is never ingested live. The offline CLI import path (ccam import rescan / ccam import path) never closed that gap, so a rescan imported the inner-agent transcripts but left them orphaned from their run (workflow_run_id = NULL), and the workflow kept showing 1 agent instead of the real N. This wires the existing workflow-journal ingest into the CLI import path so a rescan links them, mirroring what the server already does.

Changes

  • scripts/import-history.js: after each import batch, importAllSessions (backs ccam import rescan / POST /api/import/rescan) and importFromDirectory (backs ccam import path / /api/import/scan-path + /upload) now call the existing ingestWorkflowsForSession per imported session — via a small shared ingestWorkflowsForBatch helper. It runs outside the better-sqlite3 transaction (the ingest is async) at the exact spot the durable-snapshot pass already runs.
  • The ingest is reused as-is, not reimplemented (server/lib/workflow-ingest.js), and is idempotent: it converges on the same ${sessionId}-jsonl-<agentId> rows and returns early for sessions with no workflow artifacts (cheap on a rescan of an unchanged tree). Lazy-required to respect the existing import-history ⇆ workflow-ingest cycle avoidance.
  • server/__tests__/import-workflow-link.test.js: new regression test — builds a fixture session tree with a subagents/workflows/<runId>/journal.jsonl + a few agent-*.jsonl, runs the CLI import path (not the server), and asserts every inner agent ends up with workflow_run_id = <runId>, that workflows.agent_count reflects the fleet, and that re-import is idempotent (no duplicate rows).
  • Docs: ARCHITECTURE.md (adds the offline-CLI-import trigger to the Workflow-Tool Run Ingestion trigger list + the import-history.js module note) and server/README.md (import-history row).

Root cause: workflow_run_id is only ever set by the setAgentWorkflow statement, whose sole callers live in server/lib/workflow-ingest.js and are invoked only from the server (startup ingest, poll loop, live hooks). scripts/import-history.js had zero references to the workflow ingest, so the offline path could never link the fleet. Related to #167 (server-side journal ingest) — this extends the same fix to the offline import path it didn't cover.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • Documentation update

How to Test

Reproduced against real data. Point DASHBOARD_DB_PATH at a copy of a dashboard DB so nothing live is mutated.

  1. Find a completed workflow run whose inner agents were never ingested (offline). On a real 17-inner-agent run:
    sqlite3 copy.db "SELECT count(*) FROM agents WHERE workflow_run_id='wf_99d1a80e-200';"   -- 1  (bug)
    sqlite3 copy.db "SELECT agent_count,status,source FROM workflows WHERE run_id='wf_99d1a80e-200';"  -- 1|running|live
    
  2. Run the patched import over that session's tree (importAllSessions / importFromDirectory) against the copy DB.
  3. Re-check:
    sqlite3 copy.db "SELECT count(*) FROM agents WHERE workflow_run_id='wf_99d1a80e-200';"   -- 17  ✅
    sqlite3 copy.db "SELECT agent_count,status,source FROM workflows WHERE run_id='wf_99d1a80e-200';"  -- 17|killed|journal
    
    A second run is a no-op (17 inner + 1 main agent, no duplicates).
  4. npm run test:server — 655 pass (incl. the new regression test); npm run test:client — 261 pass.

Before → after (real run wf_99d1a80e-200): agents linked 1 → 17; workflows.agent_count 1 → 17.

Checklist

  • I have read the contributing guidelines
  • I have signed the CLA (the 🖋️ CLA Assistant bot will prompt me on my first PR)
  • My code follows the project's coding standards
  • I have added/updated tests that prove my fix or feature works
  • All new and existing tests pass (npm test)
  • Code is formatted (npm run format:check)
  • I have updated documentation where necessary

🤖 Generated with Claude Code

…eadless runs)

A Claude Code Workflow-tool run (dynamic workflow / fleet of sub-agents) emits
NO hooks, so when it runs where hook events never reach the dashboard server —
a headless `claude -p` run, a CI job, or an HPC/cluster compute node — its
per-run journal is never ingested live. The CLI import path never closed that
gap: `scripts/import-history.js` imported the session + sub-agent transcripts
but had zero references to the workflow-journal ingest, so no amount of
`ccam import rescan` / `ccam import path` linked the inner agents. Result: the
inner-agent rows existed but with `workflow_run_id = NULL`, leaving the run
stuck showing 1 agent instead of the real N.

Root cause: `workflow_run_id` is only ever set by `setAgentWorkflow`, whose
sole callers live in `server/lib/workflow-ingest.js` and are invoked only from
the server (startup ingest, poll loop, live hooks). The offline import path
never called them.

Fix: after each import batch, `importAllSessions` and `importFromDirectory`
now call the existing `ingestWorkflowsForSession` per imported session —
outside the SQLite transaction (the ingest is async), mirroring exactly what
the server already does. The ingest is reused as-is (not reimplemented) and is
idempotent: it converges on the same `${sessionId}-jsonl-<agentId>` rows and
returns early for sessions with no workflow artifacts.

Verified against real data: a 17-inner-agent run whose agents were orphaned
(`SELECT count(*) FROM agents WHERE workflow_run_id=?` → 1) went to 17 after a
patched rescan, with `workflows.agent_count` 1 → 17; a second run is a no-op
(no duplicate rows: 17 inner + 1 main). Adds a regression test that builds a
fixture session tree with a `subagents/workflows/<runId>/journal.jsonl` and runs
the CLI import path (not the server), asserting every inner agent ends up with
`workflow_run_id = <runId>`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@omar-merhebi
omar-merhebi requested a review from hoangsonww as a code owner July 16, 2026 04:49
@github-actions github-actions Bot added bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed question Further information is requested labels Jul 16, 2026
@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown

✅ All contributors have signed the CLA. Thank you!
Posted by the CLA Assistant Lite bot.

@omar-merhebi

Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Jul 16, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the offline CLI import path to link Workflow-tool inner agents to their corresponding runs. Specifically, it modifies scripts/import-history.js to call ingestWorkflowsForSession asynchronously and outside the SQLite transaction during both batch rescans (importAllSessions) and directory imports (importFromDirectory). This ensures that headless, CI, or cluster runs that do not emit live hooks are properly linked instead of being left orphaned. The PR also updates the documentation in ARCHITECTURE.md and server/README.md to document this new ingestion trigger, and introduces a comprehensive regression test suite in server/__tests__/import-workflow-link.test.js. No reviewer comments were provided, so there is no feedback to evaluate.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@hoangsonww
hoangsonww merged commit a3377d7 into hoangsonww:master Jul 17, 2026
11 of 12 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 17, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed question Further information is requested

Projects

Development

Successfully merging this pull request may close these issues.

2 participants