feat(#62): implement code generation layer#86
Conversation
…t and context agent
…aml for frontmatter parsing and add more tests
Resolved conflicts: - .eslintrc: kept main's rules - package.json: merged scripts (added full and example:full) - src/types/index.ts: kept development types added in this branch - Other research files: accepted main's version
- Add Claude Code CLI as drop-in LLM provider (src/llm/providers/claude-cli.ts) - Add TodoTracker utility for progress tracking (src/utils/todo-tracker.ts) - Integrate todo tracking in all 4 agents: - documentation-search-agent - context-analysis-agent - code-generation-agent - test-environment-agent - Integrate todo tracking in both supervisors: - research-supervisor - development-supervisor - Add MCP client setup (src/mcp/) - Add CLI validation script (src/scripts/validate-cli.ts) - Add cht-core context utility (src/utils/cht-core-context.ts) - Update LLM factory to support CLI provider with configurable maxTurns - Update .gitignore to exclude log folder Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove ParsedTicket alias, use IssueTemplate directly - Extract duplicated readEnv() to shared src/utils/env.ts - Move CONTRIBUTING.md from src/ to designs/ - Move test file to test/layers/code-gen/ matching src/ convention - Merge duplicate imports in test file - Add missing test cases (env var fallback, alias resolution, empty inputs) - Revert unrelated proxyquire change in context-loader.spec.ts - Regenerate clean package-lock.json removing spurious peer additions
…eneration Implement the full code generation pipeline with Claude CLI integration. Key changes: - Per-file sequential generation (one LLM call per planned file) - Python transform mode for large JSON files (>50K chars) - Search-replace mode for large non-JSON MODIFY files (>1000 lines) - LLM preamble stripping and reasoning detection - Plan reordering (JSON files generated last) - Dev CLI for development workflow - Zod schemas for plan validation - Updated tests for registry, schemas, and claude-api module
…ovements - Add multi-round tool-use loop in Anthropic provider (bindTools, ToolMessage) - Add LLMToolDefinition, ToolHandler types and capMaxTokens utility - Add read_file/list_directory tools for API provider filesystem access - Add Beads session tracking for code gen (plan, per-file progress, failures) - Add selective regeneration: carry forward passing files, only regenerate failing - Add diff-based validation using `diff` package (replaces naive positional diff) - Add per-file validation feedback (FileValidationFeedback) for targeted retries - Loosen manifest scope: plan prompt no longer restricts files to known scope - Exclude test files from code gen plan (separate agent handles tests) - Extract createModel helper, FailingFileRef type, extractTextContent utility - Fix operator precedence bug in looksLikePythonScript - Fix disableTools/tools option contradiction across providers
sugat009
changed the title
|
|
Fixing merge conflicts... |
sugat009
moved this from In Review
to In Progress
in CHT Multi-Agent System (cht-agent)
…afety net Two real code-gen modules selectable via CODE_GEN_MODULE: - claude-code-cli: tool-using agent driven by the Claude Code CLI's native Read/Write/Edit/Glob/Grep tools. Uses Claude MAX subscription. - claude-api (default): prompt-and-parse approach against the Anthropic API. Requires ANTHROPIC_API_KEY. Two-step plan-then-per-file generation. - opencode: deferred stub. Deterministic safety net at the agent layer: - Compile gate runs tsc against all cht-core tsconfigs before rollback; errors surface as cross-file issues at HC2. - Cross-file validator catches identifier mismatches (template bindings, selector references, action method calls). - AST validator complements regex passes for shape and contract checks. - Locale propagator auto-fills the 9 non-en messages-*.properties files with alphabetically inserted English placeholders. LangGraph supervisor with 3-iteration refinement loop, execute-no-op short-circuit, per-node shutdown checks, and HC2 banner grouping cross-file issues by type. Adds 4 real-issue ticket fixtures (10944, 10997, 11017, 11048) used by the runner for cross-shape regression tests. Removes superseded planning docs (BRANCH_CHANGES.md, cht-multi-agent-plan.md, tickets/9.md).
* feat(#14): Implement code generation modules (#69) * fix(#14): address PR #69 review comments - Remove ParsedTicket alias, use IssueTemplate directly - Extract duplicated readEnv() to shared src/utils/env.ts - Move CONTRIBUTING.md from src/ to designs/ - Move test file to test/layers/code-gen/ matching src/ convention - Merge duplicate imports in test file - Add missing test cases (env var fallback, alias resolution, empty inputs) - Revert unrelated proxyquire change in context-loader.spec.ts - Regenerate clean package-lock.json removing spurious peer additions * fix(#14): replace implementation plan with code scaffold in claude-api module * fix(#14): add warnings to output, reject duplicate registration, clarify repo-specific examples * fix(#14): reduce cognitive complexity and use modern string/array methods * docs(#14): refresh code-gen CONTRIBUTING to the #86 target design * fix(#14): restore scaffold fallback when phase components yield no files --------- Co-authored-by: Shashi sah <86319846+shashi-sah2003@users.noreply.github.com>
|
@witash |
There was a problem hiding this comment.
Its maybe a moot point for this PR, but in future I think we need to split these large PRs into smaller pieces; its not realistic to meaningfully do a line-by-line review of 100 files and 22,000 lines; even if it ends up being mutliple PRs with the same total changes, its easier to understand independent pieces individually.
ran manually over both the sample tickets in the repo (just to see it working) and over the datasource migration tickets.
the datasource migration tickets can't take advantage of the context database as much, and there are some inaccuracies (or maybe preferences where the AI has chosen a different preference) in the ticket descriptions, that lead to inaccuracies in the generated code, but the agent generated apparently working and correct code for each of the tickets, based off the descriptions. Although not completely read to just submit as a PR because of test failures.
a few notes:
this branch implements both beads and has its own todo-tracker.
Are both really necessary?
I looked at the prompt rules, static validators, and the LLM validation.
Also due to the large diff size, its a little hard to understand and evaluate every piece, but not many notes...
For typescript shared-libs, the compile step is important; without it, it does generate code that doesn't compile. But its only for claude-code-cli, otherwise it doesn't run compile, is there a reason?
The tickets I tested did not do much on the frontend, so the Angular checks I just skimmed; claude proposed some small tweaks but I don't think its worth it to fiddle with them too much now. I guess these will iterate after this gets merged and gets more usage anyway.
The AST signature validator detect only growth in the parameters, and is relevant only for JS files or when the compile gate is not used, because tsc would catch signature mismatches usually. Most of the cht code is not TS so that's fine, but then do we want to make the signature validator stricter?
One thing thats notably missing is running unit tests...I know that's another PR, but a big difference between using these agents and claude code in interactive mode is that claude code runs the unit tests frequently; for tickets like the datasource migrations which include a lot of modifications and not many additions, tests are a much more concrete and measurable way to validate that nothing existing broke accidentally than the validations in this branch. And small changes to tests can end up being a large portion, even the majority, of the diff. Not running them in the validation stage seems missing, and can generate code that doesn't pass tests.
So nothing to change here, but just noting that when the test branch is merged, I think it will improve the validation as well.
|
Thanks for some great feedback! @witash I'm concerned about the 2nd paragraph -- Let's fix our data-labeling (domains, descriptions, input tickets, context funneling). Some narrower ideas below
The datasource tickets struggle because it seems like a domain / context-selection gap: they route to data-sync, whose corpus is all replication memory, not the cht-datasource query-layer shape (qualifier/adapter/endpoint) these tickets would be better off with. The agent injects the wrong pattern there. Root cause is that cht-datasource has no clean home in our 9 domains (pitfall says data-sync, precedent says contacts, template says data-sync). #11174 shows the real axis is a Person/Place/Contact/Report data-access hierarchy. Proposing a first-class data-access domain + populating its component index/patterns; I'll open a follow-up and link it here. Seems the description in #11174's "same qualifier union as getUuidsPage," pushed the agent toward the rejected design. cc: @sugat009 |
…-layer # Conflicts: # .env.example # .gitignore # .nycrc.json # package-lock.json # package.json # src/agents/documentation-search-agent.ts # src/layers/code-gen/interface.ts # src/layers/code-gen/modules/claude-api/index.ts # src/layers/code-gen/registry.ts # src/llm/providers/anthropic.ts # src/llm/providers/claude-cli.ts # src/llm/types.ts # src/mcp/index.ts # src/supervisors/research-supervisor.ts # src/types/index.ts # src/utils/domain-inference.ts # test/layers/code-gen/code-gen-registry.spec.ts # test/llm/factory.spec.ts # test/llm/providers/anthropic.spec.ts # test/llm/providers/claude-cli.spec.ts
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
@witash thanks for the thorough review, the datasource-migration runs were the most useful signal. beads vs todo-tracker: different layers, not redundant. Compile gate only for Stricter AST signature validator: deliberately light (param-growth only), and with both paths now on the compile gate Running unit tests in validation: agreed it matters, but it's the downstream QA phase's job (Test Environment / Orchestration / QA Supervisor), which runs after development and isn't wired in yet. This PR is code-gen; its validation is generation-time (compile + static/AST). Your point is exactly what that phase is for. Since your review: removed ~470 lines of dead code, fixed a parser bug, added the API compile gate, raised coverage toward main (85/72/84/86), and cleared the SonarCloud findings. On PR size, fully taken, we'll keep the follow-ups (coverage to 95/85, |
|
@Hareet agreed, we settled this in the team meeting: proceed with a first-class datasource/data-access domain, go ahead and raise the PR for it. Populate it by running the #119 pipeline over the cht-datasource tickets. That lines up with Kombo's observability, once the Langfuse instrumentation (#127) merges and points at the deployed instance, the datasource run rides on the same pipeline pass, so we get traces for it and a clean before/after. And agreed the input labeling (the #11174 description that steered the agent) needs fixing alongside the domain, not the domain alone. |
github-project-automation
Bot
moved this from In Review
to Done
in CHT Multi-Agent System (cht-agent)
5 participants
Summary
diffpackage for validation prompts#62
Test plan
npm run buildpassesnpm testpasses (291 tests)default-permissions.jsnow included in plan when permission-related features are generatedUpdate: legacy test-environment-agent removed
Drops the legacy
src/agents/test-environment-agent.tsprototype that conflated test generation (#63), test environment setup (#66), and test orchestration (#67) into one LLM-driven agent. The architecture splits these across three layers under a future QA Supervisor (#64). The Development Supervisor's validation node is now code-only; the persistence helpers (writeToStaging,writeToChtCore,getAllGeneratedFiles) no longer reach intostate.testEnvironment; and thesetupTestsgraph node is gone, so the workflow runsgenerateCode → validateImpldirectly. This supersedes the "handled by separate Test Environment Agent" note in the summary above.Commits:
b461140remove legacy test-environment-agent ahead of Implement Test Generation Layer #63 implementation097f078drop test-env consumers from validation node and persistence5057169clean stale test-env comments in edited filesGates after removal: build and lint green, 629 tests passing, coverage 83.31/69.36/82.11/85.07 (thresholds 78/60/72/78), SonarCloud Quality Gate OK with 0 new issues. QA verified the rewired graph executes at runtime (
generateCode → validateImpl, nosetupTests) with no validation-score regression.🤖 Generated with Claude Code