Skip to content

coding-agents.md

Latest commit

 

History

History
167 lines (130 loc) · 9.34 KB

coding-agents.md

File metadata and controls

167 lines (130 loc) · 9.34 KB
Raw

Use JAIPilot inside your coding agent

Connect JAIPilot to an existing coding agent such as Codex or Claude Code to improve its Java testing workflow. JAIPilot prepares characterization tests before production edits, covering current behavior, boundaries and failure paths, then reruns the preserved tests after the change. Capturing expectations before the edit helps catch accidental behavior changes and gives the coding agent real failure evidence to guide repairs. Coverage can be measured when the project has coverage tooling configured.

The local stdio MCP server uses the same workflow runner, account and credits as the CLI. Builds and tests run in your checkout. The MCP server executes the local commands itself. Your coding agent invokes the tools and receives job IDs, progress, and command evidence; it makes the planned production edits once the baseline is ready.

Connect Codex

Install a JAIPilot version with the mcp command and sign in:

jaipilot auth login
codex mcp add jaipilot -- jaipilot mcp --repo /absolute/path/to/java-repository
codex mcp list

Use an absolute executable path if jaipilot is not on Codex's PATH. The server is bound to the repository passed at startup; configure a different server entry for another checkout. Start a new Codex session after configuring it. This works with Codex CLI and the IDE extension. Other MCP hosts can launch the same command using their stdio server configuration.

JAIPilot's preparation and verification tools write files or run local commands, so Codex may ask for approval. For unattended use after authorizing this server, add this to its existing configuration:

[mcp_servers.jaipilot.tools.lock_behavior]
approval_mode = "approve"

[mcp_servers.jaipilot.tools.verify_behavior]
approval_mode = "approve"

[mcp_servers.jaipilot.tools.cancel_job]
approval_mode = "approve"

Keep the normal approval setting if you prefer reviewing each invocation. Setting Codex's overall approval policy to never alone does not authorize a write-capable MCP tool. See the official Codex MCP configuration.

For a source checkout, without installing or replacing your released CLI:

go build -o dist/jaipilot ./cmd/jaipilot
codex mcp add jaipilot -- /absolute/path/to/jaipilot-cli/dist/jaipilot mcp --repo /absolute/path/to/java-repository

Connect Claude Code

After installing JAIPilot and signing in, add the same local server:

claude mcp add --transport stdio jaipilot -- jaipilot mcp --repo /absolute/path/to/java-repository

Start a new session after configuring it. See the official Claude Code MCP setup.

The testing workflow is included

The MCP server supplies the workflow in its initialization instructions and tool descriptions. No repository instruction file or extra testing prompt is required. Once connected, the coding agent receives guidance to:

  1. Identify the affected behavior and scope, the actual test directories, and a test command that runs the tests and exits nonzero on failures. Include relevant dependencies and callers in scope.
  2. Call JAIPilot's lock_behavior before production edits. Supply the scope, explicit test_paths, test_command and planned intent. Build output directories must be gitignored.
  3. Poll get_job_status with wait_seconds=30. Proceed only when ready_to_edit=true. If blocked, failed or cancelled, investigate the evidence and report the blocker before proceeding.
  4. Keep result.baseline_id and preserve the baseline tests unchanged. Make the planned source edits.
  5. Call verify_behavior with that baseline_id and poll until verified=true. Investigate failures; do not regenerate the baseline to bless a regression.

The descriptions also instruct the agent to leave the checkout alone during a running job, prepare additional original code before expanding the edit scope, and explicitly review old expectations when the requested change intentionally alters behavior. The host decides which tools to invoke; MCP instructions do not intercept its file writes.

Tool contract

All paths are relative to the server's repository root. Choose exactly one scope type: all: true, paths, classes, or selections. Selection strings use path/File.java:start-end.

Example lock_behavior arguments:

{
  "scope": { "classes": ["com.acme.OrderService"] },
  "test_paths": ["src/test/java"],
  "test_command": "./mvnw -q clean verify",
  "timeout_seconds": 3600,
  "intent": "Extract order validation while preserving observable behavior"
}

test_paths explicitly declares the files or directories where JAIPilot may create or edit tests. Every tracked and nonignored file outside those paths is protected, including pre-existing dirty files and build configuration. The chosen command must actually execute the baseline tests; JAIPilot cannot infer that an arbitrary successful shell command is a valid test suite. For projects without coverage instrumentation, configure it before preparation if you need measured coverage.

Tool Input Result
lock_behavior Scope, test paths, test command, optional intent and timeout Starts preparation; returns job_id.
get_job_status job_id, optional wait_seconds (0–30, default 10) Progress, state, readiness and final evidence.
verify_behavior baseline_id Starts a local rerun of the original command; returns job_id.
cancel_job job_id Requests cancellation; poll until cancelled.

Preparation uses the existing managed lock_behavior workflow, targeting at least 90% line coverage in the selected production code when coverage tooling is configured. It creates meaningful assertions for current behavior, boundaries and failure paths, and reports measured coverage gaps or concrete blockers. The target is included in the MCP instructions and tool description. Preparation then independently runs the supplied verification command. Only a completed workflow, unchanged protected files, nonempty test files and a passing verification command produce ready_to_edit: true and result.baseline_id. The command must leave tracked and nonignored source files unchanged during verification.

Verification does not call the LLM. It refuses added, removed or changed files in the baseline test paths, reruns the exact original command and reports real exit status and bounded output. Check verified: true; failed tests return blocked with command evidence. Passing tests demonstrate the behavior covered by those tests; they do not prove all possible behavior or complete coverage.

Jobs, storage and cancellation

  • Tool calls return promptly while a job runs. Status calls can wait up to 30 seconds; the underlying service retains the CLI's existing transient-error retries.
  • Background jobs have no overall elapsed-time cutoff. timeout_seconds defaults to 3600 seconds (1 hour), with a maximum of 7200 (2 hours). It is the minimum allowance for each local command during preparation and the limit for the independent verification command. Longer valid command limits requested by JAIPilot are honored. The original verification limit is stored in the baseline and reused after edits. A status poll returning while the job is running does not cancel the job.
  • One MCP job runs at a time per checkout, including across MCP server processes. Different Git worktrees have separate locks and baselines. Other CLI/ACP processes and host edits are not locked.
  • Baseline manifests store test and original file SHA-256 hashes, scope, command and verification evidence under the checkout's Git metadata (git rev-parse --git-path jaipilot-mcp). They survive server restarts and are not added to your source tree.
  • Jobs and their recent progress are in memory (up to 100 retained jobs). Job IDs do not survive a restart; baseline IDs do. Closing the MCP connection or stopping the server cancels active work.
  • Cancellation stops the local command process tree. It leaves partial edits for review and does not reset your working tree. Review before retrying preparation.
  • Explicitly selected test paths must contain the actual baseline tests and must not contain production code. Symbolic links in test paths are rejected. Gitignored build artifacts are not hashed or protected.
  • Long-lived MCP servers do not update themselves during a session. Run jaipilot update and restart the coding agent to use a newer installed version.

The existing local execution and privacy policy applies.

Verify a development build

Requires Go 1.26.2, Node.js 20 or later for the official protocol test clients, and a JDK for the real Java regression gate. These development dependencies are not bundled in the released executable.

npm install --ignore-scripts
sh scripts/check.sh

The checks build the native CLI and ACP adapter, exercise their real stdio transports with official SDK clients, and run Java baseline/regression, file preservation, authentication, cancellation, and verified update tests. CI runs on Linux, macOS, and Windows; release installations also run on both ARM64 targets.

The paid Codex comparison and its Deno reproduction scripts are historical evidence preserved at v1.1.1. They are not current native release gates and do not demonstrate a new native paid-service run.