feat(harness-engine): add portable runtime and BFF gateway

[sriaradhyula]

Description

Add an independent, provider-neutral Harness Engine and a BFF-owned Harness Gateway. Existing WebUI, Slack, and Webex callers continue using the established CAIPE chat endpoints; the gateway resolves each agent's selected execution harness and routes the turn either to the unchanged Dynamic Agents runtime or to Harness Engine.

AgentCore and Claude Agent SDK implement the same portable blueprint, descriptor, capability, session, and canonical-event contracts. Dynamic Agents remains the default (langchain-deepagents) and its backend source, routes, persistence, and execution are unchanged.

Design discussion

• Introducing Harness Engine: portable agent runtimes for CAIPE — architecture diagrams, sanitized UI screenshots, compatibility boundaries, and open design questions.

SpecKit design package

The design is maintained as a reviewable SpecKit package. The current PR implements the independent, opt-in Harness Engine slice; Dynamic Agents continues to serve the default langchain-deepagents harness until a separately reviewed parity and cutover phase.

Artifact	Review focus
Feature specification	User stories, requirements, compatibility guarantees, sandbox isolation, memory, sessions, tracing, and success criteria
Architecture	BFF Harness Gateway, control/execution planes, trust boundaries, durable state, and provider adapters
Implementation plan	Delivery phases, ownership boundaries, rollout, and rollback
Tasks	Completed implementation slice and explicitly deferred work
Portable abstractions	Sessions, thread state, memory, tools, prompts, sandboxes, telemetry, streaming, and delegation
Compatibility matrix	Dynamic Agents baseline and adapter capability status
API and adapter contracts	REST, canonical events, harness adapter, sandbox worker, and state/memory/tracing contracts

The companion GitHub Discussion #2405 renders the same detailed system architecture and source graph inline for broader design review.

Architecture diagrams

Detailed system architecture

This view shows the compatibility routing, unchanged Dynamic Agents default, Harness Engine control plane, durable session/run/event ownership, provider adapters, platform integrations, and clearly marked planned sandbox and broker paths.

• Editable CAIPE system architecture (.excalidraw)

High-level source graph

This view maps the public routes and UI through the BFF gateway into the Harness Engine control plane, adapters, infrastructure, contracts, deployment assets, and regression tests.

• Editable Harness Engine source graph (.excalidraw)

What changed

• Added portable contracts for prompts, models, tools, thread persistence, short/long-term memory, sandboxes, streaming, multi-agent delegation, telemetry, and limits.
• Added descriptor catalogs with operator profile aliases, bounded JSON Schema options, catalog revisions, and explicit capability levels so caipe-ui can render harness-specific forms without exposing credentials.
• Added independent Harness Engine persistence for immutable agent versions, CAIPE session bindings, detached runs, replayable canonical events, and provider-native session identifiers.
• Added CAIPEAgentSessionManager plus provider-specific session managers for deterministic AgentCore runtime sessions and Claude SDK resume sessions.
• Added detached invoke, SSE/replay, run cancellation, cancel-active, and session-clear APIs. Client disconnects stop only the subscription; provider execution and durable event capture continue.
• Added the BFF Harness Gateway to the existing browser, Slack, Webex, scheduler/invoke, cancel, and resume entry points while preserving authentication, OpenFGA, and conversation authorization.
• Added AG-UI translation for Harness Engine events, non-stream response accumulation, explicit capability failures, and no silent fallback to Dynamic Agents.
• Added editor support for selecting langchain-deepagents (default), AgentCore, or Claude Agent SDK and rendering provider options from descriptors.
• Added SpecKit spec, plan, tasks, API contracts, architecture documentation, and sandbox-pod design.

Chat identity UX

• The active chat header always shows the friendly agent name, stable avatar color, and human-readable harness label.
• Conversation history and new-chat selection reuse the same identity; explicit author themes win and unthemed agents receive deterministic distinct colors.
• The browser tab includes the selected agent name and remains correct when switching between conversations that use the same agent.
• Chat-only callers fall back to the agent#use-authorized catalog when the richer agent-detail endpoint is unavailable.

Request path

WebUI / Slack / Webex
        |
        v
Existing CAIPE BFF chat APIs
        |
        v
Harness Gateway -- langchain-deepagents --> existing Dynamic Agents
        |
        +---------- agentcore / claude-sdk --> Harness Engine
                                                    |
                                                    +--> durable CAIPE session + run + event log
                                                    +--> provider session manager
                                                    +--> selected harness adapter

The BFF stays horizontally stateless. Harness Engine owns execution after returning 202, so a browser or bot disconnect does not terminate the run. A reconnecting client can replay canonical events by cursor.

Capability status

Implemented:

• portable contracts, catalog fingerprinting, validation, and descriptor-driven primitive UI fields;
• AgentCore and Claude SDK adapters with provider-specific session managers;
• immutable versions, owner-scoped durable bindings, version pinning, clear, and epoch rotation;
• detached execution, durable event replay, streaming translation, invoke accumulation, cancellation, and cancel-active;
• routing from existing WebUI, Slack, Webex, and scheduler entry points;
• sanitized failures, owner checks, OpenFGA gates, and internal service identity isolation.

Deferred and reported as unsupported instead of silently degrading:

• resume translation for Harness Engine through the BFF;
• attachment translation for Harness Engine;
• Kubernetes Agent Sandbox worker/lease implementation;
• production ToolBroker, MemoryBroker, and DelegationBroker implementations;
• in-flight process-failure takeover and immediate cross-replica cancellation signaling;
• full interrupt/tool/reasoning translation and OpenTelemetry sink;
• Strands/Deep Agents adapters and certification.

Backward compatibility with Dynamic Agents

Dynamic Agents remains the production implementation behind the logical langchain-deepagents harness during this staged transition. This PR does not move, wrap, or rewrite the Dynamic Agents runtime.

• Existing agent documents without execution_harness_id continue to resolve to the default langchain-deepagents harness. The legacy dynamic_agents value is treated the same way.
• The Harness Gateway sends that default selection through the existing Dynamic Agents proxy and runtime, not through Harness Engine.
• Existing Web UI, Slack, Webex, CLI, and API clients keep using the same BFF endpoints and do not need to supply a harness field.
• Existing authentication, OpenFGA, agent-use, tenant, and conversation authorization checks remain in front of runtime selection.
• Existing Dynamic Agents threads, checkpoints, memory, tools, streaming behavior, and persistence stay on their current code and storage paths.
• Non-default AgentCore and Claude SDK agents opt into Harness Engine explicitly; there is no silent fallback between runtimes.

No files under ai_platform_engineering/dynamic_agents/ are changed by this PR, confirming that the existing backend source and execution path remain isolated.

Dynamic Agents will continue to provide the langchain-deepagents harness until a Harness Engine LangChain Deep Agents adapter reaches full functional parity and passes the adapter certification suite. A later cutover can then change the gateway mapping behind the same harness identifier, agent documents, client endpoints, and authorization contract. That cutover is explicitly outside this PR.

Research

• https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/runtime-sessions.html
• https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/memory.html
• https://github.com/anthropics/claude-agent-sdk-python
• https://docs.langchain.com/oss/python/deepagents/sandboxes
• https://docs.langchain.com/oss/python/deepagents/memory
• https://agent-sandbox.sigs.k8s.io/docs/

Type of Change

• Bugfix
• New Feature
• Breaking Change
• Refactor
• Documentation

Validation

• Harness Engine: uv run ruff check src tests and uv run pytest -q — passed, 18 tests.
• UI: npm run lint and npm run build — passed (two pre-existing Next.js NFT trace warnings).
• Full UI Jest suite — passed, 640 suites / 7,487 tests.
• Dynamic-agent RBAC route suite — passed, 47 tests.
• Targeted gateway/editor tests — passed, including browser, Slack, Webex, invoke accumulation, and cancellation through AgentCore.
• Live Safari E2E against caipe-oss.outshift.io — verified Harness Gateway E2E · Chat · CAIPE, the persistent Harness Gateway E2E + Amazon Bedrock AgentCore header, AgentCore/Deep Agents history badges, and distinct agent colors; existing WebUI routing still returned the AgentCore response.
• Durable E2E evidence — run run-77658dd3c06841438ccaef500bd00ea7 completed with harness_id=agentcore; its event log contained run.started, two content.delta events, usage.updated, and run.completed.
• CI covers DCO, CodeQL, UI tests, Playwright RBAC regression, Ruff, docs build, and image builds; checks from the latest push are tracked on this draft PR.
• git diff --check — passed.

The deployment retains a shared Harness Gateway E2E AgentCore demo so reviewers can repeat the browser test; it contains no credentials or production data.

Checklist

• I have read the contributing guidelines
• Existing issues have been referenced (none was provided)
• I have verified this change is not present in other open pull requests
• Functionality is documented
• All applicable code style checks pass
• New code contribution is covered by automated tests
• All applicable new and existing tests pass

Preview deployment parity

• caipe-oss.outshift.io runs the Docker Compose preview with six model definitions matching the source deployment.
• The 13 upstream MCP targets are provisioned behind AgentGateway on its data-plane port; Confluence, Jira, VictorOps, and Webex were validated from signed-in Safari with 98, 62, 11, and 8 tools respectively.
• Six OAuth connector definitions, seven provider-connection records, and 20 encrypted credential payloads were migrated without copying agents or conversations; owner subjects and callback URLs were rebound to the preview deployment.
• The config bridge now reapplies deployment JWT issuer, JWKS, and audience overrides on every reconciliation, preventing persisted bootstrap state from rejecting browser-issued tokens.
• GitLab remains remote-authorization protected and requires a linked GitLab connection, matching the source deployment where GitLab is not linked.
• The source-only Tome MCP target is intentionally excluded: it points back to the Tome product API and is not an independent upstream CAIPE service.
• Remote backups were created before credential migration and before the AgentGateway endpoint correction.

Additional validation: deploy/agentgateway unit suite passed (35 tests), Compose rendering passed, all preview containers are running, and the public HTTPS route returns 200.

Claude SDK live E2E

• Provisioned the shared Claude SDK E2E Agent (agent-claude-sdk-e2e) through a validated Harness Engine blueprint; the UI catalog row remains metadata only and Dynamic Agents runtime code is unchanged.
• Corrected the preview credential chain so the Claude SDK uses the EC2 instance profile instead of stale static AWS keys; the role is limited to invoke and streaming invoke for the selected Claude Sonnet 4.6 inference profile.
• Added Mongo-backed Claude SessionStore transcript mirroring. CAIPE persists the provider session ID and Mongo stores the provider transcript, making resume independent of a container filesystem.
• Three live turns completed with provider session 1476e362-21c7-4db3-b647-b1c9988cddc2: store run-507079ed8d894bd88d13f5682020f282, recall run-434417404e4945ce9adb6313b37ceae3, and post-restart recall run-f4f5a9aaa454429e9c649d2ff3904751. Each recall returned violet-orbit-731; 22 transcript entries were mirrored with the UUID and load-order indexes present.
• Fixed the creation wizard so platform-global agents do not incorrectly require an owner team. Targeted Jest passed (6 tests), ESLint passed, and the production Next.js image built successfully (the two pre-existing NFT trace warnings remain).

[github-actions]

✅ No proprietary content detected. This PR is clear for review!

[github-actions]

🧪 CAIPE UI Test Results

✅ All tests passed

🟡 Overall Coverage: 62%

📊 Detailed Coverage

Metric	Covered	Total	Percentage
Lines	41567	61389	67.71%
Statements	44363	67753	65.47%
Functions	7746	12717	60.91%
Branches	28313	51173	55.32%

✅ Test Suites

• ✅ auth-guard.test.tsx - Route protection & authorization
• ✅ token-expiry-guard.test.tsx - Token expiry handling
• ✅ dynamic-agent-client.test.ts - Dynamic Agents streaming client
• ✅ auth-utils.test.ts - Authentication utilities (100% coverage)
• ✅ auth-config.test.ts - OIDC configuration

📈 Coverage Thresholds

Threshold	Target	Current	Status
Minimum	40%	62%	✅ Pass
Good	60%	62%	✅ Pass
Excellent	80%	62%	⚠️ Below target

⚠️ Areas Needing Tests

High Priority:

• lib/streaming/* - Core stream event handling
• store/chat-store.ts - Chat state management
• store/agent-skills-store.ts - Agent skills
• lib/api-client.ts - API communication
• lib/storage-mode.ts - MongoDB/localStorage switching

Medium Priority:

• components/chat/ChatPanel.tsx - Main chat interface
• components/agent-builder/* - Agent builder UI
• lib/mongodb.ts - MongoDB integration

---

💡 Run locally: make caipe-ui-tests
📦 Full report: Check workflow artifacts

[subbaksh]

Before writing code, can we please discuss architecture first, we have multiple weekly syncs for this

[sriaradhyula]

This is a draft PR. It is not ready to be reviewed. Please use the discussion board for review

[sriaradhyula]

PR #2401 makes Harness Engine AG-UI-compatible and disconnect-safe, but a small AG-UI reattachment layer is still needed for automatic browser reload recovery. HITL resume and complete interrupt/tool/reasoning translation are also explicitly deferred in the current draft.

[github-actions]

ℹ️ Branch is behind main (advisory, non-blocking)

Commit: fb8e29e1

This branch does not yet contain the latest main. This does not fail the version-bump check — version files are bumped once the branch is updated, and GitHub branch protection still requires the branch to be up to date before merge. Use the Update branch button on this PR, or run:

git fetch origin main
git checkout 2026-08-17-harness-engine
git merge FETCH_HEAD
git push