feat: first-class ReasoningEngine (BYOC) support for the Gemini Enterprise Agent Platform

[joseph-wortmann]

Summary

Add first-class support for deploying adk-rust agents as ReasoningEngines on the Gemini Enterprise Agent Platform (formerly Vertex AI Agent Engine / Agent Runtime) via the bring-your-own-container path (reasoningEngines.create with spec.containerSpec), and close the integration gaps with the platform's managed services: the class-method runtime contract, Sessions event fidelity, Memory Bank, Example Store (Preview), Code Execution sandbox, RAG Engine and Agent Retrieval (formerly Vector Search 2.0), Agent Registry participation (discovery, deploy-time registration, remote ReasoningEngine invocation), Skill Registry discovery and dynamic skill loading (Preview), Cloud Trace/Logging export, env-driven Vertex configuration, Vertex context caching, and deployment tooling — all enableable at once via new agent-platform / agent-platform-full umbrella meta-features.

Scoping principle: adk-rust consumes platform-managed resources; it does not duplicate the EAP control plane. The single exception is agent creation/deployment. Managed resources — skills, RAG corpora, Example Stores — are assumed to be provisioned and lifecycle-managed by platform tooling (console, gcloud, Terraform, the platform's own APIs); adk-rust's client surfaces are read/query/invoke-only, plus runtime data-plane writes (sessions, events, memories, sandbox executions, Agent Retrieval collections/data objects, example upserts).

Motivation

The platform's BYOC deployment path is language-agnostic: any container that serves the class-method dispatch protocol (POST /api/reasoning_engine, POST /api/stream_reasoning_engine) participates fully — console Playground, reasoningEngines.query/:streamQuery, SDK clients, and managed Sessions/Memory Bank. This makes adk-rust a viable alternative to adk-python for teams that cannot use Python (supply-chain policy, HIPAA compliance, performance), but today adk-rust cannot fill that role:

• No runtime contract implementation. Nothing in the workspace implements the dispatch envelope or the Google-ADK operation set (stream_query, streaming_agent_run_with_events, session/memory methods); there are zero references to streamQuery or the GOOGLE_CLOUD_AGENT_ENGINE* env conventions. A deployed adk-rust container is invisible to the platform's query surface and console.
• Managed Sessions lose conversation content. VertexAiSessionService (adk-session/src/vertex.rs, vertex-session feature) already targets reasoningEngines/*/sessions, but appendEvent payloads omit Event.content (text/function calls), so transcripts don't round-trip. It also lacks LRO polling, TTL, and env-based engine-ID resolution.
• No Memory Bank backend in adk-memory (the analog of Python ADK's VertexAiMemoryBankService), no Example Store client, and no managed sandbox (sandboxEnvironments) client — adk-sandbox is local OS sandboxing only.
• No managed retrieval. adk-rag supports only self-hosted vector stores (InMemory, LanceDB, pgvector, Qdrant, SurrealDB): there is no RAG Engine :retrieveContexts client, no analog of Python ADK's VertexAiRagRetrieval tool, no tools[].retrieval.vertexRagStore declaration for server-side grounding in adk-gemini, and no Agent Retrieval (formerly Vector Search 2.0 — Collections/Data Objects, auto-embedding, hybrid RRF search) backend for the VectorStore trait.
• No Agent Registry / remote ReasoningEngine invocation. (Not to be confused with the Agent Retrieval product, covered above.) The platform's Govern pillar centers on Agent Registry — a regional catalog of agents, MCP servers, and endpoints (URN identifiers, keyword search, endpoint resolution) that Gemini Enterprise apps import A2A agents from via Agent Gateway, with "resolve endpoints and build orchestrators using ADK" as a documented integration pattern. adk-rust has no registry client (search/resolve, plus deploy-time self-registration), and no way to invoke another deployed ReasoningEngine (:streamQuery client) — RemoteA2aAgent exists but only reaches agents that expose A2A at a known URL. Relatedly, adk-skill's local SkillIndex is never bridged into the A2A agent-card skills[] that the registry indexes for search.
• No Skill Registry consumption (Preview). The platform's Skill Registry (Build pillar — distinct from Agent Registry) is a managed, versioned repository of SKILL.md skill packages (Skill / SkillRevision entities, semantic search) that lets agents dynamically discover and load capabilities by user intent. adk-rust's adk-skill already parses exactly this open SKILL.md standard — its SkillFrontmatter constraints map 1:1 onto the registry's validation rules — but it can only load skills from the local filesystem (.skills/, .claude/skills/). There is no client to search, resolve revisions, or dynamically load existing registry skills (adk-python integrates via its skill toolset). Skill lifecycle management (create/update/delete/publish) is explicitly out of scope — that stays in platform tooling.
• No GCP telemetry transport. adk-telemetry already emits Python-ADK-compatible gcp.vertex.agent.* attributes and OTel GenAI semconv, but only exports via plain OTLP — no authenticated telemetry.googleapis.com path, GCP resource detection, or Cloud Logging trace correlation.
• No env-driven Vertex config. GOOGLE_GENAI_USE_VERTEXAI / GOOGLE_CLOUD_PROJECT / GOOGLE_CLOUD_LOCATION are not honored by GeminiModel or provider_from_env(); regulated deployments need a guaranteed Vertex-only path (no generativelanguage.googleapis.com fallback).
• No deployment tooling. There is no Dockerfile template (the cargo-adk README documents a docker addon that isn't implemented in cargo-adk/src/registry.rs), and adk-deploy targets only the proprietary ADK Platform control plane — no reasoningEngines.create client or Terraform scaffold.
• Vertex feature ceiling. adk-gemini's Vertex backend returns GoogleCloudUnsupported for context caching even though Vertex has a cachedContents API, so runner-level prompt caching never activates on Vertex.

What already works well and should be built upon: the Vertex model backend (adk-gemini vertex: gRPC+REST, ADC/SA/WIF/express auth), VertexAiSessionService's auth/plumbing, vertex-live realtime, GCP Secret Manager (adk-auth), A2A v1.0.0 server + RemoteA2aAgent, adk-skill's SKILL.md parser and injection machinery, and the ADK-compatible POST /api/run_sse wire format in adk-server.

Proposed Solution

A phased implementation:

1. adk-server agent-engine feature — agent_engine_router() serving POST /api/reasoning_engine (unary) and POST /api/stream_reasoning_engine (newline-delimited JSON streaming), dispatching {"class_method": ..., "input": ...} onto Runner/SessionService/MemoryService with the Google-ADK operation set (create_session, list_sessions, stream_query, streaming_agent_run_with_events, async_add_session_to_memory, async_search_memory, register_operations, …), plus a turnkey serve_agent_engine(agent, opts) entrypoint that binds 0.0.0.0:$PORT and reads the platform env vars. Sketch:

   use adk_server::agent_engine::{serve_agent_engine, AgentEngineOptions};
   
   #[tokio::main]
   async fn main() -> anyhow::Result<()> {
       let agent = build_my_agent()?; // Arc<dyn Agent>
       serve_agent_engine(
           agent,
           AgentEngineOptions::default()
               .session_service(vertex_sessions_from_env()?)
               .memory_service(memory_bank_from_env()?),
       )
       .await
   }

2. adk-session fixes — round-trip Event.content through appendEvent, LRO polling on create, VertexAiSessionConfig::from_env(), TTL support.

3. adk-memory vertex-memory feature — VertexAiMemoryBankService (memories:generate / memories:retrieve) implementing MemoryService, reusing the ADC/header-caching pattern from adk-session/src/vertex.rs.

4. Deployment tooling — implement the documented-but-missing cargo-adk docker addon (multi-stage Dockerfile → distroless), an agent-engine template with Terraform (google_vertex_ai_reasoning_engine + container_spec), and a minimal reasoningEngines.create client in adk-deploy behind a gcp feature. (This is the sanctioned control-plane exception: creating agents.)

5. Config & telemetry — GeminiModel::from_env() honoring GOOGLE_GENAI_USE_VERTEXAI; adk-telemetry gcp feature with ADC-authenticated OTLP to telemetry.googleapis.com, GCP resource attributes, and JSON stdout logging with Cloud Logging trace correlation.

6. Vertex cachedContents in adk-gemini; then optional Preview services: Example Store client (adk-tool, against a pre-provisioned store), managed sandbox tool (adk-code), Gen AI Evaluation bridge (adk-eval), with shared GCP client plumbing extracted into adk-core behind a gcp feature.

7. Vertex RAG Engine + Vector Search (consume-only) — adk-rag vertex-rag feature: VertexRagEngineClient limited to read-only corpus discovery (list/get) and :retrieveContexts, plus a VertexAiRagRetrievalTool implementing adk_core::Tool (read-only, concurrency-safe); a tools[].retrieval.vertexRagStore declaration in adk-gemini/adk-model for server-side grounding on the Vertex backend; and adk-rag agent-retrieval feature: an AgentRetrievalStore implementing the existing VectorStore trait against Agent Retrieval (formerly Vector Search 2.0) — Collections/Data Objects map 1:1 onto the trait (collections are data-plane "tables" in a managed database, like the pgvector/Qdrant backends), chunk text and metadata live in the Data Object alongside the vector, with hybrid (RRF) search and Ranking-API reranking as beyond-trait extras. Legacy Vector Search 1.0 (indexes/indexEndpoints) is deliberately not implemented; corpus lifecycle and ragFiles ingestion stay in platform tooling.

8. Agent Registry + remote-agent retrieval — a skill-card bridge (adk_skill::SkillIndex → A2A agent-card skills[], which the registry indexes); AgentRegistryClient in adk-tool (feature vertex-agent-registry — distinct from the existing local agent-registry feature) with get/list/search/resolve plus an AgentSearchTool for orchestrators; and RemoteReasoningEngineAgent in adk-server (feature vertex-remote-engine, following the RemoteA2aAgent precedent) implementing adk_core::Agent over reasoningEngines:streamQuery, so a deployed engine can be consumed as a sub-agent — with optional URN resolution through the registry client. Self-registration is wired into the deploy flow only (--register, idempotent against the platform's auto-ingestion of Agent Runtime deployments); no general registry-entry management.

9. Skill Registry consumption (Preview) — adk-skill vertex-skill-registry feature, strictly read-only against the registry: SkillRegistryClient (get/list/semantic search, revision get/list, content fetch), safe payload extraction (defense-in-depth zip limits mirroring the platform's published rules), load_skill_index_from_registry() feeding existing registry skills into the unchanged SkillIndex/SkillInjector (with revision pinning and local-wins merge semantics), a SkillSearchTool for agent-driven capability discovery, and adk-rust skill search/pull CLI subcommands. No create/update/delete/publish tooling.

10. agent-platform / agent-platform-full umbrella meta-features — single switches on the adk-rust umbrella crate, composable with the tier presets: agent-platform pulls in every Vertex/EAP feature except realtime transports (existing: gemini-vertex, vertex-session, gcp-secrets; plus all features introduced by this proposal) — the right default for ReasoningEngine BYOC deployments; agent-platform-full = ["agent-platform", "vertex-live"] adds the Vertex AI Live API for voice/video agents (which drags in the adk-realtime WebSocket/audio stack — the -full suffix follows the adk-realtime full/full-webrtc precedent). Usage: adk-rust = { features = ["standard", "agent-platform"] }. Each constituent enables its base domain feature (following the postgres-session = ["sessions", ...] pattern) so both compose with minimal too. Includes an immediate fix: the umbrella currently forwards firestore-session/postgres-session/etc. but omits vertex-session, so the existing Vertex sessions backend is unreachable through the umbrella crate today. Deploy-time tooling (adk-deploy/gcp) is deliberately excluded from both (host-side concern, forwarded separately as gcp-deploy). Kept complete by convention (every PR introducing a Vertex/EAP flag appends it to the appropriate list) and enforced by a PR-tier CI check (cargo check -p adk-rust --no-default-features --features minimal,agent-platform) plus both variants in the nightly feature-matrix.

All new endpoints are mock-contract-tested (mirroring adk-session/tests/session_contract_vertex.rs), with #[ignore] live tests gated on GOOGLE_CLOUD_PROJECT/GOOGLE_CLOUD_LOCATION and pointed at pre-provisioned resources (live tests never create platform resources other than the agent itself).

Affected Crate(s)

adk-server, adk-session, adk-memory, adk-model, adk-gemini, adk-telemetry, adk-deploy, adk-cli, cargo-adk, adk-tool (Example Store, Agent Registry), adk-code (sandbox), adk-eval (eval bridge), adk-rag (RAG Engine + Agent Retrieval), adk-skill (Skill Registry consumption, skill-card bridge), adk-core (shared GCP plumbing), adk-rust (umbrella feature forwarding), examples/ (new agent_engine, vertex_rag, agent_orchestrator, and skill_registry examples), docs/official_docs/.

Alternatives Considered

• Reverse proxy/shim in front of the existing /api/run_sse server instead of implementing the class-method protocol: works for raw HTTP callers (the platform proxies arbitrary paths under .../reasoningEngines/{id}/api/*), but the agent remains a second-class citizen — no console Playground, no :query/:streamQuery, no SDK operation_schemas(), no agent_framework: "google-adk" console integration. Rejected as insufficient.
• A standalone adapter crate outside the workspace: avoids workspace churn but duplicates adk-server's auth/session wiring and would drift; the platform contract is small enough to live behind a feature flag in adk-server.
• Full lifecycle clients for skills / RAG corpora / example stores: rejected by design — adk-rust should not duplicate the EAP control plane. Provisioning stays in the console/gcloud/Terraform and the platform's own APIs; adk-rust consumes existing resources. Agent creation is the sole exception because it's the deployment path itself.
• Sidecar OTel Collector only (no native GCP telemetry): kept as the documented fallback, but a native authenticated OTLP path removes an infrastructure dependency that BYOC containers can't easily run (single-container runtime).
• Deploy via Cloud Run instead of Agent Runtime: loses managed Sessions/Memory Bank scoping, console integration, Agent Identity, and the platform's governance surface. Not equivalent.
• Self-hosted retrieval only (pgvector/Qdrant) instead of RAG Engine/Agent Retrieval: remains fully supported and may be preferable for some data-locality postures, but leaves a parity gap with adk-python (VertexAiRagRetrieval) and forgoes the platform's managed retrieval and server-side grounding — hence both are offered.
• A2A-only agent-to-agent communication (skip the registry and :streamQuery client): RemoteA2aAgent already covers agents that expose A2A at a known URL, but it cannot discover agents dynamically, reach engines that only speak the class-method protocol, or participate in Gemini Enterprise's registry-gated import flow. The registry client and RemoteReasoningEngineAgent close those paths; A2A remains the preferred transport where both ends support it.
• Filesystem-only skills (skip Skill Registry): adk-skill's local discovery keeps working and remains the default; but org-wide sharing, semantic search, versioned revisions, and dynamic capability loading require consuming the managed registry — and since both speak the same SKILL.md standard, the read-only client is almost pure transport.

Additional Context

• BYOC contract reference: Deploy an agent — Container Image and the Agent Runtime BYOC codelab (documents the /api/reasoning_engine + /api/stream_reasoning_engine dispatch endpoints, class_methods registration, IAM, and Terraform).
• Agent Registry references: Agent Registry overview, key concepts (URN identifiers include ...reasoningEngines:AGENT_ID; agent principals are SPIFFE-based and platform-assigned), and Import A2A agents from Agent Registry (registry + Agent Gateway + Gemini Enterprise app must be regionally aligned).
• Agent Retrieval references: Agent Retrieval overview (formerly Vector Search 2.0; Collections/Data Objects, auto-embedding, hybrid RRF search, ETag concurrency) and the product name-change table. RAG Engine's RagManagedVertexVectorSearch backend uses Agent Retrieval under the hood (fully managed by RAG). HIPAA note: the RAG-backend comparison lists CMEK support only for RagManagedDb (Spanner) — standalone Agent Retrieval CMEK support is a verification task.
• Skill Registry references: Skill Registry overview (Preview; Skill/SkillRevision entities, payload validation rules) and Create and manage skills. Note Skill Registry (Build pillar, skill packages) and Agent Registry (Govern pillar, agents/MCP/endpoints) are separate services; this proposal consumes both read-only (plus deploy-time agent self-registration).
• The "Python only" limitation applies to the managed build paths (agent object/source files), not BYOC — a Rust container is fully supported.
• ADC inside the Agent Runtime container resolves via the metadata server, so all existing google-cloud-auth-based integrations (adk-gemini vertex, vertex-session, vertex-live, GCP Secret Manager) work unchanged with Agent Identity (Preview) or a custom service account.
• HIPAA-relevant: CMEK (encryptionSpec), VPC-SC, and PSC-I are deployment-time configuration on reasoningEngines.create and are compatible with this plan; the env-config work includes a guard against silent fallback to the non-Vertex Gemini endpoint.
• Stability: several proposed integrations are backed by Preview platform APIs (Skill Registry, Example Store, Agent Registry, Agent Retrieval launch stage TBD, A2A on Agent Runtime). Per STABILITY.md, suggest documenting these feature flags as experimental within their (Stable) host crates — wire-format changes upstream may force breaking changes in minor releases, mirroring how other Preview-backed surfaces are handled.
• One cautionary precedent from adk-python: engines created without async-query support cannot be upgraded in place (adk deploy agent_engine provisions engines without async-query support (:asyncQuery fails FAILED_PRECONDITION 'does not support AsyncQueryReasoningEngine API') google/adk-python#6220 — adk deploy agent_engine provisions engines that reject reasoningEngines:asyncQuery). The proposed deploy tooling should decide classMethods/capability registration completely at create time; this is a verification task in the plan.
• Detailed work-package plan (14 work packages with acceptance criteria and 12 pre-coding verification tasks): posted as a follow-up comment on this issue; it will also accompany the first PR. Sub-tasks can be split into per-crate issues if maintainers prefer; suggested first PR is the adk-server adapter + sessions content fix (plus the one-line vertex-session umbrella forwarding fix, which is independent).

[joseph-wortmann]

AGENT_ENGINE_IMPLEMENTATION_PLAN.md

[joseph-wortmann]

@jkmaina I'm happy (and motivated) to do this work. My employer is building into the Gemini Enterprise Agent Platform, and due to Python supply-chain concerns wants to do it with Rust and therefore adk-rust.

I've attached the overall plan (designed for agentic implementation) to the issue.

As before, I have no pride in authorship, so any changes that you want made are fine with me as long as the end result produces agents that fit into the EAP as ReasoningAgents.

[jkmaina]

@joseph-wortmann as usual you have challenging PRs that are really good! Reviewing this in detail!

[joseph-wortmann]

@jkmaina Any comments from your review? I'm ready to get implementing if it's good to go.

[joseph-wortmann]

@jkmaina I will be beginning this this afternoon.

[joseph-wortmann]

@jkmaina Starting this today (got busy). I'm doing it in chunks because it is huge. Current plan is for 33 separate PR's. Ones that may be merged in parallel will be posted as such. PR's that require previous merges will not be posted until the required one(s) are merged.

[jkmaina]

i have created an agent platform core to adk-rust agents - adk-enterprise - for self hosting/adk-rust enterprise platform and tried to standardise with anthropic agent platform as benchmark in adk-anthropic....review those and align naming to indicate gemini specific platform, i was also adding aws agent core platform - so all these should have some uniforminity and capabilities enhanced via adk-rust...

[joseph-wortmann]

Roadmap:

AGENT_ENGINE_WAVE_0.md
AGENT_ENGINE_WAVE_1.md
AGENT_ENGINE_WAVE_2.md
AGENT_ENGINE_WAVE_3.md
AGENT_ENGINE_WAVE_4.md