diff --git a/.claude/agents/mcp-server-engineer.md b/.claude/agents/mcp-server-engineer.md new file mode 100644 index 00000000000..8d5c2abb97c --- /dev/null +++ b/.claude/agents/mcp-server-engineer.md @@ -0,0 +1,97 @@ +--- +name: mcp-server-engineer +description: Own the embedded MCP automation server in unity-explorer — design and implement MCP tools, protocol/transport changes, and the mcp-scene-iteration skill so coding agents can see and drive a running Explorer build +skills: + - code-standards + - async-programming + - ecs-system-and-component-design + - plugin-architecture + - feature-flags-and-configuration + - diagnostics-and-logging + - scene-runtime-and-crdt +--- + +# MCP Server Engineer + +You own the embedded MCP (Model Context Protocol) server inside the Decentraland Unity Explorer: the feature that lets external coding agents observe the running client (screenshots, player/scene state, scene JS logs) and control it (teleport, movement, camera, chat commands, scene reload). You design and implement new MCP tools, maintain the transport/protocol layer, and keep the agent-facing docs and skill in sync with the server. + +Read [`docs/mcp-automation.md`](../../docs/mcp-automation.md) before touching anything — it is the human-facing contract for this feature. + +## MANDATORY: plan mode before new tools + +Implementing a new MCP tool (or changing server behavior) **must go through plan mode first**: research the codebase, present the plan, and get explicit user approval before writing any code. This is a standing user instruction — it applies even when the request looks trivial. + +Tool requests from agent sessions accumulate in the **"Wanted tools"** section of [`.claude/skills/mcp-scene-iteration/SKILL.md`](../skills/mcp-scene-iteration/SKILL.md) (name, args, output shape, blocked use case). Check it when asked to extend the server, and remove entries once implemented. + +## Architecture map + +Everything lives in `Explorer/Assets/DCL/McpServer/`, its own `DCL.McpServer` assembly (root asmdef). Two subfolders are folded into other assemblies via `.asmref`: `Systems/` → `DCL.Plugins` (GUID `fc4fd35fb877e904d8cedee73b2256f6`; `DynamicWorldContainer` lives there) and `Tests/` → `DCL.EditMode.Tests`. + +| Piece | Path | Role | +|---|---|---| +| Plugin | `Systems/McpServerPlugin.cs` | Builds the tool registry in `InjectToWorld` (needs `GlobalPluginArguments.PlayerEntity/SkyboxEntity`), starts/disposes the server, wires the scene-log tap (`DiagnosticsContainer.AddDebugConsoleHandler`) | +| Core (transport + protocol + contract) | `Core/McpHttpServer.cs`, `McpJsonRpcDispatcher.cs`, `IMcpTool.cs`, `McpToolsRegistry.cs`, `McpToolResult.cs`, `McpToolAnnotations.cs`, `McpInputSchema.cs` | `HttpListener` on `http://127.0.0.1:{port}/unity-explorer-mcp` (endpoint path is single-sourced in `McpHttpServer` — commit `7a734c77b`); POST → dispatch, GET → 405, Origin allowlist, 1 MB body cap; JSON-RPC 2.0 over Streamable HTTP (spec 2025-06-18), tools-only capability | +| Tools | `Tools/*.cs` (one class per tool, 16) | The agent-facing surface | +| Components | `Components/` — `McpMovementOverride`, `McpPointerClickIntent` | ECS intents for the input-driving tools | +| Systems | `Systems/McpInputOverrideSystem.cs` (held movement), `Systems/McpPointerClickSystem.cs` (synthetic entity clicks) | Per-frame/pipeline-integrated drivers | +| Utils | `Utils/SceneLogBuffer.cs`, `JObjectExtensions.cs` | Log tap buffer, args parsing | + +Registration: `DynamicWorldContainer.CreateAsync`, gated on `FeaturesRegistry` `FeatureId.MCP_SERVER` = `appArgs.HasFlag(MCP) || appArgs.HasFlag(MCP_PORT)` — so `--mcp-port` alone implies `--mcp` (presence check; an invalid port value still enables the server and falls back to 8123). Flags accepted from CLI **or** deep link by user decision — do not add CLI-only enforcement without being asked. Log category: `ReportCategory.MCP`. + +Cross-repo launch path: `@dcl/sdk-commands` (`../js-sdk-toolchain`, `packages/@dcl/sdk-commands/src/commands/start/{index,explorer-alpha}.ts`) forwards `--mcp` / `--mcp-port` from `npm run start` into the `decentraland://` deep link (`mcp=true` / `mcp-port=`), plus arbitrary params after a second standalone `--`. Flag renames or deep-link changes must stay consistent across both repos. + +Request flow: `HttpListener` accepts on the thread pool → detached `UniTaskVoid` per request → dispatcher parses/routes → tool's `ExecuteAsync(JObject, ct)` begins with `await UniTask.SwitchToMainThread(ct)` for any ECS/Unity access → heavy encoding (base64) hops back via `DCLTask.SwitchToThreadPool()`. + +## Adding a tool — checklist + +1. One class in `Tools/`, implementing `IMcpTool` (`Name` snake_case, 1–2 sentence `Description` written for an agent, `InputSchema` as a `JObject` built with the `McpInputSchema` builder — malformed schemas fail at registration, not first use; `Annotations` behaviour hints; override the default-null `OutputSchema` only for tools returning `McpToolResult.TextWithStructured`). +2. Parse args with the `JObjectExtensions` helpers (`GetBool`/`GetInt`/`GetFloat`/`GetString` with defaults); validate before switching threads; expected failures return `McpToolResult.Error(...)` (never throw — JSON-RPC errors are for protocol-level failures only). +3. Register it in `McpServerPlugin.InjectToWorld`; dependencies must be readable from `DynamicWorldContainer.CreateAsync` scope (never mutate containers). +4. ECS writes go through **intent components** — reuse `GlobalWorldActions` (`MoveAndRotatePlayerAsync`, `RotateCamera`, `TriggerEmote`) or `IChatMessagesBus` / `ECSReloadScene` / `IWorldInfoHub` before inventing anything. A new `BaseUnityLoopSystem` is justified only when a value must be re-asserted every frame against a real-input system (see `McpInputOverrideSystem`, ordered `[UpdateAfter(typeof(UpdateInputMovementSystem))]`). +5. Long-running tools own an explicit timeout and return a truthful text result on expiry (see `TeleportTool` polling + deadline). +6. Update **all three** agent-facing surfaces: tool catalog in `docs/mcp-automation.md`, `docs/app-arguments.md` if flags changed, and the skill if the loop changes. + +## Hard rules + +- **Security invariants**: bind 127.0.0.1 only; keep the Origin allowlist in `McpHttpServer.IsAllowed` (absent Origin = CLI = allowed; non-localhost = 403). No auth token by design (v1). +- **Texture memory discipline** (standing user requirement): screenshots must never accumulate textures. Temp RTs via `GetTemporary`/`ReleaseTemporary` released in `finally`; the `ScreenCapture.CaptureScreenshotAsTexture()` result destroyed immediately after blitting; the ReadPixels fallback reuses one persistent buffer; concurrent captures rejected via an `Interlocked` gate. +- **Async rules**: ignore `OperationCanceledException`; `ReportHub.LogException(e, ReportCategory.MCP)` for the rest; no `ThrowIfCancellationRequested()` in exception-free flows. +- **No LINQ**, ReportHub not Debug.Log, nullable annotations, no `!` null-forgiving operator. + +## Known pitfalls (learned the hard way) + +- `DCL.Time` namespace shadows `UnityEngine.Time` inside any `DCL.*` namespace — always write `UnityEngine.Time.time` fully qualified. +- `CachePhysicsTick`/`GetPhysicsTickComponent` exist in BOTH `DCL.CharacterMotion` and `DCL.Input` `WorldExtensions` — importing both namespaces is a CS0121 ambiguity. Import only `DCL.Input` (needed for `InputGroup` anyway). +- `ref` locals (`TryGetRef`) are illegal in async methods (CS8177) — use `world.TryGet` copies in tools; `TryGetRef` only in synchronous system `Update`. +- `Camera.Render()` is unsupported under URP — the `worldOnly` screenshot uses a one-frame `camera.targetTexture` redirect instead. +- `UpdateInputMovementSystem` overwrites `MovementInputComponent` every frame (and zeroes it when the action map is disabled) — held input must be re-asserted by a system ordered after it, not written once. +- Complete all `ref` component reads before any structural change (`Remove`/`Add`) — copy what you need (e.g. the `Completion` source) first. +- Unity generates `.meta` files for new files on the next Editor open; you cannot compile from the CLI — the user verifies in the Editor or a manual build and pastes compile errors back. + +## Skill stewardship + +The agent-side workflow lives in `.claude/skills/mcp-scene-iteration/` (user-invokable only). Field sessions edit it with verified learnings — treat their additions as ground truth about real behavior and never revert them blindly. The bundled `scripts/screenshot.sh` captures frames to disk via raw JSON-RPC so agents don't burn context on frequent screenshots; keep it working if the tool schema changes. + +## Verification + +EditMode tests exist in `Tests/` (folded into `DCL.EditMode.Tests` via asmref): dispatcher, registry, result routing, input schema, HTTP server, state tools, pointer-click system. Run them in the Unity Test Runner — you cannot compile or run tests from the CLI. Smoke-test the protocol layer with the running client: + +```bash +curl -s -X POST http://127.0.0.1:8123/unity-explorer-mcp -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' +``` + +Editor run: add `--mcp` to `Main Scene Loader → Debug Settings → App Parameters` in `Assets/Scenes/Main.unity` and hit Play. Standalone against a local scene: `npm run start -- --mcp` in the scene folder auto-launches the installed client with the server on (`--mcp-port ` for another port, `--no-client` to serve only, `--multi-instance` + distinct `--port`/`--mcp-port` for side-by-side instances). Full launch lines are in `docs/mcp-automation.md`. + +## Git rules + +**NEVER commit or push.** All work stays as local changes — the user decides when and what to commit. + +Allowed: `git checkout -b`, `git diff`, `git status`, `git log`, `git branch` +Forbidden: `git commit`, `git push`, `git merge`, `git rebase` + +## Roadmap context + +Milestone 2 (pointer clicks) SHIPPED 2026-07-05 as `click_entity` — implemented via **semantic injection**, not the originally-scoped synthetic `InputSystem` device: `McpPointerClickSystem` raycasts camera→target, mirrors the distance gate, and fills the entity's `PBPointerEvents.AppendPointerEventResultsIntent` so the unmodified `WritePointerEventResultsSystem` emits a byte-identical `PBPointerEventsResult`. Zero production interaction code changed; approach recorded in `~/.claude/plans/wondrous-forging-fox.md`. + +Current "Wanted tools" head: **recover_scene** — force-recreate a scene that dropped out of `ScenesCache` (`get_scene_state` → `scene: null`, the LSD hard-wedge from rapid saves where every existing reload path needs the cached facade). Implementation lead is in the skill's Wanted tools entry. diff --git a/.claude/skills/mcp-scene-iteration/SKILL.md b/.claude/skills/mcp-scene-iteration/SKILL.md new file mode 100644 index 00000000000..8632b6991f3 --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/SKILL.md @@ -0,0 +1,149 @@ +--- +name: mcp-scene-iteration +description: Iterate on SDK7 scenes against a running Explorer build via its embedded MCP server. +disable-model-invocation: true +--- + +# MCP Scene Iteration + +Drive a running Explorer build through its embedded MCP server to build and test SDK7 scenes autonomously. + +Full tool catalog and flag reference: [`docs/mcp-automation.md`](../../../docs/mcp-automation.md). + +Deeper reference, loaded only when the task reaches it: + +- [`reference/camera-and-movement.md`](reference/camera-and-movement.md) — before framing screenshots, free-camera sweeps, or navigating precise lines +- [`reference/assets.md`](reference/assets.md) — before placing, downloading, converting, or exporting any 3D model +- [`reference/visuals.md`](reference/visuals.md) — before tuning emissives/bloom, UI overlays, skybox time, or judging thin geometry + +## Setup (once per session) + +0. **Load the SDK skills.** This skill only covers driving the Explorer; the SDK7 API knowledge (composite-first rule, component reference) lives in the `sdk-scenes` skill set, and parts of the API (e.g. native `TriggerArea`) are newer than training data. Try to load it: session skills first, then the filesystem — scene-local (`.claude/skills/` in the scene folder) and global (`~/.claude/skills/`). If it cannot be loaded, **MANDATORY — ask the user**: install `sdk-skills` (from https://github.com/decentraland/sdk-skills)? Recommend installing. If YES, ask at which level — scene-local or global — and run the matching command: + + ```bash + npx skills add decentraland/sdk-skills --all # scene-local (run inside the scene folder) + npx skills add decentraland/sdk-skills --all -g # global (user-level, ~/.claude/skills) + ``` + + Skills are loaded at session start, so a mid-session install may not surface until the session restarts. If NO, move forward without them — the scene can still be implemented, just less efficiently: verify any API you are not certain about against the official docs instead of writing it from memory. + +1. **Probe for an already-running MCP server, then start the scene.** Check through the harness first: if `mcp__explorer__*` tools are available in the session, call `get_scene_state` — an answer means the server is up. Fall back to curl **only if the tools are absent**: + + ```bash + curl -s -m 2 http://127.0.0.1:8123/unity-explorer-mcp -X POST \ + -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \ + -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"probe","version":"1"}}}' + ``` + + **Server found** (tool answer or `serverInfo` result) — **MANDATORY — ask the user**: use the already-running Explorer, or start the scene from scratch with the MCP flag? Never decide silently. + - *Use it*: launch nothing. If port 8000 isn't serving the target scene folder (`lsof -nP -i :8000 -sTCP:LISTEN`, then check the PID's cwd), kill whatever holds it and run `npm run start -- --no-client`. Skip step 3 if the tools are already available. + - *From scratch*: **MANDATORY — follow-up question**: kill the previously-running scene server, or keep it and run a second stack alongside? Never kill it unasked. + - *Kill it*: kill the port-8000 dev server, have the user close the running client (never kill an Editor process yourself), then continue below. + - *Keep it*: leave it and its Explorer untouched; start a second stack on its own ports — a different dev-server port (`--port`; the launched client follows it automatically), a different MCP port (`--mcp-port`, implies `--mcp`), and `--multi-instance` so a second Explorer instance can run concurrently: + + ```bash + npm install && npm run start -- --port 8666 --multi-instance --mcp-port 8124 + ``` + + From here on use the chosen ports instead of 8000/8123 — including step 3's registration, which needs a distinct server name (e.g. `claude mcp add --transport http --scope user explorer2 http://127.0.0.1:8124/unity-explorer-mcp`; the tools then surface as `mcp__explorer2__*`). + + **No server found** — serve the scene and launch the Explorer in one command from the scene folder (keep it running in the background; if something else already holds port 8000, apply the same kill-or-keep question and port overrides as above): + + ```bash + npm install && npm run start -- --mcp + ``` + + This serves the scene at `http://127.0.0.1:8000`, auto-launches the **installed** Decentraland client connected to it with the MCP server enabled (port 8123; `--mcp-port ` picks another and implies `--mcp` — adjust the 8123 URLs in steps 1 and 3 to match), and the LSD dev server hot-reloads the running Explorer on file changes. Other useful flags: `--port ` (dev-server port; the launched client follows it automatically), `--position x,y`, `--skip-auth-screen`, `-n` (new client instance), `--multi-instance` (allow concurrent Explorer instances), `--no-client` (serve only, launch nothing). Anything after a second standalone `--` is forwarded verbatim into the launch as extra Explorer params, e.g. `npm run start -- --mcp -- --windowed-mode --resolution 1280x720` (npm consumes the first `--`). If `--mcp` is rejected as an unknown option, the scene's `@dcl/sdk-commands` predates the flag — update `@dcl/sdk`, or fall back to step 2. + + **A freshly launched Explorer needs the user to log in.** The client opens on the auth screen unless a previous session's login is still cached (`--skip-auth-screen` only skips it when a valid identity exists — a missing or expired login shows it anyway, and extra `--multi-instance` instances always ask). Tell the user to log in, then wait — step 4's polling only starts succeeding once they're through, and only then can you continue working on the scene through the MCP server. + +2. **(Alternative) Launch a specific Explorer build manually** — a local build or the Editor instead of the installed client. Serve with `npm run start -- --no-client` in step 1, then: + + ```bash + # macOS + open /path/to/Decentraland.app --args \ + --realm http://127.0.0.1:8000 --local-scene true --position 0,0 \ + --debug --skip-auth-screen --skip-version-check true \ + --mcp --windowed-mode --resolution 1280x720 + ``` + + On Windows call `Decentraland.exe` with the same arguments. Add `--disable-hud --skybox-time-enabled false --landscape-terrain-enabled false` when you want deterministic screenshots without HUD noise. + +3. **Connect the MCP server** (default port 8123): + + ```bash + claude mcp add --transport http --scope user explorer http://127.0.0.1:8123/unity-explorer-mcp + ``` + + Errors with "already exists in local config" if registered by a previous session — that's fine, nothing to do. If the current session has no `mcp__explorer__*` tools, follow "Missing tools" under Scene health & recovery below — the fix is the user reconnecting via `/mcp`, not a workaround. + +4. Wait for the world to load: poll `get_scene_state` until `loadingScreenOn` is false and the scene reports `isReady: true`. + +## The iteration loop + +Repeat until **every requirement has proof**: a screenshot or state read demonstrating it, captured from a retail camera mode (`first_person`/`third_person`, not the free camera), with `get_scene_state` healthy and no unexplained errors in the logs. + +1. **Edit** the scene TypeScript in `src/` — the LSD dev server hot-reloads the running Explorer within a few seconds. If you need a deterministic reset instead, call `reload_scene`. +2. **Confirm the scene is healthy**: `get_scene_state` — a `state` of `JavaScriptError` or `EcsError` means your code crashed the scene runtime. +3. **Read the runtime output**: `get_scene_logs` with `sinceSeq` set to the last sequence number you saw. Scene `console.log` output and exceptions land here. +4. **Look and verify**: position the view (`teleport`, `move_to`, `walk`, `look_at` — for precise framing or free-camera sweeps read [`reference/camera-and-movement.md`](reference/camera-and-movement.md)), then `screenshot` and inspect the image against what the scene code should produce. +5. **Exercise behavior**: `walk` into trigger areas, `click_entity` on interactables, `send_chat` for commands, `trigger_emote`, and re-screenshot to verify reactions. `list_scene_entities` + `get_entity_details` show the scene's ECS state when visuals aren't enough. + +**Cross-examine** every conclusion: confirm each visual claim with a state read (ECS values via `get_entity_details`, logs, `get_player_state` position), and each state claim with pixels. One channel lies routinely — colliders exist that pixels don't show, entities render invisible while their state looks healthy, animations silently don't play. The reference files call out where cross-examination is mandatory. + +**MANDATORY — camera cleanup before finishing.** NEVER leave the camera in `free` mode when you stop working (end of task, handing back to the user, or pausing for their input): always restore it with `set_camera_mode third_person` as your last camera action, and confirm via `get_player_state` → `camera.mode` if anything in between could have failed. + +## Screenshot frequency & cost + +Every screenshot returned by the MCP `screenshot` tool lands in your context as an image (~1.2k tokens at 1280×720, scaling with pixel count). Occasional captures through the tool are fine; **frequent or burst captures must go through the bundled script instead**, which saves frames to disk (zero context cost) and prints only the caption: + +```bash +scripts/screenshot.sh -o shot.jpg # single frame to a file +scripts/screenshot.sh -n 10 -i 0.5 # burst: 10 frames every 0.5s into mcp-shots/ (time-based behavior: tweens, animations) +scripts/screenshot.sh -w 640 # cheap sanity-check resolution (~4x fewer tokens when you Read it) +scripts/screenshot.sh --world-only --png # UI-less lossless frame +``` + +Paths are relative to this skill's directory; requires curl + python3; pass `-p ` when not on 8123. Then `Read` only the frames you actually need to inspect — capture many, look at few. For before/after comparisons, capture both to disk and read just those two. Use `maxWidth` 640 for quick checks and 1280 only for final verification. Captures are serialized server-side (concurrent requests are rejected), so keep burst intervals ≥ 0.2s. + +## Scene health & recovery + +- Sequence-poll logs (`sinceSeq`) instead of re-reading the whole buffer; errors survive in the buffer even if they scrolled by. +- `scene.json` changes (parcels, spawn points) are not hot-reloaded — restart the `npm run start` process, then `reload_scene`. +- After `teleport` or `reload_scene`, always re-check `get_scene_state` before interacting; readiness can lag a few seconds. +- One parcel is 16×16 m; parcel `(x, y)` spans world positions `(16x..16x+16, 16y..16y+16)`. `--position 0,0` spawns at parcel 0,0. +- If the connection drops, the client probably crashed or was closed — relaunch it the same way it was started (`npm run start -- --mcp`, or the manual launch line); the MCP endpoint URL stays the same. +- **Missing tools**: `mcp__explorer__*` tools absent in-session are recoverable (typically the Explorer wasn't running when the session started, so the registered server failed its startup connection). Ask the user to run `/mcp` and reconnect the `explorer` server — an interactive command only the user can run; a successful reconnect binds all the server's tools into the running session (verified). A plain `claude mcp add` mid-session does NOT surface tools by itself. Last resort: drive the endpoint directly with curl JSON-RPC (`POST /unity-explorer-mcp`, methods `initialize` then `tools/call`; responses may be SSE-framed, tool payloads are JSON in `result.content[0].text`, screenshots are base64 in image content blocks). +- After a hot reload the player can end up off-parcel (e.g. parcel `0,-1`); `get_scene_state` then reports a null scene and `reload_scene` fails with "no scene at the current parcel". Check `get_player_state` → `parcel`, `move_to` back inside, and the scene loads again. +- Each file save triggers a rebuild: editing usage and import in separate saves produces a transient `SceneError: X is not defined` between them. Write new modules before wiring them in, and prefer a single whole-file write for multi-part edits to one file. +- **Rapid successive saves can HARD-WEDGE the client (verified 2026-07-10).** Two saves seconds apart made the Explorer load a mid-write bundle → `SyntaxError: Invalid or unexpected token` at scene start → the scene facade is torn down and drops out of `ScenesCache`, and `get_scene_state` reports `scene: null` while standing on the parcel. From that state NOTHING recovers in-session: `reload_scene` errors ("no scene at the current parcel" — its guard and every underlying reload path need the scene still cached), `/reload` hangs until cancelled, the minimap RELOAD SCENE button just sends `/reload`, LSD file-save pushes no-op (`TryGetBySceneId` misses), and moving far off-parcel and back does not recreate the facade. Only exiting/re-entering play mode (editor) or relaunching the standalone build recovers. Prevention: batch multi-edit changes into ONE file write, and after any save landing seconds after a previous one, verify `get_scene_state` still shows a scene before saving again. +- The `teleport` tool silently no-ops in local-scene-development mode: `/goto` teleports are disallowed there (chat shows "Teleport is not allowed in local scene development mode") but the tool still answers "Arrived at (x,y)". Use `move_to` for repositioning in LSD sessions. +- The Explorer under test may be the **Unity Editor in play mode**, not a standalone Decentraland.app — check `ps aux | grep -i unity` before considering a relaunch. Never kill the editor process; recovery from a wedged client is then a user action (exit/re-enter play mode). + +## Interaction testing + +- `click_entity` presses a pointer button on a scene entity (get ids from `list_scene_entities`). The target needs a `PointerEvents` component and a collider; the aim is validated by a real camera-origin raycast, so occluders return `hit:false` + `blockedBy*` (reposition and retry) and the entity's `maxDistance` (default 10 m) applies — get close first. `upRayMissed: true` means the target moved between press and release (e.g. a door starting to swing) and the release was delivered with the press-frame hit. For GLTF entities whose collider sits away from the pivot, pass an explicit `x/y/z` aim point. The player must be standing on the scene's parcel — off-parcel clicks fail with "no running current scene". +- `walk` moves relative to the camera and requires an explicit direction: pass `directionY: 1` for forward (`directionX` strafes); omitting both errors with "directionX and directionY must not both be zero". +- Collider checks beat pixels for physics (cross-examine): `look_at` straight at the target, `walk` forward, then compare `get_player_state` positions to prove passage or blockage. +- Trigger areas fire `onTriggerEnter` immediately after `reload_scene` if the player is already standing inside one — reposition the player outside all triggers before testing enter/exit sequencing (and treat post-reload trigger logs as stale state, not gameplay). + +## Improving this skill + +Two rules: + +**1. Learned something the skill didn't tell you?** A flag that turned out to be required, a timing quirk, a better verification pattern, or information here that proved wrong — edit this skill in place before finishing your session. Keep additions terse and verified (facts you observed, not speculation), and file them where their branch lives: setup, health/recovery, and interaction facts in this SKILL.md; camera and navigation facts in `reference/camera-and-movement.md`; model and import facts in `reference/assets.md`; rendering and visual-tuning facts in `reference/visuals.md`. The canonical copy lives in the unity-explorer repo at `.claude/skills/mcp-scene-iteration/`; if you are running from a copy (e.g. `~/.claude/skills/`), apply the same edit to the canonical copy too when the repo is accessible, or tell the user to sync it. + +**2. Missing a capability?** If the loop is blocked because no existing MCP tool can do what you need (e.g. pressing a specific key, reading a value no tool exposes), do NOT work around it by modifying the Explorer client, the MCP server, or the unity-explorer repo yourself. Stop and prompt the user with a concrete tool proposal: + +- proposed tool name and one-line purpose +- input arguments (names, types, defaults) +- expected output shape +- the blocked use case, and why the existing tools can't cover it + +The user decides whether and when to implement it. **MANDATORY: implementing an approved tool must go through plan mode first** — whichever session does the implementation starts in plan mode, researches the unity-explorer codebase (the server lives under `Explorer/Assets/DCL/McpServer/` — see `docs/mcp-automation.md` → Implementation map), and presents the plan for user approval before writing any code. Also append the proposal to the "Wanted tools" list below so it isn't lost if the user defers. + +## Wanted tools + +Proposals from agent sessions — name, purpose, blocked use case. Remove entries once implemented. + +- **recover_scene** — force-recreate the scene at the player's parcel when it has dropped out of `ScenesCache` (`get_scene_state` → `scene: null`; the hard-wedge state described above, where every existing reload path needs the cached facade and the session is dead until the user restarts play mode). Inputs: `timeoutSec?: number` (default 30). Output: same shape as `reload_scene`. Implementation lead: clear failed `AssetPromise` state on the definition entity and reset `StaticScenePointers.Promise` on the realm entity so the static-pointer systems re-resolve — the same reset `ECSReloadScene.DisposeAndRestartAsync` already performs for LSD, minus the requirement that a live scene exists. diff --git a/.claude/skills/mcp-scene-iteration/reference/assets.md b/.claude/skills/mcp-scene-iteration/reference/assets.md new file mode 100644 index 00000000000..c0a4cd8183a --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/reference/assets.md @@ -0,0 +1,22 @@ +# 3D asset reference + +Read before placing, downloading, converting, or exporting any 3D model. + +## Picking from the sdk-skills catalog + +- Fetch the `preview:` thumbnail BEFORE downloading — some models render near-black or broken even in their own previews (e.g. arcade-cabinet-atari). Use the exact `[anim: ...]` clip names; a wrong clip name fails silently (no error, no motion) — cross-examine: burst-capture (`-n 3 -i 1`) and diff frames to prove an animation is actually running. +- Catalog dims/pivots lie: bounding boxes can include baked animation paths or outline meshes, pivots can sit mid-crown (palms) or nowhere near the mesh (a baked drive path carried one car's mesh entirely off-parcel — swap such models rather than debug them). Place → screenshot → adjust is faster than trusting listed sizes. + +## Downloading models + +- Free CC0 model sources that download cleanly via curl: kenney.nl (zip URL is on the asset page, FBX+OBJ+GLTF inside), and itch.io free packs via the scripted flow: POST `/download_url` with the page's csrf_token → GET the returned key URL → grab `data-upload_id` → POST `/file/?source=game_download` → signed CDN URL (expires ~60s, download immediately). +- Downloaded GLBs into the scene folder hot-load without restarting the dev server. Many props ship with no colliders — cross-examine solidity: walk onto them and check the player's `y` via `get_player_state`; add `visibleMeshesCollisionMask: 3` for anything that should be solid. + +## Blender authoring & conversion + +- Blender-authored GLBs work end-to-end: export with `bpy.ops.export_scene.gltf(use_selection=True)` straight into the scene's Models folder (hot-loads like any file change). Set the object origin to bottom-center before export (`cursor to (0,0,0)` + `origin_set(type='ORIGIN_CURSOR')` with geometry built up from z=0) so `position.y = 0` grounds the model, and `transform_apply` rotation/scale for clean transforms. Principled BSDF emissive (Emission Color + Strength) renders as expected in Explorer, including the zero-channel neon saturation rule (see `visuals.md`). +- Converting downloaded FBX/OBJ to GLB in Blender works, with three verified traps: (1) FBX materials can import with Principled Alpha = 0 (FBX transparency-factor quirk, seen on Quaternius packs) — the GLB then has `alphaMode: MASK` with baseColor alpha 0 and the model is INVISIBLE in Explorer while its entity, tween and logs all look healthy; force Alpha=1 + `blend_method='OPAQUE'` before export, and when a GLB renders nothing, parse its JSON chunk (nodes/materials) instead of guessing. (2) The glTF exporter's default animation mode exports EVERY action in the .blend that fits the armature, so clips from other imported models leak into each GLB; use `export_animation_mode='ACTIVE_ACTIONS'` with the right action active (exports one clip named `Animation`). (3) Some kits (e.g. Kenney furniture) ship ASCII FBX which Blender refuses — run `file *.fbx` first and fall back to the kit's OBJ folder. Skinned meshes respect entity Transform scale, so oversize rigs can be scaled at the entity. + +## Composite gotchas + +- Composite-authored box primitives need `"box": {"uvs": []}` in `core::MeshRenderer` — a bare `"box": {}` crashes `sdk-commands build` with `TypeError: message.uvs is not iterable`. diff --git a/.claude/skills/mcp-scene-iteration/reference/camera-and-movement.md b/.claude/skills/mcp-scene-iteration/reference/camera-and-movement.md new file mode 100644 index 00000000000..54139de0a53 --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/reference/camera-and-movement.md @@ -0,0 +1,18 @@ +# Camera & movement reference + +Read before framing screenshots, navigating precise lines, or inspecting a build from multiple viewpoints. + +## Aiming the camera + +- `move_to`'s `lookAt*` params orient the avatar but NOT the third-person camera; `screenshot` and `walk` follow the camera. Call the standalone `look_at` tool (it aligns camera yaw — confirm via `get_player_state` → `camera.rotationEuler.y`) before walking a precise line or framing a screenshot. +- `look_at` lines the third-person camera up through the avatar, so the avatar occludes exactly the thing you framed. To photograph a subject, first `move_to` a spot offset sideways from the camera→subject line, then `look_at`. + +## Free camera + +- The free camera is the fastest way to inspect a scene from many points of view: `set_camera_pose` places it at any absolute position, optionally aims it (`lookAt*`) and sets `fov`, auto-entering free mode. Repositioning while already free is instant (~200ms), so sweep a build cheaply — aerial plan view, each facade, eye-level details, interiors — capturing to disk between calls, instead of walking the player around. `look_at` also works in free mode (aims from the camera's own position), and the free camera stays put while the player moves, so you can even watch the avatar walk through the scene from a fixed vantage. Entering free from another mode blends over ~2-3s (the tool waits and reports `settled`). +- The free camera is a debug view, not what players see. To confirm the end-user experience, switch back to the real modes — `set_camera_mode` `first_person` / `third_person` / `drone` are exactly the cameras retail users have — and re-check framing, avatar occlusion, and interaction reach from there (e.g. verify a hover target is actually visible and clickable at player height, not just from a flattering freecam angle). **MANDATORY: never leave the camera in free mode when you finish working** — always restore `set_camera_mode third_person` as your last camera action. `set_camera_mode` respects scene locks and errors truthfully — check `get_player_state` → `camera.modeChangeAllowed` first; `false` inside a `CameraModeArea`/scene virtual camera is correct behavior worth verifying, not a tool failure. `screenshot` works in any mode. + +## Precise navigation + +- Collider-bump navigation moves the player along precise lines reliably: `look_at` a point past where you want to end up, `walk` with generous `seconds`, and let a blocking collider stop the player — the returned `endPosition` lands ~0.38m (capsule radius) short of the collider face, a deterministic waypoint for the next leg with no duration tuning. Timing legs precisely is fragile (jog ramp-up varies effective speed); overshooting into a collider is not. Only a final leg through a gap with nothing beyond it needs a tight duration, or the player runs off-parcel. +- Measured locomotion speeds (flat ground): jog ≈6.5 m/s once ramped, `kind: "walk"` ≈1.4 m/s. To stop at a precise point no collider will stop you at, take one timed jog leg, read `endPosition`, then micro-correct with 0.5-1.2s `kind: "walk"` legs — position feedback plus slow corrections converges in 1-2 iterations where a single timed jog leg won't. diff --git a/.claude/skills/mcp-scene-iteration/reference/visuals.md b/.claude/skills/mcp-scene-iteration/reference/visuals.md new file mode 100644 index 00000000000..8325bac0895 --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/reference/visuals.md @@ -0,0 +1,9 @@ +# Visual tuning reference + +Read before tuning emissives/bloom, UI overlays, skybox time, or judging visibility of thin geometry. + +- `screenshot`'s `worldOnly` (and the script's `--world-only`) renders with full post-processing, bloom included (fixed + verified 2026-07-06: the capture target was LDR, which clamped emissives and starved bloom — it is HDR now). Emissive/glow looks are judgeable from world-only frames, with one caveat: halo spread reads slightly tighter than the full-view capture (render-resolution difference), so do final pixel-exact glow tuning on full-view frames. +- Neon emissive recipe (verified with bloom, sunset skybox): pin one emissive channel to exactly 0 (e.g. `(1, 0, 0.6)` magenta) — a small minor channel (0.15-0.2) times `emissiveIntensity` 6+ whites the whole surface out. Then weight by luminance and background: green-heavy hues (cyan) bloom to white far faster than magenta and additively desaturate against the pink haze (cyan + pink = white), so dim or blue-shift them (`(0, 0.35, 0.8)` works); magenta reinforces the sunset and stays saturated at intensity 6-15. A large on-screen emitter always whites out at its core — the hue lives in the halo, which is what a real neon tube looks like; judge hue from gameplay distance, not close-ups. Bloom already provides the halo, so an emitter needs no alpha-blended "glow shell" box around it — a shell just reads as a grey display case. +- UiBackground texture entities render far stronger than a near-zero `color` alpha suggests (a 0.05-alpha noise overlay still fully obscured the image beneath). For flicker/glitch overlays, mount/unmount the entity conditionally (`{cond ? : null}`) instead of animating alpha near zero. +- Thin geometry (a door panel ~0.1m) is invisible edge-on: a panel "open" at ~90-110° reads as a sliver from the front. Cross-examine pose via ECS rotation (`get_entity_details` quaternion) as well as pixels, and prefer open angles like ~135° when front visibility matters. +- `SkyboxTime.create(engine.RootEntity, { fixedTime })` pins the scene to a permanent time of day regardless of launch flags. diff --git a/.claude/skills/mcp-scene-iteration/scripts/screenshot.sh b/.claude/skills/mcp-scene-iteration/scripts/screenshot.sh new file mode 100755 index 00000000000..1b4ed6a879b --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/scripts/screenshot.sh @@ -0,0 +1,120 @@ +#!/usr/bin/env bash +# Capture Explorer screenshots to disk via the embedded MCP server, without spending agent context. +# Frames are saved as files; only the caption (resolution + parcel) is printed. Read a frame file +# only when you actually need to inspect it. +# +# Usage: screenshot.sh [options] +# -o, --out FILE output file (single capture; extension follows quality) +# -d, --dir DIR output directory (default: mcp-shots/; used for bursts and when -o is omitted) +# -n, --count N number of frames to capture (default: 1) +# -i, --interval SEC seconds between frames in a burst (default: 0.5; keep >= 0.2, captures are serialized) +# -w, --max-width PX maxWidth passed to the tool (default: 1280; use 640 for cheap sanity checks) +# --png capture PNG instead of JPG +# --world-only exclude UI overlays (worldOnly: true) +# -p, --port PORT MCP server port (default: 8123) +# +# Requires: curl, python3. The Explorer must be running with --mcp. + +set -euo pipefail + +PORT=8123 +MAX_WIDTH=1280 +QUALITY=jpg +WORLD_ONLY=false +COUNT=1 +INTERVAL=0.5 +OUT="" +OUT_DIR="mcp-shots" + +while [[ $# -gt 0 ]]; do + case "$1" in + -o|--out) OUT="$2"; shift 2 ;; + -d|--dir) OUT_DIR="$2"; shift 2 ;; + -n|--count) COUNT="$2"; shift 2 ;; + -i|--interval) INTERVAL="$2"; shift 2 ;; + -w|--max-width) MAX_WIDTH="$2"; shift 2 ;; + --png) QUALITY=png; shift ;; + --world-only) WORLD_ONLY=true; shift ;; + -p|--port) PORT="$2"; shift 2 ;; + -h|--help) sed -n '2,17p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; + *) echo "unknown argument: $1 (see --help)" >&2; exit 2 ;; + esac +done + +capture_one() { + local target_file="$1" + + local payload + payload=$(printf '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"screenshot","arguments":{"maxWidth":%s,"quality":"%s","worldOnly":%s}}}' \ + "$MAX_WIDTH" "$QUALITY" "$WORLD_ONLY") + + # NOTE: response must land in a file, not a pipe into `python3 - </dev/null || true + capture_one "$OUT" + exit 0 +fi + +mkdir -p "$OUT_DIR" +stamp=$(date +%Y%m%d-%H%M%S) + +for ((frame = 1; frame <= COUNT; frame++)); do + capture_one "${OUT_DIR}/shot-${stamp}-$(printf '%03d' "$frame").${QUALITY}" + + if (( frame < COUNT )); then + sleep "$INTERVAL" + fi +done diff --git a/Explorer/Assets/DCL/Character/CharacterCamera/CinemachineExtensions.cs b/Explorer/Assets/DCL/Character/CharacterCamera/CinemachineExtensions.cs index 26aaa886bb3..8034a14805d 100644 --- a/Explorer/Assets/DCL/Character/CharacterCamera/CinemachineExtensions.cs +++ b/Explorer/Assets/DCL/Character/CharacterCamera/CinemachineExtensions.cs @@ -1,4 +1,5 @@ -using DCL.CharacterCamera.Components; +using Cinemachine; +using DCL.CharacterCamera.Components; using UnityEngine; namespace DCL.CharacterCamera @@ -16,9 +17,10 @@ public static void ForceFirstPersonCameraLookAt(this ICinemachinePreset cinemach { if (cinemachinePreset.FirstPersonCameraData.POV == null) return; - (float horizontalAxis, float verticalAxis) = GetHorizontalAndVerticalAxisForIntent(lookAtIntent); - cinemachinePreset.FirstPersonCameraData.POV.m_HorizontalAxis.Value = horizontalAxis; - cinemachinePreset.FirstPersonCameraData.POV.m_VerticalAxis.Value = verticalAxis; + // POV axes are in degrees, unlike the 0..1 orbit value the FreeLook rigs consume. + (float yawDegrees, float pitchDegrees) = GetYawPitchDegrees(lookAtIntent.LookAtTarget, lookAtIntent.PlayerPosition); + cinemachinePreset.FirstPersonCameraData.POV.m_HorizontalAxis.Value = yawDegrees; + cinemachinePreset.FirstPersonCameraData.POV.m_VerticalAxis.Value = pitchDegrees; } public static void ForceDroneCameraLookAt(this ICinemachinePreset cinemachinePreset, CameraLookAtIntent lookAtIntent) @@ -28,23 +30,58 @@ public static void ForceDroneCameraLookAt(this ICinemachinePreset cinemachinePre cinemachinePreset.DroneViewCameraData.Camera.m_YAxis.Value = verticalAxis; } - private static (float, float) GetHorizontalAndVerticalAxisForIntent(CameraLookAtIntent lookAtIntent) + /// + /// Places the free camera at an absolute world position (and optionally sets its field of view). + /// The free camera's position is its vcam transform, the same mechanism free-fly input moves it through. + /// + public static void ForceFreeCameraPose(this ICinemachinePreset cinemachinePreset, Vector3 position, float? fov = null) { - var eulerDir = Vector3.zero; - var cameraTarget = lookAtIntent.LookAtTarget; - float horizontalAxisLookAt = lookAtIntent.PlayerPosition.y - cameraTarget.y; - var verticalAxisLookAt = new Vector3(cameraTarget.x - lookAtIntent.PlayerPosition.x, 0, cameraTarget.z - lookAtIntent.PlayerPosition.z); + cinemachinePreset.FreeCameraData.Camera.transform.position = position; - if (verticalAxisLookAt is { x: 0, y: 0, z: 0 }) - verticalAxisLookAt = Vector3.forward; + if (fov.HasValue) + cinemachinePreset.FreeCameraData.Camera.m_Lens.FieldOfView = fov.Value; + } + + public static void ForceFreeCameraLookAt(this ICinemachinePreset cinemachinePreset, CameraLookAtIntent lookAtIntent) + { + CinemachinePOV? pov = cinemachinePreset.FreeCameraData.POV; - eulerDir.y = Vector3.SignedAngle(Vector3.forward, verticalAxisLookAt, Vector3.up); - eulerDir.x = Mathf.Atan2(horizontalAxisLookAt, verticalAxisLookAt.magnitude) * Mathf.Rad2Deg; + if (pov == null) + return; + + // The free camera is detached from the player, so the aim originates at the camera itself. + Vector3 origin = cinemachinePreset.FreeCameraData.Camera.transform.position; + (float yawDegrees, float pitchDegrees) = GetYawPitchDegrees(lookAtIntent.LookAtTarget, origin); + + pov.m_HorizontalAxis.Value = yawDegrees; + pov.m_VerticalAxis.Value = pitchDegrees; + } + + private static (float, float) GetHorizontalAndVerticalAxisForIntent(CameraLookAtIntent lookAtIntent) + { + (float yawDegrees, float pitchDegrees) = GetYawPitchDegrees(lookAtIntent.LookAtTarget, lookAtIntent.PlayerPosition); //value range 0 to 1, being 0 the bottom orbit and 1 the top orbit - float yValue = Mathf.InverseLerp(-90, 90, eulerDir.x); + float yValue = Mathf.InverseLerp(-90, 90, pitchDegrees); + + return (yawDegrees, yValue); + } + + /// + /// Yaw/pitch in degrees (Unity euler convention: positive pitch looks down) that point from origin at target. + /// + private static (float yawDegrees, float pitchDegrees) GetYawPitchDegrees(Vector3 target, Vector3 origin) + { + float heightDelta = origin.y - target.y; + var flatDirection = new Vector3(target.x - origin.x, 0, target.z - origin.z); + + if (flatDirection is { x: 0, y: 0, z: 0 }) + flatDirection = Vector3.forward; + + float yawDegrees = Vector3.SignedAngle(Vector3.forward, flatDirection, Vector3.up); + float pitchDegrees = Mathf.Atan2(heightDelta, flatDirection.magnitude) * Mathf.Rad2Deg; - return (eulerDir.y, yValue); + return (yawDegrees, pitchDegrees); } } } diff --git a/Explorer/Assets/DCL/Character/CharacterCamera/Systems/ApplyCinemachineCameraInputSystem.cs b/Explorer/Assets/DCL/Character/CharacterCamera/Systems/ApplyCinemachineCameraInputSystem.cs index a89d0102fb8..57bd1e99ac7 100644 --- a/Explorer/Assets/DCL/Character/CharacterCamera/Systems/ApplyCinemachineCameraInputSystem.cs +++ b/Explorer/Assets/DCL/Character/CharacterCamera/Systems/ApplyCinemachineCameraInputSystem.cs @@ -81,6 +81,8 @@ private void ForceLookAt(in Entity entity, in CameraComponent camera, ref ICinem cinemachinePreset.ForceFirstPersonCameraLookAt(lookAtIntent); break; case CameraMode.Free: + cinemachinePreset.ForceFreeCameraLookAt(lookAtIntent); + break; case CameraMode.ThirdPerson: cinemachinePreset.ForceThirdPersonCameraLookAt(lookAtIntent); break; diff --git a/Explorer/Assets/DCL/FeatureFlags/FeaturesRegistry.cs b/Explorer/Assets/DCL/FeatureFlags/FeaturesRegistry.cs index ec0c6946612..3088c98eccc 100644 --- a/Explorer/Assets/DCL/FeatureFlags/FeaturesRegistry.cs +++ b/Explorer/Assets/DCL/FeatureFlags/FeaturesRegistry.cs @@ -71,6 +71,7 @@ public FeaturesRegistry( [FeatureId.AB_DEPS_DIGEST_CACHE_KEY] = featureFlags.IsEnabled(FeatureFlagsStrings.AB_DEPS_DIGEST_CACHE_KEY), [FeatureId.BYTE_WEIGHTED_LOADING_PROGRESS] = appArgs.ResolveFeatureFlagArg(AppArgsFlags.BYTE_WEIGHTED_LOADING_PROGRESS, featureFlags.IsEnabled(FeatureFlagsStrings.BYTE_WEIGHTED_LOADING_PROGRESS) || isEditor), [FeatureId.HARDWARE_FINGERPRINT] = appArgs.ResolveFeatureFlagArg(AppArgsFlags.HARDWARE_FINGERPRINT, featureFlags.IsEnabled(FeatureFlagsStrings.HARDWARE_FINGERPRINT)), + [FeatureId.MCP_SERVER] = appArgs.HasFlag(AppArgsFlags.MCP) || appArgs.HasFlag(AppArgsFlags.MCP_PORT), // Note: COMMUNITIES feature is not cached here because it depends on user identity }); @@ -207,5 +208,6 @@ public enum FeatureId HARDWARE_FINGERPRINT = 66, USER_CREDITS = 67, CREDITS_WEARABLE_PURCHASE = 68, + MCP_SERVER = 69, } } diff --git a/Explorer/Assets/DCL/Infrastructure/Global/AppArgs/AppArgsFlags.cs b/Explorer/Assets/DCL/Infrastructure/Global/AppArgs/AppArgsFlags.cs index 9c3992f08c3..b24203b5eaf 100644 --- a/Explorer/Assets/DCL/Infrastructure/Global/AppArgs/AppArgsFlags.cs +++ b/Explorer/Assets/DCL/Infrastructure/Global/AppArgs/AppArgsFlags.cs @@ -104,6 +104,16 @@ public static class AppArgsFlags public const string MULTIPLE_RUNNING_INSTANCES = "multi-instance"; public const string ALTTESTER = "alttester"; + /// + /// Starts the embedded MCP (Model Context Protocol) server on 127.0.0.1 so coding agents can drive the client. + /// + public const string MCP = "mcp"; + + /// + /// Overrides the port the embedded MCP server listens on (implies ). + /// + public const string MCP_PORT = "mcp-port"; + public const string REPORT_USER = "report-user"; public const string AVATAR_CONTEXT_MENU = "avatar-context-menu"; diff --git a/Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs b/Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs index 91ff480e4d4..4127fd97124 100644 --- a/Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs +++ b/Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs @@ -1,4 +1,5 @@ using Arch.Core; +using CrdtEcsBridge.RestrictedActions; using Cysharp.Threading.Tasks; using DCL.ApplicationGuards; using DCL.AssetsProvision; @@ -24,6 +25,7 @@ using DCL.LOD.Systems; using DCL.MarketplaceCredits; using DCL.MarketplaceCredits.Purchase; +using DCL.McpServer.Systems; using DCL.Multiplayer.Connections.Messaging.Hubs; using DCL.Multiplayer.Connections.RoomHubs; using DCL.Multiplayer.Emotes; @@ -842,6 +844,24 @@ await MapRendererContainer globalPlugins.Add(lodContainer.RoadPlugin); } + if (FeaturesRegistry.Instance.IsEnabled(FeatureId.MCP_SERVER)) + globalPlugins.Add(new McpServerPlugin( + appArgs, + new GlobalWorldActions(globalWorld, playerEntity, localSceneDevelopment, bootstrapContainer.UseRemoteAssetBundles, FeaturesRegistry.Instance.IsEnabled(FeatureId.SELF_PREVIEW_BUILDER_COLLECTIONS)), + chatContainer.ChatMessagesBus, + staticContainer.ScenesCache, + commsContainer.CurrentSceneInfo, + staticContainer.LoadingStatus, + realmNavigatorContainer.WorldInfoHub, + realmContainer.ReloadSceneController, + bootstrapContainer.DiagnosticsContainer, + exposedGlobalDataContainer.ExposedCameraData, + staticContainer.EntityCollidersGlobalCache, + coroutineRunner, + globalWorld, + staticContainer.WebRequestsContainer.NetworkLogBuffer, + localSceneDevelopment)); + if (FeaturesRegistry.Instance.IsEnabled(FeatureId.LOCAL_SCENE_DEVELOPMENT) || FeaturesRegistry.Instance.IsEnabled(FeatureId.SELF_PREVIEW_BUILDER_COLLECTIONS)) globalPlugins.Add(new GlobalGLTFLoadingPlugin(staticContainer.WebRequestsContainer.WebRequestController, staticContainer.RealmData, wearableContainer.BuilderContentURL.Value, localSceneDevelopment, staticContainer.ComponentsContainer.ComponentPoolsRegistry.RootContainerTransform())); diff --git a/Explorer/Assets/DCL/McpServer.meta b/Explorer/Assets/DCL/McpServer.meta new file mode 100644 index 00000000000..f4a20233b14 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: 99595f3e5da16754fa6419d428a933e8 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Components.meta b/Explorer/Assets/DCL/McpServer/Components.meta new file mode 100644 index 00000000000..3168cc8ff37 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: edabe99ee7c5e413e92d095e4480a477 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs b/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs new file mode 100644 index 00000000000..5511aaa8930 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs @@ -0,0 +1,35 @@ +using Cysharp.Threading.Tasks; +using DCL.CharacterMotion.Components; +using UnityEngine; + +namespace DCL.McpServer.Components +{ + /// + /// Held movement input requested by the MCP walk tool. While present on the player entity, + /// re-asserts it into every frame. + /// + public struct McpMovementOverride + { + /// + /// Normalized camera-relative axes (x = strafe, y = forward). + /// + public Vector2 Axes; + + public MovementKind Kind; + + /// + /// Value of Time.time at which the hold expires. + /// + public float EndTime; + + /// + /// Requests a single jump; consumed on the first frame of the hold. + /// + public bool JumpRequested; + + /// + /// Completed by the system when the hold expires or is preempted by a newer request. + /// + public UniTaskCompletionSource? Completion; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs.meta b/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs.meta new file mode 100644 index 00000000000..0f86dbd6f92 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 64ba45e1952cb4631b46b2d0dd9688e2 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Components/McpPointerClickIntent.cs b/Explorer/Assets/DCL/McpServer/Components/McpPointerClickIntent.cs new file mode 100644 index 00000000000..8cb42e08283 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components/McpPointerClickIntent.cs @@ -0,0 +1,69 @@ +using Cysharp.Threading.Tasks; +using DCL.ECSComponents; +using UnityEngine; +using RaycastHit = UnityEngine.RaycastHit; + +namespace DCL.McpServer.Components +{ + /// + /// Present on the player entity while an agent-requested pointer click is in flight. + /// validates the aim with a physics raycast each frame, + /// delivers the press through the target's + /// and removes the component once the click completes or fails. + /// + public struct McpPointerClickIntent + { + public enum ClickKind : byte + { + /// Pointer down, then pointer up on the next scene tick. + CLICK, + DOWN, + UP, + } + + public enum ClickPhase : byte + { + DOWN, + WAIT_TICK, + UP, + } + + /// Arch entity id in the current scene world; -1 when aiming at an explicit world point. + public int TargetEntityId; + public Vector3 AimPoint; + public bool HasExplicitAimPoint; + public InputAction Button; + public ClickKind Kind; + public ClickPhase Phase; + + /// Time.time after which the click is abandoned. + public float Deadline; + public UniTaskCompletionSource? Completion; + + // In-flight state owned by McpPointerClickSystem. + public Arch.Core.World? SceneWorld; + public Arch.Core.Entity ResolvedEntity; + public uint DownTick; + public RaycastHit DownHit; + public Ray DownRay; + public McpPointerClickResult? DownResult; + } + + /// Outcome of a synthetic pointer click, serialized by the click_entity tool. + public class McpPointerClickResult + { + public bool Hit; + public string? FailureReason; + public int SceneEntityId; + public int CrdtEntityId; + public string? HoverText; + public Vector3 HitPoint; + public float Distance; + public int? BlockedByEntityId; + public int? BlockedByCrdtId; + public string? BlockedByColliderName; + + /// The release ray no longer hit the target (it moved after the press); PetUp was delivered with the press-frame hit. + public bool UpRayMissed; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Components/McpPointerClickIntent.cs.meta b/Explorer/Assets/DCL/McpServer/Components/McpPointerClickIntent.cs.meta new file mode 100644 index 00000000000..8e6d348bae3 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components/McpPointerClickIntent.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 203dec8323e545c8a510e5883a59a4f6 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core.meta b/Explorer/Assets/DCL/McpServer/Core.meta new file mode 100644 index 00000000000..df86f87de1c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: ea5698d1ae0de4f699345c74dc6f99b6 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Core/IMcpTool.cs b/Explorer/Assets/DCL/McpServer/Core/IMcpTool.cs new file mode 100644 index 00000000000..2c498b62dd1 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/IMcpTool.cs @@ -0,0 +1,41 @@ +using Cysharp.Threading.Tasks; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Core +{ + /// + /// A single MCP tool exposed to connected coding agents via tools/list and tools/call. + /// + public interface IMcpTool + { + string Name { get; } + + string Description { get; } + + /// + /// JSON Schema of the tool arguments. Must be a JSON Schema object (type=object); build it with + /// so a malformed schema is caught at registration, not on first use. + /// + JObject InputSchema { get; } + + /// + /// Behaviour hints (read-only, destructive, idempotent, open-world) surfaced in tools/list. + /// + McpToolAnnotations Annotations { get; } + + /// + /// JSON Schema of this tool's structuredContent, surfaced as outputSchema in tools/list. Build it with + /// . Null (the default) when the tool returns only unstructured text; tools + /// that emit override this to describe that payload. + /// + JObject? OutputSchema => null; + + /// + /// Invoked on the main thread, so implementations may touch ECS and Unity state directly. Offload + /// heavy CPU work to the thread pool yourself and hop back before touching that state again. Expected + /// failures are reported through , not exceptions. + /// + UniTask ExecuteAsync(JObject arguments, CancellationToken ct); + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/IMcpTool.cs.meta b/Explorer/Assets/DCL/McpServer/Core/IMcpTool.cs.meta new file mode 100644 index 00000000000..f41607472fd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/IMcpTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: e258f7df9608348d98dffe25a3ae3bb6 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs b/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs new file mode 100644 index 00000000000..9e89e6cab7f --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs @@ -0,0 +1,235 @@ +using Cysharp.Threading.Tasks; +using DCL.Diagnostics; +using System; +using System.IO; +using System.Net; +using System.Text; +using System.Threading; +using UnityEngine; +using Utility.Multithreading; + +namespace DCL.McpServer.Core +{ + /// + /// Minimal MCP Streamable HTTP transport on top of , bound to 127.0.0.1 only. + /// POST carries a single JSON-RPC message; GET (server-initiated stream) is not supported and returns 405. + /// + public class McpHttpServer : IDisposable + { + private const int MAX_BODY_BYTES = 1024 * 1024; + + // Single source of truth for the endpoint path. Docs and scripts restate it (they cannot reference this + // const): docs/mcp-automation.md, docs/app-arguments.md, .claude/skills/mcp-scene-iteration/ — keep in sync. + private const string ENDPOINT_PATH = "unity-explorer-mcp"; + + private readonly McpJsonRpcDispatcher dispatcher; + private readonly int port; + + // The session maps to the Explorer process, not to a client: this server is stateless and drives one + // shared world, so every request operates on the same state and there is nothing session-scoped to isolate. + // Multiple agents on one server intentionally share this id because they drive the same world; a separate + // session means a separate Explorer instance (separate port, separate id). Hence one id for the process + // lifetime, echoed on every response, and no need to validate the incoming Mcp-Session-Id. + private readonly string sessionId = Guid.NewGuid().ToString("N"); + + private HttpListener? listener; + + /// The localhost URL the server listens on, e.g. http://127.0.0.1:8123/unity-explorer-mcp. + public string EndpointUrl => $"http://127.0.0.1:{port}/{ENDPOINT_PATH}"; + + public McpHttpServer(McpToolsRegistry toolsRegistry, int port) + { + this.dispatcher = new McpJsonRpcDispatcher(toolsRegistry, Application.version); + this.port = port; + } + + public void Dispose() + { + if (listener == null) return; + + try + { + listener.Stop(); + listener.Close(); + } + catch (ObjectDisposedException) { } + + listener = null; + } + + public bool TryStart() + { + var newListener = new HttpListener(); + newListener.Prefixes.Add($"{EndpointUrl}/"); // HttpListener requires the trailing slash + + try { newListener.Start(); } + catch (Exception e) when (e is HttpListenerException or InvalidOperationException) + { + ReportHub.LogError(ReportCategory.MCP, $"Cannot start the MCP server on port {port} (already in use? pass a different --mcp-port): {e.Message}"); + newListener.Close(); + return false; + } + + listener = newListener; + ReportHub.Log(LogType.Log, ReportCategory.MCP, $"MCP server listening on {EndpointUrl}"); + return true; + } + + public async UniTaskVoid RunAsync(CancellationToken ct) + { + await DCLTask.SwitchToThreadPool(); + + // Capture once: a concurrent Dispose() on the main thread nulls the field, and re-reading it between + // the guard and GetContextAsync would throw an unfiltered NRE. Dispose still stops/closes this same + // instance, so a parked GetContextAsync surfaces as one of the caught exceptions below. + HttpListener? local = listener; + + while (!ct.IsCancellationRequested && local is { IsListening: true }) + { + HttpListenerContext context; + + try { context = await local.GetContextAsync(); } + catch (Exception e) when (e is HttpListenerException or ObjectDisposedException or InvalidOperationException) + { + // The listener was stopped or disposed; end the accept loop. + break; + } + + HandleRequestAsync(context, ct).Forget(); + } + } + + private async UniTaskVoid HandleRequestAsync(HttpListenerContext context, CancellationToken ct) + { + try + { + if (!IsAllowed(context.Request.Headers["Origin"])) + { + context.Response.WriteEmptyAndClose(HttpStatusCode.Forbidden, sessionId); + return; + } + + switch (context.Request.HttpMethod) + { + case "POST": + await HandlePostAsync(context, ct); + break; + case "DELETE": + // Session termination is accepted but stateless: nothing to clean up. + context.Response.WriteEmptyAndClose(HttpStatusCode.OK, sessionId); + break; + default: + context.Response.WriteEmptyAndClose(HttpStatusCode.MethodNotAllowed, sessionId); + break; + } + } + catch (OperationCanceledException) + { + TryAbort(context); + } + catch (Exception e) + { + ReportHub.LogException(e, ReportCategory.MCP); + TryWriteInternalError(context); + } + } + + private async UniTask HandlePostAsync(HttpListenerContext context, CancellationToken ct) + { + if (context.Request.ContentLength64 > MAX_BODY_BYTES) + { + context.Response.WriteEmptyAndClose(HttpStatusCode.RequestEntityTooLarge, sessionId); + return; + } + + string requestJson; + + // The Content-Length check above is only a fast reject; a chunked request reports ContentLength64 == -1 + // and bypasses it, so cap the read itself into a fixed buffer to keep the body bounded regardless. + using (var reader = new StreamReader(context.Request.InputStream, Encoding.UTF8)) + { + var buffer = new char[MAX_BODY_BYTES + 1]; + int charsRead = await reader.ReadBlockAsync(buffer, 0, buffer.Length); + + if (charsRead > MAX_BODY_BYTES) + { + context.Response.WriteEmptyAndClose(HttpStatusCode.RequestEntityTooLarge, sessionId); + return; + } + + requestJson = new string(buffer, 0, charsRead); + } + + string? responseJson = await dispatcher.DispatchAsync(requestJson, ct); + + if (responseJson == null) + { + // Notifications get 202 Accepted with no body. + context.Response.WriteEmptyAndClose(HttpStatusCode.Accepted, sessionId); + return; + } + + byte[] payload = Encoding.UTF8.GetBytes(responseJson); + + context.Response.WithMcpHeaders(sessionId); + context.Response.StatusCode = (int)HttpStatusCode.OK; + context.Response.ContentType = "application/json; charset=utf-8"; + context.Response.ContentLength64 = payload.Length; + await context.Response.OutputStream.WriteAsync(payload, 0, payload.Length, CancellationToken.None); + context.Response.Close(); + } + + private void TryWriteInternalError(HttpListenerContext context) + { + try + { + context.Response.WriteEmptyAndClose(HttpStatusCode.InternalServerError, sessionId); + } + catch (Exception) + { + // The response may already be closed or the client gone; nothing else to do. + } + } + + private static void TryAbort(HttpListenerContext context) + { + try { context.Response.Abort(); } + catch (Exception) + { + // Ignored: aborting a torn-down connection during shutdown. + } + } + + private static bool IsAllowed(string? origin) + { + if (string.IsNullOrEmpty(origin)) + return true; + + if (!Uri.TryCreate(origin, UriKind.Absolute, out Uri? originUri)) + return false; + + if (originUri.Scheme != Uri.UriSchemeHttp && originUri.Scheme != Uri.UriSchemeHttps) + return false; + + return originUri.Host is "localhost" or "127.0.0.1" or "::1"; + } + } + + internal static class HttpListenerResponseExtensions + { + public static void WriteEmptyAndClose(this HttpListenerResponse response, HttpStatusCode status, string sessionId) + { + response.StatusCode = (int)status; + response.ContentLength64 = 0; + response.WithMcpHeaders(sessionId) + .Close(); + } + + public static HttpListenerResponse WithMcpHeaders(this HttpListenerResponse response, string sessionId) + { + response.AddHeader("Mcp-Session-Id", sessionId); + response.AddHeader("MCP-Protocol-Version", McpJsonRpcDispatcher.PROTOCOL_VERSION); + return response; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs.meta new file mode 100644 index 00000000000..6609b181173 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 4767a6ddc27e9455fbb1a982904fa578 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs b/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs new file mode 100644 index 00000000000..d629ac04cd8 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs @@ -0,0 +1,182 @@ +using Cysharp.Threading.Tasks; +using DCL.Diagnostics; +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; + +namespace DCL.McpServer.Core +{ + /// + /// Routes JSON-RPC 2.0 messages of the MCP Streamable HTTP transport to the tool registry. + /// Only the tools capability is implemented; resources and prompts are not declared. + /// + public class McpJsonRpcDispatcher + { + /// Version of the MCP specification this server implements, declared in the initialize handshake. + public const string PROTOCOL_VERSION = "2025-06-18"; + + private const string SERVER_NAME = "dcl-unity-explorer"; + + private const int PARSE_ERROR = -32700; + private const int INVALID_REQUEST = -32600; + private const int METHOD_NOT_FOUND = -32601; + private const int INVALID_PARAMS = -32602; + + // A hung tool (e.g. an asset promise that never resolves) would otherwise hold its HttpListenerContext + // open until the server stops. Cap each call so one stuck tool can't tie up the agent's connection. + private static readonly TimeSpan TOOL_CALL_TIMEOUT = TimeSpan.FromSeconds(30); + + private readonly McpToolsRegistry tools; + private readonly string serverVersion; + + // Lets an agent orchestrating several Explorer instances confirm which process answers on this port. + private readonly int processId = System.Diagnostics.Process.GetCurrentProcess().Id; + + public McpJsonRpcDispatcher(McpToolsRegistry tools, string serverVersion) + { + this.tools = tools; + this.serverVersion = serverVersion; + } + + /// + /// Returns the serialized JSON-RPC response, or null when the message is a notification + /// (or a response relayed by the client) and no reply must be sent. + /// + public async UniTask DispatchAsync(string requestJson, CancellationToken ct) + { + if (ParseRoutableRequest(requestJson, out string? earlyResponse) is not { } routable) + return earlyResponse; + + var (id, method, callParams) = routable; + + return method switch + { + "initialize" => JsonRpcEnvelope.Result(id, InitializeResult(callParams)), + "ping" => JsonRpcEnvelope.Result(id, new JObject()), + "tools/list" => JsonRpcEnvelope.Result(id, tools.ToolsListPayload()), + "tools/call" => await CallToolAsync(id, + toolName: callParams?["name"]?.Value(), + arguments: callParams?["arguments"] as JObject ?? new JObject(), + ct), + _ => JsonRpcEnvelope.Error(id, METHOD_NOT_FOUND, $"Method not found: {method}") + }; + } + + /// + /// Parses the raw message and returns the id, method and params to route on. + /// Returns null when there is nothing to route: then + /// carries the reply to send back (a JSON-RPC error) or null for a notification that gets no response. + /// + private static (JToken id, string method, JObject? callParams)? ParseRoutableRequest(string requestJson, out string? earlyResponse) + { + earlyResponse = null; + + JToken parsed; + try { parsed = JToken.Parse(requestJson); } + catch (JsonException) + { + earlyResponse = JsonRpcEnvelope.Error(null, PARSE_ERROR, "Parse error"); + return null; + } + + // -32700 is reserved for unparseable JSON; well-formed JSON that is not a JSON-RPC object + // (a bare array, number or string) is a valid document but an invalid request: -32600. + if (parsed is not JObject request) + { + earlyResponse = JsonRpcEnvelope.Error(null, INVALID_REQUEST, "Invalid request: expected a JSON-RPC object"); + return null; + } + + JToken? id = request["id"]; + string? method = request["method"]?.Value(); + + if (string.IsNullOrEmpty(method)) + { + earlyResponse = id == null ? null : JsonRpcEnvelope.Error(id, INVALID_REQUEST, "Invalid request: missing method"); + return null; + } + + // Messages without an id are notifications ("notifications/initialized" et al.) and get no response. + if (id == null) + return null; + + return (id, method, request["params"] as JObject); + } + + private JObject InitializeResult(JObject? initializeParams) + { + // We implement a single MCP revision. Per the handshake rules we answer with our version regardless, + // but a client pinned to a different revision may abort on a strict mismatch — surface it so the + // otherwise-silent interop failure is diagnosable from the server logs. + string? requestedVersion = initializeParams?["protocolVersion"]?.Value(); + + if (!string.IsNullOrEmpty(requestedVersion) && requestedVersion != PROTOCOL_VERSION) + ReportHub.LogWarning(ReportCategory.MCP, $"MCP client requested protocol version '{requestedVersion}', server responding with '{PROTOCOL_VERSION}'"); + + return new JObject + { + ["protocolVersion"] = PROTOCOL_VERSION, + ["capabilities"] = new JObject { ["tools"] = new JObject() }, + ["serverInfo"] = new JObject + { + ["name"] = SERVER_NAME, + ["version"] = serverVersion, + ["pid"] = processId, + }, + }; + } + + private async UniTask CallToolAsync(JToken id, string? toolName, JObject arguments, CancellationToken ct) + { + if (!tools.TryGet(toolName, out IMcpTool? tool)) + return JsonRpcEnvelope.Error(id, INVALID_PARAMS, $"Unknown tool: {toolName ?? ""}"); + + using var timeout = CancellationTokenSource.CreateLinkedTokenSource(ct); + timeout.CancelAfter(TOOL_CALL_TIMEOUT); + + try + { + // Tools run on the main thread: the transport accepts and reads requests on a thread-pool thread, + // and the single hop here lets every tool touch ECS/Unity state without repeating the switch. + await UniTask.SwitchToMainThread(timeout.Token); + McpToolResult result = await tool.ExecuteAsync(arguments, timeout.Token); + return JsonRpcEnvelope.Result(id, result.Payload); + } + catch (OperationCanceledException) when (timeout.IsCancellationRequested && !ct.IsCancellationRequested) + { + ReportHub.LogWarning(ReportCategory.MCP, $"Tool '{toolName}' timed out after {TOOL_CALL_TIMEOUT.TotalSeconds:0}s"); + return JsonRpcEnvelope.Result(id, McpToolResult.Error($"Tool '{toolName}' timed out after {TOOL_CALL_TIMEOUT.TotalSeconds:0}s").Payload); + } + catch (OperationCanceledException) { throw; } + catch (Exception e) + { + ReportHub.LogException(e, ReportCategory.MCP); + return JsonRpcEnvelope.Result(id, McpToolResult.Error($"Tool '{toolName}' failed: {e.Message}").Payload); + } + } + + private static class JsonRpcEnvelope + { + public static string Result(JToken id, JToken result) => + new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = id, + ["result"] = result, + }.ToString(Formatting.None); + + public static string Error(JToken? id, int code, string message) => + new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = id ?? JValue.CreateNull(), + ["error"] = new JObject + { + ["code"] = code, + ["message"] = message, + }, + }.ToString(Formatting.None); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs.meta new file mode 100644 index 00000000000..112c4546ada --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: ade5e0f3aea214bd593ed7be6eaaaee9 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs b/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs new file mode 100644 index 00000000000..8d298786ebd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs @@ -0,0 +1,114 @@ +using Newtonsoft.Json.Linq; + +namespace DCL.McpServer.Core +{ + /// + /// Fluent builder for a tool's input or output JSON Schema. Declaring each field through a typed method keeps + /// the schema typo-safe — a mistyped type name can't slip through the way it could in a raw JSON string — and + /// produces the same { type: object, properties, required } shape a tools/list entry expects. Nested objects, + /// integer arrays and nullable fields extend the same style to the richer shapes an outputSchema needs. + /// + public sealed class McpJsonSchema + { + private readonly JObject properties = new (); + private readonly JArray required = new (); + + private McpJsonSchema() { } + + /// Starts a schema describing an object; chain the field methods and finish with . + public static McpJsonSchema Object() => + new (); + + public McpJsonSchema String(string name, string? description = null, string[]? enumValues = null, bool required = false, bool nullable = false) => + Property(name, "string", description, enumValues, required, nullable); + + public McpJsonSchema Number(string name, string? description = null, bool required = false, bool nullable = false) => + Property(name, "number", description, null, required, nullable); + + public McpJsonSchema Integer(string name, string? description = null, bool required = false, bool nullable = false) => + Property(name, "integer", description, null, required, nullable); + + public McpJsonSchema Boolean(string name, string? description = null, bool required = false, bool nullable = false) => + Property(name, "boolean", description, null, required, nullable); + + /// + /// Adds a nested object field described by its own builder. A + /// field admits null in place of the object (JSON Schema "type": ["object", "null"]). + /// + public McpJsonSchema Object(string name, McpJsonSchema schema, string? description = null, bool required = false, bool nullable = false) + { + JObject field = schema.Build(); + + if (nullable) + field["type"] = TypeToken("object", true); + + if (description != null) + field["description"] = description; + + return AddField(name, field, required); + } + + /// Adds an array field whose items are all integers. + public McpJsonSchema IntegerArray(string name, string? description = null, bool required = false) + { + var field = new JObject + { + ["type"] = "array", + ["items"] = new JObject { ["type"] = "integer" }, + }; + + if (description != null) + field["description"] = description; + + return AddField(name, field, required); + } + + /// Materializes the accumulated fields into the JSON Schema object. + public JObject Build() + { + var schema = new JObject + { + ["type"] = "object", + ["properties"] = properties, + }; + + if (required.Count > 0) + schema["required"] = required; + + return schema; + } + + private McpJsonSchema Property(string name, string type, string? description, string[]? enumValues, bool isRequired, bool nullable) + { + var field = new JObject { ["type"] = TypeToken(type, nullable) }; + + if (description != null) + field["description"] = description; + + if (enumValues != null) + { + var values = new JArray(); + + foreach (string value in enumValues) + values.Add(value); + + field["enum"] = values; + } + + return AddField(name, field, isRequired); + } + + private static JToken TypeToken(string type, bool nullable) => + nullable ? new JArray { type, "null" } : type; + + private McpJsonSchema AddField(string name, JObject field, bool isRequired) + { + properties[name] = field; + + if (isRequired) + required.Add(name); + + return this; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs.meta new file mode 100644 index 00000000000..abfcab1d96b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 05bd14397a93a8242be559c114683862 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs b/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs new file mode 100644 index 00000000000..e5dc37d9b17 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs @@ -0,0 +1,54 @@ +using Newtonsoft.Json.Linq; + +namespace DCL.McpServer.Core +{ + /// + /// Behaviour hints for a tool (MCP spec 2025-06-18) surfaced in tools/list so an agent can reason + /// about a tool before calling it. Only hints set through the factory methods are emitted; the rest + /// are omitted rather than defaulted, because the spec's implicit defaults are surprising + /// (destructiveHint and openWorldHint default to true, readOnlyHint to false). + /// + public readonly struct McpToolAnnotations + { + private readonly bool? readOnlyHint; + private readonly bool? destructiveHint; + private readonly bool? idempotentHint; + private readonly bool? openWorldHint; + + private McpToolAnnotations(bool? readOnlyHint, bool? destructiveHint, bool? idempotentHint, bool? openWorldHint) + { + this.readOnlyHint = readOnlyHint; + this.destructiveHint = destructiveHint; + this.idempotentHint = idempotentHint; + this.openWorldHint = openWorldHint; + } + + /// + /// A tool that only reads state. The spec ignores destructiveHint/idempotentHint when readOnlyHint + /// is true, so they are left unset. stays false for the local Explorer. + /// + public static McpToolAnnotations ReadOnly(bool openWorld = false) => + new (readOnlyHint: true, destructiveHint: null, idempotentHint: null, openWorldHint: openWorld); + + /// + /// A tool that changes state. flags irreversible or data-losing + /// effects; flags that repeating the call with the same arguments + /// has no additional effect. stays false for the local Explorer. + /// + public static McpToolAnnotations Mutating(bool destructive, bool idempotent, bool openWorld = false) => + new (readOnlyHint: false, destructiveHint: destructive, idempotentHint: idempotent, openWorldHint: openWorld); + + /// Serializes the set hints to the MCP annotations object embedded in a tools/list entry. + public JObject ToJObject() + { + var json = new JObject(); + + if (readOnlyHint.HasValue) json["readOnlyHint"] = readOnlyHint.Value; + if (destructiveHint.HasValue) json["destructiveHint"] = destructiveHint.Value; + if (idempotentHint.HasValue) json["idempotentHint"] = idempotentHint.Value; + if (openWorldHint.HasValue) json["openWorldHint"] = openWorldHint.Value; + + return json; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs.meta new file mode 100644 index 00000000000..41c1e9b50ed --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 6d6f99687071d4842921cf17924e34b2 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs b/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs new file mode 100644 index 00000000000..a2dd0615e94 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs @@ -0,0 +1,82 @@ +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using System; + +namespace DCL.McpServer.Core +{ + /// + /// The payload of a tools/call result: a content array of text/image items, + /// with expected failures flagged via isError instead of JSON-RPC errors. + /// + public readonly struct McpToolResult + { + public readonly JObject Payload; + + private McpToolResult(JObject payload) + { + Payload = payload; + } + + public static McpToolResult Text(string text) => + new (new JObject + { + ["content"] = new JArray { TextItem(text) }, + }); + + /// + /// A text result whose body is rendered as indented JSON. Centralizes the + /// Formatting.Indented serialization so tools never render JSON differently or forget the formatting. + /// + public static McpToolResult Json(JObject payload) => + Text(payload.ToString(Formatting.Indented)); + + /// + /// A result that surfaces both as structuredContent (validated against the + /// tool's outputSchema) and as its indented-JSON text duplicate — the spec requires the text mirror so + /// clients without structured support still read the result. This is the common path; it serializes the + /// mirror itself so the two copies cannot drift. + /// + public static McpToolResult JsonWithStructured(JObject structured) => + TextWithStructured(structured.ToString(Formatting.Indented), structured); + + /// + /// Like but with an explicit mirror, for the rare + /// case where the human-readable text is deliberately not the raw serialization of . + /// + public static McpToolResult TextWithStructured(string text, JObject structured) => + new (new JObject + { + ["content"] = new JArray { TextItem(text) }, + ["structuredContent"] = structured, + }); + + public static McpToolResult Error(string message) => + new (new JObject + { + ["content"] = new JArray { TextItem(message) }, + ["isError"] = true, + }); + + public static McpToolResult Image(byte[] imageBytes, string mimeType, string caption) => + new (new JObject + { + ["content"] = new JArray + { + new JObject + { + ["type"] = "image", + ["data"] = Convert.ToBase64String(imageBytes), + ["mimeType"] = mimeType, + }, + TextItem(caption), + }, + }); + + private static JObject TextItem(string text) => + new () + { + ["type"] = "text", + ["text"] = text, + }; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs.meta new file mode 100644 index 00000000000..075d762862a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 35469ba78d1a54f148c476efbc3b062f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs b/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs new file mode 100644 index 00000000000..a68567096a9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs @@ -0,0 +1,72 @@ +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using System; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; + +namespace DCL.McpServer.Core +{ + public class McpToolsRegistry + { + private readonly Dictionary tools = new (); + private string toolsListJson = null!; + + /// + /// The tools/list payload, serialized once at Build(). Each dispatch wraps the shared immutable + /// JSON string in a fresh JRaw, so there is no shared JToken tree to clone or re-parent when + /// concurrent responses serialize on the thread pool. + /// + public JRaw ToolsListPayload() => new (toolsListJson); + + public McpToolsRegistry Add(IMcpTool tool) + { + tools.Add(tool.Name, tool); + return this; + } + + public McpToolsRegistry Build() + { + var toolsArray = new JArray(); + + foreach (IMcpTool tool in tools.Values) + { + JObject inputSchema = tool.InputSchema; + + if (inputSchema == null || !IsObjectSchema(inputSchema)) + throw new InvalidOperationException($"MCP tool '{tool.Name}' produced an invalid input schema: expected a JSON Schema object (\"type\": \"object\"). Build it with McpJsonSchema."); + + if (tool.OutputSchema != null && !IsObjectSchema(tool.OutputSchema)) + throw new InvalidOperationException($"MCP tool '{tool.Name}' produced an invalid output schema: expected a JSON Schema object (\"type\": \"object\"). Build it with McpJsonSchema."); + + var entry = new JObject + { + ["name"] = tool.Name, + ["description"] = tool.Description, + ["inputSchema"] = inputSchema, + ["annotations"] = tool.Annotations.ToJObject(), + }; + + if (tool.OutputSchema != null) + entry["outputSchema"] = tool.OutputSchema; + + toolsArray.Add(entry); + } + + toolsListJson = new JObject { ["tools"] = toolsArray }.ToString(Formatting.None); + return this; + } + + private static bool IsObjectSchema(JObject schema) => + schema["type"]?.Value() == "object"; + + public bool TryGet(string? name, [NotNullWhen(true)] out IMcpTool? tool) + { + tool = null; + + if (string.IsNullOrEmpty(name)) + return false; + + return tools.TryGetValue(name, out tool); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs.meta new file mode 100644 index 00000000000..d9966b0e97c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 3bf2f57f26a5e44fd86d2b7f1201d27e \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef b/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef new file mode 100644 index 00000000000..b2a8207f0d6 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef @@ -0,0 +1,33 @@ +{ + "name": "DCL.McpServer", + "rootNamespace": "", + "references": [ + "GUID:d832748739a186646b8656bdbd447ad0", + "GUID:fa7b3fdbb04d67549916da7bd2af58ab", + "GUID:0b3eab7834a09c24ca4e84fe0d8a43ce", + "GUID:4794e238ed0f65142a4aea5848b513e5", + "GUID:1d2c76eb8b48e0b40940e8b31a679ce1", + "GUID:c80c82a8f4e04453b85fbab973d6774a", + "GUID:54d33bbd50a28174e8ba0110106203c2", + "GUID:f56000518aba31544aa96f4739e50a64", + "GUID:f3634757d00dab2429c6c11e69404e97", + "GUID:286980af24684da6acc1caa413039811", + "GUID:809870cbfd80a7b46bcf72b178ab210b", + "GUID:1b8e1e1bd01505f478f0369c04a4fb2f", + "GUID:571dc9f8bded0034f98595106462e3d0", + "GUID:0df5180c0c3a0594fbfa11f83736de9f", + "GUID:3c7b57a14671040bd8c549056adc04f5", + "GUID:f51ebe6a0ceec4240a699833d6309b23", + "GUID:e25ef972de004615a22937e739de2def", + "GUID:2bafac87e7f4b9b418d9448d219b01ab" + ], + "includePlatforms": [], + "excludePlatforms": [], + "allowUnsafeCode": false, + "overrideReferences": false, + "precompiledReferences": [], + "autoReferenced": true, + "defineConstraints": [], + "versionDefines": [], + "noEngineReferences": false +} \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef.meta b/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef.meta new file mode 100644 index 00000000000..092bf9535d9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef.meta @@ -0,0 +1,3 @@ +fileFormatVersion: 2 +guid: f254b2009e854709a0ef0022a7ca697c +timeCreated: 1784219963 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Systems.meta b/Explorer/Assets/DCL/McpServer/Systems.meta new file mode 100644 index 00000000000..b7964b54fef --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: c6ba4ac038d1a4f32b295c8fe24fd984 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref b/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref new file mode 100644 index 00000000000..b24e4ef9e3d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref @@ -0,0 +1,3 @@ +{ + "reference": "GUID:fc4fd35fb877e904d8cedee73b2256f6" +} \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref.meta b/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref.meta new file mode 100644 index 00000000000..7d2760a40bd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref.meta @@ -0,0 +1,7 @@ +fileFormatVersion: 2 +guid: 38580450dd18841eb82d84648bb3694e +AssemblyDefinitionReferenceImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs b/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs new file mode 100644 index 00000000000..c718244e95f --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs @@ -0,0 +1,84 @@ +using Arch.Core; +using Arch.SystemGroups; +using Cysharp.Threading.Tasks; +using DCL.Character.CharacterMotion.Components; +using DCL.CharacterMotion.Components; +using DCL.CharacterMotion.Systems; +using DCL.Diagnostics; +using DCL.Input; +using DCL.McpServer.Components; +using ECS.Abstract; +using UnityEngine; + +namespace DCL.McpServer.Systems +{ + /// + /// While an is present on the player entity, re-asserts its axes into + /// after the real-input systems have written it, so an agent-requested + /// walk survives the per-frame overwrite performed by . + /// + [UpdateInGroup(typeof(InputGroup))] + [UpdateAfter(typeof(UpdateInputMovementSystem))] + [UpdateAfter(typeof(UpdateInputJumpSystem))] + [LogCategory(ReportCategory.MCP)] + public partial class McpInputOverrideSystem : BaseUnityLoopSystem + { + private readonly Entity playerEntity; + + private SingleInstanceEntity physicsTick; + + internal McpInputOverrideSystem(World world, Entity playerEntity) : base(world) + { + this.playerEntity = playerEntity; + } + + public override void Initialize() + { + base.Initialize(); + physicsTick = World.CachePhysicsTick(); + } + + protected override void Update(float t) + { + ref McpMovementOverride movementOverride = ref World.TryGetRef(playerEntity, out bool overrideExists); + + if (!overrideExists) + return; + + ref MovementInputComponent movement = ref World.TryGetRef(playerEntity, out bool hasMovement); + + if (UnityEngine.Time.time < movementOverride.EndTime) + { + if (hasMovement) + { + movement.Axes = movementOverride.Axes; + movement.Kind = movementOverride.Kind; + } + + if (movementOverride.JumpRequested) + { + movementOverride.JumpRequested = false; + + ref JumpInputComponent jump = ref World.TryGetRef(playerEntity, out bool hasJump); + + if (hasJump) + jump.Trigger.TickWhenJumpOccurred = physicsTick.GetPhysicsTickComponent(World).Tick + 1; + } + } + else + { + UniTaskCompletionSource? completion = movementOverride.Completion; + + if (hasMovement) + { + movement.Axes = Vector2.zero; + movement.Kind = MovementKind.IDLE; + } + + // Structural change only after all outstanding component refs are done. + World.Remove(playerEntity); + completion?.TrySetResult(); + } + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs.meta b/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs.meta new file mode 100644 index 00000000000..6d1c1b9ca8c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 1d8aa7853be9e41d6920f15f5468914b \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpPointerClickSystem.cs b/Explorer/Assets/DCL/McpServer/Systems/McpPointerClickSystem.cs new file mode 100644 index 00000000000..a4555ae81d2 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpPointerClickSystem.cs @@ -0,0 +1,379 @@ +using Arch.Core; +using Arch.SystemGroups; +using Arch.SystemGroups.DefaultSystemGroups; +using CRDT; +using CrdtEcsBridge.Physics; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.Diagnostics; +using DCL.ECSComponents; +using DCL.Interaction.PlayerOriginated.Components; +using DCL.Interaction.PlayerOriginated.Utility; +using DCL.Interaction.Systems; +using DCL.Interaction.Utility; +using DCL.McpServer.Components; +using ECS.Abstract; +using ECS.SceneLifeCycle; +using ECS.Unity.PrimitiveColliders.Components; +using ECS.Unity.Transforms.Components; +using SceneRunner.Scene; +using UnityEngine; +using RaycastHit = UnityEngine.RaycastHit; + +namespace DCL.McpServer.Systems +{ + /// + /// + /// Delivers an agent-requested pointer press to a scene entity while a + /// is present on the player entity. The aim is validated with the same physics raycast the reticle + /// pipeline uses (camera origin, , occlusion and + /// max-distance rules apply), then the target's + /// is filled exactly as fills it for a real click, so the + /// unmodified scene-world write-back emits an identical PBPointerEventsResult. + /// + /// + /// Runs after so its per-frame intent Initialize cannot wipe + /// the synthetic press before the scene-world flush, which happens later in the same frame. + /// + /// + [UpdateInGroup(typeof(PresentationSystemGroup))] + [UpdateAfter(typeof(ProcessPointerEventsSystem))] + [LogCategory(ReportCategory.MCP)] + public partial class McpPointerClickSystem : BaseUnityLoopSystem + { + private const float MAX_RAYCAST_DISTANCE = 100f; + private const float AIM_DIRECTION_EPSILON_SQR = 0.0001f; + + private static readonly QueryDescription ALL_ENTITIES = new (); + + private readonly IScenesCache scenesCache; + private readonly IEntityCollidersGlobalCache collidersGlobalCache; + private readonly Entity playerEntity; + + private SingleInstanceEntity playerCamera; + + internal McpPointerClickSystem(World world, + IScenesCache scenesCache, + IEntityCollidersGlobalCache collidersGlobalCache, + Entity playerEntity) : base(world) + { + this.scenesCache = scenesCache; + this.collidersGlobalCache = collidersGlobalCache; + this.playerEntity = playerEntity; + } + + public override void Initialize() + { + base.Initialize(); + playerCamera = World.CacheCamera(); + } + + protected override void Update(float t) + { + ref McpPointerClickIntent intent = ref World.TryGetRef(playerEntity, out bool exists); + + if (!exists) + return; + + if (UnityEngine.Time.time > intent.Deadline) + { + CompleteAndRemove(intent.Completion, Failure(in intent, "click timed out before it could be delivered (is the simulation paused?)")); + return; + } + + ISceneFacade? scene = scenesCache.CurrentScene.Value; + + if (scene == null || !scene.SceneStateProvider.IsCurrent || scene.SceneStateProvider.IsNotRunningState()) + { + CompleteAndRemove(intent.Completion, Failure(in intent, "no running current scene to deliver the click to")); + return; + } + + World sceneWorld = scene.EcsExecutor.World; + + if (intent.SceneWorld != null && !ReferenceEquals(sceneWorld, intent.SceneWorld)) + { + CompleteAndRemove(intent.Completion, Failure(in intent, "the scene reloaded mid-click; only the press may have been delivered")); + return; + } + + switch (intent.Phase) + { + case McpPointerClickIntent.ClickPhase.DOWN: + PointerEventType pressType = intent.Kind == McpPointerClickIntent.ClickKind.UP + ? PointerEventType.PetUp + : PointerEventType.PetDown; + + if (!TryDeliver(ref intent, sceneWorld, pressType, out McpPointerClickResult result)) + { + CompleteAndRemove(intent.Completion, result); + return; + } + + if (intent.Kind != McpPointerClickIntent.ClickKind.CLICK) + { + CompleteAndRemove(intent.Completion, result); + return; + } + + intent.SceneWorld = sceneWorld; + intent.DownTick = scene.SceneStateProvider.TickNumber; + intent.DownResult = result; + intent.Phase = McpPointerClickIntent.ClickPhase.WAIT_TICK; + return; + + case McpPointerClickIntent.ClickPhase.WAIT_TICK: + // The scene must observe PetDown on an earlier tick than PetUp, otherwise ordering is ambiguous. + if (scene.SceneStateProvider.TickNumber > intent.DownTick) + intent.Phase = McpPointerClickIntent.ClickPhase.UP; + + return; + + case McpPointerClickIntent.ClickPhase.UP: + DeliverUp(ref intent, sceneWorld, out McpPointerClickResult upResult); + CompleteAndRemove(intent.Completion, upResult); + return; + } + } + + /// Structural removal happens only after every read of the intent ref is done. + private void CompleteAndRemove(UniTaskCompletionSource? completion, McpPointerClickResult result) + { + World.Remove(playerEntity); + completion?.TrySetResult(result); + } + + private static McpPointerClickResult Failure(in McpPointerClickIntent intent, string reason) => + new () + { + Hit = false, + FailureReason = reason, + SceneEntityId = intent.TargetEntityId, + }; + + private bool TryDeliver(ref McpPointerClickIntent intent, World sceneWorld, PointerEventType eventType, out McpPointerClickResult result) + { + if (!TryResolveTarget(ref intent, sceneWorld, out result)) + return false; + + Entity targetEntity = intent.ResolvedEntity; + bool requireTarget = intent.TargetEntityId >= 0; + + Vector3 aimPoint = intent.HasExplicitAimPoint + ? intent.AimPoint + : ResolveEntityAimPoint(sceneWorld, targetEntity); + + CameraComponent camera = playerCamera.GetCameraComponent(World); + Vector3 origin = camera.Camera.transform.position; + Vector3 direction = aimPoint - origin; + + if (direction.sqrMagnitude < AIM_DIRECTION_EPSILON_SQR) + { + result = Failure(in intent, "the camera is on top of the aim point; move back and retry"); + return false; + } + + var ray = new Ray(origin, direction.normalized); + + if (!Physics.Raycast(ray, out RaycastHit hit, MAX_RAYCAST_DISTANCE, PhysicsLayers.PLAYER_ORIGIN_RAYCAST_MASK)) + { + result = Failure(in intent, "the ray from the camera hit nothing (target may lack a collider)"); + return false; + } + + if (!collidersGlobalCache.TryGetSceneEntity(hit.collider, out GlobalColliderSceneEntityInfo hitInfo) + || !ReferenceEquals(hitInfo.EcsExecutor.World, sceneWorld)) + { + result = Failure(in intent, $"the ray hit a non-scene collider '{hit.collider.name}'"); + return false; + } + + Entity hitEntity = hitInfo.ColliderSceneEntityInfo.EntityReference; + + if (requireTarget && hitEntity != targetEntity) + { + result = Failure(in intent, "another collider blocks the line of sight to the target"); + result.BlockedByEntityId = hitEntity.Id; + result.BlockedByCrdtId = hitInfo.ColliderSceneEntityInfo.SDKEntity.Id; + result.BlockedByColliderName = hit.collider.name; + return false; + } + + // In pure aim-point mode the raycast decides the target. + targetEntity = hitEntity; + intent.ResolvedEntity = hitEntity; + + if (!hitInfo.TryGetPointerEvents(out PBPointerEvents? pbPointerEvents)) + { + result = Failure(in intent, $"entity {targetEntity.Id} has no PointerEvents component (not clickable)"); + result.SceneEntityId = targetEntity.Id; + return false; + } + + if (!sceneWorld.TryGet(targetEntity, out CRDTEntity crdtEntity)) + { + result = Failure(in intent, $"entity {targetEntity.Id} has no CRDTEntity; the scene cannot receive results for it"); + result.SceneEntityId = targetEntity.Id; + return false; + } + + if (!IsQualified(pbPointerEvents!, ray, hit, camera, out float distance, out string? hoverText, out bool hasCursorEntry)) + { + result = Failure(in intent, hasCursorEntry + ? $"target is out of range for its pointer events (hit distance {distance:F2}m)" + : "the target's pointer events are proximity-type only; a cursor click cannot trigger them"); + + result.SceneEntityId = targetEntity.Id; + result.CrdtEntityId = crdtEntity.Id; + result.Distance = distance; + return false; + } + + pbPointerEvents!.AppendPointerEventResultsIntent.Initialize(hit, ray); + pbPointerEvents.AppendPointerEventResultsIntent.AddInputAction(intent.Button, eventType); + + intent.DownHit = hit; + intent.DownRay = ray; + + result = new McpPointerClickResult + { + Hit = true, + SceneEntityId = targetEntity.Id, + CrdtEntityId = crdtEntity.Id, + HoverText = hoverText, + HitPoint = hit.point, + Distance = distance, + }; + + return true; + } + + /// + /// Delivers the release. If the target moved out from under the ray after the press (or its distance gate + /// no longer qualifies), the press-frame hit is reused so the entity still receives an ordered PetUp, + /// and the divergence is reported via . + /// + private void DeliverUp(ref McpPointerClickIntent intent, World sceneWorld, out McpPointerClickResult result) + { + McpPointerClickResult downResult = intent.DownResult!; + + if (TryDeliver(ref intent, sceneWorld, PointerEventType.PetUp, out McpPointerClickResult freshResult)) + { + result = freshResult; + return; + } + + // Fresh delivery failed: fall back to the press-frame hit if the component is still reachable. + if (sceneWorld.IsAlive(intent.ResolvedEntity) && sceneWorld.TryGet(intent.ResolvedEntity, out PBPointerEvents? pbPointerEvents) && pbPointerEvents != null) + { + pbPointerEvents.AppendPointerEventResultsIntent.Initialize(intent.DownHit, intent.DownRay); + pbPointerEvents.AppendPointerEventResultsIntent.AddInputAction(intent.Button, PointerEventType.PetUp); + + downResult.UpRayMissed = true; + result = downResult; + return; + } + + downResult.UpRayMissed = true; + downResult.FailureReason = $"the entity disappeared after the press ({freshResult.FailureReason}); only PetDown was delivered"; + result = downResult; + } + + private bool TryResolveTarget(ref McpPointerClickIntent intent, World sceneWorld, out McpPointerClickResult result) + { + result = null!; + + // Aim-point mode: the validation raycast picks the entity. + if (intent.HasExplicitAimPoint && intent.TargetEntityId < 0) + return true; + + if (intent.SceneWorld != null) + { + if (sceneWorld.IsAlive(intent.ResolvedEntity)) + return true; + + result = Failure(in intent, "the target entity was destroyed mid-click"); + return false; + } + + Entity found = Entity.Null; + int targetId = intent.TargetEntityId; + + sceneWorld.Query(in ALL_ENTITIES, entity => + { + if (entity.Id == targetId) + found = entity; + }); + + if (found == Entity.Null) + { + result = Failure(in intent, $"no entity with id {targetId} in the current scene world"); + return false; + } + + intent.ResolvedEntity = found; + return true; + } + + /// Aim at the collider volume when available; entity pivots can sit at hinges or bases and miss. + private static Vector3 ResolveEntityAimPoint(World sceneWorld, Entity entity) + { + if (sceneWorld.TryGet(entity, out PrimitiveColliderComponent primitiveCollider) && primitiveCollider.Collider != null) + return primitiveCollider.Collider.bounds.center; + + if (sceneWorld.TryGet(entity, out TransformComponent transformComponent) && transformComponent.Transform != null) + return transformComponent.Transform.position; + + return Vector3.zero; + } + + /// + /// Mirrors the cursor-entry qualification of : entries get their + /// defaults prepared and the distance gate is evaluated per entry, the last cursor entry winning, exactly + /// like the production loop. Also picks the hover text a real reticle hover would show for this button. + /// + private bool IsQualified(PBPointerEvents pbPointerEvents, in Ray ray, in RaycastHit hit, in CameraComponent camera, out float distance, out string? hoverText, out bool hasCursorEntry) + { + distance = camera.Mode == CameraMode.FirstPerson + ? hit.distance + : Vector3.Distance(hit.point, camera.PlayerFocus.position); + + float? playerDistance = null; + + if (World.TryGet(playerEntity, out CharacterTransform characterTransform)) + playerDistance = Vector3.Distance(hit.point, characterTransform.Position); + + var raycastResult = new PlayerOriginRaycastResultForSceneEntities(); + raycastResult.SetRay(ray); + raycastResult.SetupHit(hit, default(GlobalColliderSceneEntityInfo), distance, playerDistance); + + var isAtDistance = false; + hoverText = null; + hasCursorEntry = false; + + for (var i = 0; i < pbPointerEvents.PointerEvents!.Count; i++) + { + PBPointerEvents.Types.Entry entry = pbPointerEvents.PointerEvents[i]!; + + if (entry.InteractionType != InteractionType.Cursor) + continue; + + hasCursorEntry = true; + + PBPointerEvents.Types.Info info = entry.EventInfo!; + info.PrepareDefaultValues(); + + isAtDistance = InteractionInputUtils.IsQualifiedByDistance(in raycastResult, info); + + if (!isAtDistance) + continue; + + if (hoverText == null && info.HasHoverText && !string.IsNullOrEmpty(info.HoverText)) + hoverText = info.HoverText; + } + + return isAtDistance; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpPointerClickSystem.cs.meta b/Explorer/Assets/DCL/McpServer/Systems/McpPointerClickSystem.cs.meta new file mode 100644 index 00000000000..821b204664e --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpPointerClickSystem.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 45b28969f78d4a59b05fdd15adf4041b \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs b/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs new file mode 100644 index 00000000000..e5633f02dfe --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs @@ -0,0 +1,179 @@ +using Arch.SystemGroups; +using CrdtEcsBridge.RestrictedActions; +using Cysharp.Threading.Tasks; +using DCL.CharacterCamera; +using DCL.Chat.MessageBus; +using DCL.Diagnostics; +using DCL.Interaction.Utility; +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using DCL.McpServer.Utils; +using DCL.PluginSystem.Global; +using DCL.RealmNavigation; +using DCL.UI.DebugMenu.MessageBus; +using DCL.WebRequests.Analytics; +using ECS.SceneLifeCycle; +using ECS.SceneLifeCycle.CurrentScene; +using Global.AppArgs; +using SceneRunner.Debugging.Hub; +using System; +using System.Threading; +using UnityEngine; +using Utility; + +namespace DCL.McpServer.Systems +{ + /// + /// Hosts the embedded MCP server so external coding agents can observe and drive the client. + /// Registered only when the mcp/mcp-port app arg is present (command line or deep link); + /// the server binds to 127.0.0.1 exclusively and validates browser Origins. + /// + public class McpServerPlugin : IDCLGlobalPluginWithoutSettings + { + private const int DEFAULT_PORT = 8123; + + private const int MIN_PORT = 1024; + private const int MAX_PORT = 65535; + + private readonly int port; + + private readonly ICoroutineRunner coroutineRunner; + private readonly ILoadingStatus loadingStatus; + + private readonly IChatMessagesBus chatMessagesBus; + private readonly ExposedCameraData exposedCameraData; + + private readonly Arch.Core.World globalWorld; + private readonly IGlobalWorldActions globalWorldActions; + private readonly IEntityCollidersGlobalCache entityCollidersGlobalCache; + private readonly IWorldInfoHub worldInfoHub; + + private readonly IScenesCache scenesCache; + private readonly ICurrentSceneInfo currentSceneInfo; + private readonly ECSReloadScene reloadSceneController; + private readonly bool localSceneDevelopment; + + private readonly SceneLogBuffer logBuffer; + private readonly McpNetworkLogBuffer networkLogBuffer; + private readonly DebugMenuConsoleLogEntryBus logEntryBus; + + private McpHttpServer? server; + private CancellationTokenSource? serverCts; + + private ScreenshotTool? screenshotTool; + + public McpServerPlugin( + IAppArgs appArgs, + IGlobalWorldActions globalWorldActions, + IChatMessagesBus chatMessagesBus, + IScenesCache scenesCache, + ICurrentSceneInfo currentSceneInfo, + ILoadingStatus loadingStatus, + IWorldInfoHub worldInfoHub, + ECSReloadScene reloadSceneController, + DiagnosticsContainer diagnosticsContainer, + ExposedCameraData exposedCameraData, + IEntityCollidersGlobalCache entityCollidersGlobalCache, + ICoroutineRunner coroutineRunner, + Arch.Core.World globalWorld, + McpNetworkLogBuffer networkLogBuffer, + bool localSceneDevelopment) + { + port = appArgs.TryGetValue(AppArgsFlags.MCP_PORT, out string? portValue) + && int.TryParse(portValue, out int parsedPort) + && parsedPort is >= MIN_PORT and <= MAX_PORT + ? parsedPort + : DEFAULT_PORT; + + this.globalWorldActions = globalWorldActions; + this.chatMessagesBus = chatMessagesBus; + this.scenesCache = scenesCache; + this.currentSceneInfo = currentSceneInfo; + this.loadingStatus = loadingStatus; + this.worldInfoHub = worldInfoHub; + this.reloadSceneController = reloadSceneController; + this.exposedCameraData = exposedCameraData; + this.entityCollidersGlobalCache = entityCollidersGlobalCache; + this.coroutineRunner = coroutineRunner; + this.globalWorld = globalWorld; + this.networkLogBuffer = networkLogBuffer; + this.localSceneDevelopment = localSceneDevelopment; + + logBuffer = new SceneLogBuffer(); + logEntryBus = new DebugMenuConsoleLogEntryBus(); + logEntryBus.MessageAdded += logBuffer.Append; + diagnosticsContainer.AddDebugConsoleHandler(logEntryBus); + } + + public void Dispose() + { + logEntryBus.MessageAdded -= logBuffer.Append; + screenshotTool?.Dispose(); + server?.Dispose(); + serverCts.SafeCancelAndDispose(); + } + + public void InjectToWorld(ref ArchSystemsWorldBuilder builder, in GlobalPluginArguments arguments) + { + McpInputOverrideSystem.InjectToWorld(ref builder, arguments.PlayerEntity); + McpPointerClickSystem.InjectToWorld(ref builder, scenesCache, entityCollidersGlobalCache, arguments.PlayerEntity); + + screenshotTool = new ScreenshotTool(coroutineRunner, globalWorld, arguments.PlayerEntity); + + var toolsRegistry = new McpToolsRegistry() + .Add(screenshotTool) + .Add(new GetPlayerStateTool(globalWorld, arguments.PlayerEntity, exposedCameraData, currentSceneInfo)) + .Add(new GetSceneStateTool(scenesCache, currentSceneInfo, loadingStatus, localSceneDevelopment)) + .Add(new GetSceneLogsTool(logBuffer)) + .Add(new GetNetworkLogTool(networkLogBuffer)) + .Add(new TeleportTool(chatMessagesBus, scenesCache, loadingStatus)) + .Add(new MoveToTool(globalWorldActions, globalWorld, arguments.PlayerEntity)) + .Add(new LookAtTool(globalWorldActions, globalWorld, arguments.PlayerEntity, exposedCameraData)) + .Add(new SetCameraModeTool(globalWorld, exposedCameraData)) + .Add(new SetCameraPoseTool(globalWorld, arguments.PlayerEntity, exposedCameraData)) + .Add(new WalkTool(globalWorld, arguments.PlayerEntity)) + .Add(new SendChatTool(chatMessagesBus)) + .Add(new ReloadSceneTool(reloadSceneController, scenesCache, globalWorld, arguments.PlayerEntity, arguments.SkyboxEntity)) + .Add(new ListSceneEntitiesTool(worldInfoHub)) + .Add(new GetEntityDetailsTool(worldInfoHub)) + .Add(new TriggerEmoteTool(globalWorldActions)) + .Add(new ClickEntityTool(globalWorld, arguments.PlayerEntity)) + .Add(new ListUiElementsTool()) + .Add(new GetUiStateTool()) + .Add(new ClickUiTool()) + .Build(); + + server = new McpHttpServer(toolsRegistry, port); + serverCts = serverCts.SafeRestart(); + + bool started = server.TryStart(); + + if (started) + server.RunAsync(serverCts.Token).Forget(); + + AnnounceStatusWhenLoadedAsync(started, server.EndpointUrl, serverCts.Token).Forget(); + } + + /// + /// Reports the server address (or the startup failure) once loading completes, so the message + /// reaches the scene debug console: its UI subscribes to log entries only after this plugin runs, + /// and a line logged at server start would be dropped. + /// + private async UniTaskVoid AnnounceStatusWhenLoadedAsync(bool started, string endpointUrl, CancellationToken ct) + { + try + { + await UniTask.WaitUntil(() => loadingStatus.CurrentStage.Value == LoadingStatus.LoadingStage.Completed, cancellationToken: ct); + + if (started) + ReportHub.Log(LogType.Log, ReportCategory.MCP, $"MCP server listening on {endpointUrl}"); + else + ReportHub.LogError(ReportCategory.MCP, $"MCP server failed to start on port {port} — agent connections unavailable (pass a different --mcp-port)"); + } + catch (OperationCanceledException) + { + ReportHub.Log(LogType.Log, ReportCategory.MCP, "MCP server status announcement cancelled before loading completed"); + } + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs.meta b/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs.meta new file mode 100644 index 00000000000..9336efcf1cd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 5d7cb65f2d6064e8f98e97c77b2ccb7e \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests.meta b/Explorer/Assets/DCL/McpServer/Tests.meta new file mode 100644 index 00000000000..309d70874e3 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: 85f2f58b3ff94ddfa14bb5bda0658b63 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref b/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref new file mode 100644 index 00000000000..9c56917b757 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref @@ -0,0 +1,3 @@ +{ + "reference": "GUID:da80994a355e49d5b84f91c0a84a721f" +} \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref.meta b/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref.meta new file mode 100644 index 00000000000..b2e1703bffa --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref.meta @@ -0,0 +1,7 @@ +fileFormatVersion: 2 +guid: b6d1d98221344ffa8441fb344268804f +AssemblyDefinitionReferenceImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs b/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs new file mode 100644 index 00000000000..ba7e07a4046 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs @@ -0,0 +1,68 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; + +namespace DCL.McpServer.Tests +{ + /// + /// A configurable test double: its ExecuteAsync returns a preset result, + /// or throws a preset exception, and records the arguments and cancellation token it was called with. + /// Keeps the routing tests independent of the real tool implementations. + /// + internal sealed class FakeMcpTool : IMcpTool + { + private readonly Func execute; + + public string Name { get; } + + public string Description { get; } + + public JObject InputSchema { get; } + + public McpToolAnnotations Annotations { get; } + + public int CallCount { get; private set; } + + public JObject? LastArguments { get; private set; } + + public CancellationToken LastCancellationToken { get; private set; } + + private FakeMcpTool(string name, string description, JObject inputSchema, McpToolAnnotations annotations, + Func execute) + { + Name = name; + Description = description; + InputSchema = inputSchema; + Annotations = annotations; + this.execute = execute; + } + + /// A tool whose ExecuteAsync returns (defaults to a text result). + public static FakeMcpTool Returning(string name, McpToolResult? result = null, JObject? inputSchema = null, + McpToolAnnotations? annotations = null) + { + McpToolResult toReturn = result ?? McpToolResult.Text($"{name} ran"); + return new FakeMcpTool(name, $"{name} description", inputSchema ?? DefaultSchema(), + annotations ?? McpToolAnnotations.ReadOnly(), (_, _) => toReturn); + } + + /// A tool whose ExecuteAsync throws . + public static FakeMcpTool Throwing(string name, Exception exception, JObject? inputSchema = null, + McpToolAnnotations? annotations = null) => + new (name, $"{name} description", inputSchema ?? DefaultSchema(), + annotations ?? McpToolAnnotations.ReadOnly(), (_, _) => throw exception); + + private static JObject DefaultSchema() => + McpJsonSchema.Object().String("value", "Any value.").Build(); + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + CallCount++; + LastArguments = arguments; + LastCancellationToken = ct; + return UniTask.FromResult(execute(arguments, ct)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs.meta new file mode 100644 index 00000000000..1d1f4a9775a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: e7a2d21be39bd5441ba0b52f68f67b34 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs b/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs new file mode 100644 index 00000000000..6ed339b06b7 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs @@ -0,0 +1,88 @@ +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using Newtonsoft.Json.Linq; +using NSubstitute; +using NUnit.Framework; +using SceneRunner.Debugging; +using SceneRunner.Debugging.Hub; +using System.Threading; + +namespace DCL.McpServer.Tests +{ + public class GetEntityDetailsToolShould + { + private const string CURRENT_SCENE = "CURRENT"; + + private IWorldInfoHub worldInfoHub = null!; + private IWorldInfo worldInfo = null!; + private GetEntityDetailsTool tool = null!; + + [SetUp] + public void Setup() + { + worldInfo = Substitute.For(); + worldInfoHub = Substitute.For(); + worldInfoHub.WorldInfo(CURRENT_SCENE).Returns(worldInfo); + tool = new GetEntityDetailsTool(worldInfoHub); + } + + [Test] + public void ReturnTheDumpWholeWhenItFitsTheBudget() + { + // Arrange + const string DUMP = "Components of entity 5, total count: 1\n1) PBTransform"; + worldInfo.EntityComponentsInfo(5).Returns(DUMP); + + // Act + string text = TextOf(Execute(5)); + + // Assert + Assert.That(text, Is.EqualTo(DUMP)); + Assert.That(text, Does.Not.Contain("truncated")); + } + + [Test] + public void TruncateWithANoteWhenTheDumpExceedsTheBudget() + { + // Arrange + string dump = new ('x', 20000); + worldInfo.EntityComponentsInfo(5).Returns(dump); + + // Act + string text = TextOf(Execute(5)); + + // Assert + Assert.That(text.Length, Is.LessThan(dump.Length)); + Assert.That(text, Does.Contain($"output truncated at 8000/{dump.Length} chars")); + } + + [Test] + public void ErrorWhenNoSceneWorldIsFound() + { + // Arrange + worldInfoHub.WorldInfo(CURRENT_SCENE).Returns((IWorldInfo?)null); + + // Act + McpToolResult result = Execute(5); + + // Assert + Assert.That(result.Payload["isError"]!.Value(), Is.True); + } + + [Test] + public void ErrorWhenEntityIdIsMissing() + { + // Act + McpToolResult result = tool.ExecuteAsync(new JObject(), CancellationToken.None).GetAwaiter().GetResult(); + + // Assert + Assert.That(result.Payload["isError"]!.Value(), Is.True); + } + + private McpToolResult Execute(int entityId) => + tool.ExecuteAsync(new JObject { ["entityId"] = entityId }, CancellationToken.None).GetAwaiter().GetResult(); + + private static string TextOf(McpToolResult result) => + result.Payload["content"]![0]!["text"]!.Value()!; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs.meta new file mode 100644 index 00000000000..5f8d40e23e5 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: c5c1a3c1b1573d04dae5c672589e81de \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs b/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs new file mode 100644 index 00000000000..55b33d059e2 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs @@ -0,0 +1,76 @@ +using Arch.Core; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.CharacterCamera.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using ECS.SceneLifeCycle.CurrentScene; +using Newtonsoft.Json.Linq; +using NSubstitute; +using NUnit.Framework; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tests +{ + public class GetPlayerStateToolShould + { + private World world = null!; + private GameObject playerGameObject = null!; + private GetPlayerStateTool tool = null!; + + [SetUp] + public void Setup() + { + world = World.Create(); + playerGameObject = new GameObject(nameof(GetPlayerStateToolShould)); + + Entity playerEntity = world.Create(new CharacterTransform(playerGameObject.transform)); + world.Create(new CameraComponent()); // the tool reads the camera mode through CacheCamera() + + tool = new GetPlayerStateTool(world, playerEntity, new ExposedCameraData(), Substitute.For()); + } + + [TearDown] + public void TearDown() + { + world.Dispose(); + Object.DestroyImmediate(playerGameObject); + } + + [Test] + public void DeclareAnObjectOutputSchema() + { + Assert.That(tool.OutputSchema!["type"]!.Value(), Is.EqualTo("object")); + } + + [Test] + public void ModelTheAddressAsANullableString() + { + var addressType = (JArray)tool.OutputSchema!["properties"]!["address"]!["type"]!; + Assert.That(addressType.ToObject(), Is.EqualTo(new[] { "string", "null" })); + } + + [Test] + public void ModelTheCameraAsANestedObject() + { + JToken camera = tool.OutputSchema!["properties"]!["camera"]!; + + Assert.That(camera["type"]!.Value(), Is.EqualTo("object")); + Assert.That(camera["properties"]!["mode"]!["type"]!.Value(), Is.EqualTo("string")); + } + + [Test] + public void KeepTheOutputSchemaInSyncWithTheStructuredPayload() + { + // Act + var structured = (JObject)Execute().Payload["structuredContent"]!; + + // Assert + McpSchemaAssert.KeysMatch(tool.OutputSchema!, structured); + } + + private McpToolResult Execute() => + tool.ExecuteAsync(new JObject(), CancellationToken.None).GetAwaiter().GetResult(); + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs.meta new file mode 100644 index 00000000000..c474cb9df5b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 486751286fc8bcc47af7880e22187283 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs b/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs new file mode 100644 index 00000000000..7d81ed80854 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs @@ -0,0 +1,105 @@ +using DCL.Diagnostics; +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using DCL.RealmNavigation; +using DCL.Utilities; +using ECS.SceneLifeCycle; +using ECS.SceneLifeCycle.CurrentScene; +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using NSubstitute; +using NUnit.Framework; +using SceneRunner.Scene; +using System.Threading; +using UnityEngine; +using Utility.Multithreading; + +namespace DCL.McpServer.Tests +{ + public class GetSceneStateToolShould + { + private IScenesCache scenesCache = null!; + private ICurrentSceneInfo currentSceneInfo = null!; + private ILoadingStatus loadingStatus = null!; + private GetSceneStateTool tool = null!; + + [SetUp] + public void Setup() + { + scenesCache = Substitute.For(); + currentSceneInfo = Substitute.For(); + loadingStatus = Substitute.For(); + + scenesCache.CurrentParcel.Returns(new ReactiveProperty(new Vector2Int(1, 2))); + scenesCache.CurrentScene.Returns(new ReactiveProperty(null)); + loadingStatus.CurrentStage.Returns(new ReactiveProperty(default)); + + tool = new GetSceneStateTool(scenesCache, currentSceneInfo, loadingStatus, localSceneDevelopment: false); + } + + [Test] + public void MirrorTheStateInBothTextAndStructuredContent() + { + // Act + McpToolResult result = Execute(); + + // Assert + var structured = (JObject)result.Payload["structuredContent"]!; + Assert.That(structured, Is.Not.Null); + Assert.That(result.Payload["content"]![0]!["text"]!.Value(), Is.EqualTo(structured.ToString(Formatting.Indented))); + } + + [Test] + public void ReportAnAbsentSceneAsAJsonNull() + { + // Act + McpToolResult result = Execute(); + + // Assert + var structured = (JObject)result.Payload["structuredContent"]!; + Assert.That(structured["scene"]!.Type, Is.EqualTo(JTokenType.Null)); + } + + [Test] + public void DeclareAnObjectOutputSchemaThatAdmitsANullScene() + { + // Act + JObject schema = tool.OutputSchema!; + + // Assert + Assert.That(schema["type"]!.Value(), Is.EqualTo("object")); + + var sceneType = (JArray)schema["properties"]!["scene"]!["type"]!; + Assert.That(sceneType.ToObject(), Is.EqualTo(new[] { "object", "null" })); + } + + [Test] + public void KeepTheOutputSchemaInSyncWithTheStructuredPayload() + { + // Arrange — a populated scene so the nested "scene" object is covered, not just the top level. + ISceneStateProvider sceneStateProvider = Substitute.For(); + sceneStateProvider.State.Returns(new Atomic(SceneState.Running)); + + ISceneData sceneData = Substitute.For(); + sceneData.SceneLoadingConcluded.Returns(true); + + ISceneFacade scene = Substitute.For(); + scene.Info.Returns(new SceneShortInfo(new Vector2Int(1, 2), "Test scene", "7")); + scene.SceneStateProvider.Returns(sceneStateProvider); + scene.SceneData.Returns(sceneData); + scene.IsSceneReady().Returns(true); + + scenesCache.CurrentScene.Returns(new ReactiveProperty(scene)); + currentSceneInfo.SceneStatus.Returns(new ReactiveProperty(null)); + + // Act + var structured = (JObject)Execute().Payload["structuredContent"]!; + + // Assert + McpSchemaAssert.KeysMatch(tool.OutputSchema!, structured); + } + + private McpToolResult Execute() => + tool.ExecuteAsync(new JObject(), CancellationToken.None).GetAwaiter().GetResult(); + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs.meta new file mode 100644 index 00000000000..c1d5e41b5ff --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 523603bbeedddae44833a5a53f9793ae \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs b/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs new file mode 100644 index 00000000000..a9550ea8d56 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs @@ -0,0 +1,107 @@ +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using Newtonsoft.Json.Linq; +using NSubstitute; +using NUnit.Framework; +using SceneRunner.Debugging; +using SceneRunner.Debugging.Hub; +using System.Collections.Generic; +using System.Threading; + +namespace DCL.McpServer.Tests +{ + public class ListSceneEntitiesToolShould + { + private const string CURRENT_SCENE = "CURRENT"; + + private IWorldInfoHub worldInfoHub = null!; + private IWorldInfo worldInfo = null!; + private ListSceneEntitiesTool tool = null!; + + [SetUp] + public void Setup() + { + worldInfo = Substitute.For(); + worldInfoHub = Substitute.For(); + worldInfoHub.WorldInfo(CURRENT_SCENE).Returns(worldInfo); + tool = new ListSceneEntitiesTool(worldInfoHub); + } + + [Test] + public void AddAnActionableLineWhenNotEverythingIsShown() + { + // Arrange + worldInfo.EntityIds().Returns(Ids(10)); + + // Act + string text = TextOf(Execute(limit: 3)); + + // Assert + Assert.That(text, Does.Contain("total=10 returned=3")); + Assert.That(text, Does.Contain("3 of 10 shown")); + } + + [Test] + public void OmitTheActionableLineWhenEverythingFits() + { + // Arrange + worldInfo.EntityIds().Returns(Ids(3)); + + // Act + string text = TextOf(Execute(limit: 200)); + + // Assert + Assert.That(text, Does.Contain("total=3 returned=3")); + Assert.That(text, Does.Not.Contain("shown")); + } + + [Test] + public void MirrorTheListingInStructuredContentWhileKeepingTheText() + { + // Arrange + worldInfo.EntityIds().Returns(Ids(10)); + + // Act + McpToolResult result = Execute(limit: 3); + + // Assert — text output is untouched + Assert.That(TextOf(result), Does.Contain("total=10 returned=3")); + + // Assert — structured mirror carries the same figures + var structured = (JObject)result.Payload["structuredContent"]!; + Assert.That(structured["total"]!.Value(), Is.EqualTo(10)); + Assert.That(structured["returned"]!.Value(), Is.EqualTo(3)); + Assert.That(structured["truncated"]!.Value(), Is.True); + Assert.That(((JArray)structured["entityIds"]!).ToObject(), Is.EqualTo(new[] { 0, 1, 2 })); + } + + [Test] + public void KeepTheOutputSchemaInSyncWithTheStructuredPayload() + { + // Arrange + worldInfo.EntityIds().Returns(Ids(3)); + + // Act + var structured = (JObject)Execute(limit: 200).Payload["structuredContent"]!; + + // Assert + McpSchemaAssert.KeysMatch(tool.OutputSchema!, structured); + } + + private McpToolResult Execute(int limit) => + tool.ExecuteAsync(new JObject { ["limit"] = limit }, CancellationToken.None).GetAwaiter().GetResult(); + + private static IReadOnlyList Ids(int count) + { + var ids = new List(count); + + for (var i = 0; i < count; i++) + ids.Add(i); + + return ids; + } + + private static string TextOf(McpToolResult result) => + result.Payload["content"]![0]!["text"]!.Value()!; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs.meta new file mode 100644 index 00000000000..11c4db393c3 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: fe60d9fd78ad61247aa09f22ac6ecab7 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs new file mode 100644 index 00000000000..d5e95c7581e --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs @@ -0,0 +1,50 @@ +using DCL.McpServer.Core; +using NUnit.Framework; +using System.Reflection; + +namespace DCL.McpServer.Tests +{ + /// + /// Covers McpHttpServer.IsAllowed, the DNS-rebinding guard that decides which Origin headers may + /// reach the JSON-RPC dispatcher. The method is private static and lives in Core, which these tests do + /// not modify, so it is exercised through reflection. If IsAllowed is later exposed as internal + /// (with InternalsVisibleTo), replace the reflection shim with a direct call. + /// + public class McpHttpServerShould + { + private static readonly MethodInfo IS_ALLOWED = + typeof(McpHttpServer).GetMethod("IsAllowed", BindingFlags.NonPublic | BindingFlags.Static)!; + + private static bool IsAllowed(string? origin) => + (bool)IS_ALLOWED.Invoke(null, new object?[] { origin })!; + + [TestCase(null)] + [TestCase("")] + public void AllowRequestsWithoutAnOrigin(string? origin) => + Assert.That(IsAllowed(origin), Is.True); + + [TestCase("http://localhost")] + [TestCase("http://localhost:8080")] + [TestCase("http://127.0.0.1")] + [TestCase("http://127.0.0.1:9001")] + [TestCase("https://127.0.0.1:9001")] + public void AllowLoopbackHttpOrigins(string origin) => + Assert.That(IsAllowed(origin), Is.True); + + [TestCase("http://evil.com")] + [TestCase("https://attacker.example:9001")] + [TestCase("http://127.0.0.1.evil.com")] // rebinding: a foreign host that merely starts with the loopback IP + public void RejectNonLoopbackHosts(string origin) => + Assert.That(IsAllowed(origin), Is.False); + + [TestCase("ftp://127.0.0.1")] + [TestCase("file:///etc/passwd")] + public void RejectNonHttpSchemes(string origin) => + Assert.That(IsAllowed(origin), Is.False); + + [TestCase("not-a-valid-origin")] + [TestCase("://missing-scheme")] + public void RejectMalformedOrigins(string origin) => + Assert.That(IsAllowed(origin), Is.False); + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs.meta new file mode 100644 index 00000000000..800c6157bd0 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 5b82a6ebaa2cb0148808b9bc41a981d7 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs new file mode 100644 index 00000000000..54c074968ce --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs @@ -0,0 +1,256 @@ +using DCL.McpServer.Core; +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System; +using System.Text.RegularExpressions; +using System.Threading; +using UnityEngine; +using UnityEngine.TestTools; + +namespace DCL.McpServer.Tests +{ + public class McpJsonRpcDispatcherShould + { + private const string SERVER_VERSION = "9.9.9-test"; + + [Test] + public void AnswerInitializeWithProtocolCapabilitiesAndServerInfo() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + JObject result = ResultOf(Dispatch(dispatcher, Request(1, "initialize"))); + + Assert.That(result["protocolVersion"]!.Value(), Is.EqualTo("2025-06-18")); + Assert.That(result["protocolVersion"]!.Value(), Is.EqualTo(McpJsonRpcDispatcher.PROTOCOL_VERSION)); + + var capabilities = (JObject)result["capabilities"]!; + Assert.That(capabilities.ContainsKey("tools"), Is.True); + + var serverInfo = (JObject)result["serverInfo"]!; + Assert.That(serverInfo["version"]!.Value(), Is.EqualTo(SERVER_VERSION)); + Assert.That(serverInfo.ContainsKey("pid"), Is.True); + Assert.That(serverInfo["pid"]!.Type, Is.EqualTo(JTokenType.Integer)); + } + + [Test] + public void AnswerPingWithAnEmptyResult() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + JObject result = ResultOf(Dispatch(dispatcher, Request(2, "ping"))); + + Assert.That(result.Count, Is.EqualTo(0)); + } + + [Test] + public void ListToolsWithInputSchemaAndAnnotations() + { + FakeMcpTool tool = FakeMcpTool.Returning("reader", annotations: McpToolAnnotations.ReadOnly()); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + JObject result = ResultOf(Dispatch(dispatcher, Request(3, "tools/list"))); + + var tools = (JArray)result["tools"]!; + Assert.That(tools.Count, Is.EqualTo(1)); + + JObject entry = ToolEntry(result, "reader"); + Assert.That(entry["description"]!.Value(), Is.EqualTo("reader description")); + + var inputSchema = (JObject)entry["inputSchema"]!; + Assert.That(inputSchema["type"]!.Value(), Is.EqualTo("object")); + Assert.That(inputSchema.ContainsKey("properties"), Is.True); + + var annotations = (JObject)entry["annotations"]!; + Assert.That(annotations["readOnlyHint"]!.Value(), Is.True); + } + + [Test] + public void CallAKnownToolAndWrapItsResult() + { + FakeMcpTool tool = FakeMcpTool.Returning("echo", McpToolResult.Text("done")); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + var arguments = new JObject { ["value"] = "hi" }; + JObject result = ResultOf(Dispatch(dispatcher, CallRequest(4, "echo", arguments))); + + Assert.That(tool.CallCount, Is.EqualTo(1)); + Assert.That(tool.LastArguments!["value"]!.Value(), Is.EqualTo("hi")); + + var content = (JArray)result["content"]!; + Assert.That(content[0]!["type"]!.Value(), Is.EqualTo("text")); + Assert.That(content[0]!["text"]!.Value(), Is.EqualTo("done")); + Assert.That(result.ContainsKey("isError"), Is.False); + } + + [Test] + public void CallWithEmptyArgumentsWhenNoneAreProvided() + { + FakeMcpTool tool = FakeMcpTool.Returning("echo"); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + // A tools/call request carrying a name but no arguments object. + string request = new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = 5, + ["method"] = "tools/call", + ["params"] = new JObject { ["name"] = "echo" }, + }.ToString(Formatting.None); + + ResultOf(Dispatch(dispatcher, request)); + + Assert.That(tool.CallCount, Is.EqualTo(1)); + Assert.That(tool.LastArguments, Is.Not.Null); + Assert.That(tool.LastArguments!.Count, Is.EqualTo(0)); + } + + [Test] + public void RejectAnUnknownToolWithInvalidParams() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(FakeMcpTool.Returning("known")); + + JObject error = ErrorOf(Dispatch(dispatcher, CallRequest(6, "missing", new JObject()))); + + Assert.That(error["code"]!.Value(), Is.EqualTo(-32602)); + Assert.That(error["message"]!.Value(), Does.Contain("missing")); + } + + [Test] + public void ReportAToolFailureAsAnIsErrorResultNotAJsonRpcError() + { + FakeMcpTool tool = FakeMcpTool.Throwing("boom", new InvalidOperationException("kaboom")); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + // ReportHub routes the caught exception to the Unity logger; expect it so the test does not fail. + LogAssert.Expect(LogType.Exception, new Regex("kaboom")); + + JObject response = JObject.Parse(Dispatch(dispatcher, CallRequest(7, "boom", new JObject()))!); + + // The failure is delivered inside result (isError), not as a top-level JSON-RPC error. + Assert.That(response.ContainsKey("error"), Is.False); + + var result = (JObject)response["result"]!; + Assert.That(result["isError"]!.Value(), Is.True); + Assert.That(result["content"]![0]!["text"]!.Value(), Does.Contain("boom")); + } + + [Test] + public void RethrowWhenAToolIsCancelled() + { + FakeMcpTool tool = FakeMcpTool.Throwing("cancelled", new OperationCanceledException()); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + Assert.Throws(() => Dispatch(dispatcher, CallRequest(8, "cancelled", new JObject()))); + } + + [Test] + public void RejectAnUnknownMethodWithMethodNotFound() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + JObject error = ErrorOf(Dispatch(dispatcher, Request(9, "resources/list"))); + + Assert.That(error["code"]!.Value(), Is.EqualTo(-32601)); + Assert.That(error["message"]!.Value(), Does.Contain("resources/list")); + } + + [Test] + public void DropNotificationsWithoutAnId() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + string request = new JObject + { + ["jsonrpc"] = "2.0", + ["method"] = "notifications/initialized", + }.ToString(Formatting.None); + + Assert.That(Dispatch(dispatcher, request), Is.Null); + } + + [Test] + public void ReplyWithAParseErrorOnMalformedJson() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + JObject error = ErrorOf(Dispatch(dispatcher, "{ this is not json")); + + Assert.That(error["code"]!.Value(), Is.EqualTo(-32700)); + } + + [Test] + public void RejectARequestMissingAMethodWithInvalidRequest() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + string request = new JObject { ["jsonrpc"] = "2.0", ["id"] = 10 }.ToString(Formatting.None); + + JObject error = ErrorOf(Dispatch(dispatcher, request)); + + Assert.That(error["code"]!.Value(), Is.EqualTo(-32600)); + } + + private static McpJsonRpcDispatcher DispatcherWith(params IMcpTool[] tools) + { + var registry = new McpToolsRegistry(); + + foreach (IMcpTool tool in tools) + registry.Add(tool); + + registry.Build(); + return new McpJsonRpcDispatcher(registry, SERVER_VERSION); + } + + private static string? Dispatch(McpJsonRpcDispatcher dispatcher, string requestJson) => + dispatcher.DispatchAsync(requestJson, CancellationToken.None).GetAwaiter().GetResult(); + + private static string Request(int id, string method) => + new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = id, + ["method"] = method, + }.ToString(Formatting.None); + + private static string CallRequest(int id, string toolName, JObject arguments) => + new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = id, + ["method"] = "tools/call", + ["params"] = new JObject + { + ["name"] = toolName, + ["arguments"] = arguments, + }, + }.ToString(Formatting.None); + + private static JObject ResultOf(string? response) + { + Assert.That(response, Is.Not.Null, "expected a response, got a dropped message"); + var parsed = JObject.Parse(response!); + Assert.That(parsed["jsonrpc"]!.Value(), Is.EqualTo("2.0")); + Assert.That(parsed.ContainsKey("error"), Is.False, $"expected a result, got an error: {response}"); + return (JObject)parsed["result"]!; + } + + private static JObject ErrorOf(string? response) + { + Assert.That(response, Is.Not.Null, "expected an error response, got a dropped message"); + var parsed = JObject.Parse(response!); + Assert.That(parsed["jsonrpc"]!.Value(), Is.EqualTo("2.0")); + return (JObject)parsed["error"]!; + } + + private static JObject ToolEntry(JObject toolsListResult, string name) + { + foreach (JToken entry in (JArray)toolsListResult["tools"]!) + if (entry["name"]!.Value() == name) + return (JObject)entry; + + Assert.Fail($"tool '{name}' not found in tools/list"); + return null!; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs.meta new file mode 100644 index 00000000000..cd4bb0fe2a8 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: e447afff3fddc6746922385663f764be \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs new file mode 100644 index 00000000000..c1e66af8e5c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs @@ -0,0 +1,98 @@ +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using NUnit.Framework; + +namespace DCL.McpServer.Tests +{ + public class McpJsonSchemaShould + { + [Test] + public void EmitTheDeclaredJsonSchemaTypeForEachFieldKind() + { + JObject schema = McpJsonSchema.Object() + .Number("ratio") + .Boolean("flag") + .Build(); + + var properties = (JObject)schema["properties"]!; + Assert.That(properties["ratio"]!["type"]!.Value(), Is.EqualTo("number")); + Assert.That(properties["flag"]!["type"]!.Value(), Is.EqualTo("boolean")); + } + + [Test] + public void OmitDescriptionAndEnumWhenNotProvided() + { + JObject schema = McpJsonSchema.Object().String("name").Build(); + + var field = (JObject)schema["properties"]!["name"]!; + Assert.That(field.ContainsKey("description"), Is.False); + Assert.That(field.ContainsKey("enum"), Is.False); + } + + [Test] + public void CollectEveryRequiredFieldInDeclarationOrder() + { + JObject schema = McpJsonSchema.Object() + .String("first", required: true) + .Integer("skipped") + .Boolean("second", required: true) + .Build(); + + Assert.That(((JArray)schema["required"]!).ToObject(), Is.EqualTo(new[] { "first", "second" })); + } + + [Test] + public void NestAnObjectFieldWithItsOwnProperties() + { + JObject schema = McpJsonSchema.Object() + .Object("camera", McpJsonSchema.Object().String("mode"), "The camera.", required: true) + .Build(); + + var camera = (JObject)schema["properties"]!["camera"]!; + Assert.That(camera["type"]!.Value(), Is.EqualTo("object")); + Assert.That(camera["description"]!.Value(), Is.EqualTo("The camera.")); + Assert.That(camera["properties"]!["mode"]!["type"]!.Value(), Is.EqualTo("string")); + Assert.That(((JArray)schema["required"]!).ToObject(), Is.EqualTo(new[] { "camera" })); + } + + [Test] + public void AdmitNullAlongsideAnObjectForANullableNestedField() + { + JObject schema = McpJsonSchema.Object() + .Object("scene", McpJsonSchema.Object().String("name"), nullable: true) + .Build(); + + var sceneType = (JArray)schema["properties"]!["scene"]!["type"]!; + Assert.That(sceneType.ToObject(), Is.EqualTo(new[] { "object", "null" })); + } + + [Test] + public void AdmitNullAlongsideTheDeclaredTypeForANullableScalar() + { + JObject schema = McpJsonSchema.Object().String("address", nullable: true).Build(); + + var addressType = (JArray)schema["properties"]!["address"]!["type"]!; + Assert.That(addressType.ToObject(), Is.EqualTo(new[] { "string", "null" })); + } + + [Test] + public void DescribeAnArrayOfIntegerItems() + { + JObject schema = McpJsonSchema.Object().IntegerArray("entityIds").Build(); + + var field = (JObject)schema["properties"]!["entityIds"]!; + Assert.That(field["type"]!.Value(), Is.EqualTo("array")); + Assert.That(field["items"]!["type"]!.Value(), Is.EqualTo("integer")); + } + + [Test] + public void ProduceAnEmptyPropertiesObjectForAnArgumentlessTool() + { + JObject schema = McpJsonSchema.Object().Build(); + + Assert.That(schema["type"]!.Value(), Is.EqualTo("object")); + Assert.That(((JObject)schema["properties"]!).Count, Is.EqualTo(0)); + Assert.That(schema.ContainsKey("required"), Is.False); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs.meta new file mode 100644 index 00000000000..0b62cbacd67 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: bcaebf033b5a3154b84bfdffe0aa383e \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpNetworkLogBufferShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpNetworkLogBufferShould.cs new file mode 100644 index 00000000000..ea8f93de2e3 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpNetworkLogBufferShould.cs @@ -0,0 +1,117 @@ +using DCL.WebRequests.Analytics; +using NUnit.Framework; +using System.Collections.Generic; + +namespace DCL.McpServer.Tests +{ + public class McpNetworkLogBufferShould + { + private McpNetworkLogBuffer buffer = null!; + + [SetUp] + public void Setup() + { + buffer = new McpNetworkLogBuffer(); + } + + private void AppendOk(string url, int status = 200) => + buffer.Append(url, "GET", status, "application/json", 100, 12.5, failed: false, failureReason: null); + + private void AppendFailed(string url) => + buffer.Append(url, "GET", 0, "application/octet-stream", 0, 5, failed: true, failureReason: "Cancelled"); + + private List Copy(long sinceSeq = -1, bool failedOnly = false, int status = -1, int limit = 100) + { + var list = new List(); + buffer.CopyTo(list, sinceSeq, failedOnly, status, limit); + return list; + } + + [Test] + public void ReportMinusOneLatestSeqWhenEmpty() + { + Assert.That(buffer.LatestSeq, Is.EqualTo(-1)); + Assert.That(Copy(), Is.Empty); + } + + [Test] + public void AssignMonotonicSequenceNumbers() + { + AppendOk("a"); + AppendOk("b"); + AppendOk("c"); + + List entries = Copy(); + + Assert.That(buffer.LatestSeq, Is.EqualTo(2)); + Assert.That(entries.ConvertAll(e => e.Seq), Is.EqualTo(new long[] { 0, 1, 2 })); + Assert.That(entries.ConvertAll(e => e.Url), Is.EqualTo(new[] { "a", "b", "c" })); + } + + [Test] + public void ReturnOnlyEntriesNewerThanSinceSeq() + { + AppendOk("a"); + AppendOk("b"); + AppendOk("c"); + + List entries = Copy(sinceSeq: 0); + + Assert.That(entries.ConvertAll(e => e.Seq), Is.EqualTo(new long[] { 1, 2 })); + } + + [Test] + public void TreatBothTransportFailuresAndErrorStatusesAsUnsuccessfulUnderFailedOnly() + { + AppendOk("ok", 200); + AppendOk("notfound", 404); + AppendFailed("boom"); + + List entries = Copy(failedOnly: true); + + Assert.That(entries.ConvertAll(e => e.Url), Is.EqualTo(new[] { "notfound", "boom" })); + } + + [Test] + public void FilterByExactStatus() + { + AppendOk("a", 200); + AppendOk("b", 404); + AppendOk("c", 404); + + List entries = Copy(status: 404); + + Assert.That(entries.ConvertAll(e => e.Url), Is.EqualTo(new[] { "b", "c" })); + } + + [Test] + public void KeepOnlyTheNewestLimitEntries() + { + for (var i = 0; i < 10; i++) + AppendOk($"u{i}"); + + List entries = Copy(limit: 3); + + Assert.That(entries.Count, Is.EqualTo(3)); + Assert.That(entries.ConvertAll(e => e.Url), Is.EqualTo(new[] { "u7", "u8", "u9" })); + } + + [Test] + public void DropOldEntriesOnceCapacityIsExceededButKeepSequenceMonotonic() + { + // CAPACITY is 512; overrun it so the oldest entries are evicted. + const int TOTAL = 600; + + for (var i = 0; i < TOTAL; i++) + AppendOk($"u{i}"); + + Assert.That(buffer.LatestSeq, Is.EqualTo(TOTAL - 1)); + + List entries = Copy(limit: 1000); + + // The oldest surviving entry is TOTAL-512, and sinceSeq below that is clamped to what's available. + Assert.That(entries[0].Seq, Is.EqualTo(TOTAL - 512)); + Assert.That(entries[^1].Seq, Is.EqualTo(TOTAL - 1)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpNetworkLogBufferShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpNetworkLogBufferShould.cs.meta new file mode 100644 index 00000000000..b2c475eb5dd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpNetworkLogBufferShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: aab164e0d4c54a03bc9598b53b245b3d \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpPointerClickSystemShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpPointerClickSystemShould.cs new file mode 100644 index 00000000000..42d52792da9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpPointerClickSystemShould.cs @@ -0,0 +1,303 @@ +using Arch.Core; +using CRDT; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.ECSComponents; +using DCL.Interaction.Utility; +using DCL.McpServer.Components; +using DCL.McpServer.Systems; +using DCL.Utilities; +using ECS.SceneLifeCycle; +using ECS.TestSuite; +using ECS.Unity.Transforms.Components; +using NSubstitute; +using NUnit.Framework; +using SceneRunner.Scene; +using UnityEngine; +using Utility.Multithreading; + +namespace DCL.McpServer.Tests +{ + public class McpPointerClickSystemShould : UnitySystemTestBase + { + private const int TARGET_CRDT_ID = 512; + private const int BLOCKER_CRDT_ID = 513; + + private World sceneWorld = null!; + private Entity playerEntity; + private Entity targetEntity; + + private GameObject cameraGo = null!; + private GameObject playerGo = null!; + private GameObject targetGo = null!; + private GameObject blockerGo = null!; + + private BoxCollider targetCollider = null!; + private PBPointerEvents targetPointerEvents = null!; + private ISceneStateProvider sceneStateProvider = null!; + private uint tick; + + [SetUp] + public void SetUp() + { + sceneWorld = World.Create(); + + cameraGo = new GameObject("mcp-click-test-camera"); + Camera camera = cameraGo.AddComponent(); + world.Create(new CameraComponent(camera)); // Mode defaults to FirstPerson + + playerGo = new GameObject("mcp-click-test-player"); + playerEntity = world.Create(new CharacterTransform(playerGo.transform)); + + targetGo = new GameObject("mcp-click-test-target") + { + transform = { position = new Vector3(0f, 0f, 5f), }, + }; + + targetCollider = targetGo.AddComponent(); + + targetPointerEvents = new PBPointerEvents + { + PointerEvents = + { + new PBPointerEvents.Types.Entry + { + EventType = PointerEventType.PetDown, + EventInfo = new PBPointerEvents.Types.Info + { + Button = InputAction.IaPointer, + HoverText = "Open", + MaxDistance = 10f, + }, + }, + }, + }; + + targetPointerEvents.AppendPointerEventResultsIntent.InitializeWithAlloc(); + + // The entity needs a TransformComponent so ResolveEntityAimPoint can aim the validation ray at it; + // without it the aim point is Vector3.zero and the click bails out before the raycast. + targetEntity = sceneWorld.Create(targetPointerEvents, new CRDTEntity(TARGET_CRDT_ID), new TransformComponent(targetGo.transform)); + + // Colliders created/moved this frame are not in the PhysX scene until transforms are synced (no physics step runs in EditMode). + Physics.SyncTransforms(); + + tick = 100u; + sceneStateProvider = Substitute.For(); + sceneStateProvider.IsCurrent.Returns(true); + sceneStateProvider.State.Returns(new Atomic(SceneState.Running)); + sceneStateProvider.TickNumber.Returns(_ => tick); + + ISceneFacade sceneFacade = Substitute.For(); + sceneFacade.SceneStateProvider.Returns(sceneStateProvider); + sceneFacade.EcsExecutor.Returns(new SceneEcsExecutor(sceneWorld)); + + IReadonlyReactiveProperty currentScene = Substitute.For>(); + currentScene.Value.Returns(sceneFacade); + + IScenesCache scenesCache = Substitute.For(); + scenesCache.CurrentScene.Returns(currentScene); + + IEntityCollidersGlobalCache collidersCache = Substitute.For(); + + collidersCache.TryGetSceneEntity(Arg.Any(), out Arg.Any()) + .Returns(call => + { + var collider = call.ArgAt(0); + + if (collider == targetCollider) + { + call[1] = new GlobalColliderSceneEntityInfo( + new SceneEcsExecutor(sceneWorld), + new ColliderSceneEntityInfo(targetEntity, new CRDTEntity(TARGET_CRDT_ID), ColliderLayer.ClPointer)); + + return true; + } + + if (blockerGo != null && collider == blockerGo.GetComponent()) + { + call[1] = new GlobalColliderSceneEntityInfo( + new SceneEcsExecutor(sceneWorld), + new ColliderSceneEntityInfo(sceneWorld.Create(new CRDTEntity(BLOCKER_CRDT_ID)), new CRDTEntity(BLOCKER_CRDT_ID), ColliderLayer.ClPhysics)); + + return true; + } + + return false; + }); + + system = new McpPointerClickSystem(world, scenesCache, collidersCache, playerEntity); + system.Initialize(); + } + + protected override void OnTearDown() + { + Object.DestroyImmediate(cameraGo); + Object.DestroyImmediate(playerGo); + Object.DestroyImmediate(targetGo); + + if (blockerGo != null) + Object.DestroyImmediate(blockerGo); + + sceneWorld.Dispose(); + } + + private UniTaskCompletionSource AddIntent( + McpPointerClickIntent.ClickKind kind = McpPointerClickIntent.ClickKind.CLICK, + int? targetId = null) + { + var completion = new UniTaskCompletionSource(); + + world.Add(playerEntity, new McpPointerClickIntent + { + TargetEntityId = targetId ?? targetEntity.Id, + Button = InputAction.IaPointer, + Kind = kind, + Phase = McpPointerClickIntent.ClickPhase.DOWN, + Deadline = UnityEngine.Time.time + 5f, + Completion = completion, + }); + + return completion; + } + + private static McpPointerClickResult ResultOf(UniTaskCompletionSource completion) + { + Assert.That(completion.Task.Status, Is.EqualTo(UniTaskStatus.Succeeded)); + return completion.Task.GetAwaiter().GetResult(); + } + + [Test] + public void DeliverDownThenUpOnNextTick() + { + UniTaskCompletionSource completion = AddIntent(); + + system!.Update(0); + + var actions = targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions; + Assert.That(actions.Count, Is.EqualTo(1)); + Assert.That(actions[0], Is.EqualTo((InputAction.IaPointer, PointerEventType.PetDown))); + Assert.That(completion.Task.Status, Is.EqualTo(UniTaskStatus.Pending)); + + // The scene-world flush clears the intent at the end of a real frame. + targetPointerEvents.AppendPointerEventResultsIntent.Clear(); + + system.Update(0); // same tick: keeps waiting so PetUp lands on a later tick than PetDown + Assert.That(completion.Task.Status, Is.EqualTo(UniTaskStatus.Pending)); + + tick++; + system.Update(0); // observes the tick advance + system.Update(0); // delivers PetUp + + actions = targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions; + Assert.That(actions.Count, Is.EqualTo(1)); + Assert.That(actions[0], Is.EqualTo((InputAction.IaPointer, PointerEventType.PetUp))); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.True); + Assert.That(result.CrdtEntityId, Is.EqualTo(TARGET_CRDT_ID)); + Assert.That(result.HoverText, Is.EqualTo("Open")); + Assert.That(result.UpRayMissed, Is.False); + Assert.That(world.Has(playerEntity), Is.False); + } + + [Test] + public void DeliverSingleDownWithoutWaiting() + { + UniTaskCompletionSource completion = AddIntent(McpPointerClickIntent.ClickKind.DOWN); + + system!.Update(0); + + var actions = targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions; + Assert.That(actions.Count, Is.EqualTo(1)); + Assert.That(actions[0], Is.EqualTo((InputAction.IaPointer, PointerEventType.PetDown))); + + Assert.That(ResultOf(completion).Hit, Is.True); + Assert.That(world.Has(playerEntity), Is.False); + } + + [Test] + public void FailWhenAnotherColliderBlocksTheRay() + { + blockerGo = new GameObject("mcp-click-test-blocker") { transform = { position = new Vector3(0f, 0f, 2f) }}; + + blockerGo.AddComponent(); + Physics.SyncTransforms(); + + UniTaskCompletionSource completion = AddIntent(); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.BlockedByCrdtId, Is.EqualTo(BLOCKER_CRDT_ID)); + Assert.That(targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions.Count, Is.EqualTo(0)); + Assert.That(world.Has(playerEntity), Is.False); + } + + [Test] + public void FailWhenOutOfRange() + { + targetPointerEvents.PointerEvents[0].EventInfo.MaxDistance = 2f; + + UniTaskCompletionSource completion = AddIntent(); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.FailureReason, Does.Contain("out of range")); + Assert.That(targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions.Count, Is.EqualTo(0)); + } + + [Test] + public void FailWhenEntityHasNoPointerEvents() + { + sceneWorld.Remove(targetEntity); + + UniTaskCompletionSource completion = AddIntent(); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.FailureReason, Does.Contain("PointerEvents")); + } + + [Test] + public void FailWhenEntityIdIsUnknown() + { + UniTaskCompletionSource completion = AddIntent(targetId: 987654); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.FailureReason, Does.Contain("no entity")); + } + + [Test] + public void FailWhenDeadlinePassed() + { + var completion = new UniTaskCompletionSource(); + + world.Add(playerEntity, new McpPointerClickIntent + { + TargetEntityId = targetEntity.Id, + Button = InputAction.IaPointer, + Kind = McpPointerClickIntent.ClickKind.CLICK, + Phase = McpPointerClickIntent.ClickPhase.DOWN, + Deadline = UnityEngine.Time.time - 1f, + Completion = completion, + }); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.FailureReason, Does.Contain("timed out")); + Assert.That(world.Has(playerEntity), Is.False); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpPointerClickSystemShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpPointerClickSystemShould.cs.meta new file mode 100644 index 00000000000..31ec606ea26 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpPointerClickSystemShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 3245eba0487e4b8a8c8f125b10268888 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs b/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs new file mode 100644 index 00000000000..851ebca12fc --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs @@ -0,0 +1,57 @@ +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System.Collections.Generic; + +namespace DCL.McpServer.Tests +{ + /// + /// Guards against output-schema ↔ structuredContent drift: a tool declares its OutputSchema by hand and + /// builds the matching payload by hand, so a field added to one and forgotten in the other lets the schema + /// silently misdescribe the payload to the agent. Asserting the two carry the same property names + /// (recursively, into nested objects the payload populates) turns that drift into a failing test. + /// + internal static class McpSchemaAssert + { + public static void KeysMatch(JObject schema, JObject payload) + { + var properties = schema["properties"] as JObject ?? new JObject(); + + CollectionAssert.AreEquivalent(NamesOf(properties), NamesOf(payload), + $"Output schema and payload disagree on the keys of '{PathOf(schema)}'."); + + foreach (JProperty property in properties.Properties()) + if (DeclaresObject(property.Value) && payload[property.Name] is JObject nested) + KeysMatch((JObject)property.Value, nested); + } + + private static List NamesOf(JObject obj) + { + var names = new List(); + + foreach (JProperty property in obj.Properties()) + names.Add(property.Name); + + return names; + } + + // A property models an object both as type "object" and as a nullable ["object", "null"] union. + private static bool DeclaresObject(JToken schema) + { + JToken? type = schema["type"]; + + if (type is JArray union) + { + foreach (JToken member in union) + if (member.Value() == "object") + return true; + + return false; + } + + return type?.Value() == "object"; + } + + private static string PathOf(JObject schema) => + string.IsNullOrEmpty(schema.Path) ? "" : schema.Path; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs.meta new file mode 100644 index 00000000000..a88c5ac22bd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: b0e1283f51dcb2948b8bb95431335e2f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs new file mode 100644 index 00000000000..ab91a05c4be --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs @@ -0,0 +1,78 @@ +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System; +using System.Text; + +namespace DCL.McpServer.Tests +{ + public class McpToolResultShould + { + [Test] + public void WrapTextInASingleTextContentItemWithoutAnErrorFlag() + { + JObject payload = McpToolResult.Text("hello").Payload; + + var content = (JArray)payload["content"]!; + Assert.That(content.Count, Is.EqualTo(1)); + Assert.That(content[0]!["type"]!.Value(), Is.EqualTo("text")); + Assert.That(content[0]!["text"]!.Value(), Is.EqualTo("hello")); + Assert.That(payload.ContainsKey("isError"), Is.False); + } + + [Test] + public void CarryBothAMirroringTextItemAndStructuredContent() + { + var structured = new JObject + { + ["count"] = 3, + ["nested"] = new JObject { ["ok"] = true }, + }; + + JObject payload = McpToolResult.TextWithStructured("mirror", structured).Payload; + + var content = (JArray)payload["content"]!; + Assert.That(content.Count, Is.EqualTo(1)); + Assert.That(content[0]!["type"]!.Value(), Is.EqualTo("text")); + Assert.That(content[0]!["text"]!.Value(), Is.EqualTo("mirror")); + + var structuredContent = (JObject)payload["structuredContent"]!; + Assert.That(structuredContent["count"]!.Value(), Is.EqualTo(3)); + Assert.That(structuredContent["nested"]!["ok"]!.Value(), Is.True); + Assert.That(payload.ContainsKey("isError"), Is.False); + } + + [Test] + public void FlagErrorsWithIsErrorAndCarryTheMessageAsText() + { + JObject payload = McpToolResult.Error("it broke").Payload; + + Assert.That(payload["isError"]!.Value(), Is.True); + + var content = (JArray)payload["content"]!; + Assert.That(content[0]!["type"]!.Value(), Is.EqualTo("text")); + Assert.That(content[0]!["text"]!.Value(), Is.EqualTo("it broke")); + } + + [Test] + public void EncodeImageBytesAsBase64AlongsideACaption() + { + byte[] bytes = Encoding.UTF8.GetBytes("pixels"); + + JObject payload = McpToolResult.Image(bytes, "image/png", "a screenshot").Payload; + + var content = (JArray)payload["content"]!; + Assert.That(content.Count, Is.EqualTo(2)); + + var image = (JObject)content[0]!; + Assert.That(image["type"]!.Value(), Is.EqualTo("image")); + Assert.That(image["mimeType"]!.Value(), Is.EqualTo("image/png")); + Assert.That(Convert.FromBase64String(image["data"]!.Value()!), Is.EqualTo(bytes)); + + var caption = (JObject)content[1]!; + Assert.That(caption["type"]!.Value(), Is.EqualTo("text")); + Assert.That(caption["text"]!.Value(), Is.EqualTo("a screenshot")); + Assert.That(payload.ContainsKey("isError"), Is.False); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs.meta new file mode 100644 index 00000000000..85c2acbac61 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 746fe351b002c704ea5996a66566492f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs new file mode 100644 index 00000000000..11541284936 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs @@ -0,0 +1,211 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System; +using System.Collections.Generic; +using System.Threading; + +namespace DCL.McpServer.Tests +{ + public class McpToolsRegistryShould + { + [Test] + public void BuildAnObjectSchemaWithTypedPropertiesAndRequired() + { + // Act + JObject schema = McpJsonSchema.Object() + .Integer("count", "How many.") + .String("mode", "Pick one.", enumValues: new[] { "a", "b" }, required: true) + .Build(); + + // Assert + Assert.That(schema["type"]!.Value(), Is.EqualTo("object")); + + var properties = (JObject)schema["properties"]!; + Assert.That(properties["count"]!["type"]!.Value(), Is.EqualTo("integer")); + Assert.That(properties["count"]!["description"]!.Value(), Is.EqualTo("How many.")); + Assert.That(properties["mode"]!["type"]!.Value(), Is.EqualTo("string")); + Assert.That(((JArray)properties["mode"]!["enum"]!).ToObject(), Is.EqualTo(new[] { "a", "b" })); + + Assert.That(((JArray)schema["required"]!).ToObject(), Is.EqualTo(new[] { "mode" })); + } + + [Test] + public void OmitRequiredWhenNoFieldIsRequired() + { + // Act + JObject schema = McpJsonSchema.Object().Boolean("flag").Build(); + + // Assert + Assert.That(schema.ContainsKey("required"), Is.False); + } + + [Test] + public void FailRegistrationNamingTheToolWithAnInvalidSchema() + { + // Arrange + var registry = new McpToolsRegistry() + .Add(new FakeTool("broken", McpToolAnnotations.ReadOnly(), new JObject())); + + // Act & Assert + InvalidOperationException error = Assert.Throws(() => registry.Build()); + Assert.That(error!.Message, Does.Contain("broken")); + } + + [Test] + public void EmitReadOnlyAnnotationsWithoutStateChangeHints() + { + // Arrange + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("reader", McpToolAnnotations.ReadOnly())) + .Build()); + + // Act + JObject annotations = AnnotationsOf(toolsList, "reader"); + + // Assert + Assert.That(annotations["readOnlyHint"]!.Value(), Is.True); + Assert.That(annotations["openWorldHint"]!.Value(), Is.False); + Assert.That(annotations.ContainsKey("destructiveHint"), Is.False); + Assert.That(annotations.ContainsKey("idempotentHint"), Is.False); + } + + [Test] + public void EmitMutatingAnnotationsWithAllStateChangeHints() + { + // Arrange + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("mutator", McpToolAnnotations.Mutating(destructive: true, idempotent: false))) + .Build()); + + // Act + JObject annotations = AnnotationsOf(toolsList, "mutator"); + + // Assert + Assert.That(annotations["readOnlyHint"]!.Value(), Is.False); + Assert.That(annotations["destructiveHint"]!.Value(), Is.True); + Assert.That(annotations["idempotentHint"]!.Value(), Is.False); + Assert.That(annotations["openWorldHint"]!.Value(), Is.False); + } + + [Test] + public void IncludeOutputSchemaWhenTheToolDeclaresOne() + { + // Arrange + JObject outputSchema = McpJsonSchema.Object().Integer("total").Build(); + + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("structured", McpToolAnnotations.ReadOnly(), outputSchema: outputSchema)) + .Build()); + + // Act + JObject entry = EntryOf(toolsList, "structured"); + + // Assert + Assert.That(entry.ContainsKey("outputSchema"), Is.True); + Assert.That(entry["outputSchema"]!["type"]!.Value(), Is.EqualTo("object")); + } + + [Test] + public void OmitOutputSchemaWhenTheToolDeclaresNone() + { + // Arrange + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("plain", McpToolAnnotations.ReadOnly())) + .Build()); + + // Act & Assert + Assert.That(EntryOf(toolsList, "plain").ContainsKey("outputSchema"), Is.False); + } + + [Test] + public void FindARegisteredToolByName() + { + var tool = new FakeTool("known", McpToolAnnotations.ReadOnly()); + var registry = new McpToolsRegistry().Add(tool); + + bool found = registry.TryGet("known", out IMcpTool? resolved); + + Assert.That(found, Is.True); + Assert.That(resolved, Is.SameAs(tool)); + } + + [Test] + public void NotFindAnUnknownTool() + { + var registry = new McpToolsRegistry().Add(new FakeTool("known", McpToolAnnotations.ReadOnly())); + + Assert.That(registry.TryGet("missing", out IMcpTool? resolved), Is.False); + Assert.That(resolved, Is.Null); + } + + [Test] + public void NotFindAToolForANullOrEmptyName() + { + var registry = new McpToolsRegistry().Add(new FakeTool("known", McpToolAnnotations.ReadOnly())); + + Assert.That(registry.TryGet(null, out IMcpTool? byNull), Is.False); + Assert.That(byNull, Is.Null); + + Assert.That(registry.TryGet(string.Empty, out IMcpTool? byEmpty), Is.False); + Assert.That(byEmpty, Is.Null); + } + + [Test] + public void ReflectTheRegisteredSetInTheToolsList() + { + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("first", McpToolAnnotations.ReadOnly())) + .Add(new FakeTool("second", McpToolAnnotations.Mutating(destructive: false, idempotent: true))) + .Build()); + + var names = new List(); + + foreach (JToken entry in (JArray)toolsList["tools"]!) + names.Add(entry["name"]!.Value()); + + Assert.That(names, Is.EquivalentTo(new[] { "first", "second" })); + } + + private static JObject Payload(McpToolsRegistry registry) => + JObject.Parse(registry.ToolsListPayload().ToString()); + + private static JObject AnnotationsOf(JObject toolsList, string name) => + (JObject)EntryOf(toolsList, name)["annotations"]!; + + private static JObject EntryOf(JObject toolsList, string name) + { + foreach (JToken entry in (JArray)toolsList["tools"]!) + if (entry["name"]!.Value() == name) + return (JObject)entry; + + Assert.Fail($"tool '{name}' not found in tools/list"); + return null!; + } + + private class FakeTool : IMcpTool + { + public string Name { get; } + + public McpToolAnnotations Annotations { get; } + + public string Description => "fake"; + + public JObject InputSchema { get; } + + public JObject? OutputSchema { get; } + + public FakeTool(string name, McpToolAnnotations annotations, JObject? inputSchema = null, JObject? outputSchema = null) + { + Name = name; + Annotations = annotations; + InputSchema = inputSchema ?? McpJsonSchema.Object().Build(); + OutputSchema = outputSchema; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) => + UniTask.FromResult(McpToolResult.Text("fake")); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs.meta new file mode 100644 index 00000000000..94d6bf30834 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 1e7c91136f9767a4b9b3922fef87f114 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/UiElementPathShould.cs b/Explorer/Assets/DCL/McpServer/Tests/UiElementPathShould.cs new file mode 100644 index 00000000000..e0a672cfe3c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/UiElementPathShould.cs @@ -0,0 +1,53 @@ +using DCL.McpServer.Utils; +using NUnit.Framework; + +namespace DCL.McpServer.Tests +{ + public class UiElementPathShould + { + [Test] + public void RecognizeTheSystemPrefix() + { + Assert.That(UiElementPath.IsUgui("ugui:/Canvas/Button"), Is.True); + Assert.That(UiElementPath.IsUgui("uitk:/Doc/button"), Is.False); + Assert.That(UiElementPath.IsUitk("uitk:/Doc/button"), Is.True); + Assert.That(UiElementPath.IsUitk("ugui:/Canvas/Button"), Is.False); + } + + [Test] + public void UseTheNameAsSegmentWhenPresentAndATypeIndexTokenOtherwise() + { + Assert.That(UiElementPath.Segment("Play", "Button", 3), Is.EqualTo("Play")); + Assert.That(UiElementPath.Segment(null, "VisualElement", 3), Is.EqualTo("VisualElement[3]")); + Assert.That(UiElementPath.Segment("", "Label", 0), Is.EqualTo("Label[0]")); + } + + [Test] + public void JoinSegmentsWithASingleSeparator() + { + Assert.That(UiElementPath.Join("uitk:/Doc", "Panel"), Is.EqualTo("uitk:/Doc/Panel")); + } + + [TestCase(null, ExpectedResult = true)] + [TestCase("", ExpectedResult = true)] + [TestCase("play", ExpectedResult = true)] // matches name (case-insensitive) + [TestCase("Canvas", ExpectedResult = true)] // matches path + [TestCase("missing", ExpectedResult = false)] + public bool MatchOnEitherNameOrPathCaseInsensitively(string? filter) => + UiElementPath.Matches("PlayButton", "ugui:/Canvas/PlayButton", filter); + + [Test] + public void ScoreAnExactPathHigherThanANameHigherThanASuffixHigherThanAContains() + { + const string PATH = "ugui:/Canvas/Menu/PlayButton"; + const string NAME = "PlayButton"; + + Assert.That(UiElementPath.MatchScore(PATH, NAME, PATH), Is.EqualTo(4)); + Assert.That(UiElementPath.MatchScore(PATH, NAME, "playbutton"), Is.EqualTo(3)); // exact name, case-insensitive + Assert.That(UiElementPath.MatchScore(PATH, "Other", "PlayButton"), Is.EqualTo(2)); // path suffix + Assert.That(UiElementPath.MatchScore(PATH, "Other", "Menu"), Is.EqualTo(1)); // path contains + Assert.That(UiElementPath.MatchScore(PATH, NAME, "nope"), Is.EqualTo(0)); + Assert.That(UiElementPath.MatchScore(PATH, NAME, ""), Is.EqualTo(0)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/UiElementPathShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/UiElementPathShould.cs.meta new file mode 100644 index 00000000000..54d15cd2c3c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/UiElementPathShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 796b59c5818d43bd815566bfb4534a8a \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools.meta b/Explorer/Assets/DCL/McpServer/Tools.meta new file mode 100644 index 00000000000..ea160a5cffd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: 9a8b8d76e6c14409697c08478f574549 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs new file mode 100644 index 00000000000..1d10faa15c3 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs @@ -0,0 +1,161 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.ECSComponents; +using DCL.McpServer.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; +using UnityEngine; +using Utility.Arch; + +namespace DCL.McpServer.Tools +{ + /// + /// Presses a pointer button on a scene entity through / + /// McpPointerClickSystem, which validates the aim with the same raycast rules as the reticle + /// pipeline before filling the entity's pointer-event intent like a real click. + /// + public class ClickEntityTool : IMcpTool + { + private const float DEFAULT_TIMEOUT_SEC = 3f; + private const float MIN_TIMEOUT_SEC = 0.5f; + private const float MAX_TIMEOUT_SEC = 15f; + private const float COMPLETION_GRACE_SEC = 2f; + + private readonly World world; + private readonly Entity playerEntity; + + public string Name => "click_entity"; + + public string Description => + "Press and release a pointer button on a scene entity so its PointerEvents fire exactly like a real click. " + + "The aim is validated by a physics raycast from the camera: occluders and the entity's maxDistance apply, and a miss " + + "returns hit:false with the blocking entity. Ids come from list_scene_entities. For entities whose collider " + + "sits away from their pivot (e.g. GLTF meshes), pass an explicit x/y/z world point to aim at."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Integer("entityId", "Target entity id in the current scene world (from list_scene_entities). Omit only when x/y/z are given, then the ray decides the target.") + .Number("x", "World-space aim point; overrides the automatic aim at the entity's collider center.") + .Number("y") + .Number("z") + .String("button", "Which input action to press. Default pointer (left click / IA_POINTER).", enumValues: new[] { "pointer", "primary", "secondary" }) + .String("eventType", "click = down, then up on the next scene tick. Default click.", enumValues: new[] { "click", "down", "up" }) + .Number("timeoutSec", "Seconds to wait for delivery. Default 3, max 15.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: false); + + public ClickEntityTool(World world, Entity playerEntity) + { + this.world = world; + this.playerEntity = playerEntity; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + bool hasEntityId = arguments.TryGetInt("entityId", out int entityId); + + bool hasAimPoint = arguments.TryGetFloat("x", out float x) + & arguments.TryGetFloat("y", out float y) + & arguments.TryGetFloat("z", out float z); + + if (!hasEntityId && !hasAimPoint) + return McpToolResult.Error("Provide entityId, or a full x/y/z world aim point, or both."); + + InputAction button; + + switch (arguments.GetString("button", "pointer")) + { + case "pointer": button = InputAction.IaPointer; break; + case "primary": button = InputAction.IaPrimary; break; + case "secondary": button = InputAction.IaSecondary; break; + default: return McpToolResult.Error("button must be one of: pointer, primary, secondary."); + } + + McpPointerClickIntent.ClickKind kind; + + switch (arguments.GetString("eventType", "click")) + { + case "click": kind = McpPointerClickIntent.ClickKind.CLICK; break; + case "down": kind = McpPointerClickIntent.ClickKind.DOWN; break; + case "up": kind = McpPointerClickIntent.ClickKind.UP; break; + default: return McpToolResult.Error("eventType must be one of: click, down, up."); + } + + float timeoutSec = Mathf.Clamp(arguments.GetFloat("timeoutSec", DEFAULT_TIMEOUT_SEC), MIN_TIMEOUT_SEC, MAX_TIMEOUT_SEC); + + // A newer click preempts a pending one; release its awaiter before replacing the intent. + if (world.TryGet(playerEntity, out McpPointerClickIntent existingIntent)) + existingIntent.Completion?.TrySetResult(new McpPointerClickResult + { + Hit = false, + FailureReason = "preempted by a newer click_entity call", + }); + + var completion = new UniTaskCompletionSource(); + + world.AddOrSet(playerEntity, new McpPointerClickIntent + { + TargetEntityId = hasEntityId ? entityId : -1, + AimPoint = new Vector3(x, y, z), + HasExplicitAimPoint = hasAimPoint, + Button = button, + Kind = kind, + Phase = McpPointerClickIntent.ClickPhase.DOWN, + Deadline = UnityEngine.Time.time + timeoutSec, + Completion = completion, + }); + + McpPointerClickResult result; + + try + { + result = await completion.Task.AttachExternalCancellation(ct) + .Timeout(TimeSpan.FromSeconds(timeoutSec + COMPLETION_GRACE_SEC)); + } + catch (TimeoutException) + { + await UniTask.SwitchToMainThread(); + + if (world.Has(playerEntity)) + world.Remove(playerEntity); + + return McpToolResult.Error($"click_entity did not complete within {timeoutSec + COMPLETION_GRACE_SEC}s (is the simulation paused?)."); + } + + var json = new JObject + { + ["hit"] = result.Hit, + ["entityId"] = result.SceneEntityId, + ["crdtEntityId"] = result.CrdtEntityId, + }; + + if (result.FailureReason != null) + json["reason"] = result.FailureReason; + + if (result.Hit) + { + json["hitPoint"] = result.HitPoint.ToVector(); + json["distance"] = Math.Round(result.Distance, 2); + } + + if (result.HoverText != null) + json["hoverText"] = result.HoverText; + + if (result.BlockedByEntityId != null) + { + json["blockedByEntityId"] = result.BlockedByEntityId; + json["blockedByCrdtId"] = result.BlockedByCrdtId; + json["blockedByCollider"] = result.BlockedByColliderName; + } + + if (result.UpRayMissed) + json["upRayMissed"] = true; + + return McpToolResult.Json(json); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs.meta new file mode 100644 index 00000000000..8b943172934 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: d28c7ef4de5c429198677e5f3fb0776d \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/ClickUiTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ClickUiTool.cs new file mode 100644 index 00000000000..01621690d7a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ClickUiTool.cs @@ -0,0 +1,40 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + public class ClickUiTool : IMcpTool + { + public string Name => "click_ui"; + + public string Description => + "Click a real UI element, the way AltTester activates a live control. Identify it by a path from " + + "list_ui_elements or a plain element name (exact path, then exact name, then a loose match). uGUI Selectables " + + "receive a full pointer down/up/click; UI-Toolkit controls receive a navigation-submit (what activating a " + + "focused Button/Toggle does). Returns the element's post-click state with clicked:true. Read back with " + + "get_ui_state or screenshot to confirm the effect."; + + public JObject InputSchema => + McpJsonSchema.Object() + .String("element", "Path from list_ui_elements, or a plain element name.", required: true) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: false); + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + string element = arguments.GetString("element", string.Empty); + + if (string.IsNullOrWhiteSpace(element)) + return UniTask.FromResult(McpToolResult.Error("Provide element (a path from list_ui_elements or a plain element name).")); + + if (!UiAutomation.TryClick(element, out JObject result)) + return UniTask.FromResult(McpToolResult.Error($"No interactable UI element matched '{element}'. Call list_ui_elements to see current names/paths.")); + + return UniTask.FromResult(McpToolResult.Json(result)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ClickUiTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ClickUiTool.cs.meta new file mode 100644 index 00000000000..0243c025f7d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ClickUiTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 34157332362a4c71818f75a8c6ce016a \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs new file mode 100644 index 00000000000..1758384d2a4 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs @@ -0,0 +1,59 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using SceneRunner.Debugging; +using SceneRunner.Debugging.Hub; +using System.Text; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + public class GetEntityDetailsTool : IMcpTool + { + private const int MAX_CHARS = 8000; + private const string CURRENT_SCENE = "CURRENT"; + + private readonly IWorldInfoHub worldInfoHub; + + public string Name => "get_entity_details"; + + public string Description => + "Dump all components of one entity in the current scene's ECS world (ids come from list_scene_entities)."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Integer("entityId", "Entity id within the current scene world.", required: true) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public GetEntityDetailsTool(IWorldInfoHub worldInfoHub) + { + this.worldInfoHub = worldInfoHub; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetInt("entityId", out int entityId)) + return UniTask.FromResult(McpToolResult.Error("entityId is required.")); + + IWorldInfo? worldInfo = worldInfoHub.WorldInfo(CURRENT_SCENE); + + if (worldInfo == null) + return UniTask.FromResult(McpToolResult.Error("No scene world found at the current parcel.")); + + string dump = worldInfo.EntityComponentsInfo(entityId); + + if (dump.Length <= MAX_CHARS) + return UniTask.FromResult(McpToolResult.Text(dump)); + + var output = new StringBuilder(MAX_CHARS + 64); + output.Append(dump, 0, MAX_CHARS); + output.AppendLine(); + output.Append($"... output truncated at {MAX_CHARS}/{dump.Length} chars"); + + return UniTask.FromResult(McpToolResult.Text(output.ToString())); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs.meta new file mode 100644 index 00000000000..bd9c419498a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 8db7c68630cf74ae7a2fa4153b29875f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetNetworkLogTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetNetworkLogTool.cs new file mode 100644 index 00000000000..2a2d118f62d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetNetworkLogTool.cs @@ -0,0 +1,88 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.WebRequests.Analytics; +using Newtonsoft.Json.Linq; +using System; +using System.Collections.Generic; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + public class GetNetworkLogTool : IMcpTool + { + private const int DEFAULT_LIMIT = 50; + private const int MAX_LIMIT = 200; + + private readonly McpNetworkLogBuffer buffer; + + public string Name => "get_network_log"; + + public string Description => + "Read the client's recent HTTP activity — the same requests the Chrome DevTools Network domain would show. " + + "Covers every request that flows through the shared WebRequestController (content, realm, asset bundles, textures, " + + "audio, wearables/emotes, profiles); direct UnityWebRequest/HttpClient calls outside the controller are not captured. " + + "Each entry has {url, method, status, mimeType, sizeBytes, durationMs, failed, reason?} and a monotonic sequence " + + "number; pass the last seen sequence as sinceSeq to poll incrementally. Use failedOnly to see only transport failures " + + "and HTTP error statuses (>=400), or status for an exact HTTP status."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Integer("limit", "Maximum entries to return (newest win). Default 50, max 200.") + .Integer("sinceSeq", "Only return entries with a sequence number greater than this.") + .Boolean("failedOnly", "Only return failed requests: a transport failure or an HTTP status >= 400. Default false.") + .Integer("status", "Only return entries with this exact HTTP status code (e.g. 404).") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public GetNetworkLogTool(McpNetworkLogBuffer buffer) + { + this.buffer = buffer; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + int limit = Mathf.Clamp(arguments.GetInt("limit", DEFAULT_LIMIT), 1, MAX_LIMIT); + long sinceSeq = arguments.GetLong("sinceSeq", -1); + bool failedOnly = arguments.GetBool("failedOnly", false); + int status = arguments.GetInt("status", -1); + + var entries = new List(limit); + buffer.CopyTo(entries, sinceSeq, failedOnly, status, limit); + + var array = new JArray(); + + foreach (McpNetworkLogBuffer.Entry entry in entries) + { + var item = new JObject + { + ["seq"] = entry.Seq, + ["timestamp"] = entry.TimestampUtc.ToString("O"), + ["method"] = entry.Method, + ["status"] = entry.Status, + ["url"] = entry.Url, + ["mimeType"] = entry.MimeType, + ["sizeBytes"] = entry.SizeBytes, + ["durationMs"] = Math.Round(entry.DurationMs, 1), + ["failed"] = entry.Failed, + }; + + if (entry.FailureReason != null) + item["reason"] = entry.FailureReason; + + array.Add(item); + } + + var output = new JObject + { + ["latestSeq"] = buffer.LatestSeq, + ["returned"] = entries.Count, + ["entries"] = array, + }; + + return UniTask.FromResult(McpToolResult.Json(output)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetNetworkLogTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetNetworkLogTool.cs.meta new file mode 100644 index 00000000000..1d5332760a3 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetNetworkLogTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 026529c4bd044531a0249598bc61bba2 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs new file mode 100644 index 00000000000..a483f2b5a79 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs @@ -0,0 +1,89 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.CharacterMotion.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.Profiles; +using ECS.SceneLifeCycle.CurrentScene; +using Newtonsoft.Json.Linq; +using System.Threading; +using UnityEngine; +using Utility; + +namespace DCL.McpServer.Tools +{ + public class GetPlayerStateTool : IMcpTool + { + private readonly World world; + private readonly Entity playerEntity; + private readonly ExposedCameraData exposedCameraData; + private readonly ICurrentSceneInfo currentSceneInfo; + + public string Name => "get_player_state"; + + public string Description => + "Read the player's current world position, rotation, parcel, velocity and grounded state, the camera position, rotation and mode, " + + "and the wallet address — use the address to tell Explorer instances apart when several run at once."; + + public JObject InputSchema => McpJsonSchema.Object().Build(); + + public JObject? OutputSchema => + McpJsonSchema.Object() + .Object("position", JObjectExtensions.VectorSchema()) + .Object("rotationEuler", JObjectExtensions.VectorSchema()) + .Object("parcel", JObjectExtensions.ParcelSchema()) + .Object("velocity", JObjectExtensions.VectorSchema()) + .Boolean("isGrounded") + .Boolean("isPlayerStandingOnScene") + .String("address", "Wallet address of the logged-in player, or null when no profile is loaded.", nullable: true) + .Object("camera", McpJsonSchema.Object() + .Object("position", JObjectExtensions.VectorSchema()) + .Object("rotationEuler", JObjectExtensions.VectorSchema()) + .String("mode") + .Boolean("modeChangeAllowed") + .Boolean("pointerLocked")) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public GetPlayerStateTool(World world, Entity playerEntity, ExposedCameraData exposedCameraData, ICurrentSceneInfo currentSceneInfo) + { + this.world = world; + this.playerEntity = playerEntity; + this.exposedCameraData = exposedCameraData; + this.currentSceneInfo = currentSceneInfo; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + CharacterTransform characterTransform = world.Get(playerEntity); + Vector3 position = characterTransform.Position; + + world.TryGet(playerEntity, out CharacterRigidTransform? rigidTransform); + world.TryGet(playerEntity, out Profile? profile); + + var state = new JObject + { + ["position"] = position.ToVector(), + ["rotationEuler"] = characterTransform.Rotation.eulerAngles.ToVector(), + ["parcel"] = position.ToParcel().ToParcel(), + ["velocity"] = (rigidTransform?.MoveVelocity.Velocity ?? Vector3.zero).ToVector(), + ["isGrounded"] = rigidTransform?.IsGrounded ?? false, + ["isPlayerStandingOnScene"] = currentSceneInfo.IsPlayerStandingOnScene, + ["address"] = profile == null ? JValue.CreateNull() : profile.Compact.UserId, + ["camera"] = new JObject + { + ["position"] = exposedCameraData.WorldPosition.Value.ToVector(), + ["rotationEuler"] = exposedCameraData.WorldRotation.Value.eulerAngles.ToVector(), + ["mode"] = exposedCameraData.CameraMode.ToString(), + ["modeChangeAllowed"] = SetCameraModeTool.IsModeChangeAllowed(world), + ["pointerLocked"] = exposedCameraData.PointerIsLocked.Value, + }, + }; + + return UniTask.FromResult(McpToolResult.JsonWithStructured(state)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs.meta new file mode 100644 index 00000000000..c42e70d65de --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 0c34c7751a4c04382810aefd640df2c1 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs new file mode 100644 index 00000000000..da6be244a51 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs @@ -0,0 +1,57 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Collections.Generic; +using System.Text; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + public class GetSceneLogsTool : IMcpTool + { + private const int DEFAULT_LIMIT = 100; + private const int MAX_LIMIT = 500; + + private readonly SceneLogBuffer logBuffer; + + public string Name => "get_scene_logs"; + + public string Description => + "Read the scene's JavaScript console output (logs, warnings, errors and exceptions). Entries carry monotonic sequence numbers; " + + "pass the last seen sequence as sinceSeq to poll incrementally."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Integer("limit", "Maximum entries to return (newest win). Default 100.") + .String("severity", "Filter by severity. Default all.", enumValues: new[] { "all", "error" }) + .Integer("sinceSeq", "Only return entries with a sequence number greater than this.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public GetSceneLogsTool(SceneLogBuffer logBuffer) + { + this.logBuffer = logBuffer; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + int limit = Mathf.Clamp(arguments.GetInt("limit", DEFAULT_LIMIT), 1, MAX_LIMIT); + bool errorsOnly = arguments.GetString("severity", "all") == "error"; + long sinceSeq = arguments.GetLong("sinceSeq", -1); + + var entries = new List(limit); + logBuffer.CopyTo(entries, sinceSeq, errorsOnly, limit); + + var output = new StringBuilder(); + output.AppendLine($"latestSeq={logBuffer.LatestSeq} returned={entries.Count}"); + + foreach (SceneLogBuffer.Entry entry in entries) + output.AppendLine($"#{entry.Seq} {entry.Message}"); + + return UniTask.FromResult(McpToolResult.Text(output.ToString())); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs.meta new file mode 100644 index 00000000000..7e2d718b638 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 0ef4e7ee221c646b1ad9e254b9326614 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs new file mode 100644 index 00000000000..274630831ac --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs @@ -0,0 +1,84 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.RealmNavigation; +using ECS.SceneLifeCycle; +using ECS.SceneLifeCycle.CurrentScene; +using Newtonsoft.Json.Linq; +using SceneRunner.Scene; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + public class GetSceneStateTool : IMcpTool + { + private readonly IScenesCache scenesCache; + private readonly ICurrentSceneInfo currentSceneInfo; + private readonly ILoadingStatus loadingStatus; + private readonly bool localSceneDevelopment; + + public string Name => "get_scene_state"; + + public string Description => + "Read the state of the scene at the player's current parcel: name, base parcel, runtime state (including JavaScript/ECS errors), " + + "readiness, asset loading progress and the global loading-screen stage. Call this after teleporting or reloading before interacting."; + + public JObject InputSchema => McpJsonSchema.Object().Build(); + + public JObject? OutputSchema => + McpJsonSchema.Object() + .Object("currentParcel", JObjectExtensions.ParcelSchema()) + .String("loadingStage") + .Boolean("loadingScreenOn") + .Boolean("localSceneDevelopment") + .Object("scene", McpJsonSchema.Object() + .String("name") + .Object("baseParcel", JObjectExtensions.ParcelSchema()) + .String("sdkVersion", "SDK version reported by the scene, or null when unknown.", nullable: true) + .String("state") + .Boolean("isReady") + .Boolean("assetsLoadingConcluded") + .String("runningStatus"), + "The scene at the player's current parcel, or null when no scene is loaded there.", nullable: true) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public GetSceneStateTool(IScenesCache scenesCache, ICurrentSceneInfo currentSceneInfo, ILoadingStatus loadingStatus, bool localSceneDevelopment) + { + this.scenesCache = scenesCache; + this.currentSceneInfo = currentSceneInfo; + this.loadingStatus = loadingStatus; + this.localSceneDevelopment = localSceneDevelopment; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + Vector2Int currentParcel = scenesCache.CurrentParcel.Value; + ISceneFacade? scene = scenesCache.CurrentScene.Value; + + var state = new JObject + { + ["currentParcel"] = currentParcel.ToParcel(), + ["loadingStage"] = loadingStatus.CurrentStage.Value.ToString(), + ["loadingScreenOn"] = loadingStatus.IsLoadingScreenOn(), + ["localSceneDevelopment"] = localSceneDevelopment, + ["scene"] = scene == null + ? JValue.CreateNull() + : new JObject + { + ["name"] = scene.Info.Name, + ["baseParcel"] = scene.Info.BaseParcel.ToParcel(), + ["sdkVersion"] = scene.Info.SdkVersion, + ["state"] = scene.SceneStateProvider.State.Value().ToString(), + ["isReady"] = scene.IsSceneReady(), + ["assetsLoadingConcluded"] = scene.SceneData.SceneLoadingConcluded, + ["runningStatus"] = currentSceneInfo.SceneStatus.Value?.ToString() ?? "Unknown", + }, + }; + + return UniTask.FromResult(McpToolResult.JsonWithStructured(state)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs.meta new file mode 100644 index 00000000000..2fdf38209b6 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: f9c6cd0d753384d63b9e283c8cd1eabb \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetUiStateTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetUiStateTool.cs new file mode 100644 index 00000000000..3e87ba52b8d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetUiStateTool.cs @@ -0,0 +1,38 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + public class GetUiStateTool : IMcpTool + { + public string Name => "get_ui_state"; + + public string Description => + "Read one UI element's current state: {path, name, type, system, interactable, visible, text?}. Identify it by " + + "a path from list_ui_elements or a plain element name (an exact path wins, then an exact name, then a loose " + + "path-suffix/contains match). Use it to confirm a toggle flipped or a field's text before or after click_ui."; + + public JObject InputSchema => + McpJsonSchema.Object() + .String("element", "Path from list_ui_elements, or a plain element name.", required: true) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + string element = arguments.GetString("element", string.Empty); + + if (string.IsNullOrWhiteSpace(element)) + return UniTask.FromResult(McpToolResult.Error("Provide element (a path from list_ui_elements or a plain element name).")); + + if (!UiAutomation.TryGetState(element, out JObject state)) + return UniTask.FromResult(McpToolResult.Error($"No interactable UI element matched '{element}'. Call list_ui_elements to see current names/paths.")); + + return UniTask.FromResult(McpToolResult.Json(state)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetUiStateTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetUiStateTool.cs.meta new file mode 100644 index 00000000000..f039e563b02 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetUiStateTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 316c8c77139d402e85a5b77c0954250d \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs new file mode 100644 index 00000000000..3ff757a6f68 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs @@ -0,0 +1,92 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using SceneRunner.Debugging; +using SceneRunner.Debugging.Hub; +using System.Collections.Generic; +using System.Text; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + /// + /// Lists the entity ids of the current scene's ECS world through the same unsynchronized + /// path the existing debug tooling uses. + /// + public class ListSceneEntitiesTool : IMcpTool + { + private const int DEFAULT_LIMIT = 200; + private const int MAX_LIMIT = 2000; + private const string CURRENT_SCENE = "CURRENT"; + + private readonly IWorldInfoHub worldInfoHub; + + public string Name => "list_scene_entities"; + + public string Description => + "List the ECS entity ids of the scene at the player's current parcel. Feed an id into get_entity_details to inspect its components."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Integer("limit", "Maximum ids to return. Default 200.") + .Build(); + + public JObject? OutputSchema => + McpJsonSchema.Object() + .Integer("total") + .Integer("returned") + .Boolean("truncated") + .IntegerArray("entityIds") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public ListSceneEntitiesTool(IWorldInfoHub worldInfoHub) + { + this.worldInfoHub = worldInfoHub; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + int limit = Mathf.Clamp(arguments.GetInt("limit", DEFAULT_LIMIT), 1, MAX_LIMIT); + + IWorldInfo? worldInfo = worldInfoHub.WorldInfo(CURRENT_SCENE); + + if (worldInfo == null) + return UniTask.FromResult(McpToolResult.Error("No scene world found at the current parcel.")); + + IReadOnlyList entityIds = worldInfo.EntityIds(); + int returned = Mathf.Min(limit, entityIds.Count); + bool truncated = returned < entityIds.Count; + + var ids = new JArray(); + var output = new StringBuilder(); + output.AppendLine($"total={entityIds.Count} returned={returned}"); + + for (var i = 0; i < entityIds.Count && i < limit; i++) + { + ids.Add(entityIds[i]); + output.Append(entityIds[i]); + output.Append(i < entityIds.Count - 1 && i < limit - 1 ? ", " : string.Empty); + } + + if (truncated) + { + output.AppendLine(); + output.Append($"{returned} of {entityIds.Count} shown; raise limit (max {MAX_LIMIT}) to see the rest."); + } + + var structured = new JObject + { + ["total"] = entityIds.Count, + ["returned"] = returned, + ["truncated"] = truncated, + ["entityIds"] = ids, + }; + + return UniTask.FromResult(McpToolResult.TextWithStructured(output.ToString(), structured)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs.meta new file mode 100644 index 00000000000..0e7a23b95e6 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 76659a159375e46aabcb920885ee30a3 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/ListUiElementsTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ListUiElementsTool.cs new file mode 100644 index 00000000000..cd895c53952 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ListUiElementsTool.cs @@ -0,0 +1,42 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + public class ListUiElementsTool : IMcpTool + { + public string Name => "list_ui_elements"; + + public string Description => + "List the client's interactable UI elements, the way AltTester queries the live UI hierarchy. Covers both UI " + + "systems: uGUI Selectables (Button, Toggle, InputField, Slider, Dropdown) and UI-Toolkit focusable elements " + + "under each active UIDocument. Each entry is {path, name, type, system (ugui|uitk), interactable, visible, " + + "text?}. Pass the returned path (or a plain element name) to get_ui_state or click_ui. Optional nameFilter " + + "keeps only elements whose name or path contains it (case-insensitive)."; + + public JObject InputSchema => + McpJsonSchema.Object() + .String("nameFilter", "Case-insensitive substring; keep only elements whose name or path contains it.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + string? filter = arguments["nameFilter"]?.Type == JTokenType.String ? arguments["nameFilter"]!.Value() : null; + + JArray elements = UiAutomation.Enumerate(filter); + + var output = new JObject + { + ["count"] = elements.Count, + ["elements"] = elements, + }; + + return UniTask.FromResult(McpToolResult.Json(output)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ListUiElementsTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ListUiElementsTool.cs.meta new file mode 100644 index 00000000000..919f6ad6b84 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ListUiElementsTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 3819b3f09f6c4c25b8b66712cd18dd6f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs b/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs new file mode 100644 index 00000000000..917737c93f8 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs @@ -0,0 +1,63 @@ +using Arch.Core; +using CrdtEcsBridge.RestrictedActions; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + public class LookAtTool : IMcpTool + { + private readonly IGlobalWorldActions globalWorldActions; + private readonly World world; + private readonly Entity playerEntity; + private readonly ExposedCameraData exposedCameraData; + + public string Name => "look_at"; + + public string Description => + "Rotate the camera to look at a world-space point (x,y,z in meters). Useful to center something on screen before a screenshot."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Number("x", required: true) + .Number("y", required: true) + .Number("z", required: true) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public LookAtTool(IGlobalWorldActions globalWorldActions, World world, Entity playerEntity, ExposedCameraData exposedCameraData) + { + this.globalWorldActions = globalWorldActions; + this.world = world; + this.playerEntity = playerEntity; + this.exposedCameraData = exposedCameraData; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetFloat("x", out float x) || !arguments.TryGetFloat("y", out float y) || !arguments.TryGetFloat("z", out float z)) + return McpToolResult.Error("x, y and z world coordinates are required."); + + Vector3 playerPosition = world.Get(playerEntity).Position; + globalWorldActions.RotateCamera(new Vector3(x, y, z), playerPosition); + + // Let the Cinemachine systems apply the look-at intent before reading the camera back. + await UniTask.DelayFrame(3, cancellationToken: ct); + + var result = new JObject + { + ["cameraPosition"] = exposedCameraData.WorldPosition.Value.ToVector(), + ["cameraRotationEuler"] = exposedCameraData.WorldRotation.Value.eulerAngles.ToVector(), + }; + + return McpToolResult.Json(result); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs.meta new file mode 100644 index 00000000000..083259d660a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: f48ec52210bd7465e84233aa7ae83b13 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs b/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs new file mode 100644 index 00000000000..bc3e199e15d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs @@ -0,0 +1,84 @@ +using Arch.Core; +using CrdtEcsBridge.RestrictedActions; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; +using UnityEngine; +using Utility; + +namespace DCL.McpServer.Tools +{ + public class MoveToTool : IMcpTool + { + private const float MAX_DURATION_SEC = 30f; + private const float COMPLETION_GRACE_SEC = 5f; + + private readonly IGlobalWorldActions globalWorldActions; + private readonly World world; + private readonly Entity playerEntity; + + public string Name => "move_to"; + + public string Description => + "Move the player to a world-space position (x,y,z in meters; one parcel is 16x16m). Instant by default, or smooth over durationSec. " + + "Optionally face a look-at target on arrival. For crossing to another scene prefer the teleport tool."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Number("x", required: true) + .Number("y", required: true) + .Number("z", required: true) + .Number("lookAtX") + .Number("lookAtY") + .Number("lookAtZ") + .Number("durationSec", "Seconds to move over; 0 (default) teleports instantly.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public MoveToTool(IGlobalWorldActions globalWorldActions, World world, Entity playerEntity) + { + this.globalWorldActions = globalWorldActions; + this.world = world; + this.playerEntity = playerEntity; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetFloat("x", out float x) || !arguments.TryGetFloat("y", out float y) || !arguments.TryGetFloat("z", out float z)) + return McpToolResult.Error("x, y and z world coordinates are required."); + + Vector3? lookAtTarget = null; + + if (arguments.TryGetFloat("lookAtX", out float lookAtX) && arguments.TryGetFloat("lookAtY", out float lookAtY) && arguments.TryGetFloat("lookAtZ", out float lookAtZ)) + lookAtTarget = new Vector3(lookAtX, lookAtY, lookAtZ); + + float durationSec = Mathf.Clamp(arguments.GetFloat("durationSec", 0f), 0f, MAX_DURATION_SEC); + var targetPosition = new Vector3(x, y, z); + + try + { + await globalWorldActions.MoveAndRotatePlayerAsync(targetPosition, lookAtTarget, lookAtTarget, durationSec, ct) + .Timeout(TimeSpan.FromSeconds(durationSec + COMPLETION_GRACE_SEC)); + } + catch (TimeoutException) { return McpToolResult.Error($"move_to did not complete within {durationSec + COMPLETION_GRACE_SEC}s."); } + + // Give the teleport/rotation systems a couple of frames to apply the intents before reading back. + await UniTask.DelayFrame(2, cancellationToken: ct); + + Vector3 finalPosition = world.Get(playerEntity).Position; + + var result = new JObject + { + ["position"] = finalPosition.ToVector(), + ["parcel"] = finalPosition.ToParcel().ToParcel(), + }; + + return McpToolResult.Json(result); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs.meta new file mode 100644 index 00000000000..789ec935f21 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: e497faa837a32493bb1f287241cb4661 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs new file mode 100644 index 00000000000..d39547ba799 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs @@ -0,0 +1,86 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.CharacterMotion.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.SkyBox.Components; +using ECS.SceneLifeCycle; +using Newtonsoft.Json.Linq; +using SceneRunner.Scene; +using System; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + /// + /// Reloads the scene at the player's parcel, freezing character motion and the skybox during the reload + /// exactly like the local-scene-development hot reload does. + /// + public class ReloadSceneTool : IMcpTool + { + private const float MIN_TIMEOUT_SEC = 5f; + private const float MAX_TIMEOUT_SEC = 120f; + private const float DEFAULT_TIMEOUT_SEC = 15f; + + private readonly ECSReloadScene reloadScene; + private readonly IScenesCache scenesCache; + private readonly World world; + private readonly Entity playerEntity; + private readonly Entity skyboxEntity; + + public string Name => "reload_scene"; + + public string Description => + "Reload the scene at the player's current parcel and wait for it to restart. Use after editing scene code " + + "when hot reload didn't trigger, or to reset scene state before a test run."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Number("timeoutSec", "Maximum seconds to wait for the reload. Default 15.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: true, idempotent: false); + + public ReloadSceneTool(ECSReloadScene reloadScene, IScenesCache scenesCache, World world, Entity playerEntity, Entity skyboxEntity) + { + this.reloadScene = reloadScene; + this.scenesCache = scenesCache; + this.world = world; + this.playerEntity = playerEntity; + this.skyboxEntity = skyboxEntity; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + float timeoutSec = Mathf.Clamp(arguments.GetFloat("timeoutSec", DEFAULT_TIMEOUT_SEC), MIN_TIMEOUT_SEC, MAX_TIMEOUT_SEC); + + if (scenesCache.CurrentScene.Value == null) + return McpToolResult.Error("There is no scene at the current parcel to reload."); + + try + { + world.AddOrGet(playerEntity, new StopCharacterMotion()); + world.AddOrGet(skyboxEntity, new PauseSkyboxTimeUpdate()); + + ISceneFacade? reloadedScene = await reloadScene.TryReloadSceneAsync(ct) + .Timeout(TimeSpan.FromSeconds(timeoutSec)); + + if (reloadedScene == null) + return McpToolResult.Error("Reload failed: no reloadable scene was found at the current parcel."); + } + catch (TimeoutException) + { + return McpToolResult.Error($"Scene reload did not complete within {timeoutSec}s. Check get_scene_state and get_scene_logs."); + } + finally + { + await UniTask.SwitchToMainThread(); + world.Remove(playerEntity); + world.Remove(skyboxEntity); + } + + return McpToolResult.Text("Scene reloaded. Call get_scene_state to confirm readiness and get_scene_logs for startup output."); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs.meta new file mode 100644 index 00000000000..218fe29e9eb --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 1ca31906893da454f85da8932ebb45c9 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs new file mode 100644 index 00000000000..ee3a6ec33dd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs @@ -0,0 +1,255 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System; +using System.Collections; +using System.Threading; +using Unity.Collections; +using UnityEngine; +using UnityEngine.Experimental.Rendering; +using UnityEngine.Rendering; +using Utility; +using Utility.Multithreading; +using Object = UnityEngine.Object; + +namespace DCL.McpServer.Tools +{ + /// + /// Captures the current frame (back buffer including UI, or a world-only camera render), + /// downscales it and returns it as an MCP image content block. Textures never accumulate: + /// temporaries are released per call and the ReadPixels fallback reuses one persistent buffer. + /// + public class ScreenshotTool : IMcpTool, IDisposable + { + private const int DEFAULT_MAX_WIDTH = 1280; + private const int MIN_WIDTH = 64; + private const int MAX_WIDTH = 1920; + private const int JPG_QUALITY = 75; + + private readonly ICoroutineRunner coroutineRunner; + private readonly World world; + private readonly Entity playerEntity; + + // Reused across calls by the ReadPixels fallback so repeated captures don't allocate new textures. + private Texture2D? readPixelsBuffer; + + // 1 while a capture is running; concurrent requests are rejected so one set of buffers suffices. + private int captureGate; + + public string Name => "screenshot"; + + public string Description => + "Capture a screenshot of what the player currently sees in the Explorer, including scene UI. " + + "Use worldOnly to exclude all UI overlays. Returns a downscaled image plus a caption with the capture context."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Integer("maxWidth", "Maximum output width in pixels (aspect ratio preserved). Default 1280.") + .String("quality", "Output encoding. Default jpg.", enumValues: new[] { "jpg", "png" }) + .Boolean("worldOnly", "Render only the 3D world through the main camera, excluding UI. Default false.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public ScreenshotTool(ICoroutineRunner coroutineRunner, World world, Entity playerEntity) + { + this.coroutineRunner = coroutineRunner; + this.world = world; + this.playerEntity = playerEntity; + } + + public void Dispose() + { + if (readPixelsBuffer != null) + Object.Destroy(readPixelsBuffer); + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + int maxWidth = Mathf.Clamp(arguments.GetInt("maxWidth", DEFAULT_MAX_WIDTH), MIN_WIDTH, MAX_WIDTH); + bool asPng = arguments.GetString("quality", "jpg") == "png"; + bool worldOnly = arguments.GetBool("worldOnly", false); + + if (Interlocked.CompareExchange(ref captureGate, 1, 0) != 0) + return McpToolResult.Error("Another screenshot capture is already in progress; retry when it completes."); + + try { return await CaptureAsync(maxWidth, asPng, worldOnly, ct); } + finally { Interlocked.Exchange(ref captureGate, 0); } + } + + private async UniTask CaptureAsync(int maxWidth, bool asPng, bool worldOnly, CancellationToken ct) + { + Texture2D? backbufferCopy = null; + RenderTexture? worldRender = null; + RenderTexture? downscaled = null; + + try + { + int sourceWidth; + int sourceHeight; + + if (worldOnly) + { + Camera camera = world.CacheCamera().GetCameraComponent(world).Camera; + + // URP replaces the camera's internal color buffer with the target texture's descriptor + // (CreateRenderTextureDescriptor), so an LDR target silently downgrades the whole render + // to 8-bit and clamps emissives to 1.0, starving bloom and other HDR-dependent post effects. + // An HDR temporary keeps the pipeline HDR end-to-end; the downscale blit into the sRGB + // descriptor below performs the linear-to-sRGB conversion. + worldRender = RenderTexture.GetTemporary(camera.pixelWidth, camera.pixelHeight, 24, RenderTextureFormat.DefaultHDR); + + // Camera.Render() is unsupported under URP: redirect the camera's output into the + // render texture for exactly one frame instead, then restore it. + RenderTexture? previousTarget = camera.targetTexture; + camera.targetTexture = worldRender; + + try { await WaitForEndOfFrameAsync(ct); } + finally + { + await UniTask.SwitchToMainThread(); + camera.targetTexture = previousTarget; + } + + sourceWidth = worldRender.width; + sourceHeight = worldRender.height; + } + else + { + backbufferCopy = await CaptureBackbufferAsync(ct); + + if (backbufferCopy == null) + return McpToolResult.Error("Back buffer capture failed."); + + sourceWidth = backbufferCopy.width; + sourceHeight = backbufferCopy.height; + } + + int width = Mathf.Min(maxWidth, sourceWidth); + int height = Mathf.Max(1, Mathf.RoundToInt((float)sourceHeight * width / sourceWidth)); + + var descriptor = new RenderTextureDescriptor(width, height) + { + graphicsFormat = OutputGraphicsFormat(), sRGB = true, msaaSamples = 1, depthBufferBits = 0, + mipCount = 1, useMipMap = false, + }; + + downscaled = RenderTexture.GetTemporary(descriptor); + + if (backbufferCopy != null) + { + Graphics.Blit(backbufferCopy, downscaled); + + // The back buffer copy is a fresh Texture2D allocated by Unity on every capture: destroy it as soon as it's blitted. + Object.Destroy(backbufferCopy); + backbufferCopy = null; + } + else + { + Graphics.Blit(worldRender, downscaled); + RenderTexture.ReleaseTemporary(worldRender); + worldRender = null; + } + + byte[] encoded; + string mimeType; + + AsyncGPUReadbackRequest readback = await AsyncGPUReadback.Request(downscaled).WithCancellation(ct); + + if (readback.hasError) { (encoded, mimeType) = EncodeViaReadPixels(downscaled, asPng); } + else + { + NativeArray rawPixels = readback.GetData(); + + using (NativeArray encodedNative = asPng + ? ImageConversion.EncodeNativeArrayToPNG(rawPixels, downscaled.graphicsFormat, (uint)width, (uint)height) + : ImageConversion.EncodeNativeArrayToJPG(rawPixels, downscaled.graphicsFormat, (uint)width, (uint)height, 0, JPG_QUALITY)) + encoded = encodedNative.ToArray(); + + mimeType = asPng ? "image/png" : "image/jpeg"; + } + + Vector2Int parcel = world.Get(playerEntity).Position.ToParcel(); + var caption = $"{width}x{height} {(worldOnly ? "world-only" : "full-view")} capture at parcel ({parcel.x},{parcel.y})"; + + // Base64 conversion of the encoded image happens off the main thread. + await DCLTask.SwitchToThreadPool(); + return McpToolResult.Image(encoded, mimeType, caption); + } + finally + { + if (backbufferCopy != null || worldRender != null || downscaled != null) + { + await UniTask.SwitchToMainThread(); + + if (backbufferCopy != null) Object.Destroy(backbufferCopy); + if (worldRender != null) RenderTexture.ReleaseTemporary(worldRender); + if (downscaled != null) RenderTexture.ReleaseTemporary(downscaled); + } + } + } + + private UniTask CaptureBackbufferAsync(CancellationToken ct) + { + var completion = new UniTaskCompletionSource(); + coroutineRunner.StartCoroutine(CaptureBackbufferCoroutine(completion)); + return completion.Task.AttachExternalCancellation(ct); + } + + private UniTask WaitForEndOfFrameAsync(CancellationToken ct) + { + var completion = new UniTaskCompletionSource(); + coroutineRunner.StartCoroutine(SignalEndOfFrameCoroutine(completion)); + return completion.Task.AttachExternalCancellation(ct); + } + + private static IEnumerator SignalEndOfFrameCoroutine(UniTaskCompletionSource completion) + { + yield return GameObjectExtensions.WAIT_FOR_END_OF_FRAME; + completion.TrySetResult(); + } + + private static IEnumerator CaptureBackbufferCoroutine(UniTaskCompletionSource completion) + { + // The back buffer is only complete (UI included) at the very end of the frame. + yield return GameObjectExtensions.WAIT_FOR_END_OF_FRAME; + + Texture2D texture = ScreenCapture.CaptureScreenshotAsTexture(); + + if (!completion.TrySetResult(texture)) + Object.Destroy(texture); + } + + private (byte[] bytes, string mimeType) EncodeViaReadPixels(RenderTexture source, bool asPng) + { + if (readPixelsBuffer == null) + readPixelsBuffer = new Texture2D(source.width, source.height, TextureFormat.RGBA32, false); + else if (readPixelsBuffer.width != source.width || readPixelsBuffer.height != source.height) + readPixelsBuffer.Reinitialize(source.width, source.height, readPixelsBuffer.graphicsFormat, false); + + RenderTexture? previousActive = RenderTexture.active; + RenderTexture.active = source; + readPixelsBuffer.ReadPixels(new Rect(0, 0, source.width, source.height), 0, 0); + readPixelsBuffer.Apply(); + RenderTexture.active = previousActive; + + return asPng + ? (readPixelsBuffer.EncodeToPNG(), "image/png") + : (readPixelsBuffer.EncodeToJPG(JPG_QUALITY), "image/jpeg"); + } + + private static GraphicsFormat OutputGraphicsFormat() + { + var preferred = GraphicsFormat.R8G8B8A8_SRGB; + + if (SystemInfo.IsFormatSupported(preferred, GraphicsFormatUsage.Render)) + return preferred; + + return SystemInfo.GetCompatibleFormat(preferred, GraphicsFormatUsage.Render); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs.meta new file mode 100644 index 00000000000..7c72a9a5cc9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 781cfdc63c1c5405aa87fa01a82e4a4f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs b/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs new file mode 100644 index 00000000000..f130a7a48f9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs @@ -0,0 +1,54 @@ +using Cysharp.Threading.Tasks; +using DCL.Chat.History; +using DCL.Chat.MessageBus; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + /// + /// Sends a message to the Nearby chat channel. Messages starting with '/' run through the chat + /// command pipeline, so agents can drive commands the same way a user typing in chat would. + /// + public class SendChatTool : IMcpTool + { + private const int MAX_MESSAGE_LENGTH = 500; + + private readonly IChatMessagesBus chatMessagesBus; + + public string Name => "send_chat"; + + public string Description => + "Send a message to the Nearby chat channel. Messages starting with '/' run chat commands " + + "(e.g. /goto x,y, /reload, /help); command output appears in chat and scene logs."; + + public JObject InputSchema => + McpJsonSchema.Object() + .String("message", "The chat message or /command to send.", required: true) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: false); + + public SendChatTool(IChatMessagesBus chatMessagesBus) + { + this.chatMessagesBus = chatMessagesBus; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + string message = arguments.GetString("message", string.Empty); + + if (string.IsNullOrWhiteSpace(message)) + return UniTask.FromResult(McpToolResult.Error("message is required.")); + + if (message.Length > MAX_MESSAGE_LENGTH) + return UniTask.FromResult(McpToolResult.Error($"message exceeds the {MAX_MESSAGE_LENGTH} character limit.")); + + chatMessagesBus.SendWithUtcNowTimestamp(ChatChannel.NEARBY_CHANNEL, message, ChatMessageOrigin.CHAT); + + return UniTask.FromResult(McpToolResult.Text($"Sent to Nearby: {message}")); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs.meta new file mode 100644 index 00000000000..7da0103ce84 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 9e754e2afb7734afb8f6b5e0f735b168 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs b/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs new file mode 100644 index 00000000000..7bc2d75741b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs @@ -0,0 +1,107 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.CharacterCamera; +using DCL.InWorldCamera; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using ECS.Abstract; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + /// + /// Switches the camera mode by writing (the same pattern scene systems use; + /// ControlCinemachineVirtualCameraSystem applies it next frame). The direct write bypasses the user-input + /// gates, so this tool re-checks them itself and refuses when a scene holds the camera. + /// + public class SetCameraModeTool : IMcpTool + { + private readonly World world; + private readonly ExposedCameraData exposedCameraData; + + public string Name => "set_camera_mode"; + + public string Description => + "Switch the player camera mode (first_person, third_person, drone, or the free-fly camera), like a user pressing the camera key. " + + "Refuses with an explanation when the scene locks the mode (CameraModeArea, scene virtual camera, photo camera). " + + "Any player movement drops free back to third_person."; + + public JObject InputSchema => + McpJsonSchema.Object() + .String("mode", "Target camera mode.", enumValues: new[] { "first_person", "third_person", "drone", "free" }, required: true) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public SetCameraModeTool(World world, ExposedCameraData exposedCameraData) + { + this.world = world; + this.exposedCameraData = exposedCameraData; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + CameraMode targetMode; + + switch (arguments.GetString("mode", string.Empty)) + { + case "first_person": targetMode = CameraMode.FirstPerson; break; + case "third_person": targetMode = CameraMode.ThirdPerson; break; + case "drone": targetMode = CameraMode.DroneView; break; + case "free": targetMode = CameraMode.Free; break; + default: return McpToolResult.Error("mode must be one of: first_person, third_person, drone, free."); + } + + string? blockReason = TrySwitchMode(world, targetMode, out CameraMode previousMode); + + if (blockReason != null) + return McpToolResult.Error(blockReason); + + // Let ControlCinemachineVirtualCameraSystem activate the matching virtual camera before reading back. + await UniTask.DelayFrame(2, cancellationToken: ct); + + var result = new JObject + { + ["requestedMode"] = targetMode.ToString(), + ["currentMode"] = exposedCameraData.CameraMode.ToString(), + ["previousMode"] = previousMode.ToString(), + }; + + return McpToolResult.Json(result); + } + + /// + /// Same gates ControlCinemachineVirtualCameraSystem.HandleCameraInput applies to user input. + /// Main thread only. + /// + internal static bool IsModeChangeAllowed(World world) + { + SingleInstanceEntity cameraEntity = world.CacheCamera(); + ref readonly CameraComponent camera = ref cameraEntity.GetCameraComponent(world); + + return camera.Mode != CameraMode.SDKCamera + && camera.CameraInputChangeEnabled + && !world.Has(cameraEntity); + } + + internal static string? TrySwitchMode(World world, CameraMode targetMode, out CameraMode previousMode) + { + SingleInstanceEntity cameraEntity = world.CacheCamera(); + ref CameraComponent camera = ref cameraEntity.GetCameraComponent(world); + previousMode = camera.Mode; + + if (camera.Mode == CameraMode.SDKCamera) + return "A scene virtual camera controls the view right now (mode SDKCamera); the mode cannot change until the scene releases it."; + + if (!camera.CameraInputChangeEnabled) + return $"Camera mode is locked by the scene (CameraModeArea; current mode: {camera.Mode}). Leave the area to change modes."; + + if (world.Has(cameraEntity)) + return "The in-world photo camera is active; close it before changing the camera mode."; + + camera.Mode = targetMode; + return null; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs.meta new file mode 100644 index 00000000000..f5b3533b96f --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 297288bb84c8d4cd1af0af018a8a4123 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs b/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs new file mode 100644 index 00000000000..47ad5d72640 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs @@ -0,0 +1,139 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.CharacterCamera.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using ECS.Abstract; +using Newtonsoft.Json.Linq; +using System.Threading; +using UnityEngine; +using Utility.Arch; + +namespace DCL.McpServer.Tools +{ + /// + /// Places the free camera at an absolute world position, optionally aiming it and setting its FOV. + /// Enters Free mode when needed (same scene-lock gates as ) and waits + /// for the Cinemachine blend to reach the target before reporting the actual pose. + /// + public class SetCameraPoseTool : IMcpTool + { + private const float DEFAULT_TIMEOUT_SEC = 5f; + private const float MIN_TIMEOUT_SEC = 0.5f; + private const float MAX_TIMEOUT_SEC = 15f; + private const float SETTLE_EPSILON = 0.1f; + private const int POLL_INTERVAL_MS = 100; + private const float MIN_FOV = 10f; + private const float MAX_FOV = 120f; + + private readonly World world; + private readonly Entity playerEntity; + private readonly ExposedCameraData exposedCameraData; + + public string Name => "set_camera_pose"; + + public string Description => + "Place the free camera at an absolute world position, optionally aiming it at a point and setting its field of view. " + + "Enters the free camera mode if needed (refuses with the reason when the scene locks the camera). The camera stays " + + "put while the player moves; restore a player-following view with set_camera_mode third_person."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Number("x", "Camera world position.", required: true) + .Number("y", required: true) + .Number("z", required: true) + .Number("lookAtX", "Optional world point to aim at (all three lookAt components required together).") + .Number("lookAtY") + .Number("lookAtZ") + .Number("fov", "Optional vertical field of view in degrees (10-120).") + .Number("timeoutSec", "Seconds to wait for the camera to settle at the target. Default 5, max 15.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public SetCameraPoseTool(World world, Entity playerEntity, ExposedCameraData exposedCameraData) + { + this.world = world; + this.playerEntity = playerEntity; + this.exposedCameraData = exposedCameraData; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetFloat("x", out float x) || !arguments.TryGetFloat("y", out float y) || !arguments.TryGetFloat("z", out float z)) + return McpToolResult.Error("x, y and z world coordinates for the camera position are required."); + + bool hasLookAtX = arguments.TryGetFloat("lookAtX", out float lookAtX); + bool hasLookAtY = arguments.TryGetFloat("lookAtY", out float lookAtY); + bool hasLookAtZ = arguments.TryGetFloat("lookAtZ", out float lookAtZ); + bool hasLookAt = hasLookAtX && hasLookAtY && hasLookAtZ; + + if ((hasLookAtX || hasLookAtY || hasLookAtZ) && !hasLookAt) + return McpToolResult.Error("lookAtX, lookAtY and lookAtZ must be provided together."); + + float? fov = null; + + if (arguments.TryGetFloat("fov", out float fovValue)) + fov = Mathf.Clamp(fovValue, MIN_FOV, MAX_FOV); + + float timeoutSec = Mathf.Clamp(arguments.GetFloat("timeoutSec", DEFAULT_TIMEOUT_SEC), MIN_TIMEOUT_SEC, MAX_TIMEOUT_SEC); + var targetPosition = new Vector3(x, y, z); + + SingleInstanceEntity cameraEntity = world.CacheCamera(); + + if (cameraEntity.GetCameraComponent(world).Mode != CameraMode.Free) + { + string? blockReason = SetCameraModeTool.TrySwitchMode(world, CameraMode.Free, out CameraMode _); + + if (blockReason != null) + return McpToolResult.Error(blockReason); + + // Let ControlCinemachineVirtualCameraSystem activate the free vcam (and apply its default + // spawn position, which the pose below overrides). + await UniTask.DelayFrame(2, cancellationToken: ct); + } + + if (!world.TryGet(cameraEntity, out ICinemachinePreset? cinemachinePreset) || cinemachinePreset == null) + return McpToolResult.Error("The camera rig is not initialized yet."); + + cinemachinePreset.ForceFreeCameraPose(targetPosition, fov); + + if (hasLookAt) + { + Vector3 playerPosition = world.Get(playerEntity).Position; + world.AddOrSet(cameraEntity, new CameraLookAtIntent(new Vector3(lookAtX, lookAtY, lookAtZ), playerPosition)); + } + + // Entering Free blends the output camera toward the vcam over a couple of seconds; when the + // mode was already Free the pose applies instantly and the first poll succeeds. + var settled = false; + float deadline = UnityEngine.Time.realtimeSinceStartup + timeoutSec; + + while (UnityEngine.Time.realtimeSinceStartup < deadline) + { + if (Vector3.Distance(exposedCameraData.WorldPosition.Value, targetPosition) <= SETTLE_EPSILON) + { + settled = true; + break; + } + + await UniTask.Delay(POLL_INTERVAL_MS, cancellationToken: ct); + } + + // Let the look-at intent apply before reading the rotation back. + await UniTask.DelayFrame(2, cancellationToken: ct); + + var result = new JObject + { + ["position"] = exposedCameraData.WorldPosition.Value.ToVector(), + ["rotationEuler"] = exposedCameraData.WorldRotation.Value.eulerAngles.ToVector(), + ["mode"] = exposedCameraData.CameraMode.ToString(), + ["settled"] = settled, + }; + + return McpToolResult.Json(result); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs.meta new file mode 100644 index 00000000000..5ad7c5b677f --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 5d66df6294124c629fa19535681000e1 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs b/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs new file mode 100644 index 00000000000..14fdea46980 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs @@ -0,0 +1,93 @@ +using Cysharp.Threading.Tasks; +using DCL.Chat.Commands; +using DCL.Chat.History; +using DCL.Chat.MessageBus; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.RealmNavigation; +using ECS.SceneLifeCycle; +using Newtonsoft.Json.Linq; +using SceneRunner.Scene; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + /// + /// Teleports through the same /goto command pipeline a user teleport takes (loading screen included), + /// then polls until the destination scene is ready or the timeout elapses. + /// + public class TeleportTool : IMcpTool + { + private const int POLL_INTERVAL_MS = 500; + private const float MIN_TIMEOUT_SEC = 5f; + private const float MAX_TIMEOUT_SEC = 300f; + private const float DEFAULT_TIMEOUT_SEC = 60f; + + private readonly IChatMessagesBus chatMessagesBus; + private readonly IScenesCache scenesCache; + private readonly ILoadingStatus loadingStatus; + + public string Name => "teleport"; + + public string Description => + "Teleport the player to a parcel (x,y) through the regular /goto flow and wait until the destination scene is ready. " + + "Reports the final scene state; follow up with get_scene_state for details."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Integer("x", "Target parcel X coordinate.", required: true) + .Integer("y", "Target parcel Y coordinate.", required: true) + .Boolean("waitForReady", "Wait until the destination scene is ready. Default true.") + .Number("timeoutSec", "Maximum seconds to wait for readiness. Default 60.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public TeleportTool(IChatMessagesBus chatMessagesBus, IScenesCache scenesCache, ILoadingStatus loadingStatus) + { + this.chatMessagesBus = chatMessagesBus; + this.scenesCache = scenesCache; + this.loadingStatus = loadingStatus; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetInt("x", out int x) || !arguments.TryGetInt("y", out int y)) + return McpToolResult.Error("Both x and y parcel coordinates are required."); + + bool waitForReady = arguments.GetBool("waitForReady", true); + float timeoutSec = Mathf.Clamp(arguments.GetFloat("timeoutSec", DEFAULT_TIMEOUT_SEC), MIN_TIMEOUT_SEC, MAX_TIMEOUT_SEC); + + chatMessagesBus.SendWithUtcNowTimestamp(ChatChannel.NEARBY_CHANNEL, $"/{ChatCommandsUtils.COMMAND_GOTO} {x},{y}", ChatMessageOrigin.RESTRICTED_ACTION_API); + + if (!waitForReady) + return McpToolResult.Text($"Teleport to ({x},{y}) requested."); + + var targetParcel = new Vector2Int(x, y); + float deadline = UnityEngine.Time.realtimeSinceStartup + timeoutSec; + + while (UnityEngine.Time.realtimeSinceStartup < deadline) + { + await UniTask.Delay(POLL_INTERVAL_MS, cancellationToken: ct); + + ISceneFacade? currentScene = scenesCache.CurrentScene.Value; + bool arrived = scenesCache.CurrentParcel.Value == targetParcel || (currentScene?.Contains(targetParcel) ?? false); + + if (!arrived || loadingStatus.IsLoadingScreenOn()) + continue; + + if (currentScene == null) + return McpToolResult.Text($"Arrived at ({x},{y}); no scene is deployed at this parcel."); + + if (currentScene.SceneStateProvider.IsNotRunningState()) + return McpToolResult.Error($"Arrived at ({x},{y}) but scene '{currentScene.Info.Name}' is not running: {currentScene.SceneStateProvider.State.Value()}. Check get_scene_logs."); + + if (currentScene.IsSceneReady()) + return McpToolResult.Text($"Teleported to ({x},{y}). Scene '{currentScene.Info.Name}' is ready."); + } + + return McpToolResult.Error($"Teleport to ({x},{y}) did not reach a ready scene within {timeoutSec}s. Check get_scene_state."); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs.meta new file mode 100644 index 00000000000..094575a2b3d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 1f263b4ea6e6e489a83b9b5accd70c06 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs b/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs new file mode 100644 index 00000000000..c89e8d47d9b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs @@ -0,0 +1,58 @@ +using CrdtEcsBridge.RestrictedActions; +using Cysharp.Threading.Tasks; +using DCL.ECSComponents; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + /// + /// Plays or stops an avatar emote through , the same intent path + /// a scene uses, so agents can verify emote-driven behaviour without touching the avatar directly. + /// + public class TriggerEmoteTool : IMcpTool + { + private readonly IGlobalWorldActions globalWorldActions; + + public string Name => "trigger_emote"; + + public string Description => + "Play an avatar emote by URN (e.g. a base emote like 'wave', 'dance', 'clap'), or stop the current one with stop: true."; + + public JObject InputSchema => + McpJsonSchema.Object() + .String("urn", "Emote URN or base emote id (wave, dance, clap...).") + .Boolean("loop", "Loop the emote until stopped. Default false.") + .Boolean("stop", "Stop the currently playing emote instead of triggering one.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: false); + + public TriggerEmoteTool(IGlobalWorldActions globalWorldActions) + { + this.globalWorldActions = globalWorldActions; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + bool stop = arguments.GetBool("stop", false); + string urn = arguments.GetString("urn", string.Empty); + + if (!stop && string.IsNullOrEmpty(urn)) + return UniTask.FromResult(McpToolResult.Error("urn is required (or pass stop: true).")); + + if (stop) + { + globalWorldActions.StopEmote(); + return UniTask.FromResult(McpToolResult.Text("Emote stopped.")); + } + + bool loop = arguments.GetBool("loop", false); + globalWorldActions.TriggerEmote(urn, loop, AvatarEmoteMask.AemFullBody); + + return UniTask.FromResult(McpToolResult.Text($"Emote '{urn}' triggered{(loop ? " (looping)" : string.Empty)}.")); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs.meta new file mode 100644 index 00000000000..f77e3e314b1 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 538f89c5fefa643f3936739de46a83e1 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs b/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs new file mode 100644 index 00000000000..21579c598e7 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs @@ -0,0 +1,118 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterMotion.Components; +using DCL.McpServer.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; +using UnityEngine; +using Utility; +using Utility.Arch; + +namespace DCL.McpServer.Tools +{ + /// + /// Holds a movement input on the player for a duration via , + /// exercising the regular locomotion pipeline (velocity, collisions, jumps) instead of teleporting. + /// + public class WalkTool : IMcpTool + { + private const float MIN_SECONDS = 0.1f; + private const float MAX_SECONDS = 30f; + private const float COMPLETION_GRACE_SEC = 5f; + + private readonly World world; + private readonly Entity playerEntity; + + public string Name => "walk"; + + public string Description => + "Walk/jog/run the player in a camera-relative direction for a number of seconds through the real locomotion pipeline " + + "(collisions apply). directionY is forward, directionX is strafe right. Returns the start and end positions."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Number("directionX", "Strafe axis: 1 right, -1 left.", required: true) + .Number("directionY", "Forward axis: 1 forward, -1 backward.", required: true) + .Number("seconds", "How long to hold the movement. Default 1, max 30.") + .String("kind", "Movement speed. Default jog.", enumValues: new[] { "walk", "jog", "run" }) + .Boolean("jump", "Jump once at the start of the movement. Default false.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: false); + + public WalkTool(World world, Entity playerEntity) + { + this.world = world; + this.playerEntity = playerEntity; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + var direction = new Vector2(arguments.GetFloat("directionX", 0f), arguments.GetFloat("directionY", 0f)); + + if (direction == Vector2.zero) + return McpToolResult.Error("directionX and directionY must not both be zero."); + + direction.Normalize(); + + float seconds = Mathf.Clamp(arguments.GetFloat("seconds", 1f), MIN_SECONDS, MAX_SECONDS); + bool jump = arguments.GetBool("jump", false); + + MovementKind kind = arguments.GetString("kind", "jog") switch + { + "walk" => MovementKind.WALK, + "run" => MovementKind.RUN, + _ => MovementKind.JOG, + }; + + // A newer walk preempts a pending one; release its awaiter before replacing the override. + if (world.TryGet(playerEntity, out McpMovementOverride existingOverride)) + existingOverride.Completion?.TrySetResult(); + + Vector3 startPosition = world.Get(playerEntity).Position; + var completion = new UniTaskCompletionSource(); + + world.AddOrSet(playerEntity, new McpMovementOverride + { + Axes = direction, + Kind = kind, + EndTime = UnityEngine.Time.time + seconds, + JumpRequested = jump, + Completion = completion, + }); + + try + { + await completion.Task.AttachExternalCancellation(ct) + .Timeout(TimeSpan.FromSeconds(seconds + COMPLETION_GRACE_SEC)); + } + catch (TimeoutException) + { + await UniTask.SwitchToMainThread(); + + if (world.Has(playerEntity)) + world.Remove(playerEntity); + + return McpToolResult.Error($"walk did not complete within {seconds + COMPLETION_GRACE_SEC}s (is the simulation paused?)."); + } + + await UniTask.SwitchToMainThread(ct); + + Vector3 endPosition = world.Get(playerEntity).Position; + + var result = new JObject + { + ["startPosition"] = startPosition.ToVector(), + ["endPosition"] = endPosition.ToVector(), + ["distance"] = Math.Round(Vector3.Distance(startPosition, endPosition), 2), + ["parcel"] = endPosition.ToParcel().ToParcel(), + }; + + return McpToolResult.Json(result); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs.meta new file mode 100644 index 00000000000..797981b32b9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: c981a464b2fe440749be02bc2b01d6ff \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Utils.meta b/Explorer/Assets/DCL/McpServer/Utils.meta new file mode 100644 index 00000000000..41ef52ed9b8 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils.meta @@ -0,0 +1,3 @@ +fileFormatVersion: 2 +guid: c2c6933e4faa4a0c9bdb0f363315b0eb +timeCreated: 1784277704 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs b/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs new file mode 100644 index 00000000000..268d5a87316 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs @@ -0,0 +1,83 @@ +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using System; +using UnityEngine; + +namespace DCL.McpServer.Utils +{ + /// + /// Builders for the JSON fragments shared by tool outputs. + /// + public static class JObjectExtensions + { + public static JObject ToVector(this Vector3 value) => + new () + { + ["x"] = Math.Round(value.x, 2), + ["y"] = Math.Round(value.y, 2), + ["z"] = Math.Round(value.z, 2), + }; + + public static JObject ToParcel(this Vector2Int value) => + new () + { + ["x"] = value.x, + ["y"] = value.y, + }; + + /// Output-schema counterpart of — an { x, y, z } object of numbers. + public static McpJsonSchema VectorSchema() => + McpJsonSchema.Object() + .Number("x") + .Number("y") + .Number("z"); + + /// Output-schema counterpart of — an { x, y } object of integers. + public static McpJsonSchema ParcelSchema() => + McpJsonSchema.Object() + .Integer("x") + .Integer("y"); + + public static bool GetBool(this JObject arguments, string name, bool defaultValue) => + arguments[name]?.Type == JTokenType.Boolean ? arguments[name]!.Value() : defaultValue; + + public static int GetInt(this JObject arguments, string name, int defaultValue) => + arguments[name].IsNumber() ? arguments[name]!.Value() : defaultValue; + + public static long GetLong(this JObject arguments, string name, long defaultValue) => + arguments[name].IsNumber() ? arguments[name]!.Value() : defaultValue; + + public static float GetFloat(this JObject arguments, string name, float defaultValue) => + arguments[name].IsNumber() ? arguments[name]!.Value() : defaultValue; + + public static string GetString(this JObject arguments, string name, string defaultValue) => + arguments[name]?.Type == JTokenType.String ? arguments[name]!.Value()! : defaultValue; + + public static bool TryGetFloat(this JObject arguments, string name, out float value) + { + if (arguments[name].IsNumber()) + { + value = arguments[name]!.Value(); + return true; + } + + value = 0f; + return false; + } + + public static bool TryGetInt(this JObject arguments, string name, out int value) + { + if (arguments[name].IsNumber()) + { + value = arguments[name]!.Value(); + return true; + } + + value = 0; + return false; + } + + private static bool IsNumber(this JToken? token) => + token?.Type is JTokenType.Integer or JTokenType.Float; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs.meta b/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs.meta new file mode 100644 index 00000000000..1a87815e6b8 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 139ddf8ed9933484c92df485a5843199 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs b/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs new file mode 100644 index 00000000000..54d315b8a96 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs @@ -0,0 +1,82 @@ +using DCL.UI.DebugMenu.LogHistory; +using System.Collections.Generic; + +namespace DCL.McpServer.Utils +{ + /// + /// Thread-safe ring buffer of scene console log entries with monotonic sequence numbers, + /// so agents can poll incrementally via sinceSeq. + /// + public class SceneLogBuffer + { + private const int CAPACITY = 1000; + + private readonly object gate = new (); + private readonly Entry[] entries = new Entry[CAPACITY]; + + private long nextSeq; + + /// + /// Sequence number of the newest stored entry, or -1 when empty. + /// + public long LatestSeq + { + get + { + lock (gate) { return nextSeq - 1; } + } + } + + /// + /// Fed from ; may be invoked from any thread. + /// + public void Append(DebugMenuConsoleLogEntry logEntry) + { + lock (gate) + { + entries[nextSeq % CAPACITY] = new Entry(nextSeq, logEntry.Type, logEntry.Message); + nextSeq++; + } + } + + /// + /// Copies up to newest entries with Seq greater than + /// into in chronological order. + /// + public void CopyTo(List target, long sinceSeq, bool errorsOnly, int limit) + { + lock (gate) + { + long oldestAvailable = nextSeq >= CAPACITY ? nextSeq - CAPACITY : 0; + long from = sinceSeq + 1 > oldestAvailable ? sinceSeq + 1 : oldestAvailable; + + for (long seq = from; seq < nextSeq; seq++) + { + Entry entry = entries[seq % CAPACITY]; + + if (errorsOnly && entry.Type != LogMessageType.Error) + continue; + + target.Add(entry); + } + + if (target.Count > limit) + target.RemoveRange(0, target.Count - limit); + } + } + + public readonly struct Entry + { + public readonly long Seq; + public readonly LogMessageType Type; + public readonly string Message; + + public Entry(long seq, LogMessageType type, string message) + { + Seq = seq; + Type = type; + Message = message; + } + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs.meta b/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs.meta new file mode 100644 index 00000000000..4bf93032e48 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 91d8bd24c95a3413c800ab47ed79d7eb \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Utils/UiAutomation.cs b/Explorer/Assets/DCL/McpServer/Utils/UiAutomation.cs new file mode 100644 index 00000000000..917fb53443e --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/UiAutomation.cs @@ -0,0 +1,271 @@ +using Newtonsoft.Json.Linq; +using System.Collections.Generic; +using UnityEngine; +using UnityEngine.EventSystems; +using UnityEngine.UI; +using UITK = UnityEngine.UIElements; + +namespace DCL.McpServer.Utils +{ + /// + /// Enumerates and drives the client's live UI the way AltTester queries the running object hierarchy and clicks + /// real controls. It covers both UI systems the Explorer uses: uGUI (every registered + /// — Button, Toggle, InputField, Slider, Dropdown — reachable through + /// ) and UI-Toolkit (every focusable element under each active + /// 's rootVisualElement). Each element gets a system-prefixed path + /// (see ) that click_ui/get_ui_state re-resolve against a fresh walk. + /// + /// All members touch Unity UI objects and must run on the main thread; the MCP dispatcher already hops there + /// before a tool executes. The path-building and matching rules live in the pure so + /// they can be unit-tested; the walk itself is integration-only. + /// + public static class UiAutomation + { + private const string SYSTEM_UGUI = "ugui"; + private const string SYSTEM_UITK = "uitk"; + + /// A resolved handle to one live UI element, either uGUI or UI-Toolkit, with its computed identity. + private readonly struct Handle + { + public readonly Selectable? Ugui; + public readonly UITK.VisualElement? Uitk; + public readonly string Path; + public readonly string Name; + public readonly string Type; + + public Handle(Selectable ugui, string path, string name, string type) + { + Ugui = ugui; + Uitk = null; + Path = path; + Name = name; + Type = type; + } + + public Handle(UITK.VisualElement uitk, string path, string name, string type) + { + Ugui = null; + Uitk = uitk; + Path = path; + Name = name; + Type = type; + } + + public bool IsUgui => Ugui != null; + } + + /// Lists every currently interactable UI element, optionally filtered by name or path substring. + public static JArray Enumerate(string? filter) + { + var array = new JArray(); + + foreach (Handle handle in AllHandles()) + { + if (!UiElementPath.Matches(handle.Name, handle.Path, filter)) + continue; + + array.Add(ToInfo(handle)); + } + + return array; + } + + /// Resolves a path/name lookup to the single best-matching live element (highest match score wins). + public static bool TryGetState(string query, out JObject state) + { + if (TryResolve(query, out Handle handle)) + { + state = ToInfo(handle); + return true; + } + + state = null!; + return false; + } + + /// + /// Performs a real activation on the resolved element: a full pointer down/up/click on a uGUI Selectable, or + /// a navigation-submit on a UI-Toolkit control (what a keyboard/gamepad "activate" does, which fires a + /// Button/Toggle exactly like a mouse click). Returns the element's post-click state, or false if unresolved. + /// + public static bool TryClick(string query, out JObject result) + { + if (!TryResolve(query, out Handle handle)) + { + result = null!; + return false; + } + + if (handle.IsUgui) + ClickUgui(handle.Ugui!); + else + ClickUitk(handle.Uitk!); + + result = ToInfo(handle); + result["clicked"] = true; + return true; + } + + private static void ClickUgui(Selectable selectable) + { + GameObject go = selectable.gameObject; + var pointer = new PointerEventData(EventSystem.current); + + ExecuteEvents.Execute(go, pointer, ExecuteEvents.pointerDownHandler); + ExecuteEvents.Execute(go, pointer, ExecuteEvents.pointerUpHandler); + ExecuteEvents.Execute(go, pointer, ExecuteEvents.pointerClickHandler); + } + + private static void ClickUitk(UITK.VisualElement element) + { + using UITK.NavigationSubmitEvent evt = UITK.NavigationSubmitEvent.GetPooled(); + evt.target = element; + element.SendEvent(evt); + } + + private static bool TryResolve(string query, out Handle best) + { + best = default; + var bestScore = 0; + + foreach (Handle handle in AllHandles()) + { + int score = UiElementPath.MatchScore(handle.Path, handle.Name, query); + + if (score > bestScore) + { + bestScore = score; + best = handle; + } + } + + return bestScore > 0; + } + + private static IEnumerable AllHandles() + { + Selectable[] selectables = Selectable.allSelectablesArray; + + foreach (Selectable selectable in selectables) + { + if (selectable == null) continue; + + string name = selectable.gameObject.name; + string type = selectable.GetType().Name; + string path = UiElementPath.UGUI_PREFIX + TransformPath(selectable.transform); + yield return new Handle(selectable, path, name, type); + } + + UITK.UIDocument[] documents = Object.FindObjectsByType(FindObjectsSortMode.None); + + foreach (UITK.UIDocument document in documents) + { + UITK.VisualElement? root = document.rootVisualElement; + if (root == null) continue; + + string docPath = UiElementPath.UITK_PREFIX + "/" + (string.IsNullOrEmpty(document.name) ? "UIDocument" : document.name); + + foreach (Handle handle in WalkUitk(root, docPath)) + yield return handle; + } + } + + private static IEnumerable WalkUitk(UITK.VisualElement element, string parentPath) + { + int childCount = element.hierarchy.childCount; + + for (var i = 0; i < childCount; i++) + { + UITK.VisualElement child = element.hierarchy.ElementAt(i); + string type = child.GetType().Name; + string path = UiElementPath.Join(parentPath, UiElementPath.Segment(child.name, type, i)); + + if (IsInteractableUitk(child)) + yield return new Handle(child, path, child.name ?? string.Empty, type); + + foreach (Handle descendant in WalkUitk(child, path)) + yield return descendant; + } + } + + private static bool IsInteractableUitk(UITK.VisualElement element) => + element.focusable || element is UITK.Button; + + private static string TransformPath(Transform transform) + { + var segments = new List(); + + for (Transform? current = transform; current != null; current = current.parent) + segments.Add(current.name); + + segments.Reverse(); + return string.Join("/", segments); + } + + private static JObject ToInfo(Handle handle) => + handle.IsUgui ? UguiInfo(handle) : UitkInfo(handle); + + private static JObject UguiInfo(Handle handle) + { + Selectable selectable = handle.Ugui!; + + var info = new JObject + { + ["path"] = handle.Path, + ["name"] = handle.Name, + ["type"] = handle.Type, + ["system"] = SYSTEM_UGUI, + ["interactable"] = selectable.IsInteractable(), + ["visible"] = selectable.gameObject.activeInHierarchy, + }; + + string? text = UguiText(selectable); + if (text != null) info["text"] = text; + + return info; + } + + private static JObject UitkInfo(Handle handle) + { + UITK.VisualElement element = handle.Uitk!; + + var info = new JObject + { + ["path"] = handle.Path, + ["name"] = handle.Name, + ["type"] = handle.Type, + ["system"] = SYSTEM_UITK, + ["interactable"] = element.enabledInHierarchy, + ["visible"] = element.visible && element.resolvedStyle.display != UITK.DisplayStyle.None, + }; + + string? text = UitkText(element); + if (text != null) info["text"] = text; + + return info; + } + + private static string? UguiText(Selectable selectable) + { + switch (selectable) + { + case InputField inputField: return inputField.text; + case Toggle toggle: return toggle.isOn ? "on" : "off"; + default: + Text? label = selectable.GetComponentInChildren(); + return label != null ? label.text : null; + } + } + + private static string? UitkText(UITK.VisualElement element) + { + switch (element) + { + case UITK.TextField textField: return textField.value; + case UITK.Toggle toggle: return toggle.value ? "on" : "off"; + case UITK.TextElement textElement: return textElement.text; + default: return null; + } + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Utils/UiAutomation.cs.meta b/Explorer/Assets/DCL/McpServer/Utils/UiAutomation.cs.meta new file mode 100644 index 00000000000..ecc7471e5ce --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/UiAutomation.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 892cc696f2904ac1bbd43ef11c543f64 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Utils/UiElementPath.cs b/Explorer/Assets/DCL/McpServer/Utils/UiElementPath.cs new file mode 100644 index 00000000000..e49ca40d7b2 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/UiElementPath.cs @@ -0,0 +1,61 @@ +using System; + +namespace DCL.McpServer.Utils +{ + /// + /// Pure, engine-free helpers for the string identity ("path") that the UI-automation tools + /// (list_ui_elements / get_ui_state / click_ui) hand back and forth. A path prefixes the + /// element hierarchy with the UI system it belongs to (uGUI Canvas tree, or a UI-Toolkit UIDocument tree) so a + /// later click_ui can re-resolve it. Kept separate from the engine walk so the parsing and matching rules + /// can be unit-tested without a live Unity UI. + /// + public static class UiElementPath + { + public const string UGUI_PREFIX = "ugui:"; + public const string UITK_PREFIX = "uitk:"; + + public static bool IsUgui(string path) => + path != null && path.StartsWith(UGUI_PREFIX, StringComparison.Ordinal); + + public static bool IsUitk(string path) => + path != null && path.StartsWith(UITK_PREFIX, StringComparison.Ordinal); + + /// + /// A path segment for one node: its name when it has one, otherwise a type-and-sibling-index token so + /// unnamed nodes still get a deterministic, re-resolvable identity within a single frame. + /// + public static string Segment(string? name, string typeName, int siblingIndex) => + string.IsNullOrEmpty(name) ? $"{typeName}[{siblingIndex}]" : name!; + + /// Appends to with a single separator. + public static string Join(string parentPath, string segment) => + parentPath + "/" + segment; + + /// + /// Whether an element with at passes the optional + /// case-insensitive (matched against either its name or its full path). + /// An empty filter matches everything. + /// + public static bool Matches(string name, string path, string? filter) => + string.IsNullOrEmpty(filter) + || (name != null && name.IndexOf(filter, StringComparison.OrdinalIgnoreCase) >= 0) + || (path != null && path.IndexOf(filter, StringComparison.OrdinalIgnoreCase) >= 0); + + /// + /// How well a stored element identified by / + /// answers the lookup a click_ui/get_ui_state call passed. Higher is + /// better; 0 means no match. Lets resolution prefer an exact path, then an exact name, then a loose contains. + /// + public static int MatchScore(string candidatePath, string candidateName, string query) + { + if (string.IsNullOrEmpty(query)) return 0; + + if (string.Equals(candidatePath, query, StringComparison.Ordinal)) return 4; + if (string.Equals(candidateName, query, StringComparison.OrdinalIgnoreCase)) return 3; + if (candidatePath != null && candidatePath.EndsWith("/" + query, StringComparison.OrdinalIgnoreCase)) return 2; + if (candidatePath != null && candidatePath.IndexOf(query, StringComparison.OrdinalIgnoreCase) >= 0) return 1; + + return 0; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Utils/UiElementPath.cs.meta b/Explorer/Assets/DCL/McpServer/Utils/UiElementPath.cs.meta new file mode 100644 index 00000000000..eeb929e1135 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/UiElementPath.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: cf94d7698ae645e3b7cd34a2a6e07de1 \ No newline at end of file diff --git a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/Container/DiagnosticsContainer.cs b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/Container/DiagnosticsContainer.cs index 017891b0591..2e02c77d114 100644 --- a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/Container/DiagnosticsContainer.cs +++ b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/Container/DiagnosticsContainer.cs @@ -106,9 +106,11 @@ private static SceneDebugConsoleReportHandler AddDebugConsoleReportHandler(Debug ReportCategory.SCENE_FETCH_REQUEST, ReportCategory.PORTABLE_EXPERIENCE, ReportCategory.EMOTE, + ReportCategory.MCP, }, logType: false); entries.Add(new CategorySeverityMatrix.Entry { Category = ReportCategory.JAVASCRIPT, Severity = LogType.Log }); + entries.Add(new CategorySeverityMatrix.Entry { Category = ReportCategory.MCP, Severity = LogType.Log }); jsOnlyMatrix.entries = entries; return new SceneDebugConsoleReportHandler(jsOnlyMatrix, sceneDebugConsoleMessageBus, false); diff --git a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportCategory.cs b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportCategory.cs index b79da65000f..0ef5de51f35 100644 --- a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportCategory.cs +++ b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportCategory.cs @@ -360,6 +360,11 @@ public static class ReportCategory public const string CHROME_DEVTOOL_PROTOCOL = nameof(CHROME_DEVTOOL_PROTOCOL); + /// + /// Embedded MCP (Model Context Protocol) automation server. + /// + public const string MCP = nameof(MCP); + public const string SCENE_PERMISSIONS = nameof(SCENE_PERMISSIONS); public const string MVC_STATE_MACHINE = nameof(MVC_STATE_MACHINE); diff --git a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsDevelopment.asset b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsDevelopment.asset index c69c1771a1c..3feccb406d8 100644 --- a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsDevelopment.asset +++ b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsDevelopment.asset @@ -708,6 +708,16 @@ MonoBehaviour: Severity: 2 - Category: MULTIPLAYER Severity: 2 + - Category: MCP + Severity: 3 + - Category: MCP + Severity: 0 + - Category: MCP + Severity: 4 + - Category: MCP + Severity: 1 + - Category: MCP + Severity: 2 sentryMatrix: entries: [] debounceEnabled: 1 diff --git a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsProduction.asset b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsProduction.asset index 57f1e67ae01..93e8412a007 100644 --- a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsProduction.asset +++ b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsProduction.asset @@ -448,6 +448,16 @@ MonoBehaviour: Severity: 0 - Category: MULTIPLAYER Severity: 2 + - Category: MCP + Severity: 4 + - Category: MCP + Severity: 1 + - Category: MCP + Severity: 3 + - Category: MCP + Severity: 2 + - Category: MCP + Severity: 0 - Category: AUTHENTICATION Severity: 3 - Category: AUTHENTICATION diff --git a/Explorer/Assets/DCL/PluginSystem/DCL.Plugins.asmdef b/Explorer/Assets/DCL/PluginSystem/DCL.Plugins.asmdef index c8338aaecc6..2032d9a6b31 100644 --- a/Explorer/Assets/DCL/PluginSystem/DCL.Plugins.asmdef +++ b/Explorer/Assets/DCL/PluginSystem/DCL.Plugins.asmdef @@ -70,10 +70,10 @@ "GUID:451516970bc990e418454aa78b72586e", "GUID:13c468c9ecfc44f249fb8fb69d3365ef", "GUID:a42927d1d4a3b4cda9b076a7adecb9cc", - "GUID:f8127c6ac263abf468221dcbccbde182", "GUID:63cde67b4a5d47449392dfc49fc0c3ed", "GUID:267f743eefeddd44982a713909628102", "GUID:0e1edbec885c4350bb2ef81c6a413df5", + "GUID:f254b2009e854709a0ef0022a7ca697c", "GUID:3c1a9be47d0f42e2a5b8c6d4e9f01a27" ], "includePlatforms": [], @@ -98,4 +98,4 @@ } ], "noEngineReferences": false -} +} \ No newline at end of file diff --git a/Explorer/Assets/DCL/Tests/Editor/DCL.EditMode.Tests.asmdef b/Explorer/Assets/DCL/Tests/Editor/DCL.EditMode.Tests.asmdef index f22886fcbd1..9b998b9b1c7 100644 --- a/Explorer/Assets/DCL/Tests/Editor/DCL.EditMode.Tests.asmdef +++ b/Explorer/Assets/DCL/Tests/Editor/DCL.EditMode.Tests.asmdef @@ -61,7 +61,8 @@ "GUID:3ca842002c51d0a43b73e45298809a13", "GUID:0df5180c0c3a0594fbfa11f83736de9f", "GUID:28964ef7dc9441b6b8671b61a8106690", - "GUID:267f743eefeddd44982a713909628102" + "GUID:267f743eefeddd44982a713909628102", + "GUID:f254b2009e854709a0ef0022a7ca697c" ], "includePlatforms": [ "Editor" diff --git a/Explorer/Assets/DCL/WebRequests/Analytics/McpNetworkAnalyticsHandler.cs b/Explorer/Assets/DCL/WebRequests/Analytics/McpNetworkAnalyticsHandler.cs new file mode 100644 index 00000000000..2c91efe68f9 --- /dev/null +++ b/Explorer/Assets/DCL/WebRequests/Analytics/McpNetworkAnalyticsHandler.cs @@ -0,0 +1,62 @@ +using Cysharp.Threading.Tasks; +using System; +using UnityEngine.Networking; + +namespace DCL.WebRequests.Analytics +{ + /// + /// A lightweight that records every finished request into an + /// for the MCP get_network_log tool. It taps the exact same seam the + /// Chrome DevTools ChromeDevToolHandler uses, so its coverage is identical: every request issued through + /// the shared WebRequestController (content, realm, asset bundles, textures, audio, wearables/emotes, + /// profiles, …). Requests made with a raw or HttpClient outside the + /// controller are not seen. + /// + /// Unlike the Chrome DevTools handler it keeps no per-request scope and streams nothing: it captures the final + /// shape of a request once, at completion or failure, which is all the pull-based tool needs. + /// + public sealed class McpNetworkAnalyticsHandler : IWebRequestAnalyticsHandler + { + private const string UNKNOWN_MIME = "application/octet-stream"; + + private readonly McpNetworkLogBuffer buffer; + + public McpNetworkAnalyticsHandler(McpNetworkLogBuffer buffer) + { + this.buffer = buffer; + } + + public void Update(float dt) { } + + public void OnBeforeBudgeting(in RequestEnvelope envelope, T request) where T: struct, ITypedWebRequest where TWebRequestArgs: struct { } + + public void OnRequestStarted(in RequestEnvelope envelope, T request, DateTime startedAt) where T: struct, ITypedWebRequest where TWebRequestArgs: struct { } + + public void OnProcessDataFinished(T request) where T: ITypedWebRequest { } + + public void OnRequestFinished(T request, TimeSpan duration) where T: ITypedWebRequest + { + UnityWebRequest uwr = request.UnityWebRequest; + string mimeType = uwr.GetResponseHeader("Content-Type") ?? UNKNOWN_MIME; + + buffer.Append(uwr.url, uwr.method, (int)uwr.responseCode, mimeType, (long)uwr.downloadedBytes, duration.TotalMilliseconds, failed: false, failureReason: null); + } + + public void OnException(T request, Exception exception, TimeSpan duration) where T: ITypedWebRequest + { + bool cancelled = exception is OperationCanceledException or AggregateException { InnerException: OperationCanceledException }; + Record(request.UnityWebRequest, duration, cancelled ? "Cancelled" : $"Engine exception: {exception.Message}"); + } + + public void OnException(T request, UnityWebRequestException exception, TimeSpan duration) where T: ITypedWebRequest + { + Record(request.UnityWebRequest, duration, exception.Error); + } + + private void Record(UnityWebRequest uwr, TimeSpan duration, string failureReason) + { + string mimeType = uwr.GetResponseHeader("Content-Type") ?? UNKNOWN_MIME; + buffer.Append(uwr.url, uwr.method, (int)uwr.responseCode, mimeType, (long)uwr.downloadedBytes, duration.TotalMilliseconds, failed: true, failureReason); + } + } +} diff --git a/Explorer/Assets/DCL/WebRequests/Analytics/McpNetworkAnalyticsHandler.cs.meta b/Explorer/Assets/DCL/WebRequests/Analytics/McpNetworkAnalyticsHandler.cs.meta new file mode 100644 index 00000000000..7686834af46 --- /dev/null +++ b/Explorer/Assets/DCL/WebRequests/Analytics/McpNetworkAnalyticsHandler.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 47662b2bc8b645bfba39c2e00566b186 \ No newline at end of file diff --git a/Explorer/Assets/DCL/WebRequests/Analytics/McpNetworkLogBuffer.cs b/Explorer/Assets/DCL/WebRequests/Analytics/McpNetworkLogBuffer.cs new file mode 100644 index 00000000000..830b6e01c88 --- /dev/null +++ b/Explorer/Assets/DCL/WebRequests/Analytics/McpNetworkLogBuffer.cs @@ -0,0 +1,106 @@ +using System; +using System.Collections.Generic; + +namespace DCL.WebRequests.Analytics +{ + /// + /// Thread-safe ring buffer of completed HTTP requests with monotonic sequence numbers, so an MCP agent can + /// poll incrementally via sinceSeq. Mirrors DCL.McpServer.Utils.SceneLogBuffer: the pure, engine-free + /// storage is kept separate from the that feeds it, so this class can + /// be unit-tested in isolation. Fed from the same IWebRequestAnalyticsHandler seam the Chrome DevTools + /// Network domain taps, so it sees every request that flows through the shared WebRequestController. + /// + public sealed class McpNetworkLogBuffer + { + private const int CAPACITY = 512; + + private readonly object gate = new (); + private readonly Entry[] entries = new Entry[CAPACITY]; + + private long nextSeq; + + /// Sequence number of the newest stored entry, or -1 when empty. + public long LatestSeq + { + get + { + lock (gate) { return nextSeq - 1; } + } + } + + /// + /// Records a finished request. May be invoked from any thread (the web-request analytics callbacks run off + /// the main thread), so all mutation happens under the lock. + /// + public void Append(string url, string method, int status, string mimeType, long sizeBytes, double durationMs, bool failed, string? failureReason) + { + lock (gate) + { + entries[nextSeq % CAPACITY] = new Entry(nextSeq, DateTime.UtcNow, url, method, status, mimeType, sizeBytes, durationMs, failed, failureReason); + nextSeq++; + } + } + + /// + /// Copies up to newest entries with Seq greater than + /// into in chronological order. When is set only + /// unsuccessful entries survive (a transport failure or an HTTP status >= 400); when + /// is non-negative only entries with that exact HTTP status survive. + /// + public void CopyTo(List target, long sinceSeq, bool failedOnly, int status, int limit) + { + lock (gate) + { + long oldestAvailable = nextSeq >= CAPACITY ? nextSeq - CAPACITY : 0; + long from = sinceSeq + 1 > oldestAvailable ? sinceSeq + 1 : oldestAvailable; + + for (long seq = from; seq < nextSeq; seq++) + { + Entry entry = entries[seq % CAPACITY]; + + if (failedOnly && !entry.IsUnsuccessful) + continue; + + if (status >= 0 && entry.Status != status) + continue; + + target.Add(entry); + } + + if (target.Count > limit) + target.RemoveRange(0, target.Count - limit); + } + } + + public readonly struct Entry + { + public readonly long Seq; + public readonly DateTime TimestampUtc; + public readonly string Url; + public readonly string Method; + public readonly int Status; + public readonly string MimeType; + public readonly long SizeBytes; + public readonly double DurationMs; + public readonly bool Failed; + public readonly string? FailureReason; + + public Entry(long seq, DateTime timestampUtc, string url, string method, int status, string mimeType, long sizeBytes, double durationMs, bool failed, string? failureReason) + { + Seq = seq; + TimestampUtc = timestampUtc; + Url = url; + Method = method; + Status = status; + MimeType = mimeType; + SizeBytes = sizeBytes; + DurationMs = durationMs; + Failed = failed; + FailureReason = failureReason; + } + + /// A transport failure, or an HTTP response that carried an error status. + public bool IsUnsuccessful => Failed || Status >= 400; + } + } +} diff --git a/Explorer/Assets/DCL/WebRequests/Analytics/McpNetworkLogBuffer.cs.meta b/Explorer/Assets/DCL/WebRequests/Analytics/McpNetworkLogBuffer.cs.meta new file mode 100644 index 00000000000..4fb42d0382e --- /dev/null +++ b/Explorer/Assets/DCL/WebRequests/Analytics/McpNetworkLogBuffer.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 95aacb2188fe409782a5e7d26427f79c \ No newline at end of file diff --git a/Explorer/Assets/DCL/WebRequests/Analytics/Plugin/WebRequestsContainer.cs b/Explorer/Assets/DCL/WebRequests/Analytics/Plugin/WebRequestsContainer.cs index 3dc373d91f0..bf734321340 100644 --- a/Explorer/Assets/DCL/WebRequests/Analytics/Plugin/WebRequestsContainer.cs +++ b/Explorer/Assets/DCL/WebRequests/Analytics/Plugin/WebRequestsContainer.cs @@ -33,6 +33,13 @@ public class WebRequestsContainer : DCLGlobalContainer + /// Ring buffer of recent HTTP activity for the MCP get_network_log tool, fed by + /// from the same analytics seam the Chrome DevTools Network domain + /// taps. Always present (bounded, metadata only); the MCP server reads it only when it is enabled. + /// + public McpNetworkLogBuffer NetworkLogBuffer { get; private set; } + private WebRequestsContainer() { } public override void Dispose() => @@ -123,7 +130,11 @@ await container.InitializeContainerAsync(pluginS WebRequestsDumper.Instance.AnalyticsHandler = dumpHandler; - var analyticsContainer = new WebRequestsAnalyticsContainer(sentryWebRequestHandler, dumpHandler, debugHandler, chromeDevtoolProtocolHandler); + var mcpNetworkLogBuffer = new McpNetworkLogBuffer(); + var mcpNetworkHandler = new McpNetworkAnalyticsHandler(mcpNetworkLogBuffer); + container.NetworkLogBuffer = mcpNetworkLogBuffer; + + var analyticsContainer = new WebRequestsAnalyticsContainer(sentryWebRequestHandler, dumpHandler, debugHandler, chromeDevtoolProtocolHandler, mcpNetworkHandler); var requestCompleteDebugMetric = new ElementBinding(0); diff --git a/docs/README.md b/docs/README.md index ff13159a5d7..c837410b3de 100644 --- a/docs/README.md +++ b/docs/README.md @@ -73,6 +73,7 @@ Welcome to the official documentation for Unity Explorer — the Decentraland cl - **[Debug Container & Widgets](debug-container-and-widgets.md)** — Runtime debug panel architecture, widget builder API, bindings, and integration patterns - **[Testing Guide](testing-guide.md)** — UnitySystemTestBase, ECS test utilities, mocking, EditMode/PlayMode, async test patterns - **[Automation Testing](automation-testing.md)** — AltTester SDK setup, writing UI automation tests, running against instrumented builds and in-Editor, triggering visual regression on PRs via `/visual-tests` +- **[MCP Automation](mcp-automation.md)** — Embedded MCP server for coding agents: screenshots, player/scene state, scene logs, and player control via `--mcp` - **[Connect to Local Scene](how-to-connect-to-a-local-scene.md)** — Running and connecting to local SDK7 scenes - **[Master of Bots](master-of-bots.md)** — Simulating multiple bot users for load testing - **[Override Debug Log Matrix](override-debug-log-matrix.md)** — Runtime log severity overrides diff --git a/docs/app-arguments.md b/docs/app-arguments.md index 139655e3de4..6f71ad28464 100644 --- a/docs/app-arguments.md +++ b/docs/app-arguments.md @@ -311,6 +311,27 @@ More detailed instructions on how to test can be found in the description of rel --- +### `mcp` +**Description:** Starts the embedded MCP (Model Context Protocol) server on `http://127.0.0.1:8123/unity-explorer-mcp` so coding agents can observe and drive the client (screenshots, player/scene state, scene logs, teleport/movement, chat commands). The listener binds to localhost only and rejects non-localhost browser Origins. See [MCP Automation](mcp-automation.md). + +**Usage:** +```bash +--mcp +``` + +--- + +### `mcp-port` +**Type:** String (integer port, 1024–65535) +**Description:** Starts the embedded MCP server on a specific port (implies `mcp`). Use distinct ports when running multiple instances via `--multi-instance`. + +**Usage:** +```bash +--mcp-port 8124 +``` + +--- + ### `launch-cdp-monitor-on-start` **Type:** Boolean **Description:** Launches the Chrome DevTools Protocol (CDP) monitor on application start. Enables remote debugging capabilities. diff --git a/docs/mcp-automation.md b/docs/mcp-automation.md new file mode 100644 index 00000000000..f4de0e6412f --- /dev/null +++ b/docs/mcp-automation.md @@ -0,0 +1,149 @@ +# MCP Automation Server + +The Explorer can host an embedded [MCP (Model Context Protocol)](https://modelcontextprotocol.io/) server so coding agents (e.g. Claude Code) can **see** the running client (screenshots, player/scene state, scene console logs) and **control** it (teleport, move, walk, look, chat commands, scene reload) — closing the edit → reload → verify loop for SDK7 scene development without a human in the middle. + +The server is compiled into all builds but stays dormant unless explicitly enabled at launch. + +--- + +## Enabling + +| Flag | Effect | +|---|---| +| `--mcp` | Starts the MCP server on the default port **8123** | +| `--mcp-port ` | Starts the MCP server on a specific port (implies `--mcp`) | + +The flag is accepted from the command line or a deep link. The endpoint is `http://127.0.0.1:/unity-explorer-mcp`. + +```bash +# macOS +open Decentraland.app --args --mcp + +# Windows +Decentraland.exe --mcp-port 8124 +``` + +In the Unity Editor, add `--mcp` to `Main Scene Loader → Debug Settings → App Parameters`. + +From a scene folder, `@dcl/sdk-commands` can enable it at launch: `npm run start -- --mcp` (optionally `--mcp-port `) forwards both flags into the deep link that auto-launches the installed client. Any extra Explorer params can follow a second standalone `--` (`npm run start -- --mcp -- --windowed-mode --resolution 1280x720`; npm consumes the first `--`). + +## Security model + +- The listener binds to **127.0.0.1 only** — it is never reachable from the network. +- Browser-originated requests are rejected unless their `Origin` is localhost (defense against drive-by pages and DNS rebinding). Requests without an `Origin` header (CLI clients) are allowed. +- The server only exists while the process runs with the flag; there is no persistence and no authentication token in v1. + +## Connecting a coding agent + +```bash +claude mcp add --transport http --scope user explorer http://127.0.0.1:8123/unity-explorer-mcp +``` + +Smoke test without an agent: + +```bash +curl -s -X POST http://127.0.0.1:8123/unity-explorer-mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"0"}}}' + +curl -s -X POST http://127.0.0.1:8123/unity-explorer-mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' +``` + +## Tool catalog + +### Seeing + +| Tool | Arguments | Returns | +|---|---|---| +| `screenshot` | `maxWidth?` (default 1280), `quality?` (`jpg`\|`png`), `worldOnly?` (exclude UI; post-processing still applied) | Downscaled image of the current view (UI included by default) + caption | +| `get_player_state` | — | Player position/rotation/parcel/velocity/grounded + camera position/rotation/mode + wallet address | +| `get_scene_state` | — | Current parcel, scene name/state (incl. `JavaScriptError`/`EcsError`), readiness, loading stage | +| `get_scene_logs` | `limit?`, `severity?` (`all`\|`error`), `sinceSeq?` | Scene JS console output with monotonic sequence numbers for incremental polling | +| `get_network_log` | `limit?` (default 50), `sinceSeq?`, `failedOnly?`, `status?` | Recent client HTTP activity — `{url, method, status, mimeType, sizeBytes, durationMs, failed, reason?}` per request, with monotonic sequence numbers for incremental polling (the same data the Chrome DevTools Network domain shows) | +| `list_ui_elements` | `nameFilter?` | Interactable UI elements (uGUI + UI-Toolkit) as `{path, name, type, system, interactable, visible, text?}` | +| `get_ui_state` | `element` (path or name) | One UI element's current state | +| `list_scene_entities` | `limit?` | Entity ids of the current scene's ECS world | +| `get_entity_details` | `entityId` | All components of one scene entity | + +### Controlling + +| Tool | Arguments | Effect | +|---|---|---| +| `teleport` | `x`, `y`, `waitForReady?`, `timeoutSec?` | `/goto x,y` through the regular pipeline, waits for scene readiness | +| `move_to` | `x`, `y`, `z`, `lookAt{X,Y,Z}?`, `durationSec?` | Instant or smooth move to a world position (16 m per parcel) | +| `walk` | `directionX`, `directionY`, `seconds?`, `kind?`, `jump?` | Holds camera-relative movement through the real locomotion pipeline (collisions apply) | +| `look_at` | `x`, `y`, `z` | Rotates the camera to a world point (aim before a screenshot) | +| `set_camera_mode` | `mode` (`first_person`\|`third_person`\|`drone`\|`free`) | Switches the camera mode like the user hotkey; refuses (with the reason) while a scene locks the camera — `CameraModeArea`, scene virtual camera, or photo camera. `get_player_state` → `camera.modeChangeAllowed` reports the lock state in advance | +| `set_camera_pose` | `x`,`y`,`z`, `lookAt{X,Y,Z}?`, `fov?`, `timeoutSec?` | Places the free camera at an absolute world position, optionally aiming it and setting FOV. Auto-enters free mode (same locks as `set_camera_mode`), waits for the blend to settle (`settled` in the result), and returns the actual pose. The camera stays put while the player moves; restore with `set_camera_mode` | +| `send_chat` | `message` | Sends to Nearby chat; `/commands` run through the chat command pipeline | +| `reload_scene` | `timeoutSec?` | Reloads the current scene (motion + skybox frozen during reload) | +| `trigger_emote` | `urn` or `stop: true`, `loop?` | Plays or stops an avatar emote | +| `click_entity` | `entityId` and/or `x`,`y`,`z` aim point, `button?` (`pointer`\|`primary`\|`secondary`), `eventType?` (`click`\|`down`\|`up`), `timeoutSec?` | Presses a pointer button on a scene entity exactly like a real click: a camera-origin raycast validates the aim (occluders and the entity's `maxDistance` apply), then the entity's pointer-event intent is filled so the scene receives an identical `PBPointerEventsResult`. `click` sends down + up on consecutive scene ticks. Returns `hit`, hover text, hit point/distance, or the blocking entity | +| `click_ui` | `element` (path or name) | Clicks a real **client UI** control (not a scene entity — see below) | + +## Client HTTP activity (`get_network_log`) + +`get_network_log` reads a ring buffer of the client's recent HTTP requests, closing the parity gap with the Chrome DevTools **Network** domain. It taps the same seam the DevTools bridge does — the shared `WebRequestController`'s `IWebRequestAnalyticsHandler` pipeline — so its **coverage is identical**: content, realm, asset-bundle, texture, audio, wearable/emote and profile requests all flow through it. Requests made with a raw `UnityWebRequest` or `HttpClient` **outside** the controller are not seen. Each finished request is captured once (at completion or failure); the buffer holds the most recent 512 and records continuously (bounded, metadata only) whether or not an agent is connected. + +Poll incrementally with `sinceSeq` (like `get_scene_logs`). `failedOnly` returns only unsuccessful requests — a transport failure **or** an HTTP status ≥ 400. `status` filters to one exact HTTP status (e.g. `404`). + +## Client UI automation (`list_ui_elements` / `get_ui_state` / `click_ui`) + +These query the client's **own UI** (menus, buttons, HUD) — not SDK7 scene UI, and not scene entities (`click_entity` is for those). They mirror how AltTester drives a build: enumerate the live UI object hierarchy and click real controls. Both UI systems the Explorer uses are covered: + +- **uGUI** — every registered `Selectable` (`Button`, `Toggle`, `InputField`, `Slider`, `Dropdown`) via `Selectable.allSelectablesArray`. +- **UI-Toolkit** — every focusable element under each active `UIDocument`'s `rootVisualElement`. + +`list_ui_elements` returns `{path, name, type, system (ugui|uitk), interactable, visible, text?}` per element; `nameFilter` keeps only those whose name or path contains the substring (case-insensitive). Each `path` is system-prefixed (`ugui:/…` or `uitk:/…`) so a later call can re-resolve it against a fresh walk of the hierarchy. + +`get_ui_state` and `click_ui` take an `element` — a `path` from `list_ui_elements` or a plain element **name** (resolution prefers an exact path, then an exact name, then a loose path-suffix/contains match). `click_ui` performs a real activation: a full pointer down/up/click on a uGUI Selectable, or a navigation-submit on a UI-Toolkit control (what activating a focused `Button`/`Toggle` does). It returns the element's post-click state with `clicked: true`; read back with `get_ui_state` or `screenshot` to confirm the effect. + +> Paths are a single-frame snapshot: list, then act promptly. `text` is best-effort — uGUI reads a child `UnityEngine.UI.Text`, a `Toggle`'s on/off, or an `InputField`'s value (TextMeshPro labels are not read); UI-Toolkit reads a `TextElement`/`Toggle`/`TextField`. Enumeration and clicking touch Unity UI objects and run on the MCP main-thread hop. + +## Structured output + +`get_player_state`, `get_scene_state` and `list_scene_entities` also return `structuredContent` mirroring their text payload and declare a matching `outputSchema` in `tools/list` (MCP 2025-06-18). This is done **only as an example on the read-only state tools that benefit from it now** — every other tool returns text content only. A tool opts in by overriding `IMcpTool.OutputSchema` (default `null`); the same `McpInputSchema` builder produces the schema. + +## The scene-iteration loop + +1. Serve the scene and launch the Explorer in one step: `npm run start -- --mcp` in the scene folder (serves at `http://127.0.0.1:8000`, auto-launches the installed client against it with the MCP server on, and hot-reloads on file changes). +2. To use a specific Explorer build instead, serve with `npm run start -- --no-client` and launch manually: + +```bash +open Decentraland.app --args \ + --realm http://127.0.0.1:8000 --local-scene true --position 0,0 \ + --debug --skip-auth-screen --skip-version-check true \ + --mcp --windowed-mode --resolution 1280x720 +``` + +Optional determinism flags for stable screenshots: `--disable-hud`, `--skybox-time-enabled false`, `--landscape-terrain-enabled false`, `--skip-minimum-specs-screen`. + +3. The agent then loops: edit scene TypeScript → LSD hot reload applies it (or call `reload_scene`) → `get_scene_state` until ready → `screenshot` + `get_scene_logs` → verify → repeat. + +Once loading completes, the server announces its address in the scene debug console (available with local scene development or `--scene-console`): `MCP server listening on http://127.0.0.1:8123/unity-explorer-mcp`. A startup failure (port in use) is announced there as an error instead. The same line lands in the `get_scene_logs` buffer, so agents can confirm the server from inside the loop. + +A user-invokable Claude Code skill wrapping this loop lives at `.claude/skills/mcp-scene-iteration/` (invoke with `/mcp-scene-iteration`). + +## Troubleshooting + +- **Port already in use** — the server logs an `MCP` category error and stays inert; relaunch with a different `--mcp-port`. Multiple Explorer instances (`--multi-instance`) each need their own port. To confirm which process answers on a port, check `serverInfo.pid` in the `initialize` response and the `address` field of `get_player_state`. +- **HTTP 403** — the request carried a non-localhost `Origin` header; MCP clients and curl don't send one. +- **Server won't start on Windows** — `HttpListener` may require a URL ACL depending on machine policy: `netsh http add urlacl url=http://127.0.0.1:8123/unity-explorer-mcp/ user=Everyone` (elevated prompt), then relaunch. +- **Verbose logs** — enabling the server registers a scene-console log handler, which turns on unconditional verbose logging for the session (same behavior as `--scene-console`). +- **Scene entity dumps** — `list_scene_entities`/`get_entity_details` read the scene world without acquiring its sync lock (same as the existing `WorldInfoTool` debug tooling); treat results as a diagnostic snapshot. +- **`click_entity` returns `hit:false` with `blockedBy*`** — another collider sits on the camera→target line; `move_to`/`look_at` to a clear vantage and retry. If the reason is "out of range", close within the entity's `maxDistance` (default 10 m) first. Entities whose collider sits away from the pivot (GLTF meshes) may need an explicit `x/y/z` aim point. + +## Implementation map + +- `Explorer/Assets/DCL/McpServer/` — feature root, its own `DCL.McpServer` assembly. Two folders are folded into other assemblies via `.asmref` so they can reach code that assembly doesn't reference: + - `Core/` — protocol, transport and tool contract: `McpHttpServer` (`HttpListener` server + Origin validation), `McpJsonRpcDispatcher` (JSON-RPC 2.0 routing; `PROTOCOL_VERSION` `2025-06-18`), `IMcpTool`, `McpToolsRegistry`, `McpToolResult`, `McpToolAnnotations` (behaviour hints), `McpInputSchema` (typed input-schema builder). + - `Tools/` — one class per tool (20). + - `Components/` — ECS components for the input-driving tools: `McpMovementOverride`, `McpPointerClickIntent`. + - `Systems/` — **folded into `DCL.Plugins`** via `.asmref`: `McpServerPlugin` (builds the registry and hosts the server in `InjectToWorld`), `McpInputOverrideSystem` (held movement), `McpPointerClickSystem` (synthetic entity clicks). + - `Utils/` — `SceneLogBuffer`, `JObjectExtensions`, `UiAutomation` (uGUI + UI-Toolkit walk/click) and its pure `UiElementPath` (path building/matching). + - `Tests/` — EditMode tests **folded into `DCL.EditMode.Tests`** via `.asmref`: dispatcher / registry / result routing, the pointer-click system, the network-log ring buffer and UI-element path rules. +- Network capture: `get_network_log` reads a `McpNetworkLogBuffer` fed by an `McpNetworkAnalyticsHandler` (an `IWebRequestAnalyticsHandler`), both in `Explorer/Assets/DCL/WebRequests/Analytics/` (`DCL.Network` assembly). `WebRequestsContainer` constructs them, adds the handler to the `WebRequestsAnalyticsContainer` alongside the Chrome DevTools handler, and exposes the buffer; `DynamicWorldContainer` passes it to `McpServerPlugin`. +- Gating: `FeatureId.MCP_SERVER` in `FeaturesRegistry` (resolved as `appArgs.HasFlag(MCP) || appArgs.HasFlag(MCP_PORT)`); `DynamicWorldContainer.CreateAsync` reads `FeaturesRegistry.Instance.IsEnabled(FeatureId.MCP_SERVER)` and adds `McpServerPlugin`. +- Flags: `AppArgsFlags.MCP` / `AppArgsFlags.MCP_PORT`; log category: `ReportCategory.MCP`.