diff --git a/.claude/agents/mcp-server-engineer.md b/.claude/agents/mcp-server-engineer.md new file mode 100644 index 00000000000..8d5c2abb97c --- /dev/null +++ b/.claude/agents/mcp-server-engineer.md @@ -0,0 +1,97 @@ +--- +name: mcp-server-engineer +description: Own the embedded MCP automation server in unity-explorer — design and implement MCP tools, protocol/transport changes, and the mcp-scene-iteration skill so coding agents can see and drive a running Explorer build +skills: + - code-standards + - async-programming + - ecs-system-and-component-design + - plugin-architecture + - feature-flags-and-configuration + - diagnostics-and-logging + - scene-runtime-and-crdt +--- + +# MCP Server Engineer + +You own the embedded MCP (Model Context Protocol) server inside the Decentraland Unity Explorer: the feature that lets external coding agents observe the running client (screenshots, player/scene state, scene JS logs) and control it (teleport, movement, camera, chat commands, scene reload). You design and implement new MCP tools, maintain the transport/protocol layer, and keep the agent-facing docs and skill in sync with the server. + +Read [`docs/mcp-automation.md`](../../docs/mcp-automation.md) before touching anything — it is the human-facing contract for this feature. + +## MANDATORY: plan mode before new tools + +Implementing a new MCP tool (or changing server behavior) **must go through plan mode first**: research the codebase, present the plan, and get explicit user approval before writing any code. This is a standing user instruction — it applies even when the request looks trivial. + +Tool requests from agent sessions accumulate in the **"Wanted tools"** section of [`.claude/skills/mcp-scene-iteration/SKILL.md`](../skills/mcp-scene-iteration/SKILL.md) (name, args, output shape, blocked use case). Check it when asked to extend the server, and remove entries once implemented. + +## Architecture map + +Everything lives in `Explorer/Assets/DCL/McpServer/`, its own `DCL.McpServer` assembly (root asmdef). Two subfolders are folded into other assemblies via `.asmref`: `Systems/` → `DCL.Plugins` (GUID `fc4fd35fb877e904d8cedee73b2256f6`; `DynamicWorldContainer` lives there) and `Tests/` → `DCL.EditMode.Tests`. + +| Piece | Path | Role | +|---|---|---| +| Plugin | `Systems/McpServerPlugin.cs` | Builds the tool registry in `InjectToWorld` (needs `GlobalPluginArguments.PlayerEntity/SkyboxEntity`), starts/disposes the server, wires the scene-log tap (`DiagnosticsContainer.AddDebugConsoleHandler`) | +| Core (transport + protocol + contract) | `Core/McpHttpServer.cs`, `McpJsonRpcDispatcher.cs`, `IMcpTool.cs`, `McpToolsRegistry.cs`, `McpToolResult.cs`, `McpToolAnnotations.cs`, `McpInputSchema.cs` | `HttpListener` on `http://127.0.0.1:{port}/unity-explorer-mcp` (endpoint path is single-sourced in `McpHttpServer` — commit `7a734c77b`); POST → dispatch, GET → 405, Origin allowlist, 1 MB body cap; JSON-RPC 2.0 over Streamable HTTP (spec 2025-06-18), tools-only capability | +| Tools | `Tools/*.cs` (one class per tool, 16) | The agent-facing surface | +| Components | `Components/` — `McpMovementOverride`, `McpPointerClickIntent` | ECS intents for the input-driving tools | +| Systems | `Systems/McpInputOverrideSystem.cs` (held movement), `Systems/McpPointerClickSystem.cs` (synthetic entity clicks) | Per-frame/pipeline-integrated drivers | +| Utils | `Utils/SceneLogBuffer.cs`, `JObjectExtensions.cs` | Log tap buffer, args parsing | + +Registration: `DynamicWorldContainer.CreateAsync`, gated on `FeaturesRegistry` `FeatureId.MCP_SERVER` = `appArgs.HasFlag(MCP) || appArgs.HasFlag(MCP_PORT)` — so `--mcp-port` alone implies `--mcp` (presence check; an invalid port value still enables the server and falls back to 8123). Flags accepted from CLI **or** deep link by user decision — do not add CLI-only enforcement without being asked. Log category: `ReportCategory.MCP`. + +Cross-repo launch path: `@dcl/sdk-commands` (`../js-sdk-toolchain`, `packages/@dcl/sdk-commands/src/commands/start/{index,explorer-alpha}.ts`) forwards `--mcp` / `--mcp-port` from `npm run start` into the `decentraland://` deep link (`mcp=true` / `mcp-port=`), plus arbitrary params after a second standalone `--`. Flag renames or deep-link changes must stay consistent across both repos. + +Request flow: `HttpListener` accepts on the thread pool → detached `UniTaskVoid` per request → dispatcher parses/routes → tool's `ExecuteAsync(JObject, ct)` begins with `await UniTask.SwitchToMainThread(ct)` for any ECS/Unity access → heavy encoding (base64) hops back via `DCLTask.SwitchToThreadPool()`. + +## Adding a tool — checklist + +1. One class in `Tools/`, implementing `IMcpTool` (`Name` snake_case, 1–2 sentence `Description` written for an agent, `InputSchema` as a `JObject` built with the `McpInputSchema` builder — malformed schemas fail at registration, not first use; `Annotations` behaviour hints; override the default-null `OutputSchema` only for tools returning `McpToolResult.TextWithStructured`). +2. Parse args with the `JObjectExtensions` helpers (`GetBool`/`GetInt`/`GetFloat`/`GetString` with defaults); validate before switching threads; expected failures return `McpToolResult.Error(...)` (never throw — JSON-RPC errors are for protocol-level failures only). +3. Register it in `McpServerPlugin.InjectToWorld`; dependencies must be readable from `DynamicWorldContainer.CreateAsync` scope (never mutate containers). +4. ECS writes go through **intent components** — reuse `GlobalWorldActions` (`MoveAndRotatePlayerAsync`, `RotateCamera`, `TriggerEmote`) or `IChatMessagesBus` / `ECSReloadScene` / `IWorldInfoHub` before inventing anything. A new `BaseUnityLoopSystem` is justified only when a value must be re-asserted every frame against a real-input system (see `McpInputOverrideSystem`, ordered `[UpdateAfter(typeof(UpdateInputMovementSystem))]`). +5. Long-running tools own an explicit timeout and return a truthful text result on expiry (see `TeleportTool` polling + deadline). +6. Update **all three** agent-facing surfaces: tool catalog in `docs/mcp-automation.md`, `docs/app-arguments.md` if flags changed, and the skill if the loop changes. + +## Hard rules + +- **Security invariants**: bind 127.0.0.1 only; keep the Origin allowlist in `McpHttpServer.IsAllowed` (absent Origin = CLI = allowed; non-localhost = 403). No auth token by design (v1). +- **Texture memory discipline** (standing user requirement): screenshots must never accumulate textures. Temp RTs via `GetTemporary`/`ReleaseTemporary` released in `finally`; the `ScreenCapture.CaptureScreenshotAsTexture()` result destroyed immediately after blitting; the ReadPixels fallback reuses one persistent buffer; concurrent captures rejected via an `Interlocked` gate. +- **Async rules**: ignore `OperationCanceledException`; `ReportHub.LogException(e, ReportCategory.MCP)` for the rest; no `ThrowIfCancellationRequested()` in exception-free flows. +- **No LINQ**, ReportHub not Debug.Log, nullable annotations, no `!` null-forgiving operator. + +## Known pitfalls (learned the hard way) + +- `DCL.Time` namespace shadows `UnityEngine.Time` inside any `DCL.*` namespace — always write `UnityEngine.Time.time` fully qualified. +- `CachePhysicsTick`/`GetPhysicsTickComponent` exist in BOTH `DCL.CharacterMotion` and `DCL.Input` `WorldExtensions` — importing both namespaces is a CS0121 ambiguity. Import only `DCL.Input` (needed for `InputGroup` anyway). +- `ref` locals (`TryGetRef`) are illegal in async methods (CS8177) — use `world.TryGet` copies in tools; `TryGetRef` only in synchronous system `Update`. +- `Camera.Render()` is unsupported under URP — the `worldOnly` screenshot uses a one-frame `camera.targetTexture` redirect instead. +- `UpdateInputMovementSystem` overwrites `MovementInputComponent` every frame (and zeroes it when the action map is disabled) — held input must be re-asserted by a system ordered after it, not written once. +- Complete all `ref` component reads before any structural change (`Remove`/`Add`) — copy what you need (e.g. the `Completion` source) first. +- Unity generates `.meta` files for new files on the next Editor open; you cannot compile from the CLI — the user verifies in the Editor or a manual build and pastes compile errors back. + +## Skill stewardship + +The agent-side workflow lives in `.claude/skills/mcp-scene-iteration/` (user-invokable only). Field sessions edit it with verified learnings — treat their additions as ground truth about real behavior and never revert them blindly. The bundled `scripts/screenshot.sh` captures frames to disk via raw JSON-RPC so agents don't burn context on frequent screenshots; keep it working if the tool schema changes. + +## Verification + +EditMode tests exist in `Tests/` (folded into `DCL.EditMode.Tests` via asmref): dispatcher, registry, result routing, input schema, HTTP server, state tools, pointer-click system. Run them in the Unity Test Runner — you cannot compile or run tests from the CLI. Smoke-test the protocol layer with the running client: + +```bash +curl -s -X POST http://127.0.0.1:8123/unity-explorer-mcp -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' +``` + +Editor run: add `--mcp` to `Main Scene Loader → Debug Settings → App Parameters` in `Assets/Scenes/Main.unity` and hit Play. Standalone against a local scene: `npm run start -- --mcp` in the scene folder auto-launches the installed client with the server on (`--mcp-port ` for another port, `--no-client` to serve only, `--multi-instance` + distinct `--port`/`--mcp-port` for side-by-side instances). Full launch lines are in `docs/mcp-automation.md`. + +## Git rules + +**NEVER commit or push.** All work stays as local changes — the user decides when and what to commit. + +Allowed: `git checkout -b`, `git diff`, `git status`, `git log`, `git branch` +Forbidden: `git commit`, `git push`, `git merge`, `git rebase` + +## Roadmap context + +Milestone 2 (pointer clicks) SHIPPED 2026-07-05 as `click_entity` — implemented via **semantic injection**, not the originally-scoped synthetic `InputSystem` device: `McpPointerClickSystem` raycasts camera→target, mirrors the distance gate, and fills the entity's `PBPointerEvents.AppendPointerEventResultsIntent` so the unmodified `WritePointerEventResultsSystem` emits a byte-identical `PBPointerEventsResult`. Zero production interaction code changed; approach recorded in `~/.claude/plans/wondrous-forging-fox.md`. + +Current "Wanted tools" head: **recover_scene** — force-recreate a scene that dropped out of `ScenesCache` (`get_scene_state` → `scene: null`, the LSD hard-wedge from rapid saves where every existing reload path needs the cached facade). Implementation lead is in the skill's Wanted tools entry. diff --git a/.claude/skills/mcp-scene-iteration/SKILL.md b/.claude/skills/mcp-scene-iteration/SKILL.md new file mode 100644 index 00000000000..8632b6991f3 --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/SKILL.md @@ -0,0 +1,149 @@ +--- +name: mcp-scene-iteration +description: Iterate on SDK7 scenes against a running Explorer build via its embedded MCP server. +disable-model-invocation: true +--- + +# MCP Scene Iteration + +Drive a running Explorer build through its embedded MCP server to build and test SDK7 scenes autonomously. + +Full tool catalog and flag reference: [`docs/mcp-automation.md`](../../../docs/mcp-automation.md). + +Deeper reference, loaded only when the task reaches it: + +- [`reference/camera-and-movement.md`](reference/camera-and-movement.md) — before framing screenshots, free-camera sweeps, or navigating precise lines +- [`reference/assets.md`](reference/assets.md) — before placing, downloading, converting, or exporting any 3D model +- [`reference/visuals.md`](reference/visuals.md) — before tuning emissives/bloom, UI overlays, skybox time, or judging thin geometry + +## Setup (once per session) + +0. **Load the SDK skills.** This skill only covers driving the Explorer; the SDK7 API knowledge (composite-first rule, component reference) lives in the `sdk-scenes` skill set, and parts of the API (e.g. native `TriggerArea`) are newer than training data. Try to load it: session skills first, then the filesystem — scene-local (`.claude/skills/` in the scene folder) and global (`~/.claude/skills/`). If it cannot be loaded, **MANDATORY — ask the user**: install `sdk-skills` (from https://github.com/decentraland/sdk-skills)? Recommend installing. If YES, ask at which level — scene-local or global — and run the matching command: + + ```bash + npx skills add decentraland/sdk-skills --all # scene-local (run inside the scene folder) + npx skills add decentraland/sdk-skills --all -g # global (user-level, ~/.claude/skills) + ``` + + Skills are loaded at session start, so a mid-session install may not surface until the session restarts. If NO, move forward without them — the scene can still be implemented, just less efficiently: verify any API you are not certain about against the official docs instead of writing it from memory. + +1. **Probe for an already-running MCP server, then start the scene.** Check through the harness first: if `mcp__explorer__*` tools are available in the session, call `get_scene_state` — an answer means the server is up. Fall back to curl **only if the tools are absent**: + + ```bash + curl -s -m 2 http://127.0.0.1:8123/unity-explorer-mcp -X POST \ + -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \ + -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"probe","version":"1"}}}' + ``` + + **Server found** (tool answer or `serverInfo` result) — **MANDATORY — ask the user**: use the already-running Explorer, or start the scene from scratch with the MCP flag? Never decide silently. + - *Use it*: launch nothing. If port 8000 isn't serving the target scene folder (`lsof -nP -i :8000 -sTCP:LISTEN`, then check the PID's cwd), kill whatever holds it and run `npm run start -- --no-client`. Skip step 3 if the tools are already available. + - *From scratch*: **MANDATORY — follow-up question**: kill the previously-running scene server, or keep it and run a second stack alongside? Never kill it unasked. + - *Kill it*: kill the port-8000 dev server, have the user close the running client (never kill an Editor process yourself), then continue below. + - *Keep it*: leave it and its Explorer untouched; start a second stack on its own ports — a different dev-server port (`--port`; the launched client follows it automatically), a different MCP port (`--mcp-port`, implies `--mcp`), and `--multi-instance` so a second Explorer instance can run concurrently: + + ```bash + npm install && npm run start -- --port 8666 --multi-instance --mcp-port 8124 + ``` + + From here on use the chosen ports instead of 8000/8123 — including step 3's registration, which needs a distinct server name (e.g. `claude mcp add --transport http --scope user explorer2 http://127.0.0.1:8124/unity-explorer-mcp`; the tools then surface as `mcp__explorer2__*`). + + **No server found** — serve the scene and launch the Explorer in one command from the scene folder (keep it running in the background; if something else already holds port 8000, apply the same kill-or-keep question and port overrides as above): + + ```bash + npm install && npm run start -- --mcp + ``` + + This serves the scene at `http://127.0.0.1:8000`, auto-launches the **installed** Decentraland client connected to it with the MCP server enabled (port 8123; `--mcp-port ` picks another and implies `--mcp` — adjust the 8123 URLs in steps 1 and 3 to match), and the LSD dev server hot-reloads the running Explorer on file changes. Other useful flags: `--port ` (dev-server port; the launched client follows it automatically), `--position x,y`, `--skip-auth-screen`, `-n` (new client instance), `--multi-instance` (allow concurrent Explorer instances), `--no-client` (serve only, launch nothing). Anything after a second standalone `--` is forwarded verbatim into the launch as extra Explorer params, e.g. `npm run start -- --mcp -- --windowed-mode --resolution 1280x720` (npm consumes the first `--`). If `--mcp` is rejected as an unknown option, the scene's `@dcl/sdk-commands` predates the flag — update `@dcl/sdk`, or fall back to step 2. + + **A freshly launched Explorer needs the user to log in.** The client opens on the auth screen unless a previous session's login is still cached (`--skip-auth-screen` only skips it when a valid identity exists — a missing or expired login shows it anyway, and extra `--multi-instance` instances always ask). Tell the user to log in, then wait — step 4's polling only starts succeeding once they're through, and only then can you continue working on the scene through the MCP server. + +2. **(Alternative) Launch a specific Explorer build manually** — a local build or the Editor instead of the installed client. Serve with `npm run start -- --no-client` in step 1, then: + + ```bash + # macOS + open /path/to/Decentraland.app --args \ + --realm http://127.0.0.1:8000 --local-scene true --position 0,0 \ + --debug --skip-auth-screen --skip-version-check true \ + --mcp --windowed-mode --resolution 1280x720 + ``` + + On Windows call `Decentraland.exe` with the same arguments. Add `--disable-hud --skybox-time-enabled false --landscape-terrain-enabled false` when you want deterministic screenshots without HUD noise. + +3. **Connect the MCP server** (default port 8123): + + ```bash + claude mcp add --transport http --scope user explorer http://127.0.0.1:8123/unity-explorer-mcp + ``` + + Errors with "already exists in local config" if registered by a previous session — that's fine, nothing to do. If the current session has no `mcp__explorer__*` tools, follow "Missing tools" under Scene health & recovery below — the fix is the user reconnecting via `/mcp`, not a workaround. + +4. Wait for the world to load: poll `get_scene_state` until `loadingScreenOn` is false and the scene reports `isReady: true`. + +## The iteration loop + +Repeat until **every requirement has proof**: a screenshot or state read demonstrating it, captured from a retail camera mode (`first_person`/`third_person`, not the free camera), with `get_scene_state` healthy and no unexplained errors in the logs. + +1. **Edit** the scene TypeScript in `src/` — the LSD dev server hot-reloads the running Explorer within a few seconds. If you need a deterministic reset instead, call `reload_scene`. +2. **Confirm the scene is healthy**: `get_scene_state` — a `state` of `JavaScriptError` or `EcsError` means your code crashed the scene runtime. +3. **Read the runtime output**: `get_scene_logs` with `sinceSeq` set to the last sequence number you saw. Scene `console.log` output and exceptions land here. +4. **Look and verify**: position the view (`teleport`, `move_to`, `walk`, `look_at` — for precise framing or free-camera sweeps read [`reference/camera-and-movement.md`](reference/camera-and-movement.md)), then `screenshot` and inspect the image against what the scene code should produce. +5. **Exercise behavior**: `walk` into trigger areas, `click_entity` on interactables, `send_chat` for commands, `trigger_emote`, and re-screenshot to verify reactions. `list_scene_entities` + `get_entity_details` show the scene's ECS state when visuals aren't enough. + +**Cross-examine** every conclusion: confirm each visual claim with a state read (ECS values via `get_entity_details`, logs, `get_player_state` position), and each state claim with pixels. One channel lies routinely — colliders exist that pixels don't show, entities render invisible while their state looks healthy, animations silently don't play. The reference files call out where cross-examination is mandatory. + +**MANDATORY — camera cleanup before finishing.** NEVER leave the camera in `free` mode when you stop working (end of task, handing back to the user, or pausing for their input): always restore it with `set_camera_mode third_person` as your last camera action, and confirm via `get_player_state` → `camera.mode` if anything in between could have failed. + +## Screenshot frequency & cost + +Every screenshot returned by the MCP `screenshot` tool lands in your context as an image (~1.2k tokens at 1280×720, scaling with pixel count). Occasional captures through the tool are fine; **frequent or burst captures must go through the bundled script instead**, which saves frames to disk (zero context cost) and prints only the caption: + +```bash +scripts/screenshot.sh -o shot.jpg # single frame to a file +scripts/screenshot.sh -n 10 -i 0.5 # burst: 10 frames every 0.5s into mcp-shots/ (time-based behavior: tweens, animations) +scripts/screenshot.sh -w 640 # cheap sanity-check resolution (~4x fewer tokens when you Read it) +scripts/screenshot.sh --world-only --png # UI-less lossless frame +``` + +Paths are relative to this skill's directory; requires curl + python3; pass `-p ` when not on 8123. Then `Read` only the frames you actually need to inspect — capture many, look at few. For before/after comparisons, capture both to disk and read just those two. Use `maxWidth` 640 for quick checks and 1280 only for final verification. Captures are serialized server-side (concurrent requests are rejected), so keep burst intervals ≥ 0.2s. + +## Scene health & recovery + +- Sequence-poll logs (`sinceSeq`) instead of re-reading the whole buffer; errors survive in the buffer even if they scrolled by. +- `scene.json` changes (parcels, spawn points) are not hot-reloaded — restart the `npm run start` process, then `reload_scene`. +- After `teleport` or `reload_scene`, always re-check `get_scene_state` before interacting; readiness can lag a few seconds. +- One parcel is 16×16 m; parcel `(x, y)` spans world positions `(16x..16x+16, 16y..16y+16)`. `--position 0,0` spawns at parcel 0,0. +- If the connection drops, the client probably crashed or was closed — relaunch it the same way it was started (`npm run start -- --mcp`, or the manual launch line); the MCP endpoint URL stays the same. +- **Missing tools**: `mcp__explorer__*` tools absent in-session are recoverable (typically the Explorer wasn't running when the session started, so the registered server failed its startup connection). Ask the user to run `/mcp` and reconnect the `explorer` server — an interactive command only the user can run; a successful reconnect binds all the server's tools into the running session (verified). A plain `claude mcp add` mid-session does NOT surface tools by itself. Last resort: drive the endpoint directly with curl JSON-RPC (`POST /unity-explorer-mcp`, methods `initialize` then `tools/call`; responses may be SSE-framed, tool payloads are JSON in `result.content[0].text`, screenshots are base64 in image content blocks). +- After a hot reload the player can end up off-parcel (e.g. parcel `0,-1`); `get_scene_state` then reports a null scene and `reload_scene` fails with "no scene at the current parcel". Check `get_player_state` → `parcel`, `move_to` back inside, and the scene loads again. +- Each file save triggers a rebuild: editing usage and import in separate saves produces a transient `SceneError: X is not defined` between them. Write new modules before wiring them in, and prefer a single whole-file write for multi-part edits to one file. +- **Rapid successive saves can HARD-WEDGE the client (verified 2026-07-10).** Two saves seconds apart made the Explorer load a mid-write bundle → `SyntaxError: Invalid or unexpected token` at scene start → the scene facade is torn down and drops out of `ScenesCache`, and `get_scene_state` reports `scene: null` while standing on the parcel. From that state NOTHING recovers in-session: `reload_scene` errors ("no scene at the current parcel" — its guard and every underlying reload path need the scene still cached), `/reload` hangs until cancelled, the minimap RELOAD SCENE button just sends `/reload`, LSD file-save pushes no-op (`TryGetBySceneId` misses), and moving far off-parcel and back does not recreate the facade. Only exiting/re-entering play mode (editor) or relaunching the standalone build recovers. Prevention: batch multi-edit changes into ONE file write, and after any save landing seconds after a previous one, verify `get_scene_state` still shows a scene before saving again. +- The `teleport` tool silently no-ops in local-scene-development mode: `/goto` teleports are disallowed there (chat shows "Teleport is not allowed in local scene development mode") but the tool still answers "Arrived at (x,y)". Use `move_to` for repositioning in LSD sessions. +- The Explorer under test may be the **Unity Editor in play mode**, not a standalone Decentraland.app — check `ps aux | grep -i unity` before considering a relaunch. Never kill the editor process; recovery from a wedged client is then a user action (exit/re-enter play mode). + +## Interaction testing + +- `click_entity` presses a pointer button on a scene entity (get ids from `list_scene_entities`). The target needs a `PointerEvents` component and a collider; the aim is validated by a real camera-origin raycast, so occluders return `hit:false` + `blockedBy*` (reposition and retry) and the entity's `maxDistance` (default 10 m) applies — get close first. `upRayMissed: true` means the target moved between press and release (e.g. a door starting to swing) and the release was delivered with the press-frame hit. For GLTF entities whose collider sits away from the pivot, pass an explicit `x/y/z` aim point. The player must be standing on the scene's parcel — off-parcel clicks fail with "no running current scene". +- `walk` moves relative to the camera and requires an explicit direction: pass `directionY: 1` for forward (`directionX` strafes); omitting both errors with "directionX and directionY must not both be zero". +- Collider checks beat pixels for physics (cross-examine): `look_at` straight at the target, `walk` forward, then compare `get_player_state` positions to prove passage or blockage. +- Trigger areas fire `onTriggerEnter` immediately after `reload_scene` if the player is already standing inside one — reposition the player outside all triggers before testing enter/exit sequencing (and treat post-reload trigger logs as stale state, not gameplay). + +## Improving this skill + +Two rules: + +**1. Learned something the skill didn't tell you?** A flag that turned out to be required, a timing quirk, a better verification pattern, or information here that proved wrong — edit this skill in place before finishing your session. Keep additions terse and verified (facts you observed, not speculation), and file them where their branch lives: setup, health/recovery, and interaction facts in this SKILL.md; camera and navigation facts in `reference/camera-and-movement.md`; model and import facts in `reference/assets.md`; rendering and visual-tuning facts in `reference/visuals.md`. The canonical copy lives in the unity-explorer repo at `.claude/skills/mcp-scene-iteration/`; if you are running from a copy (e.g. `~/.claude/skills/`), apply the same edit to the canonical copy too when the repo is accessible, or tell the user to sync it. + +**2. Missing a capability?** If the loop is blocked because no existing MCP tool can do what you need (e.g. pressing a specific key, reading a value no tool exposes), do NOT work around it by modifying the Explorer client, the MCP server, or the unity-explorer repo yourself. Stop and prompt the user with a concrete tool proposal: + +- proposed tool name and one-line purpose +- input arguments (names, types, defaults) +- expected output shape +- the blocked use case, and why the existing tools can't cover it + +The user decides whether and when to implement it. **MANDATORY: implementing an approved tool must go through plan mode first** — whichever session does the implementation starts in plan mode, researches the unity-explorer codebase (the server lives under `Explorer/Assets/DCL/McpServer/` — see `docs/mcp-automation.md` → Implementation map), and presents the plan for user approval before writing any code. Also append the proposal to the "Wanted tools" list below so it isn't lost if the user defers. + +## Wanted tools + +Proposals from agent sessions — name, purpose, blocked use case. Remove entries once implemented. + +- **recover_scene** — force-recreate the scene at the player's parcel when it has dropped out of `ScenesCache` (`get_scene_state` → `scene: null`; the hard-wedge state described above, where every existing reload path needs the cached facade and the session is dead until the user restarts play mode). Inputs: `timeoutSec?: number` (default 30). Output: same shape as `reload_scene`. Implementation lead: clear failed `AssetPromise` state on the definition entity and reset `StaticScenePointers.Promise` on the realm entity so the static-pointer systems re-resolve — the same reset `ECSReloadScene.DisposeAndRestartAsync` already performs for LSD, minus the requirement that a live scene exists. diff --git a/.claude/skills/mcp-scene-iteration/reference/assets.md b/.claude/skills/mcp-scene-iteration/reference/assets.md new file mode 100644 index 00000000000..c0a4cd8183a --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/reference/assets.md @@ -0,0 +1,22 @@ +# 3D asset reference + +Read before placing, downloading, converting, or exporting any 3D model. + +## Picking from the sdk-skills catalog + +- Fetch the `preview:` thumbnail BEFORE downloading — some models render near-black or broken even in their own previews (e.g. arcade-cabinet-atari). Use the exact `[anim: ...]` clip names; a wrong clip name fails silently (no error, no motion) — cross-examine: burst-capture (`-n 3 -i 1`) and diff frames to prove an animation is actually running. +- Catalog dims/pivots lie: bounding boxes can include baked animation paths or outline meshes, pivots can sit mid-crown (palms) or nowhere near the mesh (a baked drive path carried one car's mesh entirely off-parcel — swap such models rather than debug them). Place → screenshot → adjust is faster than trusting listed sizes. + +## Downloading models + +- Free CC0 model sources that download cleanly via curl: kenney.nl (zip URL is on the asset page, FBX+OBJ+GLTF inside), and itch.io free packs via the scripted flow: POST `/download_url` with the page's csrf_token → GET the returned key URL → grab `data-upload_id` → POST `/file/?source=game_download` → signed CDN URL (expires ~60s, download immediately). +- Downloaded GLBs into the scene folder hot-load without restarting the dev server. Many props ship with no colliders — cross-examine solidity: walk onto them and check the player's `y` via `get_player_state`; add `visibleMeshesCollisionMask: 3` for anything that should be solid. + +## Blender authoring & conversion + +- Blender-authored GLBs work end-to-end: export with `bpy.ops.export_scene.gltf(use_selection=True)` straight into the scene's Models folder (hot-loads like any file change). Set the object origin to bottom-center before export (`cursor to (0,0,0)` + `origin_set(type='ORIGIN_CURSOR')` with geometry built up from z=0) so `position.y = 0` grounds the model, and `transform_apply` rotation/scale for clean transforms. Principled BSDF emissive (Emission Color + Strength) renders as expected in Explorer, including the zero-channel neon saturation rule (see `visuals.md`). +- Converting downloaded FBX/OBJ to GLB in Blender works, with three verified traps: (1) FBX materials can import with Principled Alpha = 0 (FBX transparency-factor quirk, seen on Quaternius packs) — the GLB then has `alphaMode: MASK` with baseColor alpha 0 and the model is INVISIBLE in Explorer while its entity, tween and logs all look healthy; force Alpha=1 + `blend_method='OPAQUE'` before export, and when a GLB renders nothing, parse its JSON chunk (nodes/materials) instead of guessing. (2) The glTF exporter's default animation mode exports EVERY action in the .blend that fits the armature, so clips from other imported models leak into each GLB; use `export_animation_mode='ACTIVE_ACTIONS'` with the right action active (exports one clip named `Animation`). (3) Some kits (e.g. Kenney furniture) ship ASCII FBX which Blender refuses — run `file *.fbx` first and fall back to the kit's OBJ folder. Skinned meshes respect entity Transform scale, so oversize rigs can be scaled at the entity. + +## Composite gotchas + +- Composite-authored box primitives need `"box": {"uvs": []}` in `core::MeshRenderer` — a bare `"box": {}` crashes `sdk-commands build` with `TypeError: message.uvs is not iterable`. diff --git a/.claude/skills/mcp-scene-iteration/reference/camera-and-movement.md b/.claude/skills/mcp-scene-iteration/reference/camera-and-movement.md new file mode 100644 index 00000000000..54139de0a53 --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/reference/camera-and-movement.md @@ -0,0 +1,18 @@ +# Camera & movement reference + +Read before framing screenshots, navigating precise lines, or inspecting a build from multiple viewpoints. + +## Aiming the camera + +- `move_to`'s `lookAt*` params orient the avatar but NOT the third-person camera; `screenshot` and `walk` follow the camera. Call the standalone `look_at` tool (it aligns camera yaw — confirm via `get_player_state` → `camera.rotationEuler.y`) before walking a precise line or framing a screenshot. +- `look_at` lines the third-person camera up through the avatar, so the avatar occludes exactly the thing you framed. To photograph a subject, first `move_to` a spot offset sideways from the camera→subject line, then `look_at`. + +## Free camera + +- The free camera is the fastest way to inspect a scene from many points of view: `set_camera_pose` places it at any absolute position, optionally aims it (`lookAt*`) and sets `fov`, auto-entering free mode. Repositioning while already free is instant (~200ms), so sweep a build cheaply — aerial plan view, each facade, eye-level details, interiors — capturing to disk between calls, instead of walking the player around. `look_at` also works in free mode (aims from the camera's own position), and the free camera stays put while the player moves, so you can even watch the avatar walk through the scene from a fixed vantage. Entering free from another mode blends over ~2-3s (the tool waits and reports `settled`). +- The free camera is a debug view, not what players see. To confirm the end-user experience, switch back to the real modes — `set_camera_mode` `first_person` / `third_person` / `drone` are exactly the cameras retail users have — and re-check framing, avatar occlusion, and interaction reach from there (e.g. verify a hover target is actually visible and clickable at player height, not just from a flattering freecam angle). **MANDATORY: never leave the camera in free mode when you finish working** — always restore `set_camera_mode third_person` as your last camera action. `set_camera_mode` respects scene locks and errors truthfully — check `get_player_state` → `camera.modeChangeAllowed` first; `false` inside a `CameraModeArea`/scene virtual camera is correct behavior worth verifying, not a tool failure. `screenshot` works in any mode. + +## Precise navigation + +- Collider-bump navigation moves the player along precise lines reliably: `look_at` a point past where you want to end up, `walk` with generous `seconds`, and let a blocking collider stop the player — the returned `endPosition` lands ~0.38m (capsule radius) short of the collider face, a deterministic waypoint for the next leg with no duration tuning. Timing legs precisely is fragile (jog ramp-up varies effective speed); overshooting into a collider is not. Only a final leg through a gap with nothing beyond it needs a tight duration, or the player runs off-parcel. +- Measured locomotion speeds (flat ground): jog ≈6.5 m/s once ramped, `kind: "walk"` ≈1.4 m/s. To stop at a precise point no collider will stop you at, take one timed jog leg, read `endPosition`, then micro-correct with 0.5-1.2s `kind: "walk"` legs — position feedback plus slow corrections converges in 1-2 iterations where a single timed jog leg won't. diff --git a/.claude/skills/mcp-scene-iteration/reference/visuals.md b/.claude/skills/mcp-scene-iteration/reference/visuals.md new file mode 100644 index 00000000000..8325bac0895 --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/reference/visuals.md @@ -0,0 +1,9 @@ +# Visual tuning reference + +Read before tuning emissives/bloom, UI overlays, skybox time, or judging visibility of thin geometry. + +- `screenshot`'s `worldOnly` (and the script's `--world-only`) renders with full post-processing, bloom included (fixed + verified 2026-07-06: the capture target was LDR, which clamped emissives and starved bloom — it is HDR now). Emissive/glow looks are judgeable from world-only frames, with one caveat: halo spread reads slightly tighter than the full-view capture (render-resolution difference), so do final pixel-exact glow tuning on full-view frames. +- Neon emissive recipe (verified with bloom, sunset skybox): pin one emissive channel to exactly 0 (e.g. `(1, 0, 0.6)` magenta) — a small minor channel (0.15-0.2) times `emissiveIntensity` 6+ whites the whole surface out. Then weight by luminance and background: green-heavy hues (cyan) bloom to white far faster than magenta and additively desaturate against the pink haze (cyan + pink = white), so dim or blue-shift them (`(0, 0.35, 0.8)` works); magenta reinforces the sunset and stays saturated at intensity 6-15. A large on-screen emitter always whites out at its core — the hue lives in the halo, which is what a real neon tube looks like; judge hue from gameplay distance, not close-ups. Bloom already provides the halo, so an emitter needs no alpha-blended "glow shell" box around it — a shell just reads as a grey display case. +- UiBackground texture entities render far stronger than a near-zero `color` alpha suggests (a 0.05-alpha noise overlay still fully obscured the image beneath). For flicker/glitch overlays, mount/unmount the entity conditionally (`{cond ? : null}`) instead of animating alpha near zero. +- Thin geometry (a door panel ~0.1m) is invisible edge-on: a panel "open" at ~90-110° reads as a sliver from the front. Cross-examine pose via ECS rotation (`get_entity_details` quaternion) as well as pixels, and prefer open angles like ~135° when front visibility matters. +- `SkyboxTime.create(engine.RootEntity, { fixedTime })` pins the scene to a permanent time of day regardless of launch flags. diff --git a/.claude/skills/mcp-scene-iteration/scripts/screenshot.sh b/.claude/skills/mcp-scene-iteration/scripts/screenshot.sh new file mode 100755 index 00000000000..1b4ed6a879b --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/scripts/screenshot.sh @@ -0,0 +1,120 @@ +#!/usr/bin/env bash +# Capture Explorer screenshots to disk via the embedded MCP server, without spending agent context. +# Frames are saved as files; only the caption (resolution + parcel) is printed. Read a frame file +# only when you actually need to inspect it. +# +# Usage: screenshot.sh [options] +# -o, --out FILE output file (single capture; extension follows quality) +# -d, --dir DIR output directory (default: mcp-shots/; used for bursts and when -o is omitted) +# -n, --count N number of frames to capture (default: 1) +# -i, --interval SEC seconds between frames in a burst (default: 0.5; keep >= 0.2, captures are serialized) +# -w, --max-width PX maxWidth passed to the tool (default: 1280; use 640 for cheap sanity checks) +# --png capture PNG instead of JPG +# --world-only exclude UI overlays (worldOnly: true) +# -p, --port PORT MCP server port (default: 8123) +# +# Requires: curl, python3. The Explorer must be running with --mcp. + +set -euo pipefail + +PORT=8123 +MAX_WIDTH=1280 +QUALITY=jpg +WORLD_ONLY=false +COUNT=1 +INTERVAL=0.5 +OUT="" +OUT_DIR="mcp-shots" + +while [[ $# -gt 0 ]]; do + case "$1" in + -o|--out) OUT="$2"; shift 2 ;; + -d|--dir) OUT_DIR="$2"; shift 2 ;; + -n|--count) COUNT="$2"; shift 2 ;; + -i|--interval) INTERVAL="$2"; shift 2 ;; + -w|--max-width) MAX_WIDTH="$2"; shift 2 ;; + --png) QUALITY=png; shift ;; + --world-only) WORLD_ONLY=true; shift ;; + -p|--port) PORT="$2"; shift 2 ;; + -h|--help) sed -n '2,17p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; + *) echo "unknown argument: $1 (see --help)" >&2; exit 2 ;; + esac +done + +capture_one() { + local target_file="$1" + + local payload + payload=$(printf '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"screenshot","arguments":{"maxWidth":%s,"quality":"%s","worldOnly":%s}}}' \ + "$MAX_WIDTH" "$QUALITY" "$WORLD_ONLY") + + # NOTE: response must land in a file, not a pipe into `python3 - </dev/null || true + capture_one "$OUT" + exit 0 +fi + +mkdir -p "$OUT_DIR" +stamp=$(date +%Y%m%d-%H%M%S) + +for ((frame = 1; frame <= COUNT; frame++)); do + capture_one "${OUT_DIR}/shot-${stamp}-$(printf '%03d' "$frame").${QUALITY}" + + if (( frame < COUNT )); then + sleep "$INTERVAL" + fi +done diff --git a/Explorer/Assets/DCL/Character/CharacterCamera/CinemachineExtensions.cs b/Explorer/Assets/DCL/Character/CharacterCamera/CinemachineExtensions.cs index 26aaa886bb3..8034a14805d 100644 --- a/Explorer/Assets/DCL/Character/CharacterCamera/CinemachineExtensions.cs +++ b/Explorer/Assets/DCL/Character/CharacterCamera/CinemachineExtensions.cs @@ -1,4 +1,5 @@ -using DCL.CharacterCamera.Components; +using Cinemachine; +using DCL.CharacterCamera.Components; using UnityEngine; namespace DCL.CharacterCamera @@ -16,9 +17,10 @@ public static void ForceFirstPersonCameraLookAt(this ICinemachinePreset cinemach { if (cinemachinePreset.FirstPersonCameraData.POV == null) return; - (float horizontalAxis, float verticalAxis) = GetHorizontalAndVerticalAxisForIntent(lookAtIntent); - cinemachinePreset.FirstPersonCameraData.POV.m_HorizontalAxis.Value = horizontalAxis; - cinemachinePreset.FirstPersonCameraData.POV.m_VerticalAxis.Value = verticalAxis; + // POV axes are in degrees, unlike the 0..1 orbit value the FreeLook rigs consume. + (float yawDegrees, float pitchDegrees) = GetYawPitchDegrees(lookAtIntent.LookAtTarget, lookAtIntent.PlayerPosition); + cinemachinePreset.FirstPersonCameraData.POV.m_HorizontalAxis.Value = yawDegrees; + cinemachinePreset.FirstPersonCameraData.POV.m_VerticalAxis.Value = pitchDegrees; } public static void ForceDroneCameraLookAt(this ICinemachinePreset cinemachinePreset, CameraLookAtIntent lookAtIntent) @@ -28,23 +30,58 @@ public static void ForceDroneCameraLookAt(this ICinemachinePreset cinemachinePre cinemachinePreset.DroneViewCameraData.Camera.m_YAxis.Value = verticalAxis; } - private static (float, float) GetHorizontalAndVerticalAxisForIntent(CameraLookAtIntent lookAtIntent) + /// + /// Places the free camera at an absolute world position (and optionally sets its field of view). + /// The free camera's position is its vcam transform, the same mechanism free-fly input moves it through. + /// + public static void ForceFreeCameraPose(this ICinemachinePreset cinemachinePreset, Vector3 position, float? fov = null) { - var eulerDir = Vector3.zero; - var cameraTarget = lookAtIntent.LookAtTarget; - float horizontalAxisLookAt = lookAtIntent.PlayerPosition.y - cameraTarget.y; - var verticalAxisLookAt = new Vector3(cameraTarget.x - lookAtIntent.PlayerPosition.x, 0, cameraTarget.z - lookAtIntent.PlayerPosition.z); + cinemachinePreset.FreeCameraData.Camera.transform.position = position; - if (verticalAxisLookAt is { x: 0, y: 0, z: 0 }) - verticalAxisLookAt = Vector3.forward; + if (fov.HasValue) + cinemachinePreset.FreeCameraData.Camera.m_Lens.FieldOfView = fov.Value; + } + + public static void ForceFreeCameraLookAt(this ICinemachinePreset cinemachinePreset, CameraLookAtIntent lookAtIntent) + { + CinemachinePOV? pov = cinemachinePreset.FreeCameraData.POV; - eulerDir.y = Vector3.SignedAngle(Vector3.forward, verticalAxisLookAt, Vector3.up); - eulerDir.x = Mathf.Atan2(horizontalAxisLookAt, verticalAxisLookAt.magnitude) * Mathf.Rad2Deg; + if (pov == null) + return; + + // The free camera is detached from the player, so the aim originates at the camera itself. + Vector3 origin = cinemachinePreset.FreeCameraData.Camera.transform.position; + (float yawDegrees, float pitchDegrees) = GetYawPitchDegrees(lookAtIntent.LookAtTarget, origin); + + pov.m_HorizontalAxis.Value = yawDegrees; + pov.m_VerticalAxis.Value = pitchDegrees; + } + + private static (float, float) GetHorizontalAndVerticalAxisForIntent(CameraLookAtIntent lookAtIntent) + { + (float yawDegrees, float pitchDegrees) = GetYawPitchDegrees(lookAtIntent.LookAtTarget, lookAtIntent.PlayerPosition); //value range 0 to 1, being 0 the bottom orbit and 1 the top orbit - float yValue = Mathf.InverseLerp(-90, 90, eulerDir.x); + float yValue = Mathf.InverseLerp(-90, 90, pitchDegrees); + + return (yawDegrees, yValue); + } + + /// + /// Yaw/pitch in degrees (Unity euler convention: positive pitch looks down) that point from origin at target. + /// + private static (float yawDegrees, float pitchDegrees) GetYawPitchDegrees(Vector3 target, Vector3 origin) + { + float heightDelta = origin.y - target.y; + var flatDirection = new Vector3(target.x - origin.x, 0, target.z - origin.z); + + if (flatDirection is { x: 0, y: 0, z: 0 }) + flatDirection = Vector3.forward; + + float yawDegrees = Vector3.SignedAngle(Vector3.forward, flatDirection, Vector3.up); + float pitchDegrees = Mathf.Atan2(heightDelta, flatDirection.magnitude) * Mathf.Rad2Deg; - return (eulerDir.y, yValue); + return (yawDegrees, pitchDegrees); } } } diff --git a/Explorer/Assets/DCL/Character/CharacterCamera/Systems/ApplyCinemachineCameraInputSystem.cs b/Explorer/Assets/DCL/Character/CharacterCamera/Systems/ApplyCinemachineCameraInputSystem.cs index a89d0102fb8..57bd1e99ac7 100644 --- a/Explorer/Assets/DCL/Character/CharacterCamera/Systems/ApplyCinemachineCameraInputSystem.cs +++ b/Explorer/Assets/DCL/Character/CharacterCamera/Systems/ApplyCinemachineCameraInputSystem.cs @@ -81,6 +81,8 @@ private void ForceLookAt(in Entity entity, in CameraComponent camera, ref ICinem cinemachinePreset.ForceFirstPersonCameraLookAt(lookAtIntent); break; case CameraMode.Free: + cinemachinePreset.ForceFreeCameraLookAt(lookAtIntent); + break; case CameraMode.ThirdPerson: cinemachinePreset.ForceThirdPersonCameraLookAt(lookAtIntent); break; diff --git a/Explorer/Assets/DCL/FeatureFlags/FeaturesRegistry.cs b/Explorer/Assets/DCL/FeatureFlags/FeaturesRegistry.cs index ec0c6946612..3088c98eccc 100644 --- a/Explorer/Assets/DCL/FeatureFlags/FeaturesRegistry.cs +++ b/Explorer/Assets/DCL/FeatureFlags/FeaturesRegistry.cs @@ -71,6 +71,7 @@ public FeaturesRegistry( [FeatureId.AB_DEPS_DIGEST_CACHE_KEY] = featureFlags.IsEnabled(FeatureFlagsStrings.AB_DEPS_DIGEST_CACHE_KEY), [FeatureId.BYTE_WEIGHTED_LOADING_PROGRESS] = appArgs.ResolveFeatureFlagArg(AppArgsFlags.BYTE_WEIGHTED_LOADING_PROGRESS, featureFlags.IsEnabled(FeatureFlagsStrings.BYTE_WEIGHTED_LOADING_PROGRESS) || isEditor), [FeatureId.HARDWARE_FINGERPRINT] = appArgs.ResolveFeatureFlagArg(AppArgsFlags.HARDWARE_FINGERPRINT, featureFlags.IsEnabled(FeatureFlagsStrings.HARDWARE_FINGERPRINT)), + [FeatureId.MCP_SERVER] = appArgs.HasFlag(AppArgsFlags.MCP) || appArgs.HasFlag(AppArgsFlags.MCP_PORT), // Note: COMMUNITIES feature is not cached here because it depends on user identity }); @@ -207,5 +208,6 @@ public enum FeatureId HARDWARE_FINGERPRINT = 66, USER_CREDITS = 67, CREDITS_WEARABLE_PURCHASE = 68, + MCP_SERVER = 69, } } diff --git a/Explorer/Assets/DCL/Infrastructure/Global/AppArgs/AppArgsFlags.cs b/Explorer/Assets/DCL/Infrastructure/Global/AppArgs/AppArgsFlags.cs index 9c3992f08c3..b24203b5eaf 100644 --- a/Explorer/Assets/DCL/Infrastructure/Global/AppArgs/AppArgsFlags.cs +++ b/Explorer/Assets/DCL/Infrastructure/Global/AppArgs/AppArgsFlags.cs @@ -104,6 +104,16 @@ public static class AppArgsFlags public const string MULTIPLE_RUNNING_INSTANCES = "multi-instance"; public const string ALTTESTER = "alttester"; + /// + /// Starts the embedded MCP (Model Context Protocol) server on 127.0.0.1 so coding agents can drive the client. + /// + public const string MCP = "mcp"; + + /// + /// Overrides the port the embedded MCP server listens on (implies ). + /// + public const string MCP_PORT = "mcp-port"; + public const string REPORT_USER = "report-user"; public const string AVATAR_CONTEXT_MENU = "avatar-context-menu"; diff --git a/Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs b/Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs index 91ff480e4d4..b2c97906f2b 100644 --- a/Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs +++ b/Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs @@ -1,4 +1,5 @@ using Arch.Core; +using CrdtEcsBridge.RestrictedActions; using Cysharp.Threading.Tasks; using DCL.ApplicationGuards; using DCL.AssetsProvision; @@ -24,6 +25,7 @@ using DCL.LOD.Systems; using DCL.MarketplaceCredits; using DCL.MarketplaceCredits.Purchase; +using DCL.McpServer.Systems; using DCL.Multiplayer.Connections.Messaging.Hubs; using DCL.Multiplayer.Connections.RoomHubs; using DCL.Multiplayer.Emotes; @@ -842,6 +844,23 @@ await MapRendererContainer globalPlugins.Add(lodContainer.RoadPlugin); } + if (FeaturesRegistry.Instance.IsEnabled(FeatureId.MCP_SERVER)) + globalPlugins.Add(new McpServerPlugin( + appArgs, + new GlobalWorldActions(globalWorld, playerEntity, localSceneDevelopment, bootstrapContainer.UseRemoteAssetBundles, FeaturesRegistry.Instance.IsEnabled(FeatureId.SELF_PREVIEW_BUILDER_COLLECTIONS)), + chatContainer.ChatMessagesBus, + staticContainer.ScenesCache, + commsContainer.CurrentSceneInfo, + staticContainer.LoadingStatus, + realmNavigatorContainer.WorldInfoHub, + realmContainer.ReloadSceneController, + bootstrapContainer.DiagnosticsContainer, + exposedGlobalDataContainer.ExposedCameraData, + staticContainer.EntityCollidersGlobalCache, + coroutineRunner, + globalWorld, + localSceneDevelopment)); + if (FeaturesRegistry.Instance.IsEnabled(FeatureId.LOCAL_SCENE_DEVELOPMENT) || FeaturesRegistry.Instance.IsEnabled(FeatureId.SELF_PREVIEW_BUILDER_COLLECTIONS)) globalPlugins.Add(new GlobalGLTFLoadingPlugin(staticContainer.WebRequestsContainer.WebRequestController, staticContainer.RealmData, wearableContainer.BuilderContentURL.Value, localSceneDevelopment, staticContainer.ComponentsContainer.ComponentPoolsRegistry.RootContainerTransform())); diff --git a/Explorer/Assets/DCL/McpServer.meta b/Explorer/Assets/DCL/McpServer.meta new file mode 100644 index 00000000000..f4a20233b14 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: 99595f3e5da16754fa6419d428a933e8 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Components.meta b/Explorer/Assets/DCL/McpServer/Components.meta new file mode 100644 index 00000000000..3168cc8ff37 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: edabe99ee7c5e413e92d095e4480a477 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs b/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs new file mode 100644 index 00000000000..5511aaa8930 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs @@ -0,0 +1,35 @@ +using Cysharp.Threading.Tasks; +using DCL.CharacterMotion.Components; +using UnityEngine; + +namespace DCL.McpServer.Components +{ + /// + /// Held movement input requested by the MCP walk tool. While present on the player entity, + /// re-asserts it into every frame. + /// + public struct McpMovementOverride + { + /// + /// Normalized camera-relative axes (x = strafe, y = forward). + /// + public Vector2 Axes; + + public MovementKind Kind; + + /// + /// Value of Time.time at which the hold expires. + /// + public float EndTime; + + /// + /// Requests a single jump; consumed on the first frame of the hold. + /// + public bool JumpRequested; + + /// + /// Completed by the system when the hold expires or is preempted by a newer request. + /// + public UniTaskCompletionSource? Completion; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs.meta b/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs.meta new file mode 100644 index 00000000000..0f86dbd6f92 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 64ba45e1952cb4631b46b2d0dd9688e2 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Components/McpPointerClickIntent.cs b/Explorer/Assets/DCL/McpServer/Components/McpPointerClickIntent.cs new file mode 100644 index 00000000000..8cb42e08283 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components/McpPointerClickIntent.cs @@ -0,0 +1,69 @@ +using Cysharp.Threading.Tasks; +using DCL.ECSComponents; +using UnityEngine; +using RaycastHit = UnityEngine.RaycastHit; + +namespace DCL.McpServer.Components +{ + /// + /// Present on the player entity while an agent-requested pointer click is in flight. + /// validates the aim with a physics raycast each frame, + /// delivers the press through the target's + /// and removes the component once the click completes or fails. + /// + public struct McpPointerClickIntent + { + public enum ClickKind : byte + { + /// Pointer down, then pointer up on the next scene tick. + CLICK, + DOWN, + UP, + } + + public enum ClickPhase : byte + { + DOWN, + WAIT_TICK, + UP, + } + + /// Arch entity id in the current scene world; -1 when aiming at an explicit world point. + public int TargetEntityId; + public Vector3 AimPoint; + public bool HasExplicitAimPoint; + public InputAction Button; + public ClickKind Kind; + public ClickPhase Phase; + + /// Time.time after which the click is abandoned. + public float Deadline; + public UniTaskCompletionSource? Completion; + + // In-flight state owned by McpPointerClickSystem. + public Arch.Core.World? SceneWorld; + public Arch.Core.Entity ResolvedEntity; + public uint DownTick; + public RaycastHit DownHit; + public Ray DownRay; + public McpPointerClickResult? DownResult; + } + + /// Outcome of a synthetic pointer click, serialized by the click_entity tool. + public class McpPointerClickResult + { + public bool Hit; + public string? FailureReason; + public int SceneEntityId; + public int CrdtEntityId; + public string? HoverText; + public Vector3 HitPoint; + public float Distance; + public int? BlockedByEntityId; + public int? BlockedByCrdtId; + public string? BlockedByColliderName; + + /// The release ray no longer hit the target (it moved after the press); PetUp was delivered with the press-frame hit. + public bool UpRayMissed; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Components/McpPointerClickIntent.cs.meta b/Explorer/Assets/DCL/McpServer/Components/McpPointerClickIntent.cs.meta new file mode 100644 index 00000000000..8e6d348bae3 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components/McpPointerClickIntent.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 203dec8323e545c8a510e5883a59a4f6 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core.meta b/Explorer/Assets/DCL/McpServer/Core.meta new file mode 100644 index 00000000000..df86f87de1c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: ea5698d1ae0de4f699345c74dc6f99b6 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Core/IMcpTool.cs b/Explorer/Assets/DCL/McpServer/Core/IMcpTool.cs new file mode 100644 index 00000000000..2c498b62dd1 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/IMcpTool.cs @@ -0,0 +1,41 @@ +using Cysharp.Threading.Tasks; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Core +{ + /// + /// A single MCP tool exposed to connected coding agents via tools/list and tools/call. + /// + public interface IMcpTool + { + string Name { get; } + + string Description { get; } + + /// + /// JSON Schema of the tool arguments. Must be a JSON Schema object (type=object); build it with + /// so a malformed schema is caught at registration, not on first use. + /// + JObject InputSchema { get; } + + /// + /// Behaviour hints (read-only, destructive, idempotent, open-world) surfaced in tools/list. + /// + McpToolAnnotations Annotations { get; } + + /// + /// JSON Schema of this tool's structuredContent, surfaced as outputSchema in tools/list. Build it with + /// . Null (the default) when the tool returns only unstructured text; tools + /// that emit override this to describe that payload. + /// + JObject? OutputSchema => null; + + /// + /// Invoked on the main thread, so implementations may touch ECS and Unity state directly. Offload + /// heavy CPU work to the thread pool yourself and hop back before touching that state again. Expected + /// failures are reported through , not exceptions. + /// + UniTask ExecuteAsync(JObject arguments, CancellationToken ct); + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/IMcpTool.cs.meta b/Explorer/Assets/DCL/McpServer/Core/IMcpTool.cs.meta new file mode 100644 index 00000000000..f41607472fd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/IMcpTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: e258f7df9608348d98dffe25a3ae3bb6 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs b/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs new file mode 100644 index 00000000000..d09d3e53880 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs @@ -0,0 +1,361 @@ +using Cysharp.Threading.Tasks; +using DCL.Diagnostics; +using System; +using System.IO; +using System.Net; +using System.Text; +using System.Threading; +using UnityEngine; +using Utility.Multithreading; + +namespace DCL.McpServer.Core +{ + /// + /// Minimal MCP Streamable HTTP transport on top of , bound to 127.0.0.1 only. + /// POST carries a single JSON-RPC message; GET opens the server-to-client SSE stream — a log-stream + /// subscription (?stream=scene|client, optional ?level=) delivered via . + /// + public class McpHttpServer : IDisposable + { + private const int MAX_BODY_BYTES = 1024 * 1024; + + // Single source of truth for the endpoint path. Docs and scripts restate it (they cannot reference this + // const): docs/mcp-automation.md, docs/app-arguments.md, .claude/skills/mcp-scene-iteration/ — keep in sync. + private const string ENDPOINT_PATH = "unity-explorer-mcp"; + + private readonly McpJsonRpcDispatcher dispatcher; + private readonly int port; + private readonly McpNotificationChannel notifications; + + // The session maps to the Explorer process, not to a client: this server is stateless and drives one + // shared world, so every request operates on the same state and there is nothing session-scoped to isolate. + // Multiple agents on one server intentionally share this id because they drive the same world; a separate + // session means a separate Explorer instance (separate port, separate id). Hence one id for the process + // lifetime, echoed on every response, and no need to validate the incoming Mcp-Session-Id. + private readonly string sessionId = Guid.NewGuid().ToString("N"); + + private HttpListener? listener; + + /// The localhost URL the server listens on, e.g. http://127.0.0.1:8123/unity-explorer-mcp. + public string EndpointUrl => $"http://127.0.0.1:{port}/{ENDPOINT_PATH}"; + + public McpHttpServer(McpToolsRegistry toolsRegistry, int port, McpNotificationChannel notifications) + { + this.dispatcher = new McpJsonRpcDispatcher(toolsRegistry, Application.version); + this.port = port; + this.notifications = notifications; + } + + public void Dispose() + { + if (listener == null) return; + + try + { + listener.Stop(); + listener.Close(); + } + catch (ObjectDisposedException) { } + + listener = null; + } + + public bool TryStart() + { + var newListener = new HttpListener(); + newListener.Prefixes.Add($"{EndpointUrl}/"); // HttpListener requires the trailing slash + + try { newListener.Start(); } + catch (Exception e) when (e is HttpListenerException or InvalidOperationException) + { + ReportHub.LogError(ReportCategory.MCP, $"Cannot start the MCP server on port {port} (already in use? pass a different --mcp-port): {e.Message}"); + newListener.Close(); + return false; + } + + listener = newListener; + ReportHub.Log(LogType.Log, ReportCategory.MCP, $"MCP server listening on {EndpointUrl}"); + return true; + } + + public async UniTaskVoid RunAsync(CancellationToken ct) + { + await DCLTask.SwitchToThreadPool(); + + // Capture once: a concurrent Dispose() on the main thread nulls the field, and re-reading it between + // the guard and GetContextAsync would throw an unfiltered NRE. Dispose still stops/closes this same + // instance, so a parked GetContextAsync surfaces as one of the caught exceptions below. + HttpListener? local = listener; + + while (!ct.IsCancellationRequested && local is { IsListening: true }) + { + HttpListenerContext context; + + try { context = await local.GetContextAsync(); } + catch (Exception e) when (e is HttpListenerException or ObjectDisposedException or InvalidOperationException) + { + // The listener was stopped or disposed; end the accept loop. + break; + } + + HandleRequestAsync(context, ct).Forget(); + } + } + + private async UniTaskVoid HandleRequestAsync(HttpListenerContext context, CancellationToken ct) + { + try + { + if (!IsAllowed(context.Request.Headers["Origin"])) + { + context.Response.WriteEmptyAndClose(HttpStatusCode.Forbidden, sessionId); + return; + } + + switch (context.Request.HttpMethod) + { + case "POST": + await HandlePostAsync(context, ct); + break; + case "GET": + await HandleGetAsync(context, ct); + break; + case "DELETE": + // Session termination is accepted but stateless: nothing to clean up. + context.Response.WriteEmptyAndClose(HttpStatusCode.OK, sessionId); + break; + default: + context.Response.WriteEmptyAndClose(HttpStatusCode.MethodNotAllowed, sessionId); + break; + } + } + catch (OperationCanceledException) + { + TryAbort(context); + } + catch (Exception e) + { + ReportHub.LogException(e, ReportCategory.MCP); + TryWriteInternalError(context); + } + } + + private async UniTask HandlePostAsync(HttpListenerContext context, CancellationToken ct) + { + if (context.Request.ContentLength64 > MAX_BODY_BYTES) + { + context.Response.WriteEmptyAndClose(HttpStatusCode.RequestEntityTooLarge, sessionId); + return; + } + + string requestJson; + + // The Content-Length check above is only a fast reject; a chunked request reports ContentLength64 == -1 + // and bypasses it, so cap the read itself into a fixed buffer to keep the body bounded regardless. + using (var reader = new StreamReader(context.Request.InputStream, Encoding.UTF8)) + { + var buffer = new char[MAX_BODY_BYTES + 1]; + int charsRead = await reader.ReadBlockAsync(buffer, 0, buffer.Length); + + if (charsRead > MAX_BODY_BYTES) + { + context.Response.WriteEmptyAndClose(HttpStatusCode.RequestEntityTooLarge, sessionId); + return; + } + + requestJson = new string(buffer, 0, charsRead); + } + + string? responseJson = await dispatcher.DispatchAsync(requestJson, ct); + + if (responseJson == null) + { + // Notifications get 202 Accepted with no body. + context.Response.WriteEmptyAndClose(HttpStatusCode.Accepted, sessionId); + return; + } + + byte[] payload = Encoding.UTF8.GetBytes(responseJson); + + context.Response.WithMcpHeaders(sessionId); + context.Response.StatusCode = (int)HttpStatusCode.OK; + context.Response.ContentType = "application/json; charset=utf-8"; + context.Response.ContentLength64 = payload.Length; + await context.Response.OutputStream.WriteAsync(payload, 0, payload.Length, CancellationToken.None); + context.Response.Close(); + } + + private async UniTask HandleGetAsync(HttpListenerContext context, CancellationToken ct) + { + // The standalone server-to-client SSE stream: subscribe to one log stream and receive + // notifications/message events until the client disconnects or the server stops. + string? accept = context.Request.Headers["Accept"]; + + if (accept == null || !accept.Contains("text/event-stream")) + { + context.Response.WriteEmptyAndClose(HttpStatusCode.MethodNotAllowed, sessionId); + return; + } + + string? stream = context.Request.QueryString["stream"]; + + if (!McpLogStreams.IsKnown(stream)) + { + WriteBadRequest(context, "GET requires ?stream=scene or ?stream=client"); + return; + } + + var minLevel = McpLogLevel.Debug; + string? levelParam = context.Request.QueryString["level"]; + + if (!string.IsNullOrEmpty(levelParam) && !McpLogLevelExtensions.TryParse(levelParam, out minLevel)) + { + WriteBadRequest(context, $"Unknown level '{levelParam}' (RFC 5424: debug..emergency)"); + return; + } + + HttpListenerResponse response = context.Response; + response.StatusCode = (int)HttpStatusCode.OK; + response.ContentType = "text/event-stream"; + response.SendChunked = true; + response.AddHeader("Cache-Control", "no-cache"); + response.WithMcpHeaders(sessionId); + + // A vanished client is only discovered when a write fails; the sink then cancels this token so the + // handler stops holding the connection instead of parking on ct until the server shuts down. + using var connectionCts = CancellationTokenSource.CreateLinkedTokenSource(ct); + + IDisposable? subscription = notifications.Add(stream!, minLevel, new HttpListenerSseSink(response, connectionCts)); + + if (subscription == null) + { + // The channel is shutting down. + TryAbort(context); + return; + } + + try + { + // Park until the connection token cancels (server shutdown or a write-failure via the sink). + var closed = new UniTaskCompletionSource(); + using (connectionCts.Token.Register(() => closed.TrySetResult())) + await closed.Task; + } + finally { subscription.Dispose(); } + } + + private void WriteBadRequest(HttpListenerContext context, string message) + { + byte[] payload = Encoding.UTF8.GetBytes(message); + context.Response.WithMcpHeaders(sessionId); + context.Response.StatusCode = (int)HttpStatusCode.BadRequest; + context.Response.ContentType = "text/plain; charset=utf-8"; + context.Response.ContentLength64 = payload.Length; + context.Response.OutputStream.Write(payload, 0, payload.Length); + context.Response.Close(); + } + + private void TryWriteInternalError(HttpListenerContext context) + { + try + { + context.Response.WriteEmptyAndClose(HttpStatusCode.InternalServerError, sessionId); + } + catch (Exception) + { + // The response may already be closed or the client gone; nothing else to do. + } + } + + private static void TryAbort(HttpListenerContext context) + { + try { context.Response.Abort(); } + catch (Exception) + { + // Ignored: aborting a torn-down connection during shutdown. + } + } + + private static bool IsAllowed(string? origin) + { + if (string.IsNullOrEmpty(origin)) + return true; + + if (!Uri.TryCreate(origin, UriKind.Absolute, out Uri? originUri)) + return false; + + if (originUri.Scheme != Uri.UriSchemeHttp && originUri.Scheme != Uri.UriSchemeHttps) + return false; + + return originUri.Host is "localhost" or "127.0.0.1" or "::1"; + } + + /// Writes SSE chunks to one still-open HTTP response; serialized, and self-latching once broken. + private sealed class HttpListenerSseSink : McpNotificationChannel.ISseSink + { + private readonly HttpListenerResponse response; + private readonly CancellationTokenSource connectionCts; + private readonly object gate = new (); + private bool broken; + + public HttpListenerSseSink(HttpListenerResponse response, CancellationTokenSource connectionCts) + { + this.response = response; + this.connectionCts = connectionCts; + } + + public bool TryWrite(byte[] bytes) + { + lock (gate) + { + if (broken) return false; + + try + { + response.OutputStream.Write(bytes, 0, bytes.Length); + response.OutputStream.Flush(); + return true; + } + catch (Exception) + { + broken = true; + return false; + } + } + } + + public void Close() + { + lock (gate) + { + broken = true; + try { response.Close(); } + catch (Exception) { /* already torn down */ } + } + + // Ends the parked GET handler. The linked CTS may already be disposed if that handler + // completed first (server shutdown), so tolerate it. + try { connectionCts.Cancel(); } + catch (ObjectDisposedException) { } + } + } + } + + internal static class HttpListenerResponseExtensions + { + public static void WriteEmptyAndClose(this HttpListenerResponse response, HttpStatusCode status, string sessionId) + { + response.StatusCode = (int)status; + response.ContentLength64 = 0; + response.WithMcpHeaders(sessionId) + .Close(); + } + + public static HttpListenerResponse WithMcpHeaders(this HttpListenerResponse response, string sessionId) + { + response.AddHeader("Mcp-Session-Id", sessionId); + response.AddHeader("MCP-Protocol-Version", McpJsonRpcDispatcher.PROTOCOL_VERSION); + return response; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs.meta new file mode 100644 index 00000000000..6609b181173 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 4767a6ddc27e9455fbb1a982904fa578 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs b/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs new file mode 100644 index 00000000000..bb70d6fffe4 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs @@ -0,0 +1,197 @@ +using Cysharp.Threading.Tasks; +using DCL.Diagnostics; +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; + +namespace DCL.McpServer.Core +{ + /// + /// Routes JSON-RPC 2.0 messages of the MCP Streamable HTTP transport to the tool registry. + /// Only the tools capability is implemented; resources and prompts are not declared. + /// + public class McpJsonRpcDispatcher + { + /// Version of the MCP specification this server implements, declared in the initialize handshake. + public const string PROTOCOL_VERSION = "2025-06-18"; + + private const string SERVER_NAME = "dcl-unity-explorer"; + + private const int PARSE_ERROR = -32700; + private const int INVALID_REQUEST = -32600; + private const int METHOD_NOT_FOUND = -32601; + private const int INVALID_PARAMS = -32602; + + // A hung tool (e.g. an asset promise that never resolves) would otherwise hold its HttpListenerContext + // open until the server stops. Cap each call so one stuck tool can't tie up the agent's connection. + private static readonly TimeSpan TOOL_CALL_TIMEOUT = TimeSpan.FromSeconds(30); + + private readonly McpToolsRegistry tools; + private readonly string serverVersion; + + // Lets an agent orchestrating several Explorer instances confirm which process answers on this port. + private readonly int processId = System.Diagnostics.Process.GetCurrentProcess().Id; + + public McpJsonRpcDispatcher(McpToolsRegistry tools, string serverVersion) + { + this.tools = tools; + this.serverVersion = serverVersion; + } + + /// + /// Returns the serialized JSON-RPC response, or null when the message is a notification + /// (or a response relayed by the client) and no reply must be sent. + /// + public async UniTask DispatchAsync(string requestJson, CancellationToken ct) + { + if (ParseRoutableRequest(requestJson, out string? earlyResponse) is not { } routable) + return earlyResponse; + + var (id, method, callParams) = routable; + + return method switch + { + "initialize" => JsonRpcEnvelope.Result(id, InitializeResult(callParams)), + "ping" => JsonRpcEnvelope.Result(id, new JObject()), + "tools/list" => JsonRpcEnvelope.Result(id, tools.ToolsListPayload()), + "tools/call" => await CallToolAsync(id, + toolName: callParams?["name"]?.Value(), + arguments: callParams?["arguments"] as JObject ?? new JObject(), + ct), + _ => JsonRpcEnvelope.Error(id, METHOD_NOT_FOUND, $"Method not found: {method}") + }; + } + + /// + /// Parses the raw message and returns the id, method and params to route on. + /// Returns null when there is nothing to route: then + /// carries the reply to send back (a JSON-RPC error) or null for a notification that gets no response. + /// + private static (JToken id, string method, JObject? callParams)? ParseRoutableRequest(string requestJson, out string? earlyResponse) + { + earlyResponse = null; + + JToken parsed; + try { parsed = JToken.Parse(requestJson); } + catch (JsonException) + { + earlyResponse = JsonRpcEnvelope.Error(null, PARSE_ERROR, "Parse error"); + return null; + } + + // -32700 is reserved for unparseable JSON; well-formed JSON that is not a JSON-RPC object + // (a bare array, number or string) is a valid document but an invalid request: -32600. + if (parsed is not JObject request) + { + earlyResponse = JsonRpcEnvelope.Error(null, INVALID_REQUEST, "Invalid request: expected a JSON-RPC object"); + return null; + } + + JToken? id = request["id"]; + string? method = request["method"]?.Value(); + + if (string.IsNullOrEmpty(method)) + { + earlyResponse = id == null ? null : JsonRpcEnvelope.Error(id, INVALID_REQUEST, "Invalid request: missing method"); + return null; + } + + // Messages without an id are notifications ("notifications/initialized" et al.) and get no response. + if (id == null) + return null; + + return (id, method, request["params"] as JObject); + } + + private JObject InitializeResult(JObject? initializeParams) + { + // We implement a single MCP revision. Per the handshake rules we answer with our version regardless, + // but a client pinned to a different revision may abort on a strict mismatch — surface it so the + // otherwise-silent interop failure is diagnosable from the server logs. + string? requestedVersion = initializeParams?["protocolVersion"]?.Value(); + + if (!string.IsNullOrEmpty(requestedVersion) && requestedVersion != PROTOCOL_VERSION) + ReportHub.LogWarning(ReportCategory.MCP, $"MCP client requested protocol version '{requestedVersion}', server responding with '{PROTOCOL_VERSION}'"); + + return new JObject + { + ["protocolVersion"] = PROTOCOL_VERSION, + ["capabilities"] = new JObject + { + ["tools"] = new JObject(), + + // Non-standard: the SSE GET endpoint streams notifications/message for a chosen log + // source. Advertised under experimental so generic clients ignore it and DCL tooling + // can discover it. See McpHttpServer.HandleGetAsync and docs/mcp-automation.md. + ["experimental"] = new JObject + { + ["dcl/logStreams"] = new JObject + { + ["transport"] = "sse", + ["streams"] = new JArray("scene", "client"), + }, + }, + }, + ["serverInfo"] = new JObject + { + ["name"] = SERVER_NAME, + ["version"] = serverVersion, + ["pid"] = processId, + }, + }; + } + + private async UniTask CallToolAsync(JToken id, string? toolName, JObject arguments, CancellationToken ct) + { + if (!tools.TryGet(toolName, out IMcpTool? tool)) + return JsonRpcEnvelope.Error(id, INVALID_PARAMS, $"Unknown tool: {toolName ?? ""}"); + + using var timeout = CancellationTokenSource.CreateLinkedTokenSource(ct); + timeout.CancelAfter(TOOL_CALL_TIMEOUT); + + try + { + // Tools run on the main thread: the transport accepts and reads requests on a thread-pool thread, + // and the single hop here lets every tool touch ECS/Unity state without repeating the switch. + await UniTask.SwitchToMainThread(timeout.Token); + McpToolResult result = await tool.ExecuteAsync(arguments, timeout.Token); + return JsonRpcEnvelope.Result(id, result.Payload); + } + catch (OperationCanceledException) when (timeout.IsCancellationRequested && !ct.IsCancellationRequested) + { + ReportHub.LogWarning(ReportCategory.MCP, $"Tool '{toolName}' timed out after {TOOL_CALL_TIMEOUT.TotalSeconds:0}s"); + return JsonRpcEnvelope.Result(id, McpToolResult.Error($"Tool '{toolName}' timed out after {TOOL_CALL_TIMEOUT.TotalSeconds:0}s").Payload); + } + catch (OperationCanceledException) { throw; } + catch (Exception e) + { + ReportHub.LogException(e, ReportCategory.MCP); + return JsonRpcEnvelope.Result(id, McpToolResult.Error($"Tool '{toolName}' failed: {e.Message}").Payload); + } + } + + private static class JsonRpcEnvelope + { + public static string Result(JToken id, JToken result) => + new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = id, + ["result"] = result, + }.ToString(Formatting.None); + + public static string Error(JToken? id, int code, string message) => + new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = id ?? JValue.CreateNull(), + ["error"] = new JObject + { + ["code"] = code, + ["message"] = message, + }, + }.ToString(Formatting.None); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs.meta new file mode 100644 index 00000000000..112c4546ada --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: ade5e0f3aea214bd593ed7be6eaaaee9 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs b/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs new file mode 100644 index 00000000000..8d298786ebd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs @@ -0,0 +1,114 @@ +using Newtonsoft.Json.Linq; + +namespace DCL.McpServer.Core +{ + /// + /// Fluent builder for a tool's input or output JSON Schema. Declaring each field through a typed method keeps + /// the schema typo-safe — a mistyped type name can't slip through the way it could in a raw JSON string — and + /// produces the same { type: object, properties, required } shape a tools/list entry expects. Nested objects, + /// integer arrays and nullable fields extend the same style to the richer shapes an outputSchema needs. + /// + public sealed class McpJsonSchema + { + private readonly JObject properties = new (); + private readonly JArray required = new (); + + private McpJsonSchema() { } + + /// Starts a schema describing an object; chain the field methods and finish with . + public static McpJsonSchema Object() => + new (); + + public McpJsonSchema String(string name, string? description = null, string[]? enumValues = null, bool required = false, bool nullable = false) => + Property(name, "string", description, enumValues, required, nullable); + + public McpJsonSchema Number(string name, string? description = null, bool required = false, bool nullable = false) => + Property(name, "number", description, null, required, nullable); + + public McpJsonSchema Integer(string name, string? description = null, bool required = false, bool nullable = false) => + Property(name, "integer", description, null, required, nullable); + + public McpJsonSchema Boolean(string name, string? description = null, bool required = false, bool nullable = false) => + Property(name, "boolean", description, null, required, nullable); + + /// + /// Adds a nested object field described by its own builder. A + /// field admits null in place of the object (JSON Schema "type": ["object", "null"]). + /// + public McpJsonSchema Object(string name, McpJsonSchema schema, string? description = null, bool required = false, bool nullable = false) + { + JObject field = schema.Build(); + + if (nullable) + field["type"] = TypeToken("object", true); + + if (description != null) + field["description"] = description; + + return AddField(name, field, required); + } + + /// Adds an array field whose items are all integers. + public McpJsonSchema IntegerArray(string name, string? description = null, bool required = false) + { + var field = new JObject + { + ["type"] = "array", + ["items"] = new JObject { ["type"] = "integer" }, + }; + + if (description != null) + field["description"] = description; + + return AddField(name, field, required); + } + + /// Materializes the accumulated fields into the JSON Schema object. + public JObject Build() + { + var schema = new JObject + { + ["type"] = "object", + ["properties"] = properties, + }; + + if (required.Count > 0) + schema["required"] = required; + + return schema; + } + + private McpJsonSchema Property(string name, string type, string? description, string[]? enumValues, bool isRequired, bool nullable) + { + var field = new JObject { ["type"] = TypeToken(type, nullable) }; + + if (description != null) + field["description"] = description; + + if (enumValues != null) + { + var values = new JArray(); + + foreach (string value in enumValues) + values.Add(value); + + field["enum"] = values; + } + + return AddField(name, field, isRequired); + } + + private static JToken TypeToken(string type, bool nullable) => + nullable ? new JArray { type, "null" } : type; + + private McpJsonSchema AddField(string name, JObject field, bool isRequired) + { + properties[name] = field; + + if (isRequired) + required.Add(name); + + return this; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs.meta new file mode 100644 index 00000000000..abfcab1d96b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 05bd14397a93a8242be559c114683862 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpLogLevel.cs b/Explorer/Assets/DCL/McpServer/Core/McpLogLevel.cs new file mode 100644 index 00000000000..3d420a08ab8 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpLogLevel.cs @@ -0,0 +1,53 @@ +namespace DCL.McpServer.Core +{ + /// + /// RFC 5424 severities, as the MCP logging utility names them (the "level" field of a + /// notifications/message). Ordered low-to-high so a subscription's minimum filters by comparison. + /// + public enum McpLogLevel + { + Debug = 0, + Info = 1, + Notice = 2, + Warning = 3, + Error = 4, + Critical = 5, + Alert = 6, + Emergency = 7, + } + + public static class McpLogLevelExtensions + { + /// The lowercase wire name the MCP spec uses. + public static string Wire(this McpLogLevel level) => + level switch + { + McpLogLevel.Debug => "debug", + McpLogLevel.Info => "info", + McpLogLevel.Notice => "notice", + McpLogLevel.Warning => "warning", + McpLogLevel.Error => "error", + McpLogLevel.Critical => "critical", + McpLogLevel.Alert => "alert", + McpLogLevel.Emergency => "emergency", + _ => "info", + }; + + /// Parses a spec level name; false for anything unrecognised. + public static bool TryParse(string? name, out McpLogLevel level) + { + switch (name) + { + case "debug": level = McpLogLevel.Debug; return true; + case "info": level = McpLogLevel.Info; return true; + case "notice": level = McpLogLevel.Notice; return true; + case "warning": level = McpLogLevel.Warning; return true; + case "error": level = McpLogLevel.Error; return true; + case "critical": level = McpLogLevel.Critical; return true; + case "alert": level = McpLogLevel.Alert; return true; + case "emergency": level = McpLogLevel.Emergency; return true; + default: level = McpLogLevel.Info; return false; + } + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpLogLevel.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpLogLevel.cs.meta new file mode 100644 index 00000000000..81c1ae50e30 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpLogLevel.cs.meta @@ -0,0 +1,11 @@ +fileFormatVersion: 2 +guid: d47f897e867e41cc9688d030f0f4d9ec +MonoImporter: + externalObjects: {} + serializedVersion: 2 + defaultReferences: [] + executionOrder: 0 + icon: {instanceID: 0} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Core/McpLogStreams.cs b/Explorer/Assets/DCL/McpServer/Core/McpLogStreams.cs new file mode 100644 index 00000000000..3981922fc6b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpLogStreams.cs @@ -0,0 +1,18 @@ +namespace DCL.McpServer.Core +{ + /// + /// The two log streams a client can subscribe to over the SSE GET endpoint (?stream=). + /// Shared so the transport (which validates the query) and the notifier (which publishes) agree. + /// + public static class McpLogStreams + { + /// The running SDK7 scene's JavaScript console. + public const string SCENE = "scene"; + + /// The Unity player/editor log — engine, build and editor output. + public const string CLIENT = "client"; + + public static bool IsKnown(string? stream) => + stream is SCENE or CLIENT; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpLogStreams.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpLogStreams.cs.meta new file mode 100644 index 00000000000..22cefc84dc5 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpLogStreams.cs.meta @@ -0,0 +1,11 @@ +fileFormatVersion: 2 +guid: 5cf404908032490d88b3ddc980008380 +MonoImporter: + externalObjects: {} + serializedVersion: 2 + defaultReferences: [] + executionOrder: 0 + icon: {instanceID: 0} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Core/McpNotificationChannel.cs b/Explorer/Assets/DCL/McpServer/Core/McpNotificationChannel.cs new file mode 100644 index 00000000000..1619d88553a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpNotificationChannel.cs @@ -0,0 +1,149 @@ +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using System; +using System.Collections.Generic; +using System.Text; + +namespace DCL.McpServer.Core +{ + /// + /// The server-to-client half of the MCP Streamable HTTP transport: the SSE streams a client opens with + /// GET to receive server-initiated JSON-RPC notifications. Subscribers are grouped by a named stream + /// (e.g. "scene", "client") and a minimum level, so a client subscribes to exactly one source at one + /// verbosity. Thread-safe: log callbacks publish from arbitrary threads. + /// + public sealed class McpNotificationChannel : IDisposable + { + /// The write target behind a subscriber — an SSE-carrying HTTP response, or a fake in tests. + public interface ISseSink + { + /// Writes one already-framed SSE chunk; returns false once the client is gone. + bool TryWrite(byte[] bytes); + + void Close(); + } + + private readonly object gate = new (); + private readonly List subscribers = new (); + private bool disposed; + + /// Cheap pre-check so a log source can skip formatting when no one is listening on a stream. + public bool HasSubscribers(string stream) + { + lock (gate) + { + foreach (Subscriber s in subscribers) + if (s.Stream == stream) + return true; + + return false; + } + } + + /// + /// Registers a sink under a stream + minimum level. Returns a handle that removes it (safe to call + /// more than once), or null if the channel is already disposed. + /// + public IDisposable? Add(string stream, McpLogLevel minLevel, ISseSink sink) + { + var subscriber = new Subscriber(stream, minLevel, sink); + + lock (gate) + { + if (disposed) return null; + subscribers.Add(subscriber); + } + + return new Subscription(this, subscriber); + } + + /// + /// Frames a JSON-RPC notification as one SSE event and writes it to every subscriber of + /// whose minimum level admits ; drops any that fail. + /// + public void Publish(string stream, McpLogLevel level, JObject notification) + { + byte[]? frame = null; + List? dead = null; + + lock (gate) + { + foreach (Subscriber s in subscribers) + { + if (s.Stream != stream || level < s.MinLevel) continue; + + frame ??= Frame(notification); + + if (s.Sink.TryWrite(frame)) continue; + + dead ??= new List(); + dead.Add(s); + } + + if (dead != null) + foreach (Subscriber s in dead) + { + subscribers.Remove(s); + s.Sink.Close(); + } + } + } + + // "data: \n\n" — the JSON is serialized without newlines, so one event is one line. + private static byte[] Frame(JObject notification) => + Encoding.UTF8.GetBytes($"data: {notification.ToString(Formatting.None)}\n\n"); + + public void Dispose() + { + lock (gate) + { + disposed = true; + foreach (Subscriber s in subscribers) s.Sink.Close(); + subscribers.Clear(); + } + } + + private void Remove(Subscriber subscriber) + { + lock (gate) + { + if (subscribers.Remove(subscriber)) + subscriber.Sink.Close(); + } + } + + private sealed class Subscriber + { + public readonly string Stream; + public readonly McpLogLevel MinLevel; + public readonly ISseSink Sink; + + public Subscriber(string stream, McpLogLevel minLevel, ISseSink sink) + { + Stream = stream; + MinLevel = minLevel; + Sink = sink; + } + } + + private sealed class Subscription : IDisposable + { + private readonly McpNotificationChannel channel; + private readonly Subscriber subscriber; + private bool removed; + + public Subscription(McpNotificationChannel channel, Subscriber subscriber) + { + this.channel = channel; + this.subscriber = subscriber; + } + + public void Dispose() + { + if (removed) return; + removed = true; + channel.Remove(subscriber); + } + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpNotificationChannel.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpNotificationChannel.cs.meta new file mode 100644 index 00000000000..e2e9dc998dd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpNotificationChannel.cs.meta @@ -0,0 +1,11 @@ +fileFormatVersion: 2 +guid: 6384b793d3aa46999622b25b1689c452 +MonoImporter: + externalObjects: {} + serializedVersion: 2 + defaultReferences: [] + executionOrder: 0 + icon: {instanceID: 0} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs b/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs new file mode 100644 index 00000000000..e5dc37d9b17 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs @@ -0,0 +1,54 @@ +using Newtonsoft.Json.Linq; + +namespace DCL.McpServer.Core +{ + /// + /// Behaviour hints for a tool (MCP spec 2025-06-18) surfaced in tools/list so an agent can reason + /// about a tool before calling it. Only hints set through the factory methods are emitted; the rest + /// are omitted rather than defaulted, because the spec's implicit defaults are surprising + /// (destructiveHint and openWorldHint default to true, readOnlyHint to false). + /// + public readonly struct McpToolAnnotations + { + private readonly bool? readOnlyHint; + private readonly bool? destructiveHint; + private readonly bool? idempotentHint; + private readonly bool? openWorldHint; + + private McpToolAnnotations(bool? readOnlyHint, bool? destructiveHint, bool? idempotentHint, bool? openWorldHint) + { + this.readOnlyHint = readOnlyHint; + this.destructiveHint = destructiveHint; + this.idempotentHint = idempotentHint; + this.openWorldHint = openWorldHint; + } + + /// + /// A tool that only reads state. The spec ignores destructiveHint/idempotentHint when readOnlyHint + /// is true, so they are left unset. stays false for the local Explorer. + /// + public static McpToolAnnotations ReadOnly(bool openWorld = false) => + new (readOnlyHint: true, destructiveHint: null, idempotentHint: null, openWorldHint: openWorld); + + /// + /// A tool that changes state. flags irreversible or data-losing + /// effects; flags that repeating the call with the same arguments + /// has no additional effect. stays false for the local Explorer. + /// + public static McpToolAnnotations Mutating(bool destructive, bool idempotent, bool openWorld = false) => + new (readOnlyHint: false, destructiveHint: destructive, idempotentHint: idempotent, openWorldHint: openWorld); + + /// Serializes the set hints to the MCP annotations object embedded in a tools/list entry. + public JObject ToJObject() + { + var json = new JObject(); + + if (readOnlyHint.HasValue) json["readOnlyHint"] = readOnlyHint.Value; + if (destructiveHint.HasValue) json["destructiveHint"] = destructiveHint.Value; + if (idempotentHint.HasValue) json["idempotentHint"] = idempotentHint.Value; + if (openWorldHint.HasValue) json["openWorldHint"] = openWorldHint.Value; + + return json; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs.meta new file mode 100644 index 00000000000..41c1e9b50ed --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 6d6f99687071d4842921cf17924e34b2 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs b/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs new file mode 100644 index 00000000000..a2dd0615e94 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs @@ -0,0 +1,82 @@ +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using System; + +namespace DCL.McpServer.Core +{ + /// + /// The payload of a tools/call result: a content array of text/image items, + /// with expected failures flagged via isError instead of JSON-RPC errors. + /// + public readonly struct McpToolResult + { + public readonly JObject Payload; + + private McpToolResult(JObject payload) + { + Payload = payload; + } + + public static McpToolResult Text(string text) => + new (new JObject + { + ["content"] = new JArray { TextItem(text) }, + }); + + /// + /// A text result whose body is rendered as indented JSON. Centralizes the + /// Formatting.Indented serialization so tools never render JSON differently or forget the formatting. + /// + public static McpToolResult Json(JObject payload) => + Text(payload.ToString(Formatting.Indented)); + + /// + /// A result that surfaces both as structuredContent (validated against the + /// tool's outputSchema) and as its indented-JSON text duplicate — the spec requires the text mirror so + /// clients without structured support still read the result. This is the common path; it serializes the + /// mirror itself so the two copies cannot drift. + /// + public static McpToolResult JsonWithStructured(JObject structured) => + TextWithStructured(structured.ToString(Formatting.Indented), structured); + + /// + /// Like but with an explicit mirror, for the rare + /// case where the human-readable text is deliberately not the raw serialization of . + /// + public static McpToolResult TextWithStructured(string text, JObject structured) => + new (new JObject + { + ["content"] = new JArray { TextItem(text) }, + ["structuredContent"] = structured, + }); + + public static McpToolResult Error(string message) => + new (new JObject + { + ["content"] = new JArray { TextItem(message) }, + ["isError"] = true, + }); + + public static McpToolResult Image(byte[] imageBytes, string mimeType, string caption) => + new (new JObject + { + ["content"] = new JArray + { + new JObject + { + ["type"] = "image", + ["data"] = Convert.ToBase64String(imageBytes), + ["mimeType"] = mimeType, + }, + TextItem(caption), + }, + }); + + private static JObject TextItem(string text) => + new () + { + ["type"] = "text", + ["text"] = text, + }; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs.meta new file mode 100644 index 00000000000..075d762862a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 35469ba78d1a54f148c476efbc3b062f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs b/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs new file mode 100644 index 00000000000..a68567096a9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs @@ -0,0 +1,72 @@ +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using System; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; + +namespace DCL.McpServer.Core +{ + public class McpToolsRegistry + { + private readonly Dictionary tools = new (); + private string toolsListJson = null!; + + /// + /// The tools/list payload, serialized once at Build(). Each dispatch wraps the shared immutable + /// JSON string in a fresh JRaw, so there is no shared JToken tree to clone or re-parent when + /// concurrent responses serialize on the thread pool. + /// + public JRaw ToolsListPayload() => new (toolsListJson); + + public McpToolsRegistry Add(IMcpTool tool) + { + tools.Add(tool.Name, tool); + return this; + } + + public McpToolsRegistry Build() + { + var toolsArray = new JArray(); + + foreach (IMcpTool tool in tools.Values) + { + JObject inputSchema = tool.InputSchema; + + if (inputSchema == null || !IsObjectSchema(inputSchema)) + throw new InvalidOperationException($"MCP tool '{tool.Name}' produced an invalid input schema: expected a JSON Schema object (\"type\": \"object\"). Build it with McpJsonSchema."); + + if (tool.OutputSchema != null && !IsObjectSchema(tool.OutputSchema)) + throw new InvalidOperationException($"MCP tool '{tool.Name}' produced an invalid output schema: expected a JSON Schema object (\"type\": \"object\"). Build it with McpJsonSchema."); + + var entry = new JObject + { + ["name"] = tool.Name, + ["description"] = tool.Description, + ["inputSchema"] = inputSchema, + ["annotations"] = tool.Annotations.ToJObject(), + }; + + if (tool.OutputSchema != null) + entry["outputSchema"] = tool.OutputSchema; + + toolsArray.Add(entry); + } + + toolsListJson = new JObject { ["tools"] = toolsArray }.ToString(Formatting.None); + return this; + } + + private static bool IsObjectSchema(JObject schema) => + schema["type"]?.Value() == "object"; + + public bool TryGet(string? name, [NotNullWhen(true)] out IMcpTool? tool) + { + tool = null; + + if (string.IsNullOrEmpty(name)) + return false; + + return tools.TryGetValue(name, out tool); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs.meta new file mode 100644 index 00000000000..d9966b0e97c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 3bf2f57f26a5e44fd86d2b7f1201d27e \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef b/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef new file mode 100644 index 00000000000..92b00af768d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef @@ -0,0 +1,32 @@ +{ + "name": "DCL.McpServer", + "rootNamespace": "", + "references": [ + "GUID:d832748739a186646b8656bdbd447ad0", + "GUID:fa7b3fdbb04d67549916da7bd2af58ab", + "GUID:0b3eab7834a09c24ca4e84fe0d8a43ce", + "GUID:4794e238ed0f65142a4aea5848b513e5", + "GUID:1d2c76eb8b48e0b40940e8b31a679ce1", + "GUID:c80c82a8f4e04453b85fbab973d6774a", + "GUID:54d33bbd50a28174e8ba0110106203c2", + "GUID:f56000518aba31544aa96f4739e50a64", + "GUID:f3634757d00dab2429c6c11e69404e97", + "GUID:286980af24684da6acc1caa413039811", + "GUID:809870cbfd80a7b46bcf72b178ab210b", + "GUID:1b8e1e1bd01505f478f0369c04a4fb2f", + "GUID:571dc9f8bded0034f98595106462e3d0", + "GUID:0df5180c0c3a0594fbfa11f83736de9f", + "GUID:3c7b57a14671040bd8c549056adc04f5", + "GUID:f51ebe6a0ceec4240a699833d6309b23", + "GUID:e25ef972de004615a22937e739de2def" + ], + "includePlatforms": [], + "excludePlatforms": [], + "allowUnsafeCode": false, + "overrideReferences": false, + "precompiledReferences": [], + "autoReferenced": true, + "defineConstraints": [], + "versionDefines": [], + "noEngineReferences": false +} \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef.meta b/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef.meta new file mode 100644 index 00000000000..092bf9535d9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef.meta @@ -0,0 +1,3 @@ +fileFormatVersion: 2 +guid: f254b2009e854709a0ef0022a7ca697c +timeCreated: 1784219963 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Systems.meta b/Explorer/Assets/DCL/McpServer/Systems.meta new file mode 100644 index 00000000000..b7964b54fef --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: c6ba4ac038d1a4f32b295c8fe24fd984 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref b/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref new file mode 100644 index 00000000000..b24e4ef9e3d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref @@ -0,0 +1,3 @@ +{ + "reference": "GUID:fc4fd35fb877e904d8cedee73b2256f6" +} \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref.meta b/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref.meta new file mode 100644 index 00000000000..7d2760a40bd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref.meta @@ -0,0 +1,7 @@ +fileFormatVersion: 2 +guid: 38580450dd18841eb82d84648bb3694e +AssemblyDefinitionReferenceImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs b/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs new file mode 100644 index 00000000000..c718244e95f --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs @@ -0,0 +1,84 @@ +using Arch.Core; +using Arch.SystemGroups; +using Cysharp.Threading.Tasks; +using DCL.Character.CharacterMotion.Components; +using DCL.CharacterMotion.Components; +using DCL.CharacterMotion.Systems; +using DCL.Diagnostics; +using DCL.Input; +using DCL.McpServer.Components; +using ECS.Abstract; +using UnityEngine; + +namespace DCL.McpServer.Systems +{ + /// + /// While an is present on the player entity, re-asserts its axes into + /// after the real-input systems have written it, so an agent-requested + /// walk survives the per-frame overwrite performed by . + /// + [UpdateInGroup(typeof(InputGroup))] + [UpdateAfter(typeof(UpdateInputMovementSystem))] + [UpdateAfter(typeof(UpdateInputJumpSystem))] + [LogCategory(ReportCategory.MCP)] + public partial class McpInputOverrideSystem : BaseUnityLoopSystem + { + private readonly Entity playerEntity; + + private SingleInstanceEntity physicsTick; + + internal McpInputOverrideSystem(World world, Entity playerEntity) : base(world) + { + this.playerEntity = playerEntity; + } + + public override void Initialize() + { + base.Initialize(); + physicsTick = World.CachePhysicsTick(); + } + + protected override void Update(float t) + { + ref McpMovementOverride movementOverride = ref World.TryGetRef(playerEntity, out bool overrideExists); + + if (!overrideExists) + return; + + ref MovementInputComponent movement = ref World.TryGetRef(playerEntity, out bool hasMovement); + + if (UnityEngine.Time.time < movementOverride.EndTime) + { + if (hasMovement) + { + movement.Axes = movementOverride.Axes; + movement.Kind = movementOverride.Kind; + } + + if (movementOverride.JumpRequested) + { + movementOverride.JumpRequested = false; + + ref JumpInputComponent jump = ref World.TryGetRef(playerEntity, out bool hasJump); + + if (hasJump) + jump.Trigger.TickWhenJumpOccurred = physicsTick.GetPhysicsTickComponent(World).Tick + 1; + } + } + else + { + UniTaskCompletionSource? completion = movementOverride.Completion; + + if (hasMovement) + { + movement.Axes = Vector2.zero; + movement.Kind = MovementKind.IDLE; + } + + // Structural change only after all outstanding component refs are done. + World.Remove(playerEntity); + completion?.TrySetResult(); + } + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs.meta b/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs.meta new file mode 100644 index 00000000000..6d1c1b9ca8c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 1d8aa7853be9e41d6920f15f5468914b \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpLogNotifier.cs b/Explorer/Assets/DCL/McpServer/Systems/McpLogNotifier.cs new file mode 100644 index 00000000000..e862d084a91 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpLogNotifier.cs @@ -0,0 +1,78 @@ +using DCL.McpServer.Core; +using DCL.UI.DebugMenu.LogHistory; +using DCL.UI.DebugMenu.MessageBus; +using Newtonsoft.Json.Linq; +using System; +using UnityEngine; + +namespace DCL.McpServer.Systems +{ + /// + /// Feeds the two log streams of as notifications/message: + /// the "scene" stream carries the SDK7 scene's JavaScript console (the debug-menu log bus), and the + /// "client" stream carries the whole Unity player/editor log (, + /// which also covers build and editor output). A client subscribes to one stream at a chosen level; + /// each entry is formatted and pushed only while that stream has a subscriber. + /// + public sealed class McpLogNotifier : IDisposable + { + private readonly McpNotificationChannel channel; + private readonly DebugMenuConsoleLogEntryBus sceneLogBus; + + public McpLogNotifier(McpNotificationChannel channel, DebugMenuConsoleLogEntryBus sceneLogBus) + { + this.channel = channel; + this.sceneLogBus = sceneLogBus; + + sceneLogBus.MessageAdded += OnSceneLog; + Application.logMessageReceivedThreaded += OnClientLog; + } + + private void OnSceneLog(DebugMenuConsoleLogEntry entry) => + Emit(McpLogStreams.SCENE, FromSceneType(entry.Type), entry.Message); + + private void OnClientLog(string condition, string stackTrace, LogType type) => + Emit(McpLogStreams.CLIENT, FromUnityType(type), condition); + + private void Emit(string stream, McpLogLevel level, string message) + { + if (!channel.HasSubscribers(stream)) return; + + channel.Publish(stream, level, new JObject + { + ["jsonrpc"] = "2.0", + ["method"] = "notifications/message", + ["params"] = new JObject + { + ["level"] = level.Wire(), + ["logger"] = stream, + ["data"] = message, + }, + }); + } + + private static McpLogLevel FromSceneType(LogMessageType type) => + type switch + { + LogMessageType.Error => McpLogLevel.Error, + LogMessageType.Warning => McpLogLevel.Warning, + _ => McpLogLevel.Info, + }; + + private static McpLogLevel FromUnityType(LogType type) => + type switch + { + LogType.Exception => McpLogLevel.Critical, + LogType.Assert => McpLogLevel.Error, + LogType.Error => McpLogLevel.Error, + LogType.Warning => McpLogLevel.Warning, + _ => McpLogLevel.Info, + }; + + public void Dispose() + { + sceneLogBus.MessageAdded -= OnSceneLog; + Application.logMessageReceivedThreaded -= OnClientLog; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpLogNotifier.cs.meta b/Explorer/Assets/DCL/McpServer/Systems/McpLogNotifier.cs.meta new file mode 100644 index 00000000000..27827403691 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpLogNotifier.cs.meta @@ -0,0 +1,11 @@ +fileFormatVersion: 2 +guid: ed912eed28274c9cb5f4f9397a7f1cbf +MonoImporter: + externalObjects: {} + serializedVersion: 2 + defaultReferences: [] + executionOrder: 0 + icon: {instanceID: 0} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpPointerClickSystem.cs b/Explorer/Assets/DCL/McpServer/Systems/McpPointerClickSystem.cs new file mode 100644 index 00000000000..a4555ae81d2 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpPointerClickSystem.cs @@ -0,0 +1,379 @@ +using Arch.Core; +using Arch.SystemGroups; +using Arch.SystemGroups.DefaultSystemGroups; +using CRDT; +using CrdtEcsBridge.Physics; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.Diagnostics; +using DCL.ECSComponents; +using DCL.Interaction.PlayerOriginated.Components; +using DCL.Interaction.PlayerOriginated.Utility; +using DCL.Interaction.Systems; +using DCL.Interaction.Utility; +using DCL.McpServer.Components; +using ECS.Abstract; +using ECS.SceneLifeCycle; +using ECS.Unity.PrimitiveColliders.Components; +using ECS.Unity.Transforms.Components; +using SceneRunner.Scene; +using UnityEngine; +using RaycastHit = UnityEngine.RaycastHit; + +namespace DCL.McpServer.Systems +{ + /// + /// + /// Delivers an agent-requested pointer press to a scene entity while a + /// is present on the player entity. The aim is validated with the same physics raycast the reticle + /// pipeline uses (camera origin, , occlusion and + /// max-distance rules apply), then the target's + /// is filled exactly as fills it for a real click, so the + /// unmodified scene-world write-back emits an identical PBPointerEventsResult. + /// + /// + /// Runs after so its per-frame intent Initialize cannot wipe + /// the synthetic press before the scene-world flush, which happens later in the same frame. + /// + /// + [UpdateInGroup(typeof(PresentationSystemGroup))] + [UpdateAfter(typeof(ProcessPointerEventsSystem))] + [LogCategory(ReportCategory.MCP)] + public partial class McpPointerClickSystem : BaseUnityLoopSystem + { + private const float MAX_RAYCAST_DISTANCE = 100f; + private const float AIM_DIRECTION_EPSILON_SQR = 0.0001f; + + private static readonly QueryDescription ALL_ENTITIES = new (); + + private readonly IScenesCache scenesCache; + private readonly IEntityCollidersGlobalCache collidersGlobalCache; + private readonly Entity playerEntity; + + private SingleInstanceEntity playerCamera; + + internal McpPointerClickSystem(World world, + IScenesCache scenesCache, + IEntityCollidersGlobalCache collidersGlobalCache, + Entity playerEntity) : base(world) + { + this.scenesCache = scenesCache; + this.collidersGlobalCache = collidersGlobalCache; + this.playerEntity = playerEntity; + } + + public override void Initialize() + { + base.Initialize(); + playerCamera = World.CacheCamera(); + } + + protected override void Update(float t) + { + ref McpPointerClickIntent intent = ref World.TryGetRef(playerEntity, out bool exists); + + if (!exists) + return; + + if (UnityEngine.Time.time > intent.Deadline) + { + CompleteAndRemove(intent.Completion, Failure(in intent, "click timed out before it could be delivered (is the simulation paused?)")); + return; + } + + ISceneFacade? scene = scenesCache.CurrentScene.Value; + + if (scene == null || !scene.SceneStateProvider.IsCurrent || scene.SceneStateProvider.IsNotRunningState()) + { + CompleteAndRemove(intent.Completion, Failure(in intent, "no running current scene to deliver the click to")); + return; + } + + World sceneWorld = scene.EcsExecutor.World; + + if (intent.SceneWorld != null && !ReferenceEquals(sceneWorld, intent.SceneWorld)) + { + CompleteAndRemove(intent.Completion, Failure(in intent, "the scene reloaded mid-click; only the press may have been delivered")); + return; + } + + switch (intent.Phase) + { + case McpPointerClickIntent.ClickPhase.DOWN: + PointerEventType pressType = intent.Kind == McpPointerClickIntent.ClickKind.UP + ? PointerEventType.PetUp + : PointerEventType.PetDown; + + if (!TryDeliver(ref intent, sceneWorld, pressType, out McpPointerClickResult result)) + { + CompleteAndRemove(intent.Completion, result); + return; + } + + if (intent.Kind != McpPointerClickIntent.ClickKind.CLICK) + { + CompleteAndRemove(intent.Completion, result); + return; + } + + intent.SceneWorld = sceneWorld; + intent.DownTick = scene.SceneStateProvider.TickNumber; + intent.DownResult = result; + intent.Phase = McpPointerClickIntent.ClickPhase.WAIT_TICK; + return; + + case McpPointerClickIntent.ClickPhase.WAIT_TICK: + // The scene must observe PetDown on an earlier tick than PetUp, otherwise ordering is ambiguous. + if (scene.SceneStateProvider.TickNumber > intent.DownTick) + intent.Phase = McpPointerClickIntent.ClickPhase.UP; + + return; + + case McpPointerClickIntent.ClickPhase.UP: + DeliverUp(ref intent, sceneWorld, out McpPointerClickResult upResult); + CompleteAndRemove(intent.Completion, upResult); + return; + } + } + + /// Structural removal happens only after every read of the intent ref is done. + private void CompleteAndRemove(UniTaskCompletionSource? completion, McpPointerClickResult result) + { + World.Remove(playerEntity); + completion?.TrySetResult(result); + } + + private static McpPointerClickResult Failure(in McpPointerClickIntent intent, string reason) => + new () + { + Hit = false, + FailureReason = reason, + SceneEntityId = intent.TargetEntityId, + }; + + private bool TryDeliver(ref McpPointerClickIntent intent, World sceneWorld, PointerEventType eventType, out McpPointerClickResult result) + { + if (!TryResolveTarget(ref intent, sceneWorld, out result)) + return false; + + Entity targetEntity = intent.ResolvedEntity; + bool requireTarget = intent.TargetEntityId >= 0; + + Vector3 aimPoint = intent.HasExplicitAimPoint + ? intent.AimPoint + : ResolveEntityAimPoint(sceneWorld, targetEntity); + + CameraComponent camera = playerCamera.GetCameraComponent(World); + Vector3 origin = camera.Camera.transform.position; + Vector3 direction = aimPoint - origin; + + if (direction.sqrMagnitude < AIM_DIRECTION_EPSILON_SQR) + { + result = Failure(in intent, "the camera is on top of the aim point; move back and retry"); + return false; + } + + var ray = new Ray(origin, direction.normalized); + + if (!Physics.Raycast(ray, out RaycastHit hit, MAX_RAYCAST_DISTANCE, PhysicsLayers.PLAYER_ORIGIN_RAYCAST_MASK)) + { + result = Failure(in intent, "the ray from the camera hit nothing (target may lack a collider)"); + return false; + } + + if (!collidersGlobalCache.TryGetSceneEntity(hit.collider, out GlobalColliderSceneEntityInfo hitInfo) + || !ReferenceEquals(hitInfo.EcsExecutor.World, sceneWorld)) + { + result = Failure(in intent, $"the ray hit a non-scene collider '{hit.collider.name}'"); + return false; + } + + Entity hitEntity = hitInfo.ColliderSceneEntityInfo.EntityReference; + + if (requireTarget && hitEntity != targetEntity) + { + result = Failure(in intent, "another collider blocks the line of sight to the target"); + result.BlockedByEntityId = hitEntity.Id; + result.BlockedByCrdtId = hitInfo.ColliderSceneEntityInfo.SDKEntity.Id; + result.BlockedByColliderName = hit.collider.name; + return false; + } + + // In pure aim-point mode the raycast decides the target. + targetEntity = hitEntity; + intent.ResolvedEntity = hitEntity; + + if (!hitInfo.TryGetPointerEvents(out PBPointerEvents? pbPointerEvents)) + { + result = Failure(in intent, $"entity {targetEntity.Id} has no PointerEvents component (not clickable)"); + result.SceneEntityId = targetEntity.Id; + return false; + } + + if (!sceneWorld.TryGet(targetEntity, out CRDTEntity crdtEntity)) + { + result = Failure(in intent, $"entity {targetEntity.Id} has no CRDTEntity; the scene cannot receive results for it"); + result.SceneEntityId = targetEntity.Id; + return false; + } + + if (!IsQualified(pbPointerEvents!, ray, hit, camera, out float distance, out string? hoverText, out bool hasCursorEntry)) + { + result = Failure(in intent, hasCursorEntry + ? $"target is out of range for its pointer events (hit distance {distance:F2}m)" + : "the target's pointer events are proximity-type only; a cursor click cannot trigger them"); + + result.SceneEntityId = targetEntity.Id; + result.CrdtEntityId = crdtEntity.Id; + result.Distance = distance; + return false; + } + + pbPointerEvents!.AppendPointerEventResultsIntent.Initialize(hit, ray); + pbPointerEvents.AppendPointerEventResultsIntent.AddInputAction(intent.Button, eventType); + + intent.DownHit = hit; + intent.DownRay = ray; + + result = new McpPointerClickResult + { + Hit = true, + SceneEntityId = targetEntity.Id, + CrdtEntityId = crdtEntity.Id, + HoverText = hoverText, + HitPoint = hit.point, + Distance = distance, + }; + + return true; + } + + /// + /// Delivers the release. If the target moved out from under the ray after the press (or its distance gate + /// no longer qualifies), the press-frame hit is reused so the entity still receives an ordered PetUp, + /// and the divergence is reported via . + /// + private void DeliverUp(ref McpPointerClickIntent intent, World sceneWorld, out McpPointerClickResult result) + { + McpPointerClickResult downResult = intent.DownResult!; + + if (TryDeliver(ref intent, sceneWorld, PointerEventType.PetUp, out McpPointerClickResult freshResult)) + { + result = freshResult; + return; + } + + // Fresh delivery failed: fall back to the press-frame hit if the component is still reachable. + if (sceneWorld.IsAlive(intent.ResolvedEntity) && sceneWorld.TryGet(intent.ResolvedEntity, out PBPointerEvents? pbPointerEvents) && pbPointerEvents != null) + { + pbPointerEvents.AppendPointerEventResultsIntent.Initialize(intent.DownHit, intent.DownRay); + pbPointerEvents.AppendPointerEventResultsIntent.AddInputAction(intent.Button, PointerEventType.PetUp); + + downResult.UpRayMissed = true; + result = downResult; + return; + } + + downResult.UpRayMissed = true; + downResult.FailureReason = $"the entity disappeared after the press ({freshResult.FailureReason}); only PetDown was delivered"; + result = downResult; + } + + private bool TryResolveTarget(ref McpPointerClickIntent intent, World sceneWorld, out McpPointerClickResult result) + { + result = null!; + + // Aim-point mode: the validation raycast picks the entity. + if (intent.HasExplicitAimPoint && intent.TargetEntityId < 0) + return true; + + if (intent.SceneWorld != null) + { + if (sceneWorld.IsAlive(intent.ResolvedEntity)) + return true; + + result = Failure(in intent, "the target entity was destroyed mid-click"); + return false; + } + + Entity found = Entity.Null; + int targetId = intent.TargetEntityId; + + sceneWorld.Query(in ALL_ENTITIES, entity => + { + if (entity.Id == targetId) + found = entity; + }); + + if (found == Entity.Null) + { + result = Failure(in intent, $"no entity with id {targetId} in the current scene world"); + return false; + } + + intent.ResolvedEntity = found; + return true; + } + + /// Aim at the collider volume when available; entity pivots can sit at hinges or bases and miss. + private static Vector3 ResolveEntityAimPoint(World sceneWorld, Entity entity) + { + if (sceneWorld.TryGet(entity, out PrimitiveColliderComponent primitiveCollider) && primitiveCollider.Collider != null) + return primitiveCollider.Collider.bounds.center; + + if (sceneWorld.TryGet(entity, out TransformComponent transformComponent) && transformComponent.Transform != null) + return transformComponent.Transform.position; + + return Vector3.zero; + } + + /// + /// Mirrors the cursor-entry qualification of : entries get their + /// defaults prepared and the distance gate is evaluated per entry, the last cursor entry winning, exactly + /// like the production loop. Also picks the hover text a real reticle hover would show for this button. + /// + private bool IsQualified(PBPointerEvents pbPointerEvents, in Ray ray, in RaycastHit hit, in CameraComponent camera, out float distance, out string? hoverText, out bool hasCursorEntry) + { + distance = camera.Mode == CameraMode.FirstPerson + ? hit.distance + : Vector3.Distance(hit.point, camera.PlayerFocus.position); + + float? playerDistance = null; + + if (World.TryGet(playerEntity, out CharacterTransform characterTransform)) + playerDistance = Vector3.Distance(hit.point, characterTransform.Position); + + var raycastResult = new PlayerOriginRaycastResultForSceneEntities(); + raycastResult.SetRay(ray); + raycastResult.SetupHit(hit, default(GlobalColliderSceneEntityInfo), distance, playerDistance); + + var isAtDistance = false; + hoverText = null; + hasCursorEntry = false; + + for (var i = 0; i < pbPointerEvents.PointerEvents!.Count; i++) + { + PBPointerEvents.Types.Entry entry = pbPointerEvents.PointerEvents[i]!; + + if (entry.InteractionType != InteractionType.Cursor) + continue; + + hasCursorEntry = true; + + PBPointerEvents.Types.Info info = entry.EventInfo!; + info.PrepareDefaultValues(); + + isAtDistance = InteractionInputUtils.IsQualifiedByDistance(in raycastResult, info); + + if (!isAtDistance) + continue; + + if (hoverText == null && info.HasHoverText && !string.IsNullOrEmpty(info.HoverText)) + hoverText = info.HoverText; + } + + return isAtDistance; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpPointerClickSystem.cs.meta b/Explorer/Assets/DCL/McpServer/Systems/McpPointerClickSystem.cs.meta new file mode 100644 index 00000000000..821b204664e --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpPointerClickSystem.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 45b28969f78d4a59b05fdd15adf4041b \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs b/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs new file mode 100644 index 00000000000..fd0db6b9dea --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs @@ -0,0 +1,180 @@ +using Arch.SystemGroups; +using CrdtEcsBridge.RestrictedActions; +using Cysharp.Threading.Tasks; +using DCL.CharacterCamera; +using DCL.Chat.MessageBus; +using DCL.Diagnostics; +using DCL.Interaction.Utility; +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using DCL.McpServer.Utils; +using DCL.PluginSystem.Global; +using DCL.RealmNavigation; +using DCL.UI.DebugMenu.MessageBus; +using ECS.SceneLifeCycle; +using ECS.SceneLifeCycle.CurrentScene; +using Global.AppArgs; +using SceneRunner.Debugging.Hub; +using System; +using System.Threading; +using UnityEngine; +using Utility; + +namespace DCL.McpServer.Systems +{ + /// + /// Hosts the embedded MCP server so external coding agents can observe and drive the client. + /// Registered only when the mcp/mcp-port app arg is present (command line or deep link); + /// the server binds to 127.0.0.1 exclusively and validates browser Origins. + /// + public class McpServerPlugin : IDCLGlobalPluginWithoutSettings + { + private const int DEFAULT_PORT = 8123; + + private const int MIN_PORT = 1024; + private const int MAX_PORT = 65535; + + private readonly int port; + + private readonly ICoroutineRunner coroutineRunner; + private readonly ILoadingStatus loadingStatus; + + private readonly IChatMessagesBus chatMessagesBus; + private readonly ExposedCameraData exposedCameraData; + + private readonly Arch.Core.World globalWorld; + private readonly IGlobalWorldActions globalWorldActions; + private readonly IEntityCollidersGlobalCache entityCollidersGlobalCache; + private readonly IWorldInfoHub worldInfoHub; + + private readonly IScenesCache scenesCache; + private readonly ICurrentSceneInfo currentSceneInfo; + private readonly ECSReloadScene reloadSceneController; + private readonly bool localSceneDevelopment; + + private readonly SceneLogBuffer logBuffer; + private readonly DebugMenuConsoleLogEntryBus logEntryBus; + private readonly McpNotificationChannel notificationChannel; + private readonly McpLogNotifier logNotifier; + + private McpHttpServer? server; + private CancellationTokenSource? serverCts; + + private ScreenshotTool? screenshotTool; + + public McpServerPlugin( + IAppArgs appArgs, + IGlobalWorldActions globalWorldActions, + IChatMessagesBus chatMessagesBus, + IScenesCache scenesCache, + ICurrentSceneInfo currentSceneInfo, + ILoadingStatus loadingStatus, + IWorldInfoHub worldInfoHub, + ECSReloadScene reloadSceneController, + DiagnosticsContainer diagnosticsContainer, + ExposedCameraData exposedCameraData, + IEntityCollidersGlobalCache entityCollidersGlobalCache, + ICoroutineRunner coroutineRunner, + Arch.Core.World globalWorld, + bool localSceneDevelopment) + { + port = appArgs.TryGetValue(AppArgsFlags.MCP_PORT, out string? portValue) + && int.TryParse(portValue, out int parsedPort) + && parsedPort is >= MIN_PORT and <= MAX_PORT + ? parsedPort + : DEFAULT_PORT; + + this.globalWorldActions = globalWorldActions; + this.chatMessagesBus = chatMessagesBus; + this.scenesCache = scenesCache; + this.currentSceneInfo = currentSceneInfo; + this.loadingStatus = loadingStatus; + this.worldInfoHub = worldInfoHub; + this.reloadSceneController = reloadSceneController; + this.exposedCameraData = exposedCameraData; + this.entityCollidersGlobalCache = entityCollidersGlobalCache; + this.coroutineRunner = coroutineRunner; + this.globalWorld = globalWorld; + this.localSceneDevelopment = localSceneDevelopment; + + logBuffer = new SceneLogBuffer(); + logEntryBus = new DebugMenuConsoleLogEntryBus(); + logEntryBus.MessageAdded += logBuffer.Append; + diagnosticsContainer.AddDebugConsoleHandler(logEntryBus); + + // Live push side: the same scene log bus plus the whole Unity/editor/build log, streamed over + // SSE to subscribers of McpHttpServer's GET endpoint. get_scene_logs stays as the pull side. + notificationChannel = new McpNotificationChannel(); + logNotifier = new McpLogNotifier(notificationChannel, logEntryBus); + } + + public void Dispose() + { + logEntryBus.MessageAdded -= logBuffer.Append; + logNotifier.Dispose(); + screenshotTool?.Dispose(); + server?.Dispose(); + notificationChannel.Dispose(); + serverCts.SafeCancelAndDispose(); + } + + public void InjectToWorld(ref ArchSystemsWorldBuilder builder, in GlobalPluginArguments arguments) + { + McpInputOverrideSystem.InjectToWorld(ref builder, arguments.PlayerEntity); + McpPointerClickSystem.InjectToWorld(ref builder, scenesCache, entityCollidersGlobalCache, arguments.PlayerEntity); + + screenshotTool = new ScreenshotTool(coroutineRunner, globalWorld, arguments.PlayerEntity); + + var toolsRegistry = new McpToolsRegistry() + .Add(screenshotTool) + .Add(new GetPlayerStateTool(globalWorld, arguments.PlayerEntity, exposedCameraData, currentSceneInfo)) + .Add(new GetSceneStateTool(scenesCache, currentSceneInfo, loadingStatus, localSceneDevelopment)) + .Add(new GetSceneLogsTool(logBuffer)) + .Add(new TeleportTool(chatMessagesBus, scenesCache, loadingStatus)) + .Add(new MoveToTool(globalWorldActions, globalWorld, arguments.PlayerEntity)) + .Add(new LookAtTool(globalWorldActions, globalWorld, arguments.PlayerEntity, exposedCameraData)) + .Add(new SetCameraModeTool(globalWorld, exposedCameraData)) + .Add(new SetCameraPoseTool(globalWorld, arguments.PlayerEntity, exposedCameraData)) + .Add(new WalkTool(globalWorld, arguments.PlayerEntity)) + .Add(new SendChatTool(chatMessagesBus)) + .Add(new ReloadSceneTool(reloadSceneController, scenesCache, globalWorld, arguments.PlayerEntity, arguments.SkyboxEntity)) + .Add(new ListSceneEntitiesTool(worldInfoHub)) + .Add(new GetEntityDetailsTool(worldInfoHub)) + .Add(new TriggerEmoteTool(globalWorldActions)) + .Add(new ClickEntityTool(globalWorld, arguments.PlayerEntity)) + .Build(); + + server = new McpHttpServer(toolsRegistry, port, notificationChannel); + serverCts = serverCts.SafeRestart(); + + bool started = server.TryStart(); + + if (started) + server.RunAsync(serverCts.Token).Forget(); + + AnnounceStatusWhenLoadedAsync(started, server.EndpointUrl, serverCts.Token).Forget(); + } + + /// + /// Reports the server address (or the startup failure) once loading completes, so the message + /// reaches the scene debug console: its UI subscribes to log entries only after this plugin runs, + /// and a line logged at server start would be dropped. + /// + private async UniTaskVoid AnnounceStatusWhenLoadedAsync(bool started, string endpointUrl, CancellationToken ct) + { + try + { + await UniTask.WaitUntil(() => loadingStatus.CurrentStage.Value == LoadingStatus.LoadingStage.Completed, cancellationToken: ct); + + if (started) + ReportHub.Log(LogType.Log, ReportCategory.MCP, $"MCP server listening on {endpointUrl}"); + else + ReportHub.LogError(ReportCategory.MCP, $"MCP server failed to start on port {port} — agent connections unavailable (pass a different --mcp-port)"); + } + catch (OperationCanceledException) + { + ReportHub.Log(LogType.Log, ReportCategory.MCP, "MCP server status announcement cancelled before loading completed"); + } + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs.meta b/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs.meta new file mode 100644 index 00000000000..9336efcf1cd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 5d7cb65f2d6064e8f98e97c77b2ccb7e \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests.meta b/Explorer/Assets/DCL/McpServer/Tests.meta new file mode 100644 index 00000000000..309d70874e3 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: 85f2f58b3ff94ddfa14bb5bda0658b63 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref b/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref new file mode 100644 index 00000000000..9c56917b757 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref @@ -0,0 +1,3 @@ +{ + "reference": "GUID:da80994a355e49d5b84f91c0a84a721f" +} \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref.meta b/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref.meta new file mode 100644 index 00000000000..b2e1703bffa --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref.meta @@ -0,0 +1,7 @@ +fileFormatVersion: 2 +guid: b6d1d98221344ffa8441fb344268804f +AssemblyDefinitionReferenceImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs b/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs new file mode 100644 index 00000000000..ba7e07a4046 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs @@ -0,0 +1,68 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; + +namespace DCL.McpServer.Tests +{ + /// + /// A configurable test double: its ExecuteAsync returns a preset result, + /// or throws a preset exception, and records the arguments and cancellation token it was called with. + /// Keeps the routing tests independent of the real tool implementations. + /// + internal sealed class FakeMcpTool : IMcpTool + { + private readonly Func execute; + + public string Name { get; } + + public string Description { get; } + + public JObject InputSchema { get; } + + public McpToolAnnotations Annotations { get; } + + public int CallCount { get; private set; } + + public JObject? LastArguments { get; private set; } + + public CancellationToken LastCancellationToken { get; private set; } + + private FakeMcpTool(string name, string description, JObject inputSchema, McpToolAnnotations annotations, + Func execute) + { + Name = name; + Description = description; + InputSchema = inputSchema; + Annotations = annotations; + this.execute = execute; + } + + /// A tool whose ExecuteAsync returns (defaults to a text result). + public static FakeMcpTool Returning(string name, McpToolResult? result = null, JObject? inputSchema = null, + McpToolAnnotations? annotations = null) + { + McpToolResult toReturn = result ?? McpToolResult.Text($"{name} ran"); + return new FakeMcpTool(name, $"{name} description", inputSchema ?? DefaultSchema(), + annotations ?? McpToolAnnotations.ReadOnly(), (_, _) => toReturn); + } + + /// A tool whose ExecuteAsync throws . + public static FakeMcpTool Throwing(string name, Exception exception, JObject? inputSchema = null, + McpToolAnnotations? annotations = null) => + new (name, $"{name} description", inputSchema ?? DefaultSchema(), + annotations ?? McpToolAnnotations.ReadOnly(), (_, _) => throw exception); + + private static JObject DefaultSchema() => + McpJsonSchema.Object().String("value", "Any value.").Build(); + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + CallCount++; + LastArguments = arguments; + LastCancellationToken = ct; + return UniTask.FromResult(execute(arguments, ct)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs.meta new file mode 100644 index 00000000000..1d1f4a9775a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: e7a2d21be39bd5441ba0b52f68f67b34 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs b/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs new file mode 100644 index 00000000000..6ed339b06b7 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs @@ -0,0 +1,88 @@ +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using Newtonsoft.Json.Linq; +using NSubstitute; +using NUnit.Framework; +using SceneRunner.Debugging; +using SceneRunner.Debugging.Hub; +using System.Threading; + +namespace DCL.McpServer.Tests +{ + public class GetEntityDetailsToolShould + { + private const string CURRENT_SCENE = "CURRENT"; + + private IWorldInfoHub worldInfoHub = null!; + private IWorldInfo worldInfo = null!; + private GetEntityDetailsTool tool = null!; + + [SetUp] + public void Setup() + { + worldInfo = Substitute.For(); + worldInfoHub = Substitute.For(); + worldInfoHub.WorldInfo(CURRENT_SCENE).Returns(worldInfo); + tool = new GetEntityDetailsTool(worldInfoHub); + } + + [Test] + public void ReturnTheDumpWholeWhenItFitsTheBudget() + { + // Arrange + const string DUMP = "Components of entity 5, total count: 1\n1) PBTransform"; + worldInfo.EntityComponentsInfo(5).Returns(DUMP); + + // Act + string text = TextOf(Execute(5)); + + // Assert + Assert.That(text, Is.EqualTo(DUMP)); + Assert.That(text, Does.Not.Contain("truncated")); + } + + [Test] + public void TruncateWithANoteWhenTheDumpExceedsTheBudget() + { + // Arrange + string dump = new ('x', 20000); + worldInfo.EntityComponentsInfo(5).Returns(dump); + + // Act + string text = TextOf(Execute(5)); + + // Assert + Assert.That(text.Length, Is.LessThan(dump.Length)); + Assert.That(text, Does.Contain($"output truncated at 8000/{dump.Length} chars")); + } + + [Test] + public void ErrorWhenNoSceneWorldIsFound() + { + // Arrange + worldInfoHub.WorldInfo(CURRENT_SCENE).Returns((IWorldInfo?)null); + + // Act + McpToolResult result = Execute(5); + + // Assert + Assert.That(result.Payload["isError"]!.Value(), Is.True); + } + + [Test] + public void ErrorWhenEntityIdIsMissing() + { + // Act + McpToolResult result = tool.ExecuteAsync(new JObject(), CancellationToken.None).GetAwaiter().GetResult(); + + // Assert + Assert.That(result.Payload["isError"]!.Value(), Is.True); + } + + private McpToolResult Execute(int entityId) => + tool.ExecuteAsync(new JObject { ["entityId"] = entityId }, CancellationToken.None).GetAwaiter().GetResult(); + + private static string TextOf(McpToolResult result) => + result.Payload["content"]![0]!["text"]!.Value()!; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs.meta new file mode 100644 index 00000000000..5f8d40e23e5 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: c5c1a3c1b1573d04dae5c672589e81de \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs b/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs new file mode 100644 index 00000000000..55b33d059e2 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs @@ -0,0 +1,76 @@ +using Arch.Core; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.CharacterCamera.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using ECS.SceneLifeCycle.CurrentScene; +using Newtonsoft.Json.Linq; +using NSubstitute; +using NUnit.Framework; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tests +{ + public class GetPlayerStateToolShould + { + private World world = null!; + private GameObject playerGameObject = null!; + private GetPlayerStateTool tool = null!; + + [SetUp] + public void Setup() + { + world = World.Create(); + playerGameObject = new GameObject(nameof(GetPlayerStateToolShould)); + + Entity playerEntity = world.Create(new CharacterTransform(playerGameObject.transform)); + world.Create(new CameraComponent()); // the tool reads the camera mode through CacheCamera() + + tool = new GetPlayerStateTool(world, playerEntity, new ExposedCameraData(), Substitute.For()); + } + + [TearDown] + public void TearDown() + { + world.Dispose(); + Object.DestroyImmediate(playerGameObject); + } + + [Test] + public void DeclareAnObjectOutputSchema() + { + Assert.That(tool.OutputSchema!["type"]!.Value(), Is.EqualTo("object")); + } + + [Test] + public void ModelTheAddressAsANullableString() + { + var addressType = (JArray)tool.OutputSchema!["properties"]!["address"]!["type"]!; + Assert.That(addressType.ToObject(), Is.EqualTo(new[] { "string", "null" })); + } + + [Test] + public void ModelTheCameraAsANestedObject() + { + JToken camera = tool.OutputSchema!["properties"]!["camera"]!; + + Assert.That(camera["type"]!.Value(), Is.EqualTo("object")); + Assert.That(camera["properties"]!["mode"]!["type"]!.Value(), Is.EqualTo("string")); + } + + [Test] + public void KeepTheOutputSchemaInSyncWithTheStructuredPayload() + { + // Act + var structured = (JObject)Execute().Payload["structuredContent"]!; + + // Assert + McpSchemaAssert.KeysMatch(tool.OutputSchema!, structured); + } + + private McpToolResult Execute() => + tool.ExecuteAsync(new JObject(), CancellationToken.None).GetAwaiter().GetResult(); + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs.meta new file mode 100644 index 00000000000..c474cb9df5b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 486751286fc8bcc47af7880e22187283 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs b/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs new file mode 100644 index 00000000000..7d81ed80854 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs @@ -0,0 +1,105 @@ +using DCL.Diagnostics; +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using DCL.RealmNavigation; +using DCL.Utilities; +using ECS.SceneLifeCycle; +using ECS.SceneLifeCycle.CurrentScene; +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using NSubstitute; +using NUnit.Framework; +using SceneRunner.Scene; +using System.Threading; +using UnityEngine; +using Utility.Multithreading; + +namespace DCL.McpServer.Tests +{ + public class GetSceneStateToolShould + { + private IScenesCache scenesCache = null!; + private ICurrentSceneInfo currentSceneInfo = null!; + private ILoadingStatus loadingStatus = null!; + private GetSceneStateTool tool = null!; + + [SetUp] + public void Setup() + { + scenesCache = Substitute.For(); + currentSceneInfo = Substitute.For(); + loadingStatus = Substitute.For(); + + scenesCache.CurrentParcel.Returns(new ReactiveProperty(new Vector2Int(1, 2))); + scenesCache.CurrentScene.Returns(new ReactiveProperty(null)); + loadingStatus.CurrentStage.Returns(new ReactiveProperty(default)); + + tool = new GetSceneStateTool(scenesCache, currentSceneInfo, loadingStatus, localSceneDevelopment: false); + } + + [Test] + public void MirrorTheStateInBothTextAndStructuredContent() + { + // Act + McpToolResult result = Execute(); + + // Assert + var structured = (JObject)result.Payload["structuredContent"]!; + Assert.That(structured, Is.Not.Null); + Assert.That(result.Payload["content"]![0]!["text"]!.Value(), Is.EqualTo(structured.ToString(Formatting.Indented))); + } + + [Test] + public void ReportAnAbsentSceneAsAJsonNull() + { + // Act + McpToolResult result = Execute(); + + // Assert + var structured = (JObject)result.Payload["structuredContent"]!; + Assert.That(structured["scene"]!.Type, Is.EqualTo(JTokenType.Null)); + } + + [Test] + public void DeclareAnObjectOutputSchemaThatAdmitsANullScene() + { + // Act + JObject schema = tool.OutputSchema!; + + // Assert + Assert.That(schema["type"]!.Value(), Is.EqualTo("object")); + + var sceneType = (JArray)schema["properties"]!["scene"]!["type"]!; + Assert.That(sceneType.ToObject(), Is.EqualTo(new[] { "object", "null" })); + } + + [Test] + public void KeepTheOutputSchemaInSyncWithTheStructuredPayload() + { + // Arrange — a populated scene so the nested "scene" object is covered, not just the top level. + ISceneStateProvider sceneStateProvider = Substitute.For(); + sceneStateProvider.State.Returns(new Atomic(SceneState.Running)); + + ISceneData sceneData = Substitute.For(); + sceneData.SceneLoadingConcluded.Returns(true); + + ISceneFacade scene = Substitute.For(); + scene.Info.Returns(new SceneShortInfo(new Vector2Int(1, 2), "Test scene", "7")); + scene.SceneStateProvider.Returns(sceneStateProvider); + scene.SceneData.Returns(sceneData); + scene.IsSceneReady().Returns(true); + + scenesCache.CurrentScene.Returns(new ReactiveProperty(scene)); + currentSceneInfo.SceneStatus.Returns(new ReactiveProperty(null)); + + // Act + var structured = (JObject)Execute().Payload["structuredContent"]!; + + // Assert + McpSchemaAssert.KeysMatch(tool.OutputSchema!, structured); + } + + private McpToolResult Execute() => + tool.ExecuteAsync(new JObject(), CancellationToken.None).GetAwaiter().GetResult(); + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs.meta new file mode 100644 index 00000000000..c1d5e41b5ff --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 523603bbeedddae44833a5a53f9793ae \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs b/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs new file mode 100644 index 00000000000..a9550ea8d56 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs @@ -0,0 +1,107 @@ +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using Newtonsoft.Json.Linq; +using NSubstitute; +using NUnit.Framework; +using SceneRunner.Debugging; +using SceneRunner.Debugging.Hub; +using System.Collections.Generic; +using System.Threading; + +namespace DCL.McpServer.Tests +{ + public class ListSceneEntitiesToolShould + { + private const string CURRENT_SCENE = "CURRENT"; + + private IWorldInfoHub worldInfoHub = null!; + private IWorldInfo worldInfo = null!; + private ListSceneEntitiesTool tool = null!; + + [SetUp] + public void Setup() + { + worldInfo = Substitute.For(); + worldInfoHub = Substitute.For(); + worldInfoHub.WorldInfo(CURRENT_SCENE).Returns(worldInfo); + tool = new ListSceneEntitiesTool(worldInfoHub); + } + + [Test] + public void AddAnActionableLineWhenNotEverythingIsShown() + { + // Arrange + worldInfo.EntityIds().Returns(Ids(10)); + + // Act + string text = TextOf(Execute(limit: 3)); + + // Assert + Assert.That(text, Does.Contain("total=10 returned=3")); + Assert.That(text, Does.Contain("3 of 10 shown")); + } + + [Test] + public void OmitTheActionableLineWhenEverythingFits() + { + // Arrange + worldInfo.EntityIds().Returns(Ids(3)); + + // Act + string text = TextOf(Execute(limit: 200)); + + // Assert + Assert.That(text, Does.Contain("total=3 returned=3")); + Assert.That(text, Does.Not.Contain("shown")); + } + + [Test] + public void MirrorTheListingInStructuredContentWhileKeepingTheText() + { + // Arrange + worldInfo.EntityIds().Returns(Ids(10)); + + // Act + McpToolResult result = Execute(limit: 3); + + // Assert — text output is untouched + Assert.That(TextOf(result), Does.Contain("total=10 returned=3")); + + // Assert — structured mirror carries the same figures + var structured = (JObject)result.Payload["structuredContent"]!; + Assert.That(structured["total"]!.Value(), Is.EqualTo(10)); + Assert.That(structured["returned"]!.Value(), Is.EqualTo(3)); + Assert.That(structured["truncated"]!.Value(), Is.True); + Assert.That(((JArray)structured["entityIds"]!).ToObject(), Is.EqualTo(new[] { 0, 1, 2 })); + } + + [Test] + public void KeepTheOutputSchemaInSyncWithTheStructuredPayload() + { + // Arrange + worldInfo.EntityIds().Returns(Ids(3)); + + // Act + var structured = (JObject)Execute(limit: 200).Payload["structuredContent"]!; + + // Assert + McpSchemaAssert.KeysMatch(tool.OutputSchema!, structured); + } + + private McpToolResult Execute(int limit) => + tool.ExecuteAsync(new JObject { ["limit"] = limit }, CancellationToken.None).GetAwaiter().GetResult(); + + private static IReadOnlyList Ids(int count) + { + var ids = new List(count); + + for (var i = 0; i < count; i++) + ids.Add(i); + + return ids; + } + + private static string TextOf(McpToolResult result) => + result.Payload["content"]![0]!["text"]!.Value()!; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs.meta new file mode 100644 index 00000000000..11c4db393c3 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: fe60d9fd78ad61247aa09f22ac6ecab7 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs new file mode 100644 index 00000000000..d5e95c7581e --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs @@ -0,0 +1,50 @@ +using DCL.McpServer.Core; +using NUnit.Framework; +using System.Reflection; + +namespace DCL.McpServer.Tests +{ + /// + /// Covers McpHttpServer.IsAllowed, the DNS-rebinding guard that decides which Origin headers may + /// reach the JSON-RPC dispatcher. The method is private static and lives in Core, which these tests do + /// not modify, so it is exercised through reflection. If IsAllowed is later exposed as internal + /// (with InternalsVisibleTo), replace the reflection shim with a direct call. + /// + public class McpHttpServerShould + { + private static readonly MethodInfo IS_ALLOWED = + typeof(McpHttpServer).GetMethod("IsAllowed", BindingFlags.NonPublic | BindingFlags.Static)!; + + private static bool IsAllowed(string? origin) => + (bool)IS_ALLOWED.Invoke(null, new object?[] { origin })!; + + [TestCase(null)] + [TestCase("")] + public void AllowRequestsWithoutAnOrigin(string? origin) => + Assert.That(IsAllowed(origin), Is.True); + + [TestCase("http://localhost")] + [TestCase("http://localhost:8080")] + [TestCase("http://127.0.0.1")] + [TestCase("http://127.0.0.1:9001")] + [TestCase("https://127.0.0.1:9001")] + public void AllowLoopbackHttpOrigins(string origin) => + Assert.That(IsAllowed(origin), Is.True); + + [TestCase("http://evil.com")] + [TestCase("https://attacker.example:9001")] + [TestCase("http://127.0.0.1.evil.com")] // rebinding: a foreign host that merely starts with the loopback IP + public void RejectNonLoopbackHosts(string origin) => + Assert.That(IsAllowed(origin), Is.False); + + [TestCase("ftp://127.0.0.1")] + [TestCase("file:///etc/passwd")] + public void RejectNonHttpSchemes(string origin) => + Assert.That(IsAllowed(origin), Is.False); + + [TestCase("not-a-valid-origin")] + [TestCase("://missing-scheme")] + public void RejectMalformedOrigins(string origin) => + Assert.That(IsAllowed(origin), Is.False); + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs.meta new file mode 100644 index 00000000000..800c6157bd0 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 5b82a6ebaa2cb0148808b9bc41a981d7 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs new file mode 100644 index 00000000000..54c074968ce --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs @@ -0,0 +1,256 @@ +using DCL.McpServer.Core; +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System; +using System.Text.RegularExpressions; +using System.Threading; +using UnityEngine; +using UnityEngine.TestTools; + +namespace DCL.McpServer.Tests +{ + public class McpJsonRpcDispatcherShould + { + private const string SERVER_VERSION = "9.9.9-test"; + + [Test] + public void AnswerInitializeWithProtocolCapabilitiesAndServerInfo() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + JObject result = ResultOf(Dispatch(dispatcher, Request(1, "initialize"))); + + Assert.That(result["protocolVersion"]!.Value(), Is.EqualTo("2025-06-18")); + Assert.That(result["protocolVersion"]!.Value(), Is.EqualTo(McpJsonRpcDispatcher.PROTOCOL_VERSION)); + + var capabilities = (JObject)result["capabilities"]!; + Assert.That(capabilities.ContainsKey("tools"), Is.True); + + var serverInfo = (JObject)result["serverInfo"]!; + Assert.That(serverInfo["version"]!.Value(), Is.EqualTo(SERVER_VERSION)); + Assert.That(serverInfo.ContainsKey("pid"), Is.True); + Assert.That(serverInfo["pid"]!.Type, Is.EqualTo(JTokenType.Integer)); + } + + [Test] + public void AnswerPingWithAnEmptyResult() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + JObject result = ResultOf(Dispatch(dispatcher, Request(2, "ping"))); + + Assert.That(result.Count, Is.EqualTo(0)); + } + + [Test] + public void ListToolsWithInputSchemaAndAnnotations() + { + FakeMcpTool tool = FakeMcpTool.Returning("reader", annotations: McpToolAnnotations.ReadOnly()); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + JObject result = ResultOf(Dispatch(dispatcher, Request(3, "tools/list"))); + + var tools = (JArray)result["tools"]!; + Assert.That(tools.Count, Is.EqualTo(1)); + + JObject entry = ToolEntry(result, "reader"); + Assert.That(entry["description"]!.Value(), Is.EqualTo("reader description")); + + var inputSchema = (JObject)entry["inputSchema"]!; + Assert.That(inputSchema["type"]!.Value(), Is.EqualTo("object")); + Assert.That(inputSchema.ContainsKey("properties"), Is.True); + + var annotations = (JObject)entry["annotations"]!; + Assert.That(annotations["readOnlyHint"]!.Value(), Is.True); + } + + [Test] + public void CallAKnownToolAndWrapItsResult() + { + FakeMcpTool tool = FakeMcpTool.Returning("echo", McpToolResult.Text("done")); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + var arguments = new JObject { ["value"] = "hi" }; + JObject result = ResultOf(Dispatch(dispatcher, CallRequest(4, "echo", arguments))); + + Assert.That(tool.CallCount, Is.EqualTo(1)); + Assert.That(tool.LastArguments!["value"]!.Value(), Is.EqualTo("hi")); + + var content = (JArray)result["content"]!; + Assert.That(content[0]!["type"]!.Value(), Is.EqualTo("text")); + Assert.That(content[0]!["text"]!.Value(), Is.EqualTo("done")); + Assert.That(result.ContainsKey("isError"), Is.False); + } + + [Test] + public void CallWithEmptyArgumentsWhenNoneAreProvided() + { + FakeMcpTool tool = FakeMcpTool.Returning("echo"); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + // A tools/call request carrying a name but no arguments object. + string request = new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = 5, + ["method"] = "tools/call", + ["params"] = new JObject { ["name"] = "echo" }, + }.ToString(Formatting.None); + + ResultOf(Dispatch(dispatcher, request)); + + Assert.That(tool.CallCount, Is.EqualTo(1)); + Assert.That(tool.LastArguments, Is.Not.Null); + Assert.That(tool.LastArguments!.Count, Is.EqualTo(0)); + } + + [Test] + public void RejectAnUnknownToolWithInvalidParams() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(FakeMcpTool.Returning("known")); + + JObject error = ErrorOf(Dispatch(dispatcher, CallRequest(6, "missing", new JObject()))); + + Assert.That(error["code"]!.Value(), Is.EqualTo(-32602)); + Assert.That(error["message"]!.Value(), Does.Contain("missing")); + } + + [Test] + public void ReportAToolFailureAsAnIsErrorResultNotAJsonRpcError() + { + FakeMcpTool tool = FakeMcpTool.Throwing("boom", new InvalidOperationException("kaboom")); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + // ReportHub routes the caught exception to the Unity logger; expect it so the test does not fail. + LogAssert.Expect(LogType.Exception, new Regex("kaboom")); + + JObject response = JObject.Parse(Dispatch(dispatcher, CallRequest(7, "boom", new JObject()))!); + + // The failure is delivered inside result (isError), not as a top-level JSON-RPC error. + Assert.That(response.ContainsKey("error"), Is.False); + + var result = (JObject)response["result"]!; + Assert.That(result["isError"]!.Value(), Is.True); + Assert.That(result["content"]![0]!["text"]!.Value(), Does.Contain("boom")); + } + + [Test] + public void RethrowWhenAToolIsCancelled() + { + FakeMcpTool tool = FakeMcpTool.Throwing("cancelled", new OperationCanceledException()); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + Assert.Throws(() => Dispatch(dispatcher, CallRequest(8, "cancelled", new JObject()))); + } + + [Test] + public void RejectAnUnknownMethodWithMethodNotFound() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + JObject error = ErrorOf(Dispatch(dispatcher, Request(9, "resources/list"))); + + Assert.That(error["code"]!.Value(), Is.EqualTo(-32601)); + Assert.That(error["message"]!.Value(), Does.Contain("resources/list")); + } + + [Test] + public void DropNotificationsWithoutAnId() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + string request = new JObject + { + ["jsonrpc"] = "2.0", + ["method"] = "notifications/initialized", + }.ToString(Formatting.None); + + Assert.That(Dispatch(dispatcher, request), Is.Null); + } + + [Test] + public void ReplyWithAParseErrorOnMalformedJson() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + JObject error = ErrorOf(Dispatch(dispatcher, "{ this is not json")); + + Assert.That(error["code"]!.Value(), Is.EqualTo(-32700)); + } + + [Test] + public void RejectARequestMissingAMethodWithInvalidRequest() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + string request = new JObject { ["jsonrpc"] = "2.0", ["id"] = 10 }.ToString(Formatting.None); + + JObject error = ErrorOf(Dispatch(dispatcher, request)); + + Assert.That(error["code"]!.Value(), Is.EqualTo(-32600)); + } + + private static McpJsonRpcDispatcher DispatcherWith(params IMcpTool[] tools) + { + var registry = new McpToolsRegistry(); + + foreach (IMcpTool tool in tools) + registry.Add(tool); + + registry.Build(); + return new McpJsonRpcDispatcher(registry, SERVER_VERSION); + } + + private static string? Dispatch(McpJsonRpcDispatcher dispatcher, string requestJson) => + dispatcher.DispatchAsync(requestJson, CancellationToken.None).GetAwaiter().GetResult(); + + private static string Request(int id, string method) => + new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = id, + ["method"] = method, + }.ToString(Formatting.None); + + private static string CallRequest(int id, string toolName, JObject arguments) => + new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = id, + ["method"] = "tools/call", + ["params"] = new JObject + { + ["name"] = toolName, + ["arguments"] = arguments, + }, + }.ToString(Formatting.None); + + private static JObject ResultOf(string? response) + { + Assert.That(response, Is.Not.Null, "expected a response, got a dropped message"); + var parsed = JObject.Parse(response!); + Assert.That(parsed["jsonrpc"]!.Value(), Is.EqualTo("2.0")); + Assert.That(parsed.ContainsKey("error"), Is.False, $"expected a result, got an error: {response}"); + return (JObject)parsed["result"]!; + } + + private static JObject ErrorOf(string? response) + { + Assert.That(response, Is.Not.Null, "expected an error response, got a dropped message"); + var parsed = JObject.Parse(response!); + Assert.That(parsed["jsonrpc"]!.Value(), Is.EqualTo("2.0")); + return (JObject)parsed["error"]!; + } + + private static JObject ToolEntry(JObject toolsListResult, string name) + { + foreach (JToken entry in (JArray)toolsListResult["tools"]!) + if (entry["name"]!.Value() == name) + return (JObject)entry; + + Assert.Fail($"tool '{name}' not found in tools/list"); + return null!; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs.meta new file mode 100644 index 00000000000..cd4bb0fe2a8 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: e447afff3fddc6746922385663f764be \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs new file mode 100644 index 00000000000..c1e66af8e5c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs @@ -0,0 +1,98 @@ +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using NUnit.Framework; + +namespace DCL.McpServer.Tests +{ + public class McpJsonSchemaShould + { + [Test] + public void EmitTheDeclaredJsonSchemaTypeForEachFieldKind() + { + JObject schema = McpJsonSchema.Object() + .Number("ratio") + .Boolean("flag") + .Build(); + + var properties = (JObject)schema["properties"]!; + Assert.That(properties["ratio"]!["type"]!.Value(), Is.EqualTo("number")); + Assert.That(properties["flag"]!["type"]!.Value(), Is.EqualTo("boolean")); + } + + [Test] + public void OmitDescriptionAndEnumWhenNotProvided() + { + JObject schema = McpJsonSchema.Object().String("name").Build(); + + var field = (JObject)schema["properties"]!["name"]!; + Assert.That(field.ContainsKey("description"), Is.False); + Assert.That(field.ContainsKey("enum"), Is.False); + } + + [Test] + public void CollectEveryRequiredFieldInDeclarationOrder() + { + JObject schema = McpJsonSchema.Object() + .String("first", required: true) + .Integer("skipped") + .Boolean("second", required: true) + .Build(); + + Assert.That(((JArray)schema["required"]!).ToObject(), Is.EqualTo(new[] { "first", "second" })); + } + + [Test] + public void NestAnObjectFieldWithItsOwnProperties() + { + JObject schema = McpJsonSchema.Object() + .Object("camera", McpJsonSchema.Object().String("mode"), "The camera.", required: true) + .Build(); + + var camera = (JObject)schema["properties"]!["camera"]!; + Assert.That(camera["type"]!.Value(), Is.EqualTo("object")); + Assert.That(camera["description"]!.Value(), Is.EqualTo("The camera.")); + Assert.That(camera["properties"]!["mode"]!["type"]!.Value(), Is.EqualTo("string")); + Assert.That(((JArray)schema["required"]!).ToObject(), Is.EqualTo(new[] { "camera" })); + } + + [Test] + public void AdmitNullAlongsideAnObjectForANullableNestedField() + { + JObject schema = McpJsonSchema.Object() + .Object("scene", McpJsonSchema.Object().String("name"), nullable: true) + .Build(); + + var sceneType = (JArray)schema["properties"]!["scene"]!["type"]!; + Assert.That(sceneType.ToObject(), Is.EqualTo(new[] { "object", "null" })); + } + + [Test] + public void AdmitNullAlongsideTheDeclaredTypeForANullableScalar() + { + JObject schema = McpJsonSchema.Object().String("address", nullable: true).Build(); + + var addressType = (JArray)schema["properties"]!["address"]!["type"]!; + Assert.That(addressType.ToObject(), Is.EqualTo(new[] { "string", "null" })); + } + + [Test] + public void DescribeAnArrayOfIntegerItems() + { + JObject schema = McpJsonSchema.Object().IntegerArray("entityIds").Build(); + + var field = (JObject)schema["properties"]!["entityIds"]!; + Assert.That(field["type"]!.Value(), Is.EqualTo("array")); + Assert.That(field["items"]!["type"]!.Value(), Is.EqualTo("integer")); + } + + [Test] + public void ProduceAnEmptyPropertiesObjectForAnArgumentlessTool() + { + JObject schema = McpJsonSchema.Object().Build(); + + Assert.That(schema["type"]!.Value(), Is.EqualTo("object")); + Assert.That(((JObject)schema["properties"]!).Count, Is.EqualTo(0)); + Assert.That(schema.ContainsKey("required"), Is.False); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs.meta new file mode 100644 index 00000000000..0b62cbacd67 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: bcaebf033b5a3154b84bfdffe0aa383e \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpLogLevelShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpLogLevelShould.cs new file mode 100644 index 00000000000..b56b52c4fd1 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpLogLevelShould.cs @@ -0,0 +1,43 @@ +using DCL.McpServer.Core; +using NUnit.Framework; + +namespace DCL.McpServer.Tests +{ + public class McpLogLevelShould + { + [TestCase("debug", McpLogLevel.Debug)] + [TestCase("info", McpLogLevel.Info)] + [TestCase("notice", McpLogLevel.Notice)] + [TestCase("warning", McpLogLevel.Warning)] + [TestCase("error", McpLogLevel.Error)] + [TestCase("critical", McpLogLevel.Critical)] + [TestCase("alert", McpLogLevel.Alert)] + [TestCase("emergency", McpLogLevel.Emergency)] + public void ParseEverySpecLevel(string name, McpLogLevel expected) + { + Assert.That(McpLogLevelExtensions.TryParse(name, out McpLogLevel level), Is.True); + Assert.That(level, Is.EqualTo(expected)); + } + + [TestCase(null)] + [TestCase("")] + [TestCase("verbose")] + [TestCase("Error")] // case-sensitive per spec + public void RejectUnknownLevels(string? name) => + Assert.That(McpLogLevelExtensions.TryParse(name, out _), Is.False); + + [TestCase(McpLogLevel.Debug, "debug")] + [TestCase(McpLogLevel.Warning, "warning")] + [TestCase(McpLogLevel.Emergency, "emergency")] + public void RoundTripThroughTheWireName(McpLogLevel level, string wire) + { + Assert.That(level.Wire(), Is.EqualTo(wire)); + Assert.That(McpLogLevelExtensions.TryParse(wire, out McpLogLevel parsed), Is.True); + Assert.That(parsed, Is.EqualTo(level)); + } + + [Test] + public void OrderSeveritiesLowToHigh() => + Assert.That(McpLogLevel.Error, Is.GreaterThan(McpLogLevel.Warning)); + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpLogLevelShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpLogLevelShould.cs.meta new file mode 100644 index 00000000000..b928d5a2ab9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpLogLevelShould.cs.meta @@ -0,0 +1,11 @@ +fileFormatVersion: 2 +guid: b9de2c37a3a5493697c584994e64558d +MonoImporter: + externalObjects: {} + serializedVersion: 2 + defaultReferences: [] + executionOrder: 0 + icon: {instanceID: 0} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpNotificationChannelShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpNotificationChannelShould.cs new file mode 100644 index 00000000000..f3a285ac592 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpNotificationChannelShould.cs @@ -0,0 +1,123 @@ +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System.Collections.Generic; +using System.Text; + +namespace DCL.McpServer.Tests +{ + public class McpNotificationChannelShould + { + private static JObject Note(string body) => + new () { ["jsonrpc"] = "2.0", ["method"] = "notifications/message", ["params"] = new JObject { ["data"] = body } }; + + private sealed class FakeSink : McpNotificationChannel.ISseSink + { + public readonly List Frames = new (); + public bool Closed; + public bool FailWrites; + + public bool TryWrite(byte[] bytes) + { + if (FailWrites) return false; + Frames.Add(Encoding.UTF8.GetString(bytes)); + return true; + } + + public void Close() => Closed = true; + } + + [Test] + public void DeliverToSubscribersOfTheMatchingStreamOnly() + { + using var channel = new McpNotificationChannel(); + var scene = new FakeSink(); + var client = new FakeSink(); + channel.Add(McpLogNotifierStreams.SCENE, McpLogLevel.Debug, scene); + channel.Add(McpLogNotifierStreams.CLIENT, McpLogLevel.Debug, client); + + channel.Publish(McpLogNotifierStreams.SCENE, McpLogLevel.Info, Note("hi")); + + Assert.That(scene.Frames, Has.Count.EqualTo(1)); + Assert.That(client.Frames, Is.Empty); + } + + [Test] + public void FrameNotificationsAsSingleLineSseEvents() + { + using var channel = new McpNotificationChannel(); + var sink = new FakeSink(); + channel.Add(McpLogNotifierStreams.SCENE, McpLogLevel.Debug, sink); + + channel.Publish(McpLogNotifierStreams.SCENE, McpLogLevel.Info, Note("hello")); + + Assert.That(sink.Frames[0], Does.StartWith("data: ")); + Assert.That(sink.Frames[0], Does.EndWith("\n\n")); + Assert.That(sink.Frames[0], Does.Contain("\"hello\"")); + } + + [Test] + public void DropEntriesBelowTheSubscriptionLevel() + { + using var channel = new McpNotificationChannel(); + var sink = new FakeSink(); + channel.Add(McpLogNotifierStreams.CLIENT, McpLogLevel.Warning, sink); + + channel.Publish(McpLogNotifierStreams.CLIENT, McpLogLevel.Info, Note("noise")); + channel.Publish(McpLogNotifierStreams.CLIENT, McpLogLevel.Error, Note("real")); + + Assert.That(sink.Frames, Has.Count.EqualTo(1)); + Assert.That(sink.Frames[0], Does.Contain("real")); + } + + [Test] + public void DropAndCloseASinkThatFailsToWrite() + { + using var channel = new McpNotificationChannel(); + var sink = new FakeSink { FailWrites = true }; + channel.Add(McpLogNotifierStreams.SCENE, McpLogLevel.Debug, sink); + + channel.Publish(McpLogNotifierStreams.SCENE, McpLogLevel.Error, Note("x")); + Assert.That(sink.Closed, Is.True); + + // The dead sink was removed; a second publish must not touch it again. + sink.Closed = false; + channel.Publish(McpLogNotifierStreams.SCENE, McpLogLevel.Error, Note("y")); + Assert.That(sink.Closed, Is.False); + } + + [Test] + public void StopDeliveringAfterUnsubscribe() + { + using var channel = new McpNotificationChannel(); + var sink = new FakeSink(); + var handle = channel.Add(McpLogNotifierStreams.SCENE, McpLogLevel.Debug, sink); + + handle!.Dispose(); + channel.Publish(McpLogNotifierStreams.SCENE, McpLogLevel.Error, Note("after")); + + Assert.That(sink.Frames, Is.Empty); + Assert.That(sink.Closed, Is.True); + } + + [Test] + public void CloseAllSinksAndRefuseNewOnesOnDispose() + { + var channel = new McpNotificationChannel(); + var sink = new FakeSink(); + channel.Add(McpLogNotifierStreams.SCENE, McpLogLevel.Debug, sink); + + channel.Dispose(); + + Assert.That(sink.Closed, Is.True); + Assert.That(channel.Add(McpLogNotifierStreams.SCENE, McpLogLevel.Debug, new FakeSink()), Is.Null); + } + + // Mirror of McpLogNotifier's stream names, kept local so the test asserts the channel in isolation. + private static class McpLogNotifierStreams + { + public const string SCENE = "scene"; + public const string CLIENT = "client"; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpNotificationChannelShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpNotificationChannelShould.cs.meta new file mode 100644 index 00000000000..868c164266b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpNotificationChannelShould.cs.meta @@ -0,0 +1,11 @@ +fileFormatVersion: 2 +guid: 19d0d0a3860d4841bceb9c7400569a2b +MonoImporter: + externalObjects: {} + serializedVersion: 2 + defaultReferences: [] + executionOrder: 0 + icon: {instanceID: 0} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpPointerClickSystemShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpPointerClickSystemShould.cs new file mode 100644 index 00000000000..42d52792da9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpPointerClickSystemShould.cs @@ -0,0 +1,303 @@ +using Arch.Core; +using CRDT; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.ECSComponents; +using DCL.Interaction.Utility; +using DCL.McpServer.Components; +using DCL.McpServer.Systems; +using DCL.Utilities; +using ECS.SceneLifeCycle; +using ECS.TestSuite; +using ECS.Unity.Transforms.Components; +using NSubstitute; +using NUnit.Framework; +using SceneRunner.Scene; +using UnityEngine; +using Utility.Multithreading; + +namespace DCL.McpServer.Tests +{ + public class McpPointerClickSystemShould : UnitySystemTestBase + { + private const int TARGET_CRDT_ID = 512; + private const int BLOCKER_CRDT_ID = 513; + + private World sceneWorld = null!; + private Entity playerEntity; + private Entity targetEntity; + + private GameObject cameraGo = null!; + private GameObject playerGo = null!; + private GameObject targetGo = null!; + private GameObject blockerGo = null!; + + private BoxCollider targetCollider = null!; + private PBPointerEvents targetPointerEvents = null!; + private ISceneStateProvider sceneStateProvider = null!; + private uint tick; + + [SetUp] + public void SetUp() + { + sceneWorld = World.Create(); + + cameraGo = new GameObject("mcp-click-test-camera"); + Camera camera = cameraGo.AddComponent(); + world.Create(new CameraComponent(camera)); // Mode defaults to FirstPerson + + playerGo = new GameObject("mcp-click-test-player"); + playerEntity = world.Create(new CharacterTransform(playerGo.transform)); + + targetGo = new GameObject("mcp-click-test-target") + { + transform = { position = new Vector3(0f, 0f, 5f), }, + }; + + targetCollider = targetGo.AddComponent(); + + targetPointerEvents = new PBPointerEvents + { + PointerEvents = + { + new PBPointerEvents.Types.Entry + { + EventType = PointerEventType.PetDown, + EventInfo = new PBPointerEvents.Types.Info + { + Button = InputAction.IaPointer, + HoverText = "Open", + MaxDistance = 10f, + }, + }, + }, + }; + + targetPointerEvents.AppendPointerEventResultsIntent.InitializeWithAlloc(); + + // The entity needs a TransformComponent so ResolveEntityAimPoint can aim the validation ray at it; + // without it the aim point is Vector3.zero and the click bails out before the raycast. + targetEntity = sceneWorld.Create(targetPointerEvents, new CRDTEntity(TARGET_CRDT_ID), new TransformComponent(targetGo.transform)); + + // Colliders created/moved this frame are not in the PhysX scene until transforms are synced (no physics step runs in EditMode). + Physics.SyncTransforms(); + + tick = 100u; + sceneStateProvider = Substitute.For(); + sceneStateProvider.IsCurrent.Returns(true); + sceneStateProvider.State.Returns(new Atomic(SceneState.Running)); + sceneStateProvider.TickNumber.Returns(_ => tick); + + ISceneFacade sceneFacade = Substitute.For(); + sceneFacade.SceneStateProvider.Returns(sceneStateProvider); + sceneFacade.EcsExecutor.Returns(new SceneEcsExecutor(sceneWorld)); + + IReadonlyReactiveProperty currentScene = Substitute.For>(); + currentScene.Value.Returns(sceneFacade); + + IScenesCache scenesCache = Substitute.For(); + scenesCache.CurrentScene.Returns(currentScene); + + IEntityCollidersGlobalCache collidersCache = Substitute.For(); + + collidersCache.TryGetSceneEntity(Arg.Any(), out Arg.Any()) + .Returns(call => + { + var collider = call.ArgAt(0); + + if (collider == targetCollider) + { + call[1] = new GlobalColliderSceneEntityInfo( + new SceneEcsExecutor(sceneWorld), + new ColliderSceneEntityInfo(targetEntity, new CRDTEntity(TARGET_CRDT_ID), ColliderLayer.ClPointer)); + + return true; + } + + if (blockerGo != null && collider == blockerGo.GetComponent()) + { + call[1] = new GlobalColliderSceneEntityInfo( + new SceneEcsExecutor(sceneWorld), + new ColliderSceneEntityInfo(sceneWorld.Create(new CRDTEntity(BLOCKER_CRDT_ID)), new CRDTEntity(BLOCKER_CRDT_ID), ColliderLayer.ClPhysics)); + + return true; + } + + return false; + }); + + system = new McpPointerClickSystem(world, scenesCache, collidersCache, playerEntity); + system.Initialize(); + } + + protected override void OnTearDown() + { + Object.DestroyImmediate(cameraGo); + Object.DestroyImmediate(playerGo); + Object.DestroyImmediate(targetGo); + + if (blockerGo != null) + Object.DestroyImmediate(blockerGo); + + sceneWorld.Dispose(); + } + + private UniTaskCompletionSource AddIntent( + McpPointerClickIntent.ClickKind kind = McpPointerClickIntent.ClickKind.CLICK, + int? targetId = null) + { + var completion = new UniTaskCompletionSource(); + + world.Add(playerEntity, new McpPointerClickIntent + { + TargetEntityId = targetId ?? targetEntity.Id, + Button = InputAction.IaPointer, + Kind = kind, + Phase = McpPointerClickIntent.ClickPhase.DOWN, + Deadline = UnityEngine.Time.time + 5f, + Completion = completion, + }); + + return completion; + } + + private static McpPointerClickResult ResultOf(UniTaskCompletionSource completion) + { + Assert.That(completion.Task.Status, Is.EqualTo(UniTaskStatus.Succeeded)); + return completion.Task.GetAwaiter().GetResult(); + } + + [Test] + public void DeliverDownThenUpOnNextTick() + { + UniTaskCompletionSource completion = AddIntent(); + + system!.Update(0); + + var actions = targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions; + Assert.That(actions.Count, Is.EqualTo(1)); + Assert.That(actions[0], Is.EqualTo((InputAction.IaPointer, PointerEventType.PetDown))); + Assert.That(completion.Task.Status, Is.EqualTo(UniTaskStatus.Pending)); + + // The scene-world flush clears the intent at the end of a real frame. + targetPointerEvents.AppendPointerEventResultsIntent.Clear(); + + system.Update(0); // same tick: keeps waiting so PetUp lands on a later tick than PetDown + Assert.That(completion.Task.Status, Is.EqualTo(UniTaskStatus.Pending)); + + tick++; + system.Update(0); // observes the tick advance + system.Update(0); // delivers PetUp + + actions = targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions; + Assert.That(actions.Count, Is.EqualTo(1)); + Assert.That(actions[0], Is.EqualTo((InputAction.IaPointer, PointerEventType.PetUp))); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.True); + Assert.That(result.CrdtEntityId, Is.EqualTo(TARGET_CRDT_ID)); + Assert.That(result.HoverText, Is.EqualTo("Open")); + Assert.That(result.UpRayMissed, Is.False); + Assert.That(world.Has(playerEntity), Is.False); + } + + [Test] + public void DeliverSingleDownWithoutWaiting() + { + UniTaskCompletionSource completion = AddIntent(McpPointerClickIntent.ClickKind.DOWN); + + system!.Update(0); + + var actions = targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions; + Assert.That(actions.Count, Is.EqualTo(1)); + Assert.That(actions[0], Is.EqualTo((InputAction.IaPointer, PointerEventType.PetDown))); + + Assert.That(ResultOf(completion).Hit, Is.True); + Assert.That(world.Has(playerEntity), Is.False); + } + + [Test] + public void FailWhenAnotherColliderBlocksTheRay() + { + blockerGo = new GameObject("mcp-click-test-blocker") { transform = { position = new Vector3(0f, 0f, 2f) }}; + + blockerGo.AddComponent(); + Physics.SyncTransforms(); + + UniTaskCompletionSource completion = AddIntent(); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.BlockedByCrdtId, Is.EqualTo(BLOCKER_CRDT_ID)); + Assert.That(targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions.Count, Is.EqualTo(0)); + Assert.That(world.Has(playerEntity), Is.False); + } + + [Test] + public void FailWhenOutOfRange() + { + targetPointerEvents.PointerEvents[0].EventInfo.MaxDistance = 2f; + + UniTaskCompletionSource completion = AddIntent(); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.FailureReason, Does.Contain("out of range")); + Assert.That(targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions.Count, Is.EqualTo(0)); + } + + [Test] + public void FailWhenEntityHasNoPointerEvents() + { + sceneWorld.Remove(targetEntity); + + UniTaskCompletionSource completion = AddIntent(); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.FailureReason, Does.Contain("PointerEvents")); + } + + [Test] + public void FailWhenEntityIdIsUnknown() + { + UniTaskCompletionSource completion = AddIntent(targetId: 987654); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.FailureReason, Does.Contain("no entity")); + } + + [Test] + public void FailWhenDeadlinePassed() + { + var completion = new UniTaskCompletionSource(); + + world.Add(playerEntity, new McpPointerClickIntent + { + TargetEntityId = targetEntity.Id, + Button = InputAction.IaPointer, + Kind = McpPointerClickIntent.ClickKind.CLICK, + Phase = McpPointerClickIntent.ClickPhase.DOWN, + Deadline = UnityEngine.Time.time - 1f, + Completion = completion, + }); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.FailureReason, Does.Contain("timed out")); + Assert.That(world.Has(playerEntity), Is.False); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpPointerClickSystemShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpPointerClickSystemShould.cs.meta new file mode 100644 index 00000000000..31ec606ea26 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpPointerClickSystemShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 3245eba0487e4b8a8c8f125b10268888 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs b/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs new file mode 100644 index 00000000000..851ebca12fc --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs @@ -0,0 +1,57 @@ +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System.Collections.Generic; + +namespace DCL.McpServer.Tests +{ + /// + /// Guards against output-schema ↔ structuredContent drift: a tool declares its OutputSchema by hand and + /// builds the matching payload by hand, so a field added to one and forgotten in the other lets the schema + /// silently misdescribe the payload to the agent. Asserting the two carry the same property names + /// (recursively, into nested objects the payload populates) turns that drift into a failing test. + /// + internal static class McpSchemaAssert + { + public static void KeysMatch(JObject schema, JObject payload) + { + var properties = schema["properties"] as JObject ?? new JObject(); + + CollectionAssert.AreEquivalent(NamesOf(properties), NamesOf(payload), + $"Output schema and payload disagree on the keys of '{PathOf(schema)}'."); + + foreach (JProperty property in properties.Properties()) + if (DeclaresObject(property.Value) && payload[property.Name] is JObject nested) + KeysMatch((JObject)property.Value, nested); + } + + private static List NamesOf(JObject obj) + { + var names = new List(); + + foreach (JProperty property in obj.Properties()) + names.Add(property.Name); + + return names; + } + + // A property models an object both as type "object" and as a nullable ["object", "null"] union. + private static bool DeclaresObject(JToken schema) + { + JToken? type = schema["type"]; + + if (type is JArray union) + { + foreach (JToken member in union) + if (member.Value() == "object") + return true; + + return false; + } + + return type?.Value() == "object"; + } + + private static string PathOf(JObject schema) => + string.IsNullOrEmpty(schema.Path) ? "" : schema.Path; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs.meta new file mode 100644 index 00000000000..a88c5ac22bd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: b0e1283f51dcb2948b8bb95431335e2f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs new file mode 100644 index 00000000000..ab91a05c4be --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs @@ -0,0 +1,78 @@ +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System; +using System.Text; + +namespace DCL.McpServer.Tests +{ + public class McpToolResultShould + { + [Test] + public void WrapTextInASingleTextContentItemWithoutAnErrorFlag() + { + JObject payload = McpToolResult.Text("hello").Payload; + + var content = (JArray)payload["content"]!; + Assert.That(content.Count, Is.EqualTo(1)); + Assert.That(content[0]!["type"]!.Value(), Is.EqualTo("text")); + Assert.That(content[0]!["text"]!.Value(), Is.EqualTo("hello")); + Assert.That(payload.ContainsKey("isError"), Is.False); + } + + [Test] + public void CarryBothAMirroringTextItemAndStructuredContent() + { + var structured = new JObject + { + ["count"] = 3, + ["nested"] = new JObject { ["ok"] = true }, + }; + + JObject payload = McpToolResult.TextWithStructured("mirror", structured).Payload; + + var content = (JArray)payload["content"]!; + Assert.That(content.Count, Is.EqualTo(1)); + Assert.That(content[0]!["type"]!.Value(), Is.EqualTo("text")); + Assert.That(content[0]!["text"]!.Value(), Is.EqualTo("mirror")); + + var structuredContent = (JObject)payload["structuredContent"]!; + Assert.That(structuredContent["count"]!.Value(), Is.EqualTo(3)); + Assert.That(structuredContent["nested"]!["ok"]!.Value(), Is.True); + Assert.That(payload.ContainsKey("isError"), Is.False); + } + + [Test] + public void FlagErrorsWithIsErrorAndCarryTheMessageAsText() + { + JObject payload = McpToolResult.Error("it broke").Payload; + + Assert.That(payload["isError"]!.Value(), Is.True); + + var content = (JArray)payload["content"]!; + Assert.That(content[0]!["type"]!.Value(), Is.EqualTo("text")); + Assert.That(content[0]!["text"]!.Value(), Is.EqualTo("it broke")); + } + + [Test] + public void EncodeImageBytesAsBase64AlongsideACaption() + { + byte[] bytes = Encoding.UTF8.GetBytes("pixels"); + + JObject payload = McpToolResult.Image(bytes, "image/png", "a screenshot").Payload; + + var content = (JArray)payload["content"]!; + Assert.That(content.Count, Is.EqualTo(2)); + + var image = (JObject)content[0]!; + Assert.That(image["type"]!.Value(), Is.EqualTo("image")); + Assert.That(image["mimeType"]!.Value(), Is.EqualTo("image/png")); + Assert.That(Convert.FromBase64String(image["data"]!.Value()!), Is.EqualTo(bytes)); + + var caption = (JObject)content[1]!; + Assert.That(caption["type"]!.Value(), Is.EqualTo("text")); + Assert.That(caption["text"]!.Value(), Is.EqualTo("a screenshot")); + Assert.That(payload.ContainsKey("isError"), Is.False); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs.meta new file mode 100644 index 00000000000..85c2acbac61 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 746fe351b002c704ea5996a66566492f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs new file mode 100644 index 00000000000..11541284936 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs @@ -0,0 +1,211 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System; +using System.Collections.Generic; +using System.Threading; + +namespace DCL.McpServer.Tests +{ + public class McpToolsRegistryShould + { + [Test] + public void BuildAnObjectSchemaWithTypedPropertiesAndRequired() + { + // Act + JObject schema = McpJsonSchema.Object() + .Integer("count", "How many.") + .String("mode", "Pick one.", enumValues: new[] { "a", "b" }, required: true) + .Build(); + + // Assert + Assert.That(schema["type"]!.Value(), Is.EqualTo("object")); + + var properties = (JObject)schema["properties"]!; + Assert.That(properties["count"]!["type"]!.Value(), Is.EqualTo("integer")); + Assert.That(properties["count"]!["description"]!.Value(), Is.EqualTo("How many.")); + Assert.That(properties["mode"]!["type"]!.Value(), Is.EqualTo("string")); + Assert.That(((JArray)properties["mode"]!["enum"]!).ToObject(), Is.EqualTo(new[] { "a", "b" })); + + Assert.That(((JArray)schema["required"]!).ToObject(), Is.EqualTo(new[] { "mode" })); + } + + [Test] + public void OmitRequiredWhenNoFieldIsRequired() + { + // Act + JObject schema = McpJsonSchema.Object().Boolean("flag").Build(); + + // Assert + Assert.That(schema.ContainsKey("required"), Is.False); + } + + [Test] + public void FailRegistrationNamingTheToolWithAnInvalidSchema() + { + // Arrange + var registry = new McpToolsRegistry() + .Add(new FakeTool("broken", McpToolAnnotations.ReadOnly(), new JObject())); + + // Act & Assert + InvalidOperationException error = Assert.Throws(() => registry.Build()); + Assert.That(error!.Message, Does.Contain("broken")); + } + + [Test] + public void EmitReadOnlyAnnotationsWithoutStateChangeHints() + { + // Arrange + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("reader", McpToolAnnotations.ReadOnly())) + .Build()); + + // Act + JObject annotations = AnnotationsOf(toolsList, "reader"); + + // Assert + Assert.That(annotations["readOnlyHint"]!.Value(), Is.True); + Assert.That(annotations["openWorldHint"]!.Value(), Is.False); + Assert.That(annotations.ContainsKey("destructiveHint"), Is.False); + Assert.That(annotations.ContainsKey("idempotentHint"), Is.False); + } + + [Test] + public void EmitMutatingAnnotationsWithAllStateChangeHints() + { + // Arrange + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("mutator", McpToolAnnotations.Mutating(destructive: true, idempotent: false))) + .Build()); + + // Act + JObject annotations = AnnotationsOf(toolsList, "mutator"); + + // Assert + Assert.That(annotations["readOnlyHint"]!.Value(), Is.False); + Assert.That(annotations["destructiveHint"]!.Value(), Is.True); + Assert.That(annotations["idempotentHint"]!.Value(), Is.False); + Assert.That(annotations["openWorldHint"]!.Value(), Is.False); + } + + [Test] + public void IncludeOutputSchemaWhenTheToolDeclaresOne() + { + // Arrange + JObject outputSchema = McpJsonSchema.Object().Integer("total").Build(); + + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("structured", McpToolAnnotations.ReadOnly(), outputSchema: outputSchema)) + .Build()); + + // Act + JObject entry = EntryOf(toolsList, "structured"); + + // Assert + Assert.That(entry.ContainsKey("outputSchema"), Is.True); + Assert.That(entry["outputSchema"]!["type"]!.Value(), Is.EqualTo("object")); + } + + [Test] + public void OmitOutputSchemaWhenTheToolDeclaresNone() + { + // Arrange + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("plain", McpToolAnnotations.ReadOnly())) + .Build()); + + // Act & Assert + Assert.That(EntryOf(toolsList, "plain").ContainsKey("outputSchema"), Is.False); + } + + [Test] + public void FindARegisteredToolByName() + { + var tool = new FakeTool("known", McpToolAnnotations.ReadOnly()); + var registry = new McpToolsRegistry().Add(tool); + + bool found = registry.TryGet("known", out IMcpTool? resolved); + + Assert.That(found, Is.True); + Assert.That(resolved, Is.SameAs(tool)); + } + + [Test] + public void NotFindAnUnknownTool() + { + var registry = new McpToolsRegistry().Add(new FakeTool("known", McpToolAnnotations.ReadOnly())); + + Assert.That(registry.TryGet("missing", out IMcpTool? resolved), Is.False); + Assert.That(resolved, Is.Null); + } + + [Test] + public void NotFindAToolForANullOrEmptyName() + { + var registry = new McpToolsRegistry().Add(new FakeTool("known", McpToolAnnotations.ReadOnly())); + + Assert.That(registry.TryGet(null, out IMcpTool? byNull), Is.False); + Assert.That(byNull, Is.Null); + + Assert.That(registry.TryGet(string.Empty, out IMcpTool? byEmpty), Is.False); + Assert.That(byEmpty, Is.Null); + } + + [Test] + public void ReflectTheRegisteredSetInTheToolsList() + { + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("first", McpToolAnnotations.ReadOnly())) + .Add(new FakeTool("second", McpToolAnnotations.Mutating(destructive: false, idempotent: true))) + .Build()); + + var names = new List(); + + foreach (JToken entry in (JArray)toolsList["tools"]!) + names.Add(entry["name"]!.Value()); + + Assert.That(names, Is.EquivalentTo(new[] { "first", "second" })); + } + + private static JObject Payload(McpToolsRegistry registry) => + JObject.Parse(registry.ToolsListPayload().ToString()); + + private static JObject AnnotationsOf(JObject toolsList, string name) => + (JObject)EntryOf(toolsList, name)["annotations"]!; + + private static JObject EntryOf(JObject toolsList, string name) + { + foreach (JToken entry in (JArray)toolsList["tools"]!) + if (entry["name"]!.Value() == name) + return (JObject)entry; + + Assert.Fail($"tool '{name}' not found in tools/list"); + return null!; + } + + private class FakeTool : IMcpTool + { + public string Name { get; } + + public McpToolAnnotations Annotations { get; } + + public string Description => "fake"; + + public JObject InputSchema { get; } + + public JObject? OutputSchema { get; } + + public FakeTool(string name, McpToolAnnotations annotations, JObject? inputSchema = null, JObject? outputSchema = null) + { + Name = name; + Annotations = annotations; + InputSchema = inputSchema ?? McpJsonSchema.Object().Build(); + OutputSchema = outputSchema; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) => + UniTask.FromResult(McpToolResult.Text("fake")); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs.meta new file mode 100644 index 00000000000..94d6bf30834 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 1e7c91136f9767a4b9b3922fef87f114 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools.meta b/Explorer/Assets/DCL/McpServer/Tools.meta new file mode 100644 index 00000000000..ea160a5cffd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: 9a8b8d76e6c14409697c08478f574549 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs new file mode 100644 index 00000000000..1d10faa15c3 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs @@ -0,0 +1,161 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.ECSComponents; +using DCL.McpServer.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; +using UnityEngine; +using Utility.Arch; + +namespace DCL.McpServer.Tools +{ + /// + /// Presses a pointer button on a scene entity through / + /// McpPointerClickSystem, which validates the aim with the same raycast rules as the reticle + /// pipeline before filling the entity's pointer-event intent like a real click. + /// + public class ClickEntityTool : IMcpTool + { + private const float DEFAULT_TIMEOUT_SEC = 3f; + private const float MIN_TIMEOUT_SEC = 0.5f; + private const float MAX_TIMEOUT_SEC = 15f; + private const float COMPLETION_GRACE_SEC = 2f; + + private readonly World world; + private readonly Entity playerEntity; + + public string Name => "click_entity"; + + public string Description => + "Press and release a pointer button on a scene entity so its PointerEvents fire exactly like a real click. " + + "The aim is validated by a physics raycast from the camera: occluders and the entity's maxDistance apply, and a miss " + + "returns hit:false with the blocking entity. Ids come from list_scene_entities. For entities whose collider " + + "sits away from their pivot (e.g. GLTF meshes), pass an explicit x/y/z world point to aim at."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Integer("entityId", "Target entity id in the current scene world (from list_scene_entities). Omit only when x/y/z are given, then the ray decides the target.") + .Number("x", "World-space aim point; overrides the automatic aim at the entity's collider center.") + .Number("y") + .Number("z") + .String("button", "Which input action to press. Default pointer (left click / IA_POINTER).", enumValues: new[] { "pointer", "primary", "secondary" }) + .String("eventType", "click = down, then up on the next scene tick. Default click.", enumValues: new[] { "click", "down", "up" }) + .Number("timeoutSec", "Seconds to wait for delivery. Default 3, max 15.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: false); + + public ClickEntityTool(World world, Entity playerEntity) + { + this.world = world; + this.playerEntity = playerEntity; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + bool hasEntityId = arguments.TryGetInt("entityId", out int entityId); + + bool hasAimPoint = arguments.TryGetFloat("x", out float x) + & arguments.TryGetFloat("y", out float y) + & arguments.TryGetFloat("z", out float z); + + if (!hasEntityId && !hasAimPoint) + return McpToolResult.Error("Provide entityId, or a full x/y/z world aim point, or both."); + + InputAction button; + + switch (arguments.GetString("button", "pointer")) + { + case "pointer": button = InputAction.IaPointer; break; + case "primary": button = InputAction.IaPrimary; break; + case "secondary": button = InputAction.IaSecondary; break; + default: return McpToolResult.Error("button must be one of: pointer, primary, secondary."); + } + + McpPointerClickIntent.ClickKind kind; + + switch (arguments.GetString("eventType", "click")) + { + case "click": kind = McpPointerClickIntent.ClickKind.CLICK; break; + case "down": kind = McpPointerClickIntent.ClickKind.DOWN; break; + case "up": kind = McpPointerClickIntent.ClickKind.UP; break; + default: return McpToolResult.Error("eventType must be one of: click, down, up."); + } + + float timeoutSec = Mathf.Clamp(arguments.GetFloat("timeoutSec", DEFAULT_TIMEOUT_SEC), MIN_TIMEOUT_SEC, MAX_TIMEOUT_SEC); + + // A newer click preempts a pending one; release its awaiter before replacing the intent. + if (world.TryGet(playerEntity, out McpPointerClickIntent existingIntent)) + existingIntent.Completion?.TrySetResult(new McpPointerClickResult + { + Hit = false, + FailureReason = "preempted by a newer click_entity call", + }); + + var completion = new UniTaskCompletionSource(); + + world.AddOrSet(playerEntity, new McpPointerClickIntent + { + TargetEntityId = hasEntityId ? entityId : -1, + AimPoint = new Vector3(x, y, z), + HasExplicitAimPoint = hasAimPoint, + Button = button, + Kind = kind, + Phase = McpPointerClickIntent.ClickPhase.DOWN, + Deadline = UnityEngine.Time.time + timeoutSec, + Completion = completion, + }); + + McpPointerClickResult result; + + try + { + result = await completion.Task.AttachExternalCancellation(ct) + .Timeout(TimeSpan.FromSeconds(timeoutSec + COMPLETION_GRACE_SEC)); + } + catch (TimeoutException) + { + await UniTask.SwitchToMainThread(); + + if (world.Has(playerEntity)) + world.Remove(playerEntity); + + return McpToolResult.Error($"click_entity did not complete within {timeoutSec + COMPLETION_GRACE_SEC}s (is the simulation paused?)."); + } + + var json = new JObject + { + ["hit"] = result.Hit, + ["entityId"] = result.SceneEntityId, + ["crdtEntityId"] = result.CrdtEntityId, + }; + + if (result.FailureReason != null) + json["reason"] = result.FailureReason; + + if (result.Hit) + { + json["hitPoint"] = result.HitPoint.ToVector(); + json["distance"] = Math.Round(result.Distance, 2); + } + + if (result.HoverText != null) + json["hoverText"] = result.HoverText; + + if (result.BlockedByEntityId != null) + { + json["blockedByEntityId"] = result.BlockedByEntityId; + json["blockedByCrdtId"] = result.BlockedByCrdtId; + json["blockedByCollider"] = result.BlockedByColliderName; + } + + if (result.UpRayMissed) + json["upRayMissed"] = true; + + return McpToolResult.Json(json); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs.meta new file mode 100644 index 00000000000..8b943172934 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: d28c7ef4de5c429198677e5f3fb0776d \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs new file mode 100644 index 00000000000..1758384d2a4 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs @@ -0,0 +1,59 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using SceneRunner.Debugging; +using SceneRunner.Debugging.Hub; +using System.Text; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + public class GetEntityDetailsTool : IMcpTool + { + private const int MAX_CHARS = 8000; + private const string CURRENT_SCENE = "CURRENT"; + + private readonly IWorldInfoHub worldInfoHub; + + public string Name => "get_entity_details"; + + public string Description => + "Dump all components of one entity in the current scene's ECS world (ids come from list_scene_entities)."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Integer("entityId", "Entity id within the current scene world.", required: true) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public GetEntityDetailsTool(IWorldInfoHub worldInfoHub) + { + this.worldInfoHub = worldInfoHub; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetInt("entityId", out int entityId)) + return UniTask.FromResult(McpToolResult.Error("entityId is required.")); + + IWorldInfo? worldInfo = worldInfoHub.WorldInfo(CURRENT_SCENE); + + if (worldInfo == null) + return UniTask.FromResult(McpToolResult.Error("No scene world found at the current parcel.")); + + string dump = worldInfo.EntityComponentsInfo(entityId); + + if (dump.Length <= MAX_CHARS) + return UniTask.FromResult(McpToolResult.Text(dump)); + + var output = new StringBuilder(MAX_CHARS + 64); + output.Append(dump, 0, MAX_CHARS); + output.AppendLine(); + output.Append($"... output truncated at {MAX_CHARS}/{dump.Length} chars"); + + return UniTask.FromResult(McpToolResult.Text(output.ToString())); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs.meta new file mode 100644 index 00000000000..bd9c419498a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 8db7c68630cf74ae7a2fa4153b29875f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs new file mode 100644 index 00000000000..a483f2b5a79 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs @@ -0,0 +1,89 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.CharacterMotion.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.Profiles; +using ECS.SceneLifeCycle.CurrentScene; +using Newtonsoft.Json.Linq; +using System.Threading; +using UnityEngine; +using Utility; + +namespace DCL.McpServer.Tools +{ + public class GetPlayerStateTool : IMcpTool + { + private readonly World world; + private readonly Entity playerEntity; + private readonly ExposedCameraData exposedCameraData; + private readonly ICurrentSceneInfo currentSceneInfo; + + public string Name => "get_player_state"; + + public string Description => + "Read the player's current world position, rotation, parcel, velocity and grounded state, the camera position, rotation and mode, " + + "and the wallet address — use the address to tell Explorer instances apart when several run at once."; + + public JObject InputSchema => McpJsonSchema.Object().Build(); + + public JObject? OutputSchema => + McpJsonSchema.Object() + .Object("position", JObjectExtensions.VectorSchema()) + .Object("rotationEuler", JObjectExtensions.VectorSchema()) + .Object("parcel", JObjectExtensions.ParcelSchema()) + .Object("velocity", JObjectExtensions.VectorSchema()) + .Boolean("isGrounded") + .Boolean("isPlayerStandingOnScene") + .String("address", "Wallet address of the logged-in player, or null when no profile is loaded.", nullable: true) + .Object("camera", McpJsonSchema.Object() + .Object("position", JObjectExtensions.VectorSchema()) + .Object("rotationEuler", JObjectExtensions.VectorSchema()) + .String("mode") + .Boolean("modeChangeAllowed") + .Boolean("pointerLocked")) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public GetPlayerStateTool(World world, Entity playerEntity, ExposedCameraData exposedCameraData, ICurrentSceneInfo currentSceneInfo) + { + this.world = world; + this.playerEntity = playerEntity; + this.exposedCameraData = exposedCameraData; + this.currentSceneInfo = currentSceneInfo; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + CharacterTransform characterTransform = world.Get(playerEntity); + Vector3 position = characterTransform.Position; + + world.TryGet(playerEntity, out CharacterRigidTransform? rigidTransform); + world.TryGet(playerEntity, out Profile? profile); + + var state = new JObject + { + ["position"] = position.ToVector(), + ["rotationEuler"] = characterTransform.Rotation.eulerAngles.ToVector(), + ["parcel"] = position.ToParcel().ToParcel(), + ["velocity"] = (rigidTransform?.MoveVelocity.Velocity ?? Vector3.zero).ToVector(), + ["isGrounded"] = rigidTransform?.IsGrounded ?? false, + ["isPlayerStandingOnScene"] = currentSceneInfo.IsPlayerStandingOnScene, + ["address"] = profile == null ? JValue.CreateNull() : profile.Compact.UserId, + ["camera"] = new JObject + { + ["position"] = exposedCameraData.WorldPosition.Value.ToVector(), + ["rotationEuler"] = exposedCameraData.WorldRotation.Value.eulerAngles.ToVector(), + ["mode"] = exposedCameraData.CameraMode.ToString(), + ["modeChangeAllowed"] = SetCameraModeTool.IsModeChangeAllowed(world), + ["pointerLocked"] = exposedCameraData.PointerIsLocked.Value, + }, + }; + + return UniTask.FromResult(McpToolResult.JsonWithStructured(state)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs.meta new file mode 100644 index 00000000000..c42e70d65de --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 0c34c7751a4c04382810aefd640df2c1 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs new file mode 100644 index 00000000000..da6be244a51 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs @@ -0,0 +1,57 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Collections.Generic; +using System.Text; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + public class GetSceneLogsTool : IMcpTool + { + private const int DEFAULT_LIMIT = 100; + private const int MAX_LIMIT = 500; + + private readonly SceneLogBuffer logBuffer; + + public string Name => "get_scene_logs"; + + public string Description => + "Read the scene's JavaScript console output (logs, warnings, errors and exceptions). Entries carry monotonic sequence numbers; " + + "pass the last seen sequence as sinceSeq to poll incrementally."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Integer("limit", "Maximum entries to return (newest win). Default 100.") + .String("severity", "Filter by severity. Default all.", enumValues: new[] { "all", "error" }) + .Integer("sinceSeq", "Only return entries with a sequence number greater than this.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public GetSceneLogsTool(SceneLogBuffer logBuffer) + { + this.logBuffer = logBuffer; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + int limit = Mathf.Clamp(arguments.GetInt("limit", DEFAULT_LIMIT), 1, MAX_LIMIT); + bool errorsOnly = arguments.GetString("severity", "all") == "error"; + long sinceSeq = arguments.GetLong("sinceSeq", -1); + + var entries = new List(limit); + logBuffer.CopyTo(entries, sinceSeq, errorsOnly, limit); + + var output = new StringBuilder(); + output.AppendLine($"latestSeq={logBuffer.LatestSeq} returned={entries.Count}"); + + foreach (SceneLogBuffer.Entry entry in entries) + output.AppendLine($"#{entry.Seq} {entry.Message}"); + + return UniTask.FromResult(McpToolResult.Text(output.ToString())); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs.meta new file mode 100644 index 00000000000..7e2d718b638 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 0ef4e7ee221c646b1ad9e254b9326614 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs new file mode 100644 index 00000000000..274630831ac --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs @@ -0,0 +1,84 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.RealmNavigation; +using ECS.SceneLifeCycle; +using ECS.SceneLifeCycle.CurrentScene; +using Newtonsoft.Json.Linq; +using SceneRunner.Scene; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + public class GetSceneStateTool : IMcpTool + { + private readonly IScenesCache scenesCache; + private readonly ICurrentSceneInfo currentSceneInfo; + private readonly ILoadingStatus loadingStatus; + private readonly bool localSceneDevelopment; + + public string Name => "get_scene_state"; + + public string Description => + "Read the state of the scene at the player's current parcel: name, base parcel, runtime state (including JavaScript/ECS errors), " + + "readiness, asset loading progress and the global loading-screen stage. Call this after teleporting or reloading before interacting."; + + public JObject InputSchema => McpJsonSchema.Object().Build(); + + public JObject? OutputSchema => + McpJsonSchema.Object() + .Object("currentParcel", JObjectExtensions.ParcelSchema()) + .String("loadingStage") + .Boolean("loadingScreenOn") + .Boolean("localSceneDevelopment") + .Object("scene", McpJsonSchema.Object() + .String("name") + .Object("baseParcel", JObjectExtensions.ParcelSchema()) + .String("sdkVersion", "SDK version reported by the scene, or null when unknown.", nullable: true) + .String("state") + .Boolean("isReady") + .Boolean("assetsLoadingConcluded") + .String("runningStatus"), + "The scene at the player's current parcel, or null when no scene is loaded there.", nullable: true) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public GetSceneStateTool(IScenesCache scenesCache, ICurrentSceneInfo currentSceneInfo, ILoadingStatus loadingStatus, bool localSceneDevelopment) + { + this.scenesCache = scenesCache; + this.currentSceneInfo = currentSceneInfo; + this.loadingStatus = loadingStatus; + this.localSceneDevelopment = localSceneDevelopment; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + Vector2Int currentParcel = scenesCache.CurrentParcel.Value; + ISceneFacade? scene = scenesCache.CurrentScene.Value; + + var state = new JObject + { + ["currentParcel"] = currentParcel.ToParcel(), + ["loadingStage"] = loadingStatus.CurrentStage.Value.ToString(), + ["loadingScreenOn"] = loadingStatus.IsLoadingScreenOn(), + ["localSceneDevelopment"] = localSceneDevelopment, + ["scene"] = scene == null + ? JValue.CreateNull() + : new JObject + { + ["name"] = scene.Info.Name, + ["baseParcel"] = scene.Info.BaseParcel.ToParcel(), + ["sdkVersion"] = scene.Info.SdkVersion, + ["state"] = scene.SceneStateProvider.State.Value().ToString(), + ["isReady"] = scene.IsSceneReady(), + ["assetsLoadingConcluded"] = scene.SceneData.SceneLoadingConcluded, + ["runningStatus"] = currentSceneInfo.SceneStatus.Value?.ToString() ?? "Unknown", + }, + }; + + return UniTask.FromResult(McpToolResult.JsonWithStructured(state)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs.meta new file mode 100644 index 00000000000..2fdf38209b6 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: f9c6cd0d753384d63b9e283c8cd1eabb \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs new file mode 100644 index 00000000000..3ff757a6f68 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs @@ -0,0 +1,92 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using SceneRunner.Debugging; +using SceneRunner.Debugging.Hub; +using System.Collections.Generic; +using System.Text; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + /// + /// Lists the entity ids of the current scene's ECS world through the same unsynchronized + /// path the existing debug tooling uses. + /// + public class ListSceneEntitiesTool : IMcpTool + { + private const int DEFAULT_LIMIT = 200; + private const int MAX_LIMIT = 2000; + private const string CURRENT_SCENE = "CURRENT"; + + private readonly IWorldInfoHub worldInfoHub; + + public string Name => "list_scene_entities"; + + public string Description => + "List the ECS entity ids of the scene at the player's current parcel. Feed an id into get_entity_details to inspect its components."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Integer("limit", "Maximum ids to return. Default 200.") + .Build(); + + public JObject? OutputSchema => + McpJsonSchema.Object() + .Integer("total") + .Integer("returned") + .Boolean("truncated") + .IntegerArray("entityIds") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public ListSceneEntitiesTool(IWorldInfoHub worldInfoHub) + { + this.worldInfoHub = worldInfoHub; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + int limit = Mathf.Clamp(arguments.GetInt("limit", DEFAULT_LIMIT), 1, MAX_LIMIT); + + IWorldInfo? worldInfo = worldInfoHub.WorldInfo(CURRENT_SCENE); + + if (worldInfo == null) + return UniTask.FromResult(McpToolResult.Error("No scene world found at the current parcel.")); + + IReadOnlyList entityIds = worldInfo.EntityIds(); + int returned = Mathf.Min(limit, entityIds.Count); + bool truncated = returned < entityIds.Count; + + var ids = new JArray(); + var output = new StringBuilder(); + output.AppendLine($"total={entityIds.Count} returned={returned}"); + + for (var i = 0; i < entityIds.Count && i < limit; i++) + { + ids.Add(entityIds[i]); + output.Append(entityIds[i]); + output.Append(i < entityIds.Count - 1 && i < limit - 1 ? ", " : string.Empty); + } + + if (truncated) + { + output.AppendLine(); + output.Append($"{returned} of {entityIds.Count} shown; raise limit (max {MAX_LIMIT}) to see the rest."); + } + + var structured = new JObject + { + ["total"] = entityIds.Count, + ["returned"] = returned, + ["truncated"] = truncated, + ["entityIds"] = ids, + }; + + return UniTask.FromResult(McpToolResult.TextWithStructured(output.ToString(), structured)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs.meta new file mode 100644 index 00000000000..0e7a23b95e6 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 76659a159375e46aabcb920885ee30a3 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs b/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs new file mode 100644 index 00000000000..917737c93f8 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs @@ -0,0 +1,63 @@ +using Arch.Core; +using CrdtEcsBridge.RestrictedActions; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + public class LookAtTool : IMcpTool + { + private readonly IGlobalWorldActions globalWorldActions; + private readonly World world; + private readonly Entity playerEntity; + private readonly ExposedCameraData exposedCameraData; + + public string Name => "look_at"; + + public string Description => + "Rotate the camera to look at a world-space point (x,y,z in meters). Useful to center something on screen before a screenshot."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Number("x", required: true) + .Number("y", required: true) + .Number("z", required: true) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public LookAtTool(IGlobalWorldActions globalWorldActions, World world, Entity playerEntity, ExposedCameraData exposedCameraData) + { + this.globalWorldActions = globalWorldActions; + this.world = world; + this.playerEntity = playerEntity; + this.exposedCameraData = exposedCameraData; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetFloat("x", out float x) || !arguments.TryGetFloat("y", out float y) || !arguments.TryGetFloat("z", out float z)) + return McpToolResult.Error("x, y and z world coordinates are required."); + + Vector3 playerPosition = world.Get(playerEntity).Position; + globalWorldActions.RotateCamera(new Vector3(x, y, z), playerPosition); + + // Let the Cinemachine systems apply the look-at intent before reading the camera back. + await UniTask.DelayFrame(3, cancellationToken: ct); + + var result = new JObject + { + ["cameraPosition"] = exposedCameraData.WorldPosition.Value.ToVector(), + ["cameraRotationEuler"] = exposedCameraData.WorldRotation.Value.eulerAngles.ToVector(), + }; + + return McpToolResult.Json(result); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs.meta new file mode 100644 index 00000000000..083259d660a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: f48ec52210bd7465e84233aa7ae83b13 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs b/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs new file mode 100644 index 00000000000..bc3e199e15d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs @@ -0,0 +1,84 @@ +using Arch.Core; +using CrdtEcsBridge.RestrictedActions; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; +using UnityEngine; +using Utility; + +namespace DCL.McpServer.Tools +{ + public class MoveToTool : IMcpTool + { + private const float MAX_DURATION_SEC = 30f; + private const float COMPLETION_GRACE_SEC = 5f; + + private readonly IGlobalWorldActions globalWorldActions; + private readonly World world; + private readonly Entity playerEntity; + + public string Name => "move_to"; + + public string Description => + "Move the player to a world-space position (x,y,z in meters; one parcel is 16x16m). Instant by default, or smooth over durationSec. " + + "Optionally face a look-at target on arrival. For crossing to another scene prefer the teleport tool."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Number("x", required: true) + .Number("y", required: true) + .Number("z", required: true) + .Number("lookAtX") + .Number("lookAtY") + .Number("lookAtZ") + .Number("durationSec", "Seconds to move over; 0 (default) teleports instantly.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public MoveToTool(IGlobalWorldActions globalWorldActions, World world, Entity playerEntity) + { + this.globalWorldActions = globalWorldActions; + this.world = world; + this.playerEntity = playerEntity; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetFloat("x", out float x) || !arguments.TryGetFloat("y", out float y) || !arguments.TryGetFloat("z", out float z)) + return McpToolResult.Error("x, y and z world coordinates are required."); + + Vector3? lookAtTarget = null; + + if (arguments.TryGetFloat("lookAtX", out float lookAtX) && arguments.TryGetFloat("lookAtY", out float lookAtY) && arguments.TryGetFloat("lookAtZ", out float lookAtZ)) + lookAtTarget = new Vector3(lookAtX, lookAtY, lookAtZ); + + float durationSec = Mathf.Clamp(arguments.GetFloat("durationSec", 0f), 0f, MAX_DURATION_SEC); + var targetPosition = new Vector3(x, y, z); + + try + { + await globalWorldActions.MoveAndRotatePlayerAsync(targetPosition, lookAtTarget, lookAtTarget, durationSec, ct) + .Timeout(TimeSpan.FromSeconds(durationSec + COMPLETION_GRACE_SEC)); + } + catch (TimeoutException) { return McpToolResult.Error($"move_to did not complete within {durationSec + COMPLETION_GRACE_SEC}s."); } + + // Give the teleport/rotation systems a couple of frames to apply the intents before reading back. + await UniTask.DelayFrame(2, cancellationToken: ct); + + Vector3 finalPosition = world.Get(playerEntity).Position; + + var result = new JObject + { + ["position"] = finalPosition.ToVector(), + ["parcel"] = finalPosition.ToParcel().ToParcel(), + }; + + return McpToolResult.Json(result); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs.meta new file mode 100644 index 00000000000..789ec935f21 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: e497faa837a32493bb1f287241cb4661 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs new file mode 100644 index 00000000000..d39547ba799 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs @@ -0,0 +1,86 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.CharacterMotion.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.SkyBox.Components; +using ECS.SceneLifeCycle; +using Newtonsoft.Json.Linq; +using SceneRunner.Scene; +using System; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + /// + /// Reloads the scene at the player's parcel, freezing character motion and the skybox during the reload + /// exactly like the local-scene-development hot reload does. + /// + public class ReloadSceneTool : IMcpTool + { + private const float MIN_TIMEOUT_SEC = 5f; + private const float MAX_TIMEOUT_SEC = 120f; + private const float DEFAULT_TIMEOUT_SEC = 15f; + + private readonly ECSReloadScene reloadScene; + private readonly IScenesCache scenesCache; + private readonly World world; + private readonly Entity playerEntity; + private readonly Entity skyboxEntity; + + public string Name => "reload_scene"; + + public string Description => + "Reload the scene at the player's current parcel and wait for it to restart. Use after editing scene code " + + "when hot reload didn't trigger, or to reset scene state before a test run."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Number("timeoutSec", "Maximum seconds to wait for the reload. Default 15.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: true, idempotent: false); + + public ReloadSceneTool(ECSReloadScene reloadScene, IScenesCache scenesCache, World world, Entity playerEntity, Entity skyboxEntity) + { + this.reloadScene = reloadScene; + this.scenesCache = scenesCache; + this.world = world; + this.playerEntity = playerEntity; + this.skyboxEntity = skyboxEntity; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + float timeoutSec = Mathf.Clamp(arguments.GetFloat("timeoutSec", DEFAULT_TIMEOUT_SEC), MIN_TIMEOUT_SEC, MAX_TIMEOUT_SEC); + + if (scenesCache.CurrentScene.Value == null) + return McpToolResult.Error("There is no scene at the current parcel to reload."); + + try + { + world.AddOrGet(playerEntity, new StopCharacterMotion()); + world.AddOrGet(skyboxEntity, new PauseSkyboxTimeUpdate()); + + ISceneFacade? reloadedScene = await reloadScene.TryReloadSceneAsync(ct) + .Timeout(TimeSpan.FromSeconds(timeoutSec)); + + if (reloadedScene == null) + return McpToolResult.Error("Reload failed: no reloadable scene was found at the current parcel."); + } + catch (TimeoutException) + { + return McpToolResult.Error($"Scene reload did not complete within {timeoutSec}s. Check get_scene_state and get_scene_logs."); + } + finally + { + await UniTask.SwitchToMainThread(); + world.Remove(playerEntity); + world.Remove(skyboxEntity); + } + + return McpToolResult.Text("Scene reloaded. Call get_scene_state to confirm readiness and get_scene_logs for startup output."); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs.meta new file mode 100644 index 00000000000..218fe29e9eb --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 1ca31906893da454f85da8932ebb45c9 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs new file mode 100644 index 00000000000..ee3a6ec33dd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs @@ -0,0 +1,255 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System; +using System.Collections; +using System.Threading; +using Unity.Collections; +using UnityEngine; +using UnityEngine.Experimental.Rendering; +using UnityEngine.Rendering; +using Utility; +using Utility.Multithreading; +using Object = UnityEngine.Object; + +namespace DCL.McpServer.Tools +{ + /// + /// Captures the current frame (back buffer including UI, or a world-only camera render), + /// downscales it and returns it as an MCP image content block. Textures never accumulate: + /// temporaries are released per call and the ReadPixels fallback reuses one persistent buffer. + /// + public class ScreenshotTool : IMcpTool, IDisposable + { + private const int DEFAULT_MAX_WIDTH = 1280; + private const int MIN_WIDTH = 64; + private const int MAX_WIDTH = 1920; + private const int JPG_QUALITY = 75; + + private readonly ICoroutineRunner coroutineRunner; + private readonly World world; + private readonly Entity playerEntity; + + // Reused across calls by the ReadPixels fallback so repeated captures don't allocate new textures. + private Texture2D? readPixelsBuffer; + + // 1 while a capture is running; concurrent requests are rejected so one set of buffers suffices. + private int captureGate; + + public string Name => "screenshot"; + + public string Description => + "Capture a screenshot of what the player currently sees in the Explorer, including scene UI. " + + "Use worldOnly to exclude all UI overlays. Returns a downscaled image plus a caption with the capture context."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Integer("maxWidth", "Maximum output width in pixels (aspect ratio preserved). Default 1280.") + .String("quality", "Output encoding. Default jpg.", enumValues: new[] { "jpg", "png" }) + .Boolean("worldOnly", "Render only the 3D world through the main camera, excluding UI. Default false.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public ScreenshotTool(ICoroutineRunner coroutineRunner, World world, Entity playerEntity) + { + this.coroutineRunner = coroutineRunner; + this.world = world; + this.playerEntity = playerEntity; + } + + public void Dispose() + { + if (readPixelsBuffer != null) + Object.Destroy(readPixelsBuffer); + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + int maxWidth = Mathf.Clamp(arguments.GetInt("maxWidth", DEFAULT_MAX_WIDTH), MIN_WIDTH, MAX_WIDTH); + bool asPng = arguments.GetString("quality", "jpg") == "png"; + bool worldOnly = arguments.GetBool("worldOnly", false); + + if (Interlocked.CompareExchange(ref captureGate, 1, 0) != 0) + return McpToolResult.Error("Another screenshot capture is already in progress; retry when it completes."); + + try { return await CaptureAsync(maxWidth, asPng, worldOnly, ct); } + finally { Interlocked.Exchange(ref captureGate, 0); } + } + + private async UniTask CaptureAsync(int maxWidth, bool asPng, bool worldOnly, CancellationToken ct) + { + Texture2D? backbufferCopy = null; + RenderTexture? worldRender = null; + RenderTexture? downscaled = null; + + try + { + int sourceWidth; + int sourceHeight; + + if (worldOnly) + { + Camera camera = world.CacheCamera().GetCameraComponent(world).Camera; + + // URP replaces the camera's internal color buffer with the target texture's descriptor + // (CreateRenderTextureDescriptor), so an LDR target silently downgrades the whole render + // to 8-bit and clamps emissives to 1.0, starving bloom and other HDR-dependent post effects. + // An HDR temporary keeps the pipeline HDR end-to-end; the downscale blit into the sRGB + // descriptor below performs the linear-to-sRGB conversion. + worldRender = RenderTexture.GetTemporary(camera.pixelWidth, camera.pixelHeight, 24, RenderTextureFormat.DefaultHDR); + + // Camera.Render() is unsupported under URP: redirect the camera's output into the + // render texture for exactly one frame instead, then restore it. + RenderTexture? previousTarget = camera.targetTexture; + camera.targetTexture = worldRender; + + try { await WaitForEndOfFrameAsync(ct); } + finally + { + await UniTask.SwitchToMainThread(); + camera.targetTexture = previousTarget; + } + + sourceWidth = worldRender.width; + sourceHeight = worldRender.height; + } + else + { + backbufferCopy = await CaptureBackbufferAsync(ct); + + if (backbufferCopy == null) + return McpToolResult.Error("Back buffer capture failed."); + + sourceWidth = backbufferCopy.width; + sourceHeight = backbufferCopy.height; + } + + int width = Mathf.Min(maxWidth, sourceWidth); + int height = Mathf.Max(1, Mathf.RoundToInt((float)sourceHeight * width / sourceWidth)); + + var descriptor = new RenderTextureDescriptor(width, height) + { + graphicsFormat = OutputGraphicsFormat(), sRGB = true, msaaSamples = 1, depthBufferBits = 0, + mipCount = 1, useMipMap = false, + }; + + downscaled = RenderTexture.GetTemporary(descriptor); + + if (backbufferCopy != null) + { + Graphics.Blit(backbufferCopy, downscaled); + + // The back buffer copy is a fresh Texture2D allocated by Unity on every capture: destroy it as soon as it's blitted. + Object.Destroy(backbufferCopy); + backbufferCopy = null; + } + else + { + Graphics.Blit(worldRender, downscaled); + RenderTexture.ReleaseTemporary(worldRender); + worldRender = null; + } + + byte[] encoded; + string mimeType; + + AsyncGPUReadbackRequest readback = await AsyncGPUReadback.Request(downscaled).WithCancellation(ct); + + if (readback.hasError) { (encoded, mimeType) = EncodeViaReadPixels(downscaled, asPng); } + else + { + NativeArray rawPixels = readback.GetData(); + + using (NativeArray encodedNative = asPng + ? ImageConversion.EncodeNativeArrayToPNG(rawPixels, downscaled.graphicsFormat, (uint)width, (uint)height) + : ImageConversion.EncodeNativeArrayToJPG(rawPixels, downscaled.graphicsFormat, (uint)width, (uint)height, 0, JPG_QUALITY)) + encoded = encodedNative.ToArray(); + + mimeType = asPng ? "image/png" : "image/jpeg"; + } + + Vector2Int parcel = world.Get(playerEntity).Position.ToParcel(); + var caption = $"{width}x{height} {(worldOnly ? "world-only" : "full-view")} capture at parcel ({parcel.x},{parcel.y})"; + + // Base64 conversion of the encoded image happens off the main thread. + await DCLTask.SwitchToThreadPool(); + return McpToolResult.Image(encoded, mimeType, caption); + } + finally + { + if (backbufferCopy != null || worldRender != null || downscaled != null) + { + await UniTask.SwitchToMainThread(); + + if (backbufferCopy != null) Object.Destroy(backbufferCopy); + if (worldRender != null) RenderTexture.ReleaseTemporary(worldRender); + if (downscaled != null) RenderTexture.ReleaseTemporary(downscaled); + } + } + } + + private UniTask CaptureBackbufferAsync(CancellationToken ct) + { + var completion = new UniTaskCompletionSource(); + coroutineRunner.StartCoroutine(CaptureBackbufferCoroutine(completion)); + return completion.Task.AttachExternalCancellation(ct); + } + + private UniTask WaitForEndOfFrameAsync(CancellationToken ct) + { + var completion = new UniTaskCompletionSource(); + coroutineRunner.StartCoroutine(SignalEndOfFrameCoroutine(completion)); + return completion.Task.AttachExternalCancellation(ct); + } + + private static IEnumerator SignalEndOfFrameCoroutine(UniTaskCompletionSource completion) + { + yield return GameObjectExtensions.WAIT_FOR_END_OF_FRAME; + completion.TrySetResult(); + } + + private static IEnumerator CaptureBackbufferCoroutine(UniTaskCompletionSource completion) + { + // The back buffer is only complete (UI included) at the very end of the frame. + yield return GameObjectExtensions.WAIT_FOR_END_OF_FRAME; + + Texture2D texture = ScreenCapture.CaptureScreenshotAsTexture(); + + if (!completion.TrySetResult(texture)) + Object.Destroy(texture); + } + + private (byte[] bytes, string mimeType) EncodeViaReadPixels(RenderTexture source, bool asPng) + { + if (readPixelsBuffer == null) + readPixelsBuffer = new Texture2D(source.width, source.height, TextureFormat.RGBA32, false); + else if (readPixelsBuffer.width != source.width || readPixelsBuffer.height != source.height) + readPixelsBuffer.Reinitialize(source.width, source.height, readPixelsBuffer.graphicsFormat, false); + + RenderTexture? previousActive = RenderTexture.active; + RenderTexture.active = source; + readPixelsBuffer.ReadPixels(new Rect(0, 0, source.width, source.height), 0, 0); + readPixelsBuffer.Apply(); + RenderTexture.active = previousActive; + + return asPng + ? (readPixelsBuffer.EncodeToPNG(), "image/png") + : (readPixelsBuffer.EncodeToJPG(JPG_QUALITY), "image/jpeg"); + } + + private static GraphicsFormat OutputGraphicsFormat() + { + var preferred = GraphicsFormat.R8G8B8A8_SRGB; + + if (SystemInfo.IsFormatSupported(preferred, GraphicsFormatUsage.Render)) + return preferred; + + return SystemInfo.GetCompatibleFormat(preferred, GraphicsFormatUsage.Render); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs.meta new file mode 100644 index 00000000000..7c72a9a5cc9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 781cfdc63c1c5405aa87fa01a82e4a4f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs b/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs new file mode 100644 index 00000000000..f130a7a48f9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs @@ -0,0 +1,54 @@ +using Cysharp.Threading.Tasks; +using DCL.Chat.History; +using DCL.Chat.MessageBus; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + /// + /// Sends a message to the Nearby chat channel. Messages starting with '/' run through the chat + /// command pipeline, so agents can drive commands the same way a user typing in chat would. + /// + public class SendChatTool : IMcpTool + { + private const int MAX_MESSAGE_LENGTH = 500; + + private readonly IChatMessagesBus chatMessagesBus; + + public string Name => "send_chat"; + + public string Description => + "Send a message to the Nearby chat channel. Messages starting with '/' run chat commands " + + "(e.g. /goto x,y, /reload, /help); command output appears in chat and scene logs."; + + public JObject InputSchema => + McpJsonSchema.Object() + .String("message", "The chat message or /command to send.", required: true) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: false); + + public SendChatTool(IChatMessagesBus chatMessagesBus) + { + this.chatMessagesBus = chatMessagesBus; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + string message = arguments.GetString("message", string.Empty); + + if (string.IsNullOrWhiteSpace(message)) + return UniTask.FromResult(McpToolResult.Error("message is required.")); + + if (message.Length > MAX_MESSAGE_LENGTH) + return UniTask.FromResult(McpToolResult.Error($"message exceeds the {MAX_MESSAGE_LENGTH} character limit.")); + + chatMessagesBus.SendWithUtcNowTimestamp(ChatChannel.NEARBY_CHANNEL, message, ChatMessageOrigin.CHAT); + + return UniTask.FromResult(McpToolResult.Text($"Sent to Nearby: {message}")); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs.meta new file mode 100644 index 00000000000..7da0103ce84 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 9e754e2afb7734afb8f6b5e0f735b168 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs b/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs new file mode 100644 index 00000000000..7bc2d75741b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs @@ -0,0 +1,107 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.CharacterCamera; +using DCL.InWorldCamera; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using ECS.Abstract; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + /// + /// Switches the camera mode by writing (the same pattern scene systems use; + /// ControlCinemachineVirtualCameraSystem applies it next frame). The direct write bypasses the user-input + /// gates, so this tool re-checks them itself and refuses when a scene holds the camera. + /// + public class SetCameraModeTool : IMcpTool + { + private readonly World world; + private readonly ExposedCameraData exposedCameraData; + + public string Name => "set_camera_mode"; + + public string Description => + "Switch the player camera mode (first_person, third_person, drone, or the free-fly camera), like a user pressing the camera key. " + + "Refuses with an explanation when the scene locks the mode (CameraModeArea, scene virtual camera, photo camera). " + + "Any player movement drops free back to third_person."; + + public JObject InputSchema => + McpJsonSchema.Object() + .String("mode", "Target camera mode.", enumValues: new[] { "first_person", "third_person", "drone", "free" }, required: true) + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public SetCameraModeTool(World world, ExposedCameraData exposedCameraData) + { + this.world = world; + this.exposedCameraData = exposedCameraData; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + CameraMode targetMode; + + switch (arguments.GetString("mode", string.Empty)) + { + case "first_person": targetMode = CameraMode.FirstPerson; break; + case "third_person": targetMode = CameraMode.ThirdPerson; break; + case "drone": targetMode = CameraMode.DroneView; break; + case "free": targetMode = CameraMode.Free; break; + default: return McpToolResult.Error("mode must be one of: first_person, third_person, drone, free."); + } + + string? blockReason = TrySwitchMode(world, targetMode, out CameraMode previousMode); + + if (blockReason != null) + return McpToolResult.Error(blockReason); + + // Let ControlCinemachineVirtualCameraSystem activate the matching virtual camera before reading back. + await UniTask.DelayFrame(2, cancellationToken: ct); + + var result = new JObject + { + ["requestedMode"] = targetMode.ToString(), + ["currentMode"] = exposedCameraData.CameraMode.ToString(), + ["previousMode"] = previousMode.ToString(), + }; + + return McpToolResult.Json(result); + } + + /// + /// Same gates ControlCinemachineVirtualCameraSystem.HandleCameraInput applies to user input. + /// Main thread only. + /// + internal static bool IsModeChangeAllowed(World world) + { + SingleInstanceEntity cameraEntity = world.CacheCamera(); + ref readonly CameraComponent camera = ref cameraEntity.GetCameraComponent(world); + + return camera.Mode != CameraMode.SDKCamera + && camera.CameraInputChangeEnabled + && !world.Has(cameraEntity); + } + + internal static string? TrySwitchMode(World world, CameraMode targetMode, out CameraMode previousMode) + { + SingleInstanceEntity cameraEntity = world.CacheCamera(); + ref CameraComponent camera = ref cameraEntity.GetCameraComponent(world); + previousMode = camera.Mode; + + if (camera.Mode == CameraMode.SDKCamera) + return "A scene virtual camera controls the view right now (mode SDKCamera); the mode cannot change until the scene releases it."; + + if (!camera.CameraInputChangeEnabled) + return $"Camera mode is locked by the scene (CameraModeArea; current mode: {camera.Mode}). Leave the area to change modes."; + + if (world.Has(cameraEntity)) + return "The in-world photo camera is active; close it before changing the camera mode."; + + camera.Mode = targetMode; + return null; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs.meta new file mode 100644 index 00000000000..f5b3533b96f --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 297288bb84c8d4cd1af0af018a8a4123 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs b/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs new file mode 100644 index 00000000000..47ad5d72640 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs @@ -0,0 +1,139 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.CharacterCamera.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using ECS.Abstract; +using Newtonsoft.Json.Linq; +using System.Threading; +using UnityEngine; +using Utility.Arch; + +namespace DCL.McpServer.Tools +{ + /// + /// Places the free camera at an absolute world position, optionally aiming it and setting its FOV. + /// Enters Free mode when needed (same scene-lock gates as ) and waits + /// for the Cinemachine blend to reach the target before reporting the actual pose. + /// + public class SetCameraPoseTool : IMcpTool + { + private const float DEFAULT_TIMEOUT_SEC = 5f; + private const float MIN_TIMEOUT_SEC = 0.5f; + private const float MAX_TIMEOUT_SEC = 15f; + private const float SETTLE_EPSILON = 0.1f; + private const int POLL_INTERVAL_MS = 100; + private const float MIN_FOV = 10f; + private const float MAX_FOV = 120f; + + private readonly World world; + private readonly Entity playerEntity; + private readonly ExposedCameraData exposedCameraData; + + public string Name => "set_camera_pose"; + + public string Description => + "Place the free camera at an absolute world position, optionally aiming it at a point and setting its field of view. " + + "Enters the free camera mode if needed (refuses with the reason when the scene locks the camera). The camera stays " + + "put while the player moves; restore a player-following view with set_camera_mode third_person."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Number("x", "Camera world position.", required: true) + .Number("y", required: true) + .Number("z", required: true) + .Number("lookAtX", "Optional world point to aim at (all three lookAt components required together).") + .Number("lookAtY") + .Number("lookAtZ") + .Number("fov", "Optional vertical field of view in degrees (10-120).") + .Number("timeoutSec", "Seconds to wait for the camera to settle at the target. Default 5, max 15.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public SetCameraPoseTool(World world, Entity playerEntity, ExposedCameraData exposedCameraData) + { + this.world = world; + this.playerEntity = playerEntity; + this.exposedCameraData = exposedCameraData; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetFloat("x", out float x) || !arguments.TryGetFloat("y", out float y) || !arguments.TryGetFloat("z", out float z)) + return McpToolResult.Error("x, y and z world coordinates for the camera position are required."); + + bool hasLookAtX = arguments.TryGetFloat("lookAtX", out float lookAtX); + bool hasLookAtY = arguments.TryGetFloat("lookAtY", out float lookAtY); + bool hasLookAtZ = arguments.TryGetFloat("lookAtZ", out float lookAtZ); + bool hasLookAt = hasLookAtX && hasLookAtY && hasLookAtZ; + + if ((hasLookAtX || hasLookAtY || hasLookAtZ) && !hasLookAt) + return McpToolResult.Error("lookAtX, lookAtY and lookAtZ must be provided together."); + + float? fov = null; + + if (arguments.TryGetFloat("fov", out float fovValue)) + fov = Mathf.Clamp(fovValue, MIN_FOV, MAX_FOV); + + float timeoutSec = Mathf.Clamp(arguments.GetFloat("timeoutSec", DEFAULT_TIMEOUT_SEC), MIN_TIMEOUT_SEC, MAX_TIMEOUT_SEC); + var targetPosition = new Vector3(x, y, z); + + SingleInstanceEntity cameraEntity = world.CacheCamera(); + + if (cameraEntity.GetCameraComponent(world).Mode != CameraMode.Free) + { + string? blockReason = SetCameraModeTool.TrySwitchMode(world, CameraMode.Free, out CameraMode _); + + if (blockReason != null) + return McpToolResult.Error(blockReason); + + // Let ControlCinemachineVirtualCameraSystem activate the free vcam (and apply its default + // spawn position, which the pose below overrides). + await UniTask.DelayFrame(2, cancellationToken: ct); + } + + if (!world.TryGet(cameraEntity, out ICinemachinePreset? cinemachinePreset) || cinemachinePreset == null) + return McpToolResult.Error("The camera rig is not initialized yet."); + + cinemachinePreset.ForceFreeCameraPose(targetPosition, fov); + + if (hasLookAt) + { + Vector3 playerPosition = world.Get(playerEntity).Position; + world.AddOrSet(cameraEntity, new CameraLookAtIntent(new Vector3(lookAtX, lookAtY, lookAtZ), playerPosition)); + } + + // Entering Free blends the output camera toward the vcam over a couple of seconds; when the + // mode was already Free the pose applies instantly and the first poll succeeds. + var settled = false; + float deadline = UnityEngine.Time.realtimeSinceStartup + timeoutSec; + + while (UnityEngine.Time.realtimeSinceStartup < deadline) + { + if (Vector3.Distance(exposedCameraData.WorldPosition.Value, targetPosition) <= SETTLE_EPSILON) + { + settled = true; + break; + } + + await UniTask.Delay(POLL_INTERVAL_MS, cancellationToken: ct); + } + + // Let the look-at intent apply before reading the rotation back. + await UniTask.DelayFrame(2, cancellationToken: ct); + + var result = new JObject + { + ["position"] = exposedCameraData.WorldPosition.Value.ToVector(), + ["rotationEuler"] = exposedCameraData.WorldRotation.Value.eulerAngles.ToVector(), + ["mode"] = exposedCameraData.CameraMode.ToString(), + ["settled"] = settled, + }; + + return McpToolResult.Json(result); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs.meta new file mode 100644 index 00000000000..5ad7c5b677f --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 5d66df6294124c629fa19535681000e1 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs b/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs new file mode 100644 index 00000000000..14fdea46980 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs @@ -0,0 +1,93 @@ +using Cysharp.Threading.Tasks; +using DCL.Chat.Commands; +using DCL.Chat.History; +using DCL.Chat.MessageBus; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.RealmNavigation; +using ECS.SceneLifeCycle; +using Newtonsoft.Json.Linq; +using SceneRunner.Scene; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + /// + /// Teleports through the same /goto command pipeline a user teleport takes (loading screen included), + /// then polls until the destination scene is ready or the timeout elapses. + /// + public class TeleportTool : IMcpTool + { + private const int POLL_INTERVAL_MS = 500; + private const float MIN_TIMEOUT_SEC = 5f; + private const float MAX_TIMEOUT_SEC = 300f; + private const float DEFAULT_TIMEOUT_SEC = 60f; + + private readonly IChatMessagesBus chatMessagesBus; + private readonly IScenesCache scenesCache; + private readonly ILoadingStatus loadingStatus; + + public string Name => "teleport"; + + public string Description => + "Teleport the player to a parcel (x,y) through the regular /goto flow and wait until the destination scene is ready. " + + "Reports the final scene state; follow up with get_scene_state for details."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Integer("x", "Target parcel X coordinate.", required: true) + .Integer("y", "Target parcel Y coordinate.", required: true) + .Boolean("waitForReady", "Wait until the destination scene is ready. Default true.") + .Number("timeoutSec", "Maximum seconds to wait for readiness. Default 60.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public TeleportTool(IChatMessagesBus chatMessagesBus, IScenesCache scenesCache, ILoadingStatus loadingStatus) + { + this.chatMessagesBus = chatMessagesBus; + this.scenesCache = scenesCache; + this.loadingStatus = loadingStatus; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetInt("x", out int x) || !arguments.TryGetInt("y", out int y)) + return McpToolResult.Error("Both x and y parcel coordinates are required."); + + bool waitForReady = arguments.GetBool("waitForReady", true); + float timeoutSec = Mathf.Clamp(arguments.GetFloat("timeoutSec", DEFAULT_TIMEOUT_SEC), MIN_TIMEOUT_SEC, MAX_TIMEOUT_SEC); + + chatMessagesBus.SendWithUtcNowTimestamp(ChatChannel.NEARBY_CHANNEL, $"/{ChatCommandsUtils.COMMAND_GOTO} {x},{y}", ChatMessageOrigin.RESTRICTED_ACTION_API); + + if (!waitForReady) + return McpToolResult.Text($"Teleport to ({x},{y}) requested."); + + var targetParcel = new Vector2Int(x, y); + float deadline = UnityEngine.Time.realtimeSinceStartup + timeoutSec; + + while (UnityEngine.Time.realtimeSinceStartup < deadline) + { + await UniTask.Delay(POLL_INTERVAL_MS, cancellationToken: ct); + + ISceneFacade? currentScene = scenesCache.CurrentScene.Value; + bool arrived = scenesCache.CurrentParcel.Value == targetParcel || (currentScene?.Contains(targetParcel) ?? false); + + if (!arrived || loadingStatus.IsLoadingScreenOn()) + continue; + + if (currentScene == null) + return McpToolResult.Text($"Arrived at ({x},{y}); no scene is deployed at this parcel."); + + if (currentScene.SceneStateProvider.IsNotRunningState()) + return McpToolResult.Error($"Arrived at ({x},{y}) but scene '{currentScene.Info.Name}' is not running: {currentScene.SceneStateProvider.State.Value()}. Check get_scene_logs."); + + if (currentScene.IsSceneReady()) + return McpToolResult.Text($"Teleported to ({x},{y}). Scene '{currentScene.Info.Name}' is ready."); + } + + return McpToolResult.Error($"Teleport to ({x},{y}) did not reach a ready scene within {timeoutSec}s. Check get_scene_state."); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs.meta new file mode 100644 index 00000000000..094575a2b3d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 1f263b4ea6e6e489a83b9b5accd70c06 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs b/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs new file mode 100644 index 00000000000..c89e8d47d9b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs @@ -0,0 +1,58 @@ +using CrdtEcsBridge.RestrictedActions; +using Cysharp.Threading.Tasks; +using DCL.ECSComponents; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + /// + /// Plays or stops an avatar emote through , the same intent path + /// a scene uses, so agents can verify emote-driven behaviour without touching the avatar directly. + /// + public class TriggerEmoteTool : IMcpTool + { + private readonly IGlobalWorldActions globalWorldActions; + + public string Name => "trigger_emote"; + + public string Description => + "Play an avatar emote by URN (e.g. a base emote like 'wave', 'dance', 'clap'), or stop the current one with stop: true."; + + public JObject InputSchema => + McpJsonSchema.Object() + .String("urn", "Emote URN or base emote id (wave, dance, clap...).") + .Boolean("loop", "Loop the emote until stopped. Default false.") + .Boolean("stop", "Stop the currently playing emote instead of triggering one.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: false); + + public TriggerEmoteTool(IGlobalWorldActions globalWorldActions) + { + this.globalWorldActions = globalWorldActions; + } + + public UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + bool stop = arguments.GetBool("stop", false); + string urn = arguments.GetString("urn", string.Empty); + + if (!stop && string.IsNullOrEmpty(urn)) + return UniTask.FromResult(McpToolResult.Error("urn is required (or pass stop: true).")); + + if (stop) + { + globalWorldActions.StopEmote(); + return UniTask.FromResult(McpToolResult.Text("Emote stopped.")); + } + + bool loop = arguments.GetBool("loop", false); + globalWorldActions.TriggerEmote(urn, loop, AvatarEmoteMask.AemFullBody); + + return UniTask.FromResult(McpToolResult.Text($"Emote '{urn}' triggered{(loop ? " (looping)" : string.Empty)}.")); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs.meta new file mode 100644 index 00000000000..f77e3e314b1 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 538f89c5fefa643f3936739de46a83e1 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs b/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs new file mode 100644 index 00000000000..21579c598e7 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs @@ -0,0 +1,118 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterMotion.Components; +using DCL.McpServer.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; +using UnityEngine; +using Utility; +using Utility.Arch; + +namespace DCL.McpServer.Tools +{ + /// + /// Holds a movement input on the player for a duration via , + /// exercising the regular locomotion pipeline (velocity, collisions, jumps) instead of teleporting. + /// + public class WalkTool : IMcpTool + { + private const float MIN_SECONDS = 0.1f; + private const float MAX_SECONDS = 30f; + private const float COMPLETION_GRACE_SEC = 5f; + + private readonly World world; + private readonly Entity playerEntity; + + public string Name => "walk"; + + public string Description => + "Walk/jog/run the player in a camera-relative direction for a number of seconds through the real locomotion pipeline " + + "(collisions apply). directionY is forward, directionX is strafe right. Returns the start and end positions."; + + public JObject InputSchema => + McpJsonSchema.Object() + .Number("directionX", "Strafe axis: 1 right, -1 left.", required: true) + .Number("directionY", "Forward axis: 1 forward, -1 backward.", required: true) + .Number("seconds", "How long to hold the movement. Default 1, max 30.") + .String("kind", "Movement speed. Default jog.", enumValues: new[] { "walk", "jog", "run" }) + .Boolean("jump", "Jump once at the start of the movement. Default false.") + .Build(); + + public McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: false); + + public WalkTool(World world, Entity playerEntity) + { + this.world = world; + this.playerEntity = playerEntity; + } + + public async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + var direction = new Vector2(arguments.GetFloat("directionX", 0f), arguments.GetFloat("directionY", 0f)); + + if (direction == Vector2.zero) + return McpToolResult.Error("directionX and directionY must not both be zero."); + + direction.Normalize(); + + float seconds = Mathf.Clamp(arguments.GetFloat("seconds", 1f), MIN_SECONDS, MAX_SECONDS); + bool jump = arguments.GetBool("jump", false); + + MovementKind kind = arguments.GetString("kind", "jog") switch + { + "walk" => MovementKind.WALK, + "run" => MovementKind.RUN, + _ => MovementKind.JOG, + }; + + // A newer walk preempts a pending one; release its awaiter before replacing the override. + if (world.TryGet(playerEntity, out McpMovementOverride existingOverride)) + existingOverride.Completion?.TrySetResult(); + + Vector3 startPosition = world.Get(playerEntity).Position; + var completion = new UniTaskCompletionSource(); + + world.AddOrSet(playerEntity, new McpMovementOverride + { + Axes = direction, + Kind = kind, + EndTime = UnityEngine.Time.time + seconds, + JumpRequested = jump, + Completion = completion, + }); + + try + { + await completion.Task.AttachExternalCancellation(ct) + .Timeout(TimeSpan.FromSeconds(seconds + COMPLETION_GRACE_SEC)); + } + catch (TimeoutException) + { + await UniTask.SwitchToMainThread(); + + if (world.Has(playerEntity)) + world.Remove(playerEntity); + + return McpToolResult.Error($"walk did not complete within {seconds + COMPLETION_GRACE_SEC}s (is the simulation paused?)."); + } + + await UniTask.SwitchToMainThread(ct); + + Vector3 endPosition = world.Get(playerEntity).Position; + + var result = new JObject + { + ["startPosition"] = startPosition.ToVector(), + ["endPosition"] = endPosition.ToVector(), + ["distance"] = Math.Round(Vector3.Distance(startPosition, endPosition), 2), + ["parcel"] = endPosition.ToParcel().ToParcel(), + }; + + return McpToolResult.Json(result); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs.meta new file mode 100644 index 00000000000..797981b32b9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: c981a464b2fe440749be02bc2b01d6ff \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Utils.meta b/Explorer/Assets/DCL/McpServer/Utils.meta new file mode 100644 index 00000000000..41ef52ed9b8 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils.meta @@ -0,0 +1,3 @@ +fileFormatVersion: 2 +guid: c2c6933e4faa4a0c9bdb0f363315b0eb +timeCreated: 1784277704 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs b/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs new file mode 100644 index 00000000000..268d5a87316 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs @@ -0,0 +1,83 @@ +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using System; +using UnityEngine; + +namespace DCL.McpServer.Utils +{ + /// + /// Builders for the JSON fragments shared by tool outputs. + /// + public static class JObjectExtensions + { + public static JObject ToVector(this Vector3 value) => + new () + { + ["x"] = Math.Round(value.x, 2), + ["y"] = Math.Round(value.y, 2), + ["z"] = Math.Round(value.z, 2), + }; + + public static JObject ToParcel(this Vector2Int value) => + new () + { + ["x"] = value.x, + ["y"] = value.y, + }; + + /// Output-schema counterpart of — an { x, y, z } object of numbers. + public static McpJsonSchema VectorSchema() => + McpJsonSchema.Object() + .Number("x") + .Number("y") + .Number("z"); + + /// Output-schema counterpart of — an { x, y } object of integers. + public static McpJsonSchema ParcelSchema() => + McpJsonSchema.Object() + .Integer("x") + .Integer("y"); + + public static bool GetBool(this JObject arguments, string name, bool defaultValue) => + arguments[name]?.Type == JTokenType.Boolean ? arguments[name]!.Value() : defaultValue; + + public static int GetInt(this JObject arguments, string name, int defaultValue) => + arguments[name].IsNumber() ? arguments[name]!.Value() : defaultValue; + + public static long GetLong(this JObject arguments, string name, long defaultValue) => + arguments[name].IsNumber() ? arguments[name]!.Value() : defaultValue; + + public static float GetFloat(this JObject arguments, string name, float defaultValue) => + arguments[name].IsNumber() ? arguments[name]!.Value() : defaultValue; + + public static string GetString(this JObject arguments, string name, string defaultValue) => + arguments[name]?.Type == JTokenType.String ? arguments[name]!.Value()! : defaultValue; + + public static bool TryGetFloat(this JObject arguments, string name, out float value) + { + if (arguments[name].IsNumber()) + { + value = arguments[name]!.Value(); + return true; + } + + value = 0f; + return false; + } + + public static bool TryGetInt(this JObject arguments, string name, out int value) + { + if (arguments[name].IsNumber()) + { + value = arguments[name]!.Value(); + return true; + } + + value = 0; + return false; + } + + private static bool IsNumber(this JToken? token) => + token?.Type is JTokenType.Integer or JTokenType.Float; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs.meta b/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs.meta new file mode 100644 index 00000000000..1a87815e6b8 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 139ddf8ed9933484c92df485a5843199 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs b/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs new file mode 100644 index 00000000000..54d315b8a96 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs @@ -0,0 +1,82 @@ +using DCL.UI.DebugMenu.LogHistory; +using System.Collections.Generic; + +namespace DCL.McpServer.Utils +{ + /// + /// Thread-safe ring buffer of scene console log entries with monotonic sequence numbers, + /// so agents can poll incrementally via sinceSeq. + /// + public class SceneLogBuffer + { + private const int CAPACITY = 1000; + + private readonly object gate = new (); + private readonly Entry[] entries = new Entry[CAPACITY]; + + private long nextSeq; + + /// + /// Sequence number of the newest stored entry, or -1 when empty. + /// + public long LatestSeq + { + get + { + lock (gate) { return nextSeq - 1; } + } + } + + /// + /// Fed from ; may be invoked from any thread. + /// + public void Append(DebugMenuConsoleLogEntry logEntry) + { + lock (gate) + { + entries[nextSeq % CAPACITY] = new Entry(nextSeq, logEntry.Type, logEntry.Message); + nextSeq++; + } + } + + /// + /// Copies up to newest entries with Seq greater than + /// into in chronological order. + /// + public void CopyTo(List target, long sinceSeq, bool errorsOnly, int limit) + { + lock (gate) + { + long oldestAvailable = nextSeq >= CAPACITY ? nextSeq - CAPACITY : 0; + long from = sinceSeq + 1 > oldestAvailable ? sinceSeq + 1 : oldestAvailable; + + for (long seq = from; seq < nextSeq; seq++) + { + Entry entry = entries[seq % CAPACITY]; + + if (errorsOnly && entry.Type != LogMessageType.Error) + continue; + + target.Add(entry); + } + + if (target.Count > limit) + target.RemoveRange(0, target.Count - limit); + } + } + + public readonly struct Entry + { + public readonly long Seq; + public readonly LogMessageType Type; + public readonly string Message; + + public Entry(long seq, LogMessageType type, string message) + { + Seq = seq; + Type = type; + Message = message; + } + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs.meta b/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs.meta new file mode 100644 index 00000000000..4bf93032e48 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 91d8bd24c95a3413c800ab47ed79d7eb \ No newline at end of file diff --git a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/Container/DiagnosticsContainer.cs b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/Container/DiagnosticsContainer.cs index 017891b0591..2e02c77d114 100644 --- a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/Container/DiagnosticsContainer.cs +++ b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/Container/DiagnosticsContainer.cs @@ -106,9 +106,11 @@ private static SceneDebugConsoleReportHandler AddDebugConsoleReportHandler(Debug ReportCategory.SCENE_FETCH_REQUEST, ReportCategory.PORTABLE_EXPERIENCE, ReportCategory.EMOTE, + ReportCategory.MCP, }, logType: false); entries.Add(new CategorySeverityMatrix.Entry { Category = ReportCategory.JAVASCRIPT, Severity = LogType.Log }); + entries.Add(new CategorySeverityMatrix.Entry { Category = ReportCategory.MCP, Severity = LogType.Log }); jsOnlyMatrix.entries = entries; return new SceneDebugConsoleReportHandler(jsOnlyMatrix, sceneDebugConsoleMessageBus, false); diff --git a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportCategory.cs b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportCategory.cs index b79da65000f..0ef5de51f35 100644 --- a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportCategory.cs +++ b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportCategory.cs @@ -360,6 +360,11 @@ public static class ReportCategory public const string CHROME_DEVTOOL_PROTOCOL = nameof(CHROME_DEVTOOL_PROTOCOL); + /// + /// Embedded MCP (Model Context Protocol) automation server. + /// + public const string MCP = nameof(MCP); + public const string SCENE_PERMISSIONS = nameof(SCENE_PERMISSIONS); public const string MVC_STATE_MACHINE = nameof(MVC_STATE_MACHINE); diff --git a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsDevelopment.asset b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsDevelopment.asset index c69c1771a1c..3feccb406d8 100644 --- a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsDevelopment.asset +++ b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsDevelopment.asset @@ -708,6 +708,16 @@ MonoBehaviour: Severity: 2 - Category: MULTIPLAYER Severity: 2 + - Category: MCP + Severity: 3 + - Category: MCP + Severity: 0 + - Category: MCP + Severity: 4 + - Category: MCP + Severity: 1 + - Category: MCP + Severity: 2 sentryMatrix: entries: [] debounceEnabled: 1 diff --git a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsProduction.asset b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsProduction.asset index 57f1e67ae01..93e8412a007 100644 --- a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsProduction.asset +++ b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsProduction.asset @@ -448,6 +448,16 @@ MonoBehaviour: Severity: 0 - Category: MULTIPLAYER Severity: 2 + - Category: MCP + Severity: 4 + - Category: MCP + Severity: 1 + - Category: MCP + Severity: 3 + - Category: MCP + Severity: 2 + - Category: MCP + Severity: 0 - Category: AUTHENTICATION Severity: 3 - Category: AUTHENTICATION diff --git a/Explorer/Assets/DCL/PluginSystem/DCL.Plugins.asmdef b/Explorer/Assets/DCL/PluginSystem/DCL.Plugins.asmdef index c8338aaecc6..2032d9a6b31 100644 --- a/Explorer/Assets/DCL/PluginSystem/DCL.Plugins.asmdef +++ b/Explorer/Assets/DCL/PluginSystem/DCL.Plugins.asmdef @@ -70,10 +70,10 @@ "GUID:451516970bc990e418454aa78b72586e", "GUID:13c468c9ecfc44f249fb8fb69d3365ef", "GUID:a42927d1d4a3b4cda9b076a7adecb9cc", - "GUID:f8127c6ac263abf468221dcbccbde182", "GUID:63cde67b4a5d47449392dfc49fc0c3ed", "GUID:267f743eefeddd44982a713909628102", "GUID:0e1edbec885c4350bb2ef81c6a413df5", + "GUID:f254b2009e854709a0ef0022a7ca697c", "GUID:3c1a9be47d0f42e2a5b8c6d4e9f01a27" ], "includePlatforms": [], @@ -98,4 +98,4 @@ } ], "noEngineReferences": false -} +} \ No newline at end of file diff --git a/Explorer/Assets/DCL/Tests/Editor/DCL.EditMode.Tests.asmdef b/Explorer/Assets/DCL/Tests/Editor/DCL.EditMode.Tests.asmdef index f22886fcbd1..9b998b9b1c7 100644 --- a/Explorer/Assets/DCL/Tests/Editor/DCL.EditMode.Tests.asmdef +++ b/Explorer/Assets/DCL/Tests/Editor/DCL.EditMode.Tests.asmdef @@ -61,7 +61,8 @@ "GUID:3ca842002c51d0a43b73e45298809a13", "GUID:0df5180c0c3a0594fbfa11f83736de9f", "GUID:28964ef7dc9441b6b8671b61a8106690", - "GUID:267f743eefeddd44982a713909628102" + "GUID:267f743eefeddd44982a713909628102", + "GUID:f254b2009e854709a0ef0022a7ca697c" ], "includePlatforms": [ "Editor" diff --git a/docs/README.md b/docs/README.md index ff13159a5d7..c837410b3de 100644 --- a/docs/README.md +++ b/docs/README.md @@ -73,6 +73,7 @@ Welcome to the official documentation for Unity Explorer — the Decentraland cl - **[Debug Container & Widgets](debug-container-and-widgets.md)** — Runtime debug panel architecture, widget builder API, bindings, and integration patterns - **[Testing Guide](testing-guide.md)** — UnitySystemTestBase, ECS test utilities, mocking, EditMode/PlayMode, async test patterns - **[Automation Testing](automation-testing.md)** — AltTester SDK setup, writing UI automation tests, running against instrumented builds and in-Editor, triggering visual regression on PRs via `/visual-tests` +- **[MCP Automation](mcp-automation.md)** — Embedded MCP server for coding agents: screenshots, player/scene state, scene logs, and player control via `--mcp` - **[Connect to Local Scene](how-to-connect-to-a-local-scene.md)** — Running and connecting to local SDK7 scenes - **[Master of Bots](master-of-bots.md)** — Simulating multiple bot users for load testing - **[Override Debug Log Matrix](override-debug-log-matrix.md)** — Runtime log severity overrides diff --git a/docs/app-arguments.md b/docs/app-arguments.md index 139655e3de4..6f71ad28464 100644 --- a/docs/app-arguments.md +++ b/docs/app-arguments.md @@ -311,6 +311,27 @@ More detailed instructions on how to test can be found in the description of rel --- +### `mcp` +**Description:** Starts the embedded MCP (Model Context Protocol) server on `http://127.0.0.1:8123/unity-explorer-mcp` so coding agents can observe and drive the client (screenshots, player/scene state, scene logs, teleport/movement, chat commands). The listener binds to localhost only and rejects non-localhost browser Origins. See [MCP Automation](mcp-automation.md). + +**Usage:** +```bash +--mcp +``` + +--- + +### `mcp-port` +**Type:** String (integer port, 1024–65535) +**Description:** Starts the embedded MCP server on a specific port (implies `mcp`). Use distinct ports when running multiple instances via `--multi-instance`. + +**Usage:** +```bash +--mcp-port 8124 +``` + +--- + ### `launch-cdp-monitor-on-start` **Type:** Boolean **Description:** Launches the Chrome DevTools Protocol (CDP) monitor on application start. Enables remote debugging capabilities. diff --git a/docs/mcp-automation.md b/docs/mcp-automation.md new file mode 100644 index 00000000000..41a173a1d61 --- /dev/null +++ b/docs/mcp-automation.md @@ -0,0 +1,162 @@ +# MCP Automation Server + +The Explorer can host an embedded [MCP (Model Context Protocol)](https://modelcontextprotocol.io/) server so coding agents (e.g. Claude Code) can **see** the running client (screenshots, player/scene state, scene console logs) and **control** it (teleport, move, walk, look, chat commands, scene reload) — closing the edit → reload → verify loop for SDK7 scene development without a human in the middle. + +The server is compiled into all builds but stays dormant unless explicitly enabled at launch. + +--- + +## Enabling + +| Flag | Effect | +|---|---| +| `--mcp` | Starts the MCP server on the default port **8123** | +| `--mcp-port ` | Starts the MCP server on a specific port (implies `--mcp`) | + +The flag is accepted from the command line or a deep link. The endpoint is `http://127.0.0.1:/unity-explorer-mcp`. + +```bash +# macOS +open Decentraland.app --args --mcp + +# Windows +Decentraland.exe --mcp-port 8124 +``` + +In the Unity Editor, add `--mcp` to `Main Scene Loader → Debug Settings → App Parameters`. + +From a scene folder, `@dcl/sdk-commands` can enable it at launch: `npm run start -- --mcp` (optionally `--mcp-port `) forwards both flags into the deep link that auto-launches the installed client. Any extra Explorer params can follow a second standalone `--` (`npm run start -- --mcp -- --windowed-mode --resolution 1280x720`; npm consumes the first `--`). + +## Security model + +- The listener binds to **127.0.0.1 only** — it is never reachable from the network. +- Browser-originated requests are rejected unless their `Origin` is localhost (defense against drive-by pages and DNS rebinding). Requests without an `Origin` header (CLI clients) are allowed. +- The server only exists while the process runs with the flag; there is no persistence and no authentication token in v1. + +## Connecting a coding agent + +```bash +claude mcp add --transport http --scope user explorer http://127.0.0.1:8123/unity-explorer-mcp +``` + +Smoke test without an agent: + +```bash +curl -s -X POST http://127.0.0.1:8123/unity-explorer-mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"0"}}}' + +curl -s -X POST http://127.0.0.1:8123/unity-explorer-mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' +``` + +## Tool catalog + +### Seeing + +| Tool | Arguments | Returns | +|---|---|---| +| `screenshot` | `maxWidth?` (default 1280), `quality?` (`jpg`\|`png`), `worldOnly?` (exclude UI; post-processing still applied) | Downscaled image of the current view (UI included by default) + caption | +| `get_player_state` | — | Player position/rotation/parcel/velocity/grounded + camera position/rotation/mode + wallet address | +| `get_scene_state` | — | Current parcel, scene name/state (incl. `JavaScriptError`/`EcsError`), readiness, loading stage | +| `get_scene_logs` | `limit?`, `severity?` (`all`\|`error`), `sinceSeq?` | Scene JS console output with monotonic sequence numbers for incremental polling | +| `list_scene_entities` | `limit?` | Entity ids of the current scene's ECS world | +| `get_entity_details` | `entityId` | All components of one scene entity | + +### Controlling + +| Tool | Arguments | Effect | +|---|---|---| +| `teleport` | `x`, `y`, `waitForReady?`, `timeoutSec?` | `/goto x,y` through the regular pipeline, waits for scene readiness | +| `move_to` | `x`, `y`, `z`, `lookAt{X,Y,Z}?`, `durationSec?` | Instant or smooth move to a world position (16 m per parcel) | +| `walk` | `directionX`, `directionY`, `seconds?`, `kind?`, `jump?` | Holds camera-relative movement through the real locomotion pipeline (collisions apply) | +| `look_at` | `x`, `y`, `z` | Rotates the camera to a world point (aim before a screenshot) | +| `set_camera_mode` | `mode` (`first_person`\|`third_person`\|`drone`\|`free`) | Switches the camera mode like the user hotkey; refuses (with the reason) while a scene locks the camera — `CameraModeArea`, scene virtual camera, or photo camera. `get_player_state` → `camera.modeChangeAllowed` reports the lock state in advance | +| `set_camera_pose` | `x`,`y`,`z`, `lookAt{X,Y,Z}?`, `fov?`, `timeoutSec?` | Places the free camera at an absolute world position, optionally aiming it and setting FOV. Auto-enters free mode (same locks as `set_camera_mode`), waits for the blend to settle (`settled` in the result), and returns the actual pose. The camera stays put while the player moves; restore with `set_camera_mode` | +| `send_chat` | `message` | Sends to Nearby chat; `/commands` run through the chat command pipeline | +| `reload_scene` | `timeoutSec?` | Reloads the current scene (motion + skybox frozen during reload) | +| `trigger_emote` | `urn` or `stop: true`, `loop?` | Plays or stops an avatar emote | +| `click_entity` | `entityId` and/or `x`,`y`,`z` aim point, `button?` (`pointer`\|`primary`\|`secondary`), `eventType?` (`click`\|`down`\|`up`), `timeoutSec?` | Presses a pointer button on a scene entity exactly like a real click: a camera-origin raycast validates the aim (occluders and the entity's `maxDistance` apply), then the entity's pointer-event intent is filled so the scene receives an identical `PBPointerEventsResult`. `click` sends down + up on consecutive scene ticks. Returns `hit`, hover text, hit point/distance, or the blocking entity | + +## Structured output + +`get_player_state`, `get_scene_state` and `list_scene_entities` also return `structuredContent` mirroring their text payload and declare a matching `outputSchema` in `tools/list` (MCP 2025-06-18). This is done **only as an example on the read-only state tools that benefit from it now** — every other tool returns text content only. A tool opts in by overriding `IMcpTool.OutputSchema` (default `null`); the same `McpInputSchema` builder produces the schema. + + +## Streaming logs (SSE subscribe) + +The tools above are request/response. For logs you can also **subscribe** and get a live push +instead of polling `get_scene_logs`. Open the server-to-client SSE stream with a `GET` and pick one +of two independent streams: + +| Stream | `GET …?stream=` | Source | +|---|---|---| +| Scene | `scene` | The running SDK7 scene's JavaScript console (same source as `get_scene_logs`) | +| Client | `client` | The whole Unity player/editor log — engine, build and editor output | + +Each event is a JSON-RPC `notifications/message` (MCP logging shape): + +```json +{"jsonrpc":"2.0","method":"notifications/message","params":{"level":"error","logger":"scene","data":"…"}} +``` + +`level` is an RFC 5424 severity (`debug`,`info`,`notice`,`warning`,`error`,`critical`,`alert`,`emergency`). +Filter with an optional `?level=` (default `debug` = everything at or above): the subscription only +receives entries at that level or higher. The two streams are independent — subscribe to one or both, +each at its own level. The capability is advertised in `initialize` under +`capabilities.experimental["dcl/logStreams"]`. + +```bash +# Stream engine/build/editor errors as they happen +curl -N -H 'Accept: text/event-stream' \ + 'http://127.0.0.1:8123/unity-explorer-mcp?stream=client&level=warning' + +# Stream the scene's JS console (all levels) +curl -N -H 'Accept: text/event-stream' \ + 'http://127.0.0.1:8123/unity-explorer-mcp?stream=scene' +``` + +The stream stays open until the client disconnects or the Explorer stops. `GET` without +`Accept: text/event-stream` still returns 405; an unknown `stream`/`level` returns 400. + +## The scene-iteration loop + +1. Serve the scene and launch the Explorer in one step: `npm run start -- --mcp` in the scene folder (serves at `http://127.0.0.1:8000`, auto-launches the installed client against it with the MCP server on, and hot-reloads on file changes). +2. To use a specific Explorer build instead, serve with `npm run start -- --no-client` and launch manually: + +```bash +open Decentraland.app --args \ + --realm http://127.0.0.1:8000 --local-scene true --position 0,0 \ + --debug --skip-auth-screen --skip-version-check true \ + --mcp --windowed-mode --resolution 1280x720 +``` + +Optional determinism flags for stable screenshots: `--disable-hud`, `--skybox-time-enabled false`, `--landscape-terrain-enabled false`, `--skip-minimum-specs-screen`. + +3. The agent then loops: edit scene TypeScript → LSD hot reload applies it (or call `reload_scene`) → `get_scene_state` until ready → `screenshot` + `get_scene_logs` → verify → repeat. + +Once loading completes, the server announces its address in the scene debug console (available with local scene development or `--scene-console`): `MCP server listening on http://127.0.0.1:8123/unity-explorer-mcp`. A startup failure (port in use) is announced there as an error instead. The same line lands in the `get_scene_logs` buffer, so agents can confirm the server from inside the loop. + +A user-invokable Claude Code skill wrapping this loop lives at `.claude/skills/mcp-scene-iteration/` (invoke with `/mcp-scene-iteration`). + +## Troubleshooting + +- **Port already in use** — the server logs an `MCP` category error and stays inert; relaunch with a different `--mcp-port`. Multiple Explorer instances (`--multi-instance`) each need their own port. To confirm which process answers on a port, check `serverInfo.pid` in the `initialize` response and the `address` field of `get_player_state`. +- **HTTP 403** — the request carried a non-localhost `Origin` header; MCP clients and curl don't send one. +- **Server won't start on Windows** — `HttpListener` may require a URL ACL depending on machine policy: `netsh http add urlacl url=http://127.0.0.1:8123/unity-explorer-mcp/ user=Everyone` (elevated prompt), then relaunch. +- **Verbose logs** — enabling the server registers a scene-console log handler, which turns on unconditional verbose logging for the session (same behavior as `--scene-console`). +- **Scene entity dumps** — `list_scene_entities`/`get_entity_details` read the scene world without acquiring its sync lock (same as the existing `WorldInfoTool` debug tooling); treat results as a diagnostic snapshot. +- **`click_entity` returns `hit:false` with `blockedBy*`** — another collider sits on the camera→target line; `move_to`/`look_at` to a clear vantage and retry. If the reason is "out of range", close within the entity's `maxDistance` (default 10 m) first. Entities whose collider sits away from the pivot (GLTF meshes) may need an explicit `x/y/z` aim point. + +## Implementation map + +- `Explorer/Assets/DCL/McpServer/` — feature root, its own `DCL.McpServer` assembly. Two folders are folded into other assemblies via `.asmref` so they can reach code that assembly doesn't reference: + - `Core/` — protocol, transport and tool contract: `McpHttpServer` (`HttpListener` server + Origin validation), `McpJsonRpcDispatcher` (JSON-RPC 2.0 routing; `PROTOCOL_VERSION` `2025-06-18`), `IMcpTool`, `McpToolsRegistry`, `McpToolResult`, `McpToolAnnotations` (behaviour hints), `McpInputSchema` (typed input-schema builder). + - `Tools/` — one class per tool (16). + - `Components/` — ECS components for the input-driving tools: `McpMovementOverride`, `McpPointerClickIntent`. + - `Systems/` — **folded into `DCL.Plugins`** via `.asmref`: `McpServerPlugin` (builds the registry and hosts the server in `InjectToWorld`), `McpInputOverrideSystem` (held movement), `McpPointerClickSystem` (synthetic entity clicks). + - `Utils/` — `SceneLogBuffer`, `JObjectExtensions`. + - `Tests/` — EditMode tests **folded into `DCL.EditMode.Tests`** via `.asmref`: dispatcher / registry / result routing and the pointer-click system. +- Gating: `FeatureId.MCP_SERVER` in `FeaturesRegistry` (resolved as `appArgs.HasFlag(MCP) || appArgs.HasFlag(MCP_PORT)`); `DynamicWorldContainer.CreateAsync` reads `FeaturesRegistry.Instance.IsEnabled(FeatureId.MCP_SERVER)` and adds `McpServerPlugin`. +- Flags: `AppArgsFlags.MCP` / `AppArgsFlags.MCP_PORT`; log category: `ReportCategory.MCP`.