Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 17 additions & 5 deletions command-center-ai/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,15 @@
## Unreleased

Compatibility axes: additive npm API on `ChatRailProps` (`resizable`, `width`, `defaultWidth`,
`minWidth`, `maxWidth`, `onWidthChange`); additive stylesheet classes
`ms-chat-rail__resize-handle` and `ms-chat-rail--resizing`; and the content and description of the
packaged skill `compose-command-center-ai-rail`. No browser storage key or value, skill name or
path, installer behavior, SDK peer range, platform route, or payload changes. No backend action or
storage migration is required: sizing is presentation state and the application owns persistence.
`minWidth`, `maxWidth`, `onWidthChange`), on `AgentIcon` (`motion`) and on
`CommandCenterAgentIcon` (optional `imageRendering`); additive stylesheet classes
`ms-chat-rail__resize-handle` and `ms-chat-rail--resizing` and a rule that stops `ms-chat-pulse`
on an animated agent icon; the read of the platform's additive `commandCenterIcon.imageRendering`
(absent means `auto`); and the content of the packaged skills `compose-command-center-ai-rail`
(also its description) and `design-agent-conversation-capabilities`. No browser storage key or
value, skill name or path, installer behavior, SDK peer range, or platform route changes. No
backend action or storage migration is required: sizing is presentation state and the application
owns persistence, and an icon the platform does not animate or mark pixelated renders as before.

- **People can resize the AI right rail.** Drag its left edge or focus "Resize chat rail" and use
Left/Right, Shift+Left/Right, Home, or End. Double-click restores its default width. It starts at
Expand All @@ -18,6 +22,14 @@ storage migration is required: sizing is presentation state and the application
fixed overlay. The rail guide, UI README, packaged skill, consumer fixture, and standalone demo
describe or exercise this API. The demo retains width across closing and expanding, docks from
1400px, and floats below it.
- **Animated agent icons move only where they help.** A colour agent icon may be an animated PNG
(the platform's ADR-033, amended 2026-10-16). `AgentIcon` shows its first frame by default, in
lists, the launcher, thread avatars and the application's agent pages, and animates it only with
`motion="animate"`, which the rail header, the connecting stage and the avatar of the working
agent pass. Reduced motion always shows the first frame, and an animated avatar does not also
pulse. The first frame is made once from the same bytes by dropping the APNG animation chunks, so
nothing is fetched or decoded twice. A colour icon the platform marks `imageRendering:
"pixelated"` keeps sharp pixels when scaled.

## 0.0.13

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,10 @@ answers so the text and the tool calls tell the story on their own.
Agent's with its icon, and labels actors when several take part.
- Agent names are shown as the platform stores them; the thread does not shorten or rewrite them.
- An Agent's icon comes from the platform's agent icon projection for the Environment, as a
theme-following mask or a colour image, with a robot as the fallback. During a live turn the avatar
theme-following mask or a colour image, with a robot as the fallback. A colour icon may be an
animated pixel-art sprite: it moves only in the rail header, the connecting stage and while its
Agent works, shows its first frame everywhere else and under reduced motion, and keeps sharp
pixels when the platform marks it pixelated. During a live turn the avatar
is the robot until the next reload; that is reported to the package's owner.

## The Session Around The Answers
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
- Status: Accepted
- Date: 2026-09-14
- Amended: 2026-09-16 for the unified Agent shortcut
- Amended: 2026-10-16 for animated and pixel-art icons (section 6), following the platform's
ADR-033 amendment of the same day
- Moved: 2026-09-23 into the chat package with its code (ADR 096, step 4); the code paths below
are the package's
- Related:
Expand Down Expand Up @@ -44,7 +46,8 @@ bytes itself and render a local object URL. This ADR does not change the backend
one projection per active Organization Environment and trusts it for five minutes, shared by every
provider on the page; a failed read is retried once. Until ADR 096 moved the engine into the
package this was a TanStack query with the same staleness and no refetch on focus. It exposes
`useAgentIconLookup()`: agent uid in, `{ rendering, url }` or null out, and `useAgentIconAuth()`:
`useAgentIconLookup()`: agent uid in, `{ rendering, imageRendering, url }` or null out
(`imageRendering` is `auto` unless the platform sends `pixelated` for a colour icon), and `useAgentIconAuth()`:
the connection and the token the icon requests use. Without a provider (tests, surfaces outside the chat provider) the lookup returns null
and everything falls back. A failed projection never blocks a surface.

Expand Down Expand Up @@ -102,6 +105,28 @@ navigation entry are not agents and do not change.
- The configured shortcut Agent keeps whatever icon it has on the platform; Command Center adds no
shortcut-specific default.

### 6. Animated and pixel-art icons

Amended 2026-10-16. The platform may now deliver a colour icon as an animated PNG (APNG) whose
frame 0 is its still, and may mark a colour icon `imageRendering: "pixelated"`.

- **Still unless asked.** `AgentIcon` takes `motion`, `"still"` by default. An animated icon
animates only with `motion="animate"`, which three places pass: the rail header badge, the
connecting stage, and the thread avatar of the agent that is working. Every other place,
including the application's own agent surfaces, shows frame 0. Lists and repeated avatars are
where many icons would move at once and keep the page from ever going idle.
- **Reduced motion.** When the person asks for reduced motion
(`useCommandCenterViewport().reducedMotion`), every place shows frame 0.
- **The still comes from the same bytes.** The cache makes it once per delivery URL: an APNG is a
PNG plus `acTL`, `fcTL` and `fdAT` chunks, and dropping them leaves a static PNG of the default
image, which the platform guarantees is frame 0 (`src/engine/apng-still.ts`). No second request,
canvas or decoding; the still object URL lives and is revoked with the icon's. The platform's
projection and delivery are unchanged.
- **One motion at a time.** An animated avatar is marked `data-agent-icon-motion="animated"` and
the stylesheet stops the working pulse (`ms-chat-pulse`) on it.
- **Pixel art stays sharp.** A colour icon marked `pixelated` is drawn with CSS
`image-rendering: pixelated`. Mask icons are always smooth.

## Non-goals

- Uploading or editing icons from the Command Center; the repository workflow owns that.
Expand All @@ -117,7 +142,11 @@ navigation entry are not agents and do not change.
de-duplication, revalidation with and without an ETag, error handling and clearing.
- `src/engine/agent-icons-context.tsx`: `AgentIconsProvider`, `AgentIconsContext`,
`AgentIconAuthContext`, `useAgentIconLookup`, `useAgentIconAuth`.
- `src/ui/AgentIcon.tsx`, exported by the package; tested for mask, colour, fallback and failure.
- `src/ui/AgentIcon.tsx`, exported by the package; tested for mask, colour, fallback and failure,
and since 2026-10-16 for the still default, `motion="animate"`, reduced motion and pixelated
scaling.
- `src/engine/apng-still.ts`: frame 0 of an APNG as a static PNG; tested with the platform's own
sanitizer output beside the cache tests.
- `ChatEngineProvider` mounts the provider and exposes `activeAgentUid`; the package's
`ChatThread.tsx` (`ActorAvatar`, the connecting stage prop) and `AgentConnectingState.tsx` use
the component, and so do the application's own agent surfaces in section 4.
Expand All @@ -128,7 +157,10 @@ navigation entry are not agents and do not change.

- One projection request per environment every five minutes of use, one bytes request per
distinct icon per page load, and a cheap revalidation per icon every five minutes.
- Object URLs live for the page; at most 512 KiB per icon.
- Object URLs live for the page; at most 512 KiB per icon, plus a still of at most the same size
for an animated one.
- Animated icons cost nothing where they are shown still; only the few single places that animate
keep the page drawing frames.
- Should the platform later add a version or digest to the projection, the cache can key on it
and skip revalidation; nothing here depends on that.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,26 @@ describe("command center agent icons api", () => {
"junk",
]);
expect(Array.from(icons.keys())).toEqual(["a1", "a5"]);
expect(icons.get("a1")).toEqual({ agentUid: "a1", rendering: "mask", url: "https://api.test/icon/a1/" });
expect(icons.get("a1")).toEqual({
agentUid: "a1",
rendering: "mask",
imageRendering: "auto",
url: "https://api.test/icon/a1/",
});
expect(icons.get("a5")?.rendering).toBe("color");
expect(normalizeCommandCenterAgentIcons({ not: "an array" }).size).toBe(0);
});

it("keeps pixelated scaling only for colour icons", () => {
const icons = normalizeCommandCenterAgentIcons([
{ agentUid: "pixel", commandCenterIcon: { rendering: "color", imageRendering: "pixelated", url: "https://api.test/p/" } },
{ agentUid: "mask", commandCenterIcon: { rendering: "mask", imageRendering: "pixelated", url: "https://api.test/m/" } },
{ agentUid: "odd", commandCenterIcon: { rendering: "color", imageRendering: "crisp", url: "https://api.test/o/" } },
]);
expect(icons.get("pixel")?.imageRendering).toBe("pixelated");
expect(icons.get("mask")?.imageRendering).toBe("auto");
expect(icons.get("odd")?.imageRendering).toBe("auto");
});
});

describe("command center agent icon bytes", () => {
Expand Down
10 changes: 9 additions & 1 deletion command-center-ai/src/backend/command-center-agent-icons-api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,14 @@ import { requestPlatform } from "./platform-request.js";
* fetched with the authenticated client and rendered from an object URL.
*/
export type CommandCenterAgentIconRendering = "mask" | "color";
/** How a color icon is scaled: smoothly, or with sharp pixels for pixel art. */
export type CommandCenterAgentIconImageRendering = "auto" | "pixelated";

export interface CommandCenterAgentIcon {
agentUid: string;
rendering: CommandCenterAgentIconRendering;
/** `pixelated` only for color icons; absent or `auto` scales smoothly. */
imageRendering?: CommandCenterAgentIconImageRendering;
url: string;
}

Expand Down Expand Up @@ -58,7 +62,11 @@ export function normalizeCommandCenterAgentIcons(payload: unknown): Map<string,
if ((rendering !== "mask" && rendering !== "color") || !url) {
continue;
}
icons.set(agentUid, { agentUid, rendering, url });
const imageRendering =
rendering === "color" && readString(iconRecord.imageRendering ?? iconRecord.image_rendering) === "pixelated"
? "pixelated"
: "auto";
icons.set(agentUid, { agentUid, rendering, imageRendering, url });
}
return icons;
}
Expand Down
51 changes: 51 additions & 0 deletions command-center-ai/src/engine/agent-icon-cache.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,22 @@ import {
getAgentIconCacheEntry,
isAgentIconRequestInFlight,
} from "./agent-icon-cache.js";
import { apngStill } from "./apng-still.js";
import { createChatBackendConnection } from "../backend/connection.js";

// What the platform serves: a 32 px two-frame APNG and a static PNG, both from its sanitizer.
const APNG = Uint8Array.from(atob("iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAACGFjVEwAAAACAAAAAPONk3AAAAAaZmNUTAAAAAAAAAAgAAAAIAAAAAAAAAAAAGQD6AAAcyZw9wAAAD1JREFUeJxjYBgFo2AUjHTASEjBBxGN/5RYIPDmBl47mCgxnBpg1AGjDhh1wKgDRh0w6oBRB4yCUTAKRgEAcagEIFRAp3wAAAAaZmNUTAAAAAEAAAAQAAAAEAAAAAgAAAAIAGQD6AAAmBjZzgAAACFmZEFUAAAAAnicYxTR+PCfgQLARInmUQNGDRg1YDAZAAANFwJLQfgfzgAAAABJRU5ErkJggg=="), (character) => character.charCodeAt(0));
const STATIC_PNG = Uint8Array.from(atob("iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAAPElEQVR42mNgGAWjYBSMdMBISMEHEY3/lFgg8OYGXjuYBjoERh0w6oBRB4w6YNQBow4YdcAoGAWjYBQAAHGoBCArKbGwAAAAAElFTkSuQmCC"), (character) => character.charCodeAt(0));

function pngChunkTypes(bytes: Uint8Array) {
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
const types: string[] = [];
for (let offset = 8; offset + 12 <= bytes.length; offset += 12 + view.getUint32(offset)) {
types.push(String.fromCharCode(...bytes.subarray(offset + 4, offset + 8)));
}
return types;
}

const connection = createChatBackendConnection({ apiBaseUrl: "http://localhost:8000" });

const url = "https://api.test/api/v1/command-center/agents/a1/icon/";
Expand Down Expand Up @@ -123,4 +137,41 @@ describe("agent icon cache", () => {
expect(revokeObjectURL).toHaveBeenCalledWith("blob:test/1");
expect(getAgentIconCacheEntry(url)).toBeUndefined();
});

it("keeps a still of frame 0 for an animated PNG and revokes it with the icon", async () => {
fetchMock.mockResolvedValueOnce(new Response(APNG, { status: 200, headers: { "Content-Type": "image/png" } }));
ensureAgentIcon({ connection, url, token: "jwt" });
await vi.waitFor(() => expect(getAgentIconCacheEntry(url)?.status).toBe("ready"));
expect(getAgentIconCacheEntry(url)).toMatchObject({ objectUrl: "blob:test/1", stillObjectUrl: "blob:test/2" });
const still = createObjectURL.mock.calls[1]?.[0];
expect(still?.type).toBe("image/png");
expect(pngChunkTypes(new Uint8Array(await still!.arrayBuffer()))).toEqual(["IHDR", "IDAT", "IEND"]);

clearAgentIconCache();
expect(revokeObjectURL).toHaveBeenCalledWith("blob:test/1");
expect(revokeObjectURL).toHaveBeenCalledWith("blob:test/2");
});

it("makes no still for a static icon", async () => {
fetchMock.mockResolvedValueOnce(new Response(STATIC_PNG, { status: 200, headers: { "Content-Type": "image/png" } }));
ensureAgentIcon({ connection, url, token: "jwt" });
await vi.waitFor(() => expect(getAgentIconCacheEntry(url)?.status).toBe("ready"));
expect(getAgentIconCacheEntry(url)?.stillObjectUrl).toBeNull();
expect(createObjectURL).toHaveBeenCalledTimes(1);
});
});

describe("apngStill", () => {
it("drops the animation chunks and keeps frame 0 byte for byte", () => {
expect(pngChunkTypes(APNG)).toEqual(["IHDR", "acTL", "fcTL", "IDAT", "fcTL", "fdAT", "IEND"]);
const still = apngStill(APNG)!;
expect(pngChunkTypes(still)).toEqual(["IHDR", "IDAT", "IEND"]);
expect(Array.from(still.subarray(0, 8))).toEqual(Array.from(APNG.subarray(0, 8)));
});

it("returns null for a static PNG, other bytes, and a truncated APNG", () => {
expect(apngStill(STATIC_PNG)).toBeNull();
expect(apngStill(new TextEncoder().encode("<svg/>"))).toBeNull();
expect(apngStill(APNG.subarray(0, 80))).toBeNull();
});
});
43 changes: 34 additions & 9 deletions command-center-ai/src/engine/agent-icon-cache.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@

import { fetchCommandCenterAgentIconBytes } from "../backend/command-center-agent-icons-api.js";
import type { ChatBackendConnection } from "../backend/connection.js";
import { apngStill } from "./apng-still.js";

/**
* Bytes of agent icons, shared by every avatar on the page (ADR 090).
Expand All @@ -11,13 +12,18 @@ import type { ChatBackendConnection } from "../backend/connection.js";
* AGENT_ICON_REVALIDATE_MS: the browser sends If-None-Match, and the object URL
* is replaced only when the bytes' fingerprint (ETag when exposed, otherwise a
* SHA-256 of the bytes) differs. Sign-out clears everything.
*
* An animated PNG also gets a still: a static PNG of its frame 0, made once from the same bytes,
* for the places that should not move (ADR 090, amended 2026-10-16).
*/
export const AGENT_ICON_REVALIDATE_MS = 5 * 60_000;
export const AGENT_ICON_RETRY_MS = 60_000;

export interface AgentIconCacheEntry {
status: "loading" | "ready" | "error";
objectUrl: string | null;
/** A static PNG of frame 0 when the icon is an animated PNG; null otherwise. */
stillObjectUrl: string | null;
fingerprint: string | null;
checkedAt: number;
}
Expand Down Expand Up @@ -64,6 +70,27 @@ function revokeObjectUrl(objectUrl: string) {
}
}

async function createStillObjectUrl(blob: Blob) {
if (blob.type && blob.type !== "image/png") {
return null;
}
try {
const still = apngStill(new Uint8Array(await blob.arrayBuffer()));
return still ? createObjectUrl(new Blob([still], { type: "image/png" })) : null;
} catch {
return null;
}
}

function revokeEntryUrls(entry: AgentIconCacheEntry) {
if (entry.objectUrl) {
revokeObjectUrl(entry.objectUrl);
}
if (entry.stillObjectUrl) {
revokeObjectUrl(entry.stillObjectUrl);
}
}

async function fingerprintBlob(blob: Blob, etag: string | null) {
if (etag) {
return etag;
Expand Down Expand Up @@ -104,7 +131,7 @@ export function ensureAgentIcon({
return;
}
if (!existing) {
entries.set(url, { status: "loading", objectUrl: null, fingerprint: null, checkedAt: now });
entries.set(url, { status: "loading", objectUrl: null, stillObjectUrl: null, fingerprint: null, checkedAt: now });
emit();
}
const request = fetchCommandCenterAgentIconBytes({ connection, url, token, tokenType })
Expand All @@ -118,12 +145,14 @@ export function ensureAgentIcon({
return;
}
const objectUrl = createObjectUrl(blob);
if (current?.objectUrl) {
revokeObjectUrl(current.objectUrl);
const stillObjectUrl = objectUrl ? await createStillObjectUrl(blob) : null;
if (current) {
revokeEntryUrls(current);
}
entries.set(url, {
status: objectUrl ? "ready" : "error",
objectUrl,
stillObjectUrl,
fingerprint: objectUrl ? fingerprint : null,
checkedAt: Date.now(),
});
Expand All @@ -137,7 +166,7 @@ export function ensureAgentIcon({
url,
current?.objectUrl
? { ...current, status: "ready", checkedAt: Date.now() }
: { status: "error", objectUrl: null, fingerprint: null, checkedAt: Date.now() },
: { status: "error", objectUrl: null, stillObjectUrl: null, fingerprint: null, checkedAt: Date.now() },
);
emit();
})
Expand All @@ -148,11 +177,7 @@ export function ensureAgentIcon({
}

export function clearAgentIconCache() {
entries.forEach((entry) => {
if (entry.objectUrl) {
revokeObjectUrl(entry.objectUrl);
}
});
entries.forEach(revokeEntryUrls);
entries.clear();
inFlight.clear();
emit();
Expand Down
14 changes: 13 additions & 1 deletion command-center-ai/src/engine/agent-icons-context.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,14 @@ const noEntry = () => undefined;

/** The object URL for an icon, or null while it loads, without auth, or after it failed. */
export function useAgentIconObjectUrl(url: string | null | undefined, auth: AgentIconAuth | null) {
return useAgentIconImage(url, auth).objectUrl;
}

/**
* The icon's object URL and, for an animated PNG, the object URL of its frame 0 still; both null
* while it loads, without auth, or after it failed.
*/
export function useAgentIconImage(url: string | null | undefined, auth: AgentIconAuth | null) {
const entry = useSyncExternalStore(
subscribeAgentIconCache,
() => getAgentIconCacheEntry(url),
Expand All @@ -176,5 +184,9 @@ export function useAgentIconObjectUrl(url: string | null | undefined, auth: Agen
ensureAgentIcon({ connection, url, token, tokenType });
}
}, [connection, token, tokenType, url]);
return entry?.status === "ready" ? entry.objectUrl : null;
const ready = entry?.status === "ready";
return {
objectUrl: ready ? entry.objectUrl : null,
stillObjectUrl: ready ? entry.stillObjectUrl : null,
};
}
Loading
Loading