Skip to content

Commit b5fe21a

Browse files
[SDK/Factories] Add argsSchema To The Factory Authoring Surface
FactoryMeta now declares an optional argsSchema, typed as the existing FactoryJsonSchema. The field already crossed the wire because defineFactory snapshots meta whole, so this is additive and type-level: it makes a runtime feature discoverable to extension authors writing against the published types. Without a declared schema nothing validates a caller's args. A malformed call starts a run, takes a user approval, spends credits, and then fails inside the factory body. With one, the CLI rejects it before the run row exists and the model retries against a correction hint. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent f75d222 commit b5fe21a

5 files changed

Lines changed: 140 additions & 8 deletions

File tree

nodejs/docs/factories.md

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,13 @@ const reviewChanged = defineFactory({
1616
"Review changed files and verify the findings. " +
1717
"args: { files: string[] } — the paths to review.",
1818
phases: [{ title: "Review" }, { title: "Verify" }],
19+
argsSchema: {
20+
type: "object",
21+
required: ["files"],
22+
properties: {
23+
files: { type: "array", items: { type: "string" } },
24+
},
25+
},
1926
limits: {
2027
maxConcurrentSubagents: 3,
2128
maxTotalSubagents: 10,
@@ -41,9 +48,17 @@ const reviewChanged = defineFactory({
4148
const session = await joinSession({ factories: [reviewChanged] });
4249
```
4350

44-
Factory metadata contains a stable `name`, a human-readable `description`, declared `phases`, and optional `limits`. Phase entries contain a `title` and optional `detail`.
51+
Factory metadata contains a stable `name`, a human-readable `description`, declared `phases`, an optional `argsSchema`, and optional `limits`. Phase entries contain a `title` and optional `detail`.
52+
53+
## Declaring an argument shape
54+
55+
A factory that reads `ctx.args` should declare `meta.argsSchema`, as the example above does. The CLI validates the caller's `args` against it **before** the run starts.
56+
57+
Declaring one turns an expensive failure into a cheap one. With a schema, a malformed call is rejected up front — the model gets a correction hint and retries, and no run row, permission prompt, or credit spend happens. Without one, nothing validates: the run starts, takes a user approval, spends credits, and then dies inside the factory body with a confusing error. Agents can read the declared shape with `factories_manage` using `operation: "inspect"`.
58+
59+
Enforcement covers structure — types, required properties, and enum or const values. Finer constraints such as `minLength`, `pattern`, or `additionalProperties` are recorded in the declaration but not enforced. The accepted vocabulary is the `FactoryJsonSchema` subset also used for subagent structured output: `type`, `required`, `enum`, `const`, recursive `properties`/`items`, and `anyOf`/`oneOf`/`allOf`. A `type` is one of `null`, `boolean`, `integer`, `number`, `string`, `array`, or `object`, or a non-empty array of those such as `["object", "null"]`. A declaration outside that subset is rejected at registration.
4560

46-
There is no declared schema for `ctx.args`. The `run_factory` tool forwards `args` verbatim and its parameter is untyped, so **the `description` is the only thing telling an agent what arguments to supply** — state the expected shape there whenever a factory reads `ctx.args`, as the example above does. Arguments supplied by an extension calling `session.factory.run(...)` directly are typed through `defineFactory<TArgs>`, but that typing does not reach the model. A factory that reads `ctx.args` should validate it rather than assume a shape.
61+
`argsSchema` is optional and backward compatible. A factory that omits it behaves exactly as before, so **the `description` is then the only thing telling an agent what arguments to supply** — state the expected shape there. Arguments supplied by an extension calling `session.factory.run(...)` directly are typed through `defineFactory<TArgs>`, but that typing does not reach the model. A factory that reads `ctx.args` should still validate it rather than assume a shape, because the declared subset does not enforce every constraint.
4762

4863
`defineFactory<TArgs, TResult>` accepts a `run(context)` function returning `Promise<TResult>`, where `TResult` is `JsonValue | void`. Objects, arrays, strings, numbers, booleans, and `null` are valid results. Returning `undefined` completes the factory with no result. Other non-JSON values are rejected.
4964

@@ -193,7 +208,7 @@ async ({ args, agent, phase }) => {
193208
};
194209
```
195210
196-
Authoring registers the factory but does not run it. Invoke it afterwards with `run_factory`. Use `factories_manage` with `operation: "list"` to see the factories already registered in the session and `operation: "inspect"` to read one factory's description, phases, and limits before running it.
211+
Authoring registers the factory but does not run it. Invoke it afterwards with `run_factory`. Use `factories_manage` with `operation: "list"` to see the factories already registered in the session and `operation: "inspect"` to read one factory's description, phases, declared argument shape, and limits before running it.
197212
198213
## Observe a run
199214

nodejs/src/factory.ts

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -62,12 +62,16 @@ export type JsonValue =
6262
| { [key: string]: JsonValue };
6363

6464
/**
65-
* Conservative JSON shape language accepted for structured factory agent output.
65+
* Conservative JSON shape language accepted by the Agent Factories surface, for
66+
* both structured factory agent output and a factory's declared `argsSchema`.
6667
*
67-
* This is a best-effort structural guard used to decide whether a subagent's
68-
* structured output should be accepted or retried — **not** a full JSON Schema
68+
* This is a best-effort structural guard — used to decide whether a subagent's
69+
* structured output should be accepted or retried, and whether a caller's
70+
* factory `args` match the declared shape — **not** a full JSON Schema
6971
* validator. Only these keywords are honored: `type`, `required`, `enum`,
70-
* `const`, recursive `properties`/`items`, and `anyOf`/`oneOf`/`allOf`.
72+
* `const`, recursive `properties`/`items`, and `anyOf`/`oneOf`/`allOf`. A `type`
73+
* is one of `null`, `boolean`, `integer`, `number`, `string`, `array`, or
74+
* `object`, or a non-empty array of those (for example `["object", "null"]`).
7175
*
7276
* Everything else is **ignored, not enforced**. In particular, string
7377
* constraints (`pattern`, `minLength`, `maxLength`, `format`), numeric ranges

nodejs/src/types.ts

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ import type {
1919
SessionEvent as GeneratedSessionEvent,
2020
} from "./generated/session-events.js";
2121
import type { CopilotSession } from "./session.js";
22-
import type { JsonValue } from "./factory.js";
22+
import type { FactoryJsonSchema, JsonValue } from "./factory.js";
2323
import type {
2424
GitHubTelemetryNotification,
2525
ModelBillingTokenPrices,
@@ -2007,6 +2007,25 @@ export interface FactoryMeta {
20072007
description: string;
20082008
/** Display metadata for the progress phases the factory may report. */
20092009
phases: Array<{ title: string; detail?: string }>;
2010+
/**
2011+
* Optional declared shape of the arguments this factory expects as `ctx.args`.
2012+
*
2013+
* Declaring one is strongly recommended for any factory that reads `ctx.args`.
2014+
* The CLI validates the caller's `args` against it **before** the run starts, so a
2015+
* malformed call from the model is rejected with a correction hint and retried
2016+
* without ever creating a run row, prompting the user for permission, or spending
2017+
* credits. A factory that declares nothing is never validated: a malformed call
2018+
* starts, takes an approval, spends credits, and then fails inside the factory
2019+
* body. `factories_manage` with `operation: "inspect"` reports the declared shape
2020+
* so an agent can read it before invoking.
2021+
*
2022+
* Enforcement covers structure — types, required properties, and enum/const
2023+
* values. Finer constraints such as `minLength`, `pattern`, and
2024+
* `additionalProperties` are recorded in the declaration but not enforced. See
2025+
* {@link FactoryJsonSchema} for the accepted subset. A declaration outside that
2026+
* subset is rejected at registration.
2027+
*/
2028+
argsSchema?: FactoryJsonSchema;
20102029
/** Optional resource ceilings presented to the user before execution. */
20112030
limits?: FactoryLimits;
20122031
}

nodejs/test/e2e/fixtures/factory-extension.mjs

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,9 @@ const argumentEcho = defineFactory({
1818
name: "argument-echo",
1919
description: "Return the invocation arguments verbatim.",
2020
phases: [],
21+
// A declared shape has to survive the SDK boundary and reach the runtime,
22+
// which validates `args` against it before a run row exists.
23+
argsSchema: { type: ["object", "null"] },
2124
},
2225
run: async ({ args }) => args,
2326
});

nodejs/test/factory.test.ts

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import {
1515
type FactoryAgentOptions,
1616
type FactoryContext,
1717
type FactoryDefinition,
18+
type FactoryJsonSchema,
1819
type JsonValue,
1920
} from "../src/factory.js";
2021

@@ -461,6 +462,96 @@ describe("factories", () => {
461462
);
462463
});
463464

465+
it("carries a declared argsSchema through defineFactory into the registration payload", async () => {
466+
const client = new CopilotClient();
467+
await client.start();
468+
onTestFinished(() => stopClient(client));
469+
470+
const argsSchema = {
471+
type: "object",
472+
required: ["repoPath"],
473+
properties: {
474+
repoPath: { type: "string" },
475+
depth: { type: ["integer", "null"] },
476+
mode: { enum: ["fast", "thorough"] },
477+
},
478+
} satisfies FactoryJsonSchema;
479+
const meta = {
480+
name: "declares-args",
481+
description: "Declares the argument shape it expects",
482+
phases: [],
483+
argsSchema,
484+
};
485+
const factory = defineFactory({ meta, run: async () => ({ ok: true }) });
486+
487+
// The declaration is snapshotted and deep-frozen like the rest of the
488+
// metadata, so it cannot be mutated after registration.
489+
expect(factory.meta.argsSchema).toEqual(argsSchema);
490+
expect(factory.meta.argsSchema).not.toBe(argsSchema);
491+
expect(Object.isFrozen(factory.meta.argsSchema)).toBe(true);
492+
expect(() => {
493+
// @ts-expect-error handle.meta.argsSchema is deeply readonly.
494+
factory.meta.argsSchema!.type = "array";
495+
}).toThrow(TypeError);
496+
497+
const omitted = defineFactory({
498+
meta: { name: "omits-args", description: "Declares nothing", phases: [] },
499+
run: async () => ({ ok: true }),
500+
});
501+
expect(omitted.meta.argsSchema).toBeUndefined();
502+
expect("argsSchema" in omitted.meta).toBe(false);
503+
504+
const sendRequest = vi
505+
.spyOn(
506+
(client as never as { connection: { sendRequest: Function } }).connection,
507+
"sendRequest"
508+
)
509+
.mockImplementation(async (method: string, params: Record<string, unknown>) => {
510+
if (method === "session.resume") {
511+
return { sessionId: params.sessionId };
512+
}
513+
throw new Error(`Unexpected method: ${method}`);
514+
});
515+
516+
await client.resumeSessionForExtension(
517+
"session-args-schema",
518+
{ onPermissionRequest: () => ({ kind: "approved" }) },
519+
[factory, omitted]
520+
);
521+
522+
const payload = sendRequest.mock.calls.find(
523+
([method]) => method === "session.resume"
524+
)![1] as { factories: Array<Record<string, unknown>> };
525+
// The schema has to survive JSON serialization to reach the runtime, which
526+
// validates `args` against it before a run row exists.
527+
expect(JSON.parse(JSON.stringify(payload.factories))[0].argsSchema).toEqual(argsSchema);
528+
expect(payload.factories[1]).not.toHaveProperty("argsSchema");
529+
});
530+
531+
it("documents argsSchema consistently with the runtime's enforced subset", () => {
532+
const publicTypes = readFileSync(new URL("../src/types.ts", import.meta.url), "utf8");
533+
const publicApi = readFileSync(new URL("../src/factory.ts", import.meta.url), "utf8");
534+
const guide = readFileSync(new URL("../docs/factories.md", import.meta.url), "utf8");
535+
const normalizeJSDoc = (document: string) =>
536+
document.replace(/\r?\n\s*\* ?/g, " ").replace(/\s+/g, " ");
537+
538+
expect(publicTypes).toContain("argsSchema?: FactoryJsonSchema;");
539+
540+
// The `run_factory` tool tells the model exactly this. The two surfaces
541+
// have to agree about what a declaration does and does not enforce.
542+
for (const document of [normalizeJSDoc(publicTypes), guide]) {
543+
expect(document).toContain("types, required properties, and enum");
544+
expect(document).toMatch(
545+
/`minLength`, `pattern`,? (?:and|or) `additionalProperties` are recorded/
546+
);
547+
}
548+
expect(normalizeJSDoc(publicTypes)).toContain("before** the run starts");
549+
expect(normalizeJSDoc(publicApi)).toContain(
550+
"`null`, `boolean`, `integer`, `number`, `string`, `array`, or `object`"
551+
);
552+
expect(guide).toContain("no run row, permission prompt, or credit spend happens");
553+
});
554+
464555
it("serializes only factory metadata in the extension resume payload", async () => {
465556
const client = new CopilotClient();
466557
await client.start();

0 commit comments

Comments
 (0)