From bf41ace6cba9cc9a93312816976b04387b04647c Mon Sep 17 00:00:00 2001 From: Seth Juarez Date: Fri, 16 Oct 2026 00:12:04 -0700 Subject: [PATCH 1/8] feat(emitter): load absent required nullable keys as null with `= null` `x: T | null = null` is now accepted on a required nullable property. Load treats an absent key as null instead of rejecting it, while save still always writes the key (as null when unset). This models documents written before the key existed (lenient load, strict save) with stock TypeSpec, no new decorator. - IR: record `hasNullDefault` on the field and load assignment; a `null` default on any other property is still a diagnostic. - Policy: add `shouldRejectAbsentNullableKey` and route every runtime's missing-key guard (TS, Python, Go, C#, Java, Rust, Swift, inherited and Swift unknown-discriminator paths) through it; save paths are unchanged. - Coverage: extend `fixtures/features/nullable` (row scalar/model, inherited base field, all-defaulted model), IR golden in validate-fixtures, per-runtime codegen assertions, and an executable Python absent-key -> null probe. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 25c59b95-6ae9-4775-bd14-86a11b625d8d --- .../docs/reference/runtime-semantics.mdx | 8 +- packages/typra-emitter/fixtures/README.md | 2 +- .../fixtures/features/nullable/main.tsp | 21 ++ .../scripts/validate-fixtures.mjs | 6 +- packages/typra-emitter/src/ir/ast.ts | 26 +- packages/typra-emitter/src/ir/declarations.ts | 8 + .../src/ir/field-emission-policy.ts | 15 +- packages/typra-emitter/src/ir/inheritance.ts | 4 +- packages/typra-emitter/src/ir/lower.ts | 4 + .../src/languages/csharp/emitter.ts | 3 +- .../typra-emitter/src/languages/go/emitter.ts | 6 +- .../src/languages/java/emitter.ts | 3 +- .../src/languages/python/emitter.ts | 3 +- .../src/languages/rust/emitter.ts | 3 +- .../src/languages/swift/emitter.ts | 5 +- .../src/languages/typescript/emitter.ts | 3 +- packages/typra-emitter/test/nullable.test.ts | 260 +++++++++++++++++- 17 files changed, 358 insertions(+), 22 deletions(-) diff --git a/docs/src/content/docs/reference/runtime-semantics.mdx b/docs/src/content/docs/reference/runtime-semantics.mdx index 2cb120a..3781739 100644 --- a/docs/src/content/docs/reference/runtime-semantics.mdx +++ b/docs/src/content/docs/reference/runtime-semantics.mdx @@ -18,7 +18,8 @@ be explicit in diagnostics, the conformance matrix, or a tracked known failure. - A required TypeSpec property must be present when loading object-shaped data. Missing required fields must fail; generated runtimes must not fabricate a value to make construction succeed. -- Defaults do not satisfy required-field presence. Until complex defaults are +- Defaults do not satisfy required-field presence (the one exception is + `x: T | null = null`, below). Until complex defaults are supported end-to-end, a default on a model-typed or collection-typed property is a portability error rather than an implicit object factory. - Optional properties have distinct wire states: absent, present with a value, @@ -33,6 +34,11 @@ be explicit in diagnostics, the conformance matrix, or a tracked known failure. ## Nulls and unknown values +- `x: T | null` is a required key whose value may be null: load rejects an + absent key and accepts `null`; save always writes the key, as `null` when + unset. `x: T | null = null` keeps that save rule but loads an absent key as + `null` (for payloads written before the key existed). `x?: T | null` behaves + like `x?: T`. - `null` is accepted only where the lowered contract explicitly permits it or where the field type is an unknown/dictionary-shaped carrier. Otherwise it is a load error. diff --git a/packages/typra-emitter/fixtures/README.md b/packages/typra-emitter/fixtures/README.md index be3c789..c76afb9 100644 --- a/packages/typra-emitter/fixtures/README.md +++ b/packages/typra-emitter/fixtures/README.md @@ -38,7 +38,7 @@ behavior is genuinely runtime-specific. | Dispatch | `features/dispatch/main.tsp` | Behavioral polymorphic dispatch via `@dispatch`, resolving a discriminator field to a deterministic parameter access path. | | Provider wire | `features/wire/main.tsp` | Provider-specific names with `@@knownAs` and provider defaults with `@@defaultFor`. | | Encoded names | `features/encoded-name/main.tsp` | `@encodedName("application/json", ...)` as the canonical dict/JSON/YAML key on load and save (scalar, nested model, collection, discriminator, inherited `@coerce` target, and `@@knownAs` interplay); non-JSON mimes are ignored. | -| Nullable fields | `features/nullable/main.tsp` | `x: T \| null` (required key, value may be null: load rejects an absent key and accepts `null`; save always writes the key, as `null` when unset) versus `x?: T` and `x?: T \| null` (both omitted when unset) across scalars, enums, models, arrays, and records. | +| Nullable fields | `features/nullable/main.tsp` | `x: T \| null` (required key, value may be null: load rejects an absent key and accepts `null`; save always writes the key, as `null` when unset) versus `x?: T` and `x?: T \| null` (both omitted when unset) across scalars, enums, models, arrays, and records. `x: T \| null = null` keeps the always-written save but loads an absent key as `null` (including inherited through a discriminated base). | | Additional properties | `features/additional-properties/main.tsp` | Unknown-field passthrough (#336): `...Record` and `is Record` models keep every undeclared key in a generated `additionalProperties` member on load and write it back on save; known keys are matched by wire name (`@encodedName`), and a polymorphic base passes its passthrough to derived variants. A plain model stays closed. | | Transport | `features/transport/main.tsp` | Path, query, header, cookie, body, status, success/error body envelopes, and wildcard status bodies. | | Namespaces | `features/namespaces/main.tsp` | Nested namespaces and cross-namespace references. | diff --git a/packages/typra-emitter/fixtures/features/nullable/main.tsp b/packages/typra-emitter/fixtures/features/nullable/main.tsp index 60006ac..cc6697a 100644 --- a/packages/typra-emitter/fixtures/features/nullable/main.tsp +++ b/packages/typra-emitter/fixtures/features/nullable/main.tsp @@ -6,6 +6,9 @@ namespace Typra.Fixtures.Features.Nullable; // but its value may be null": load rejects an absent key yet accepts an explicit // null, and save always writes the key (as null when unset). This is distinct from // `x?: T`, which is omitted when unset. `x?: T | null` behaves like `x?: T`. +// +// `x: T | null = null` keeps the same save rule but lets load treat an absent key +// as null (lenient load, strict save), e.g. for legacy documents that omit it. @serializable model Root { row: Row; @@ -14,6 +17,16 @@ model Root { // variant's own nullable field must both keep the required-nullable contract. shape: Shape; shapes: Shape[]; + + legacyOnly: LegacyOnly; +} + +// Every nullable field has a `null` default, so load emits no missing-key guard at all +// (Go must then not import an unused `fmt`). +model LegacyOnly { + @sample(#{ legacyNote: null }) + @sample(#{ legacyNote: "kept" }) + legacyNote: string | null = null; } @discriminator("kind") @@ -24,6 +37,9 @@ model Shape { @sample(#{ label: null }) @sample(#{ label: "unit" }) label: string | null; + + // Inherited required-nullable field with a `null` default: absent loads as null. + legacyLabel: string | null = null; } model Circle extends Shape { @@ -59,6 +75,11 @@ model Row { details: Detail[] | null; labels: Record | null; + // Required nullable with a `null` default: an absent key loads as null, and save + // still always writes the key. + legacy: string | null = null; + legacyDetail: Detail | null = null; + // Plain optional: omitted when unset. note?: string; diff --git a/packages/typra-emitter/scripts/validate-fixtures.mjs b/packages/typra-emitter/scripts/validate-fixtures.mjs index 87e313b..98898d5 100644 --- a/packages/typra-emitter/scripts/validate-fixtures.mjs +++ b/packages/typra-emitter/scripts/validate-fixtures.mjs @@ -840,13 +840,15 @@ function assertFocusedFeatureFixtures() { // Issue #339: `x: T | null` is a required key whose value may be null, // distinct from `x?: T` (omitted when unset). The IR must record both // flags so every runtime can require the key on load and always write it. + // `x: T | null = null` additionally records `hasNullDefault`, so load treats + // an absent key as null while save still always writes it. const row = (model.properties ?? []).find( (field) => field.name === "row", )?.type; const flags = Object.fromEntries( (row?.properties ?? []).map((field) => [ field.name, - `${field.isOptional ? "optional" : "required"}${field.isNullable ? "+nullable" : ""}`, + `${field.isOptional ? "optional" : "required"}${field.isNullable ? "+nullable" : ""}${field.hasNullDefault ? "+nullDefault" : ""}`, ]), ); const expected = { @@ -857,6 +859,8 @@ function assertFocusedFeatureFixtures() { tags: "required+nullable", details: "required+nullable", labels: "required+nullable", + legacy: "required+nullable+nullDefault", + legacyDetail: "required+nullable+nullDefault", note: "optional", hint: "optional+nullable", }; diff --git a/packages/typra-emitter/src/ir/ast.ts b/packages/typra-emitter/src/ir/ast.ts index 937e82a..17dbaee 100644 --- a/packages/typra-emitter/src/ir/ast.ts +++ b/packages/typra-emitter/src/ir/ast.ts @@ -356,6 +356,12 @@ export class PropertyNode { * stripped and `T` resolved as usual; `isOptional` keeps the TypeSpec `?`. */ public isNullable: boolean = false; + /** + * True for a required nullable property declared `= null` (`x: T | null = null`): an + * absent key loads as null instead of failing. Save still always writes the key. + * `hasExplicitDefault` stays false because there is no value to materialize. + */ + public hasNullDefault: boolean = false; public isCollection: boolean = false; public isAny: boolean = false; public isDict: boolean = false; @@ -409,6 +415,7 @@ export class PropertyNode { isScalar: this.isScalar, isOptional: this.isOptional, ...(this.isNullable ? { isNullable: true } : {}), + ...(this.hasNullDefault ? { hasNullDefault: true } : {}), isCollection: this.isCollection, isAny: this.isAny, isDict: this.isDict, @@ -1079,12 +1086,19 @@ export const resolveProperty = ( } prop.isNullable = isNullable; if (property.defaultValue?.valueKind === "NullValue") { - reportUnsupportedDefault( - program, - property, - prop, - "A `null` default is not supported; a nullable field is already null when unset, so remove the default.", - ); + if (isNullable && !property.optional) { + // `x: T | null = null`: an absent key loads as null; save still always writes it. + prop.hasNullDefault = true; + prop.hasExplicitDefault = false; + prop.defaultValue = null; + } else { + reportUnsupportedDefault( + program, + property, + prop, + "A `null` default is only supported on a required nullable property (`x: T | null = null`); an optional field is already null when unset, so remove the default.", + ); + } } if (property.defaultValue?.valueKind === "ObjectValue") { reportUnsupportedDefault( diff --git a/packages/typra-emitter/src/ir/declarations.ts b/packages/typra-emitter/src/ir/declarations.ts index a10d08f..3807d1f 100644 --- a/packages/typra-emitter/src/ir/declarations.ts +++ b/packages/typra-emitter/src/ir/declarations.ts @@ -200,6 +200,12 @@ export interface FieldDecl { * nullable fields (`x?: T | null`) lower exactly like `x?: T` and do not carry it. */ isNullable?: boolean; + /** + * Required nullable field declared `= null` (`x: T | null = null`). Present (always + * `true`) only alongside {@link FieldDecl.isNullable}: load accepts an ABSENT key as + * null instead of rejecting it; save is unchanged and always writes the key. + */ + hasNullDefault?: boolean; /** * Optional nullable field (`x?: T | null`, issue #339). Present (always `true`) only * for that shape. It behaves like `x?: T` (omitted on save when unset); the flag only @@ -459,6 +465,8 @@ export interface LoadAssignment { isOptional: boolean; /** Required nullable field (see {@link FieldDecl.isNullable}). */ isNullable?: boolean; + /** Absent key loads as null (see {@link FieldDecl.hasNullDefault}). */ + hasNullDefault?: boolean; /** Optional nullable field (see {@link FieldDecl.isOptionalNullable}). */ isOptionalNullable?: boolean; /** Parent type name (needed for calling collection helpers like Parent.load_items) */ diff --git a/packages/typra-emitter/src/ir/field-emission-policy.ts b/packages/typra-emitter/src/ir/field-emission-policy.ts index c014d0c..a33930c 100644 --- a/packages/typra-emitter/src/ir/field-emission-policy.ts +++ b/packages/typra-emitter/src/ir/field-emission-policy.ts @@ -67,8 +67,8 @@ export function shouldGuardMissingRequiredField( /** * Required nullable field (`x: T | null`, #339): the value is typed and stored like an - * optional field, but load must reject an ABSENT key (an explicit `null` is accepted and - * leaves the value unset) and save must ALWAYS write the key, as `null` when unset. + * optional field, and save must ALWAYS write the key, as `null` when unset. Whether load + * rejects an absent key is {@link shouldRejectAbsentNullableKey}. */ export function isRequiredNullable( field: { isNullable?: boolean } | undefined, @@ -76,6 +76,17 @@ export function isRequiredNullable( return field?.isNullable === true; } +/** + * Whether load must reject an ABSENT required nullable key (an explicit `null` is always + * accepted and leaves the value unset). `x: T | null` rejects it; `x: T | null = null` + * loads it as null, so payloads written before the key existed stay loadable. + */ +export function shouldRejectAbsentNullableKey( + field: { isNullable?: boolean; hasNullDefault?: boolean } | undefined, +): boolean { + return isRequiredNullable(field) && field?.hasNullDefault !== true; +} + export function saveFieldEmissionPolicy( field: FieldDecl | SaveAssignment | undefined, profile: SaveFieldEmissionProfile = "optional-only", diff --git a/packages/typra-emitter/src/ir/inheritance.ts b/packages/typra-emitter/src/ir/inheritance.ts index fb7789a..9761bd1 100644 --- a/packages/typra-emitter/src/ir/inheritance.ts +++ b/packages/typra-emitter/src/ir/inheritance.ts @@ -1,4 +1,5 @@ import { LoadAssignment, TypeDecl, WireDecl } from "./declarations.js"; +import { shouldRejectAbsentNullableKey } from "./field-emission-policy.js"; function mergeInheritedByKey(groups: T[][], key: (item: T) => string): T[] { const order: string[] = []; @@ -111,6 +112,7 @@ export function inheritedRequiredNullableAssignments( const own = new Set(type.load.assignments.map((a) => a.fieldName)); const [flattened] = flattenInheritance([type], declarationUniverse); return flattened.load.assignments.filter( - (assignment) => assignment.isNullable === true && !own.has(assignment.fieldName), + (assignment) => + shouldRejectAbsentNullableKey(assignment) && !own.has(assignment.fieldName), ); } diff --git a/packages/typra-emitter/src/ir/lower.ts b/packages/typra-emitter/src/ir/lower.ts index cfc1e8d..a49c8b8 100644 --- a/packages/typra-emitter/src/ir/lower.ts +++ b/packages/typra-emitter/src/ir/lower.ts @@ -391,6 +391,9 @@ function lowerField( // key; `x?: T | null` lowers exactly like `x?: T` (#339). isOptional: prop.isOptional || prop.isNullable, ...(prop.isNullable && !prop.isOptional ? { isNullable: true } : {}), + ...(prop.isNullable && !prop.isOptional && prop.hasNullDefault + ? { hasNullDefault: true } + : {}), ...(prop.isNullable && prop.isOptional ? { isOptionalNullable: true } : {}), defaultValue: prop.defaultValue, hasExplicitDefault: prop.hasExplicitDefault, @@ -464,6 +467,7 @@ function lowerLoad( category: f.category, isOptional: f.isOptional, ...(f.isNullable ? { isNullable: true } : {}), + ...(f.hasNullDefault ? { hasNullDefault: true } : {}), ...(f.isOptionalNullable ? { isOptionalNullable: true } : {}), hasExplicitDefault: f.hasExplicitDefault, parentTypeName: node.typeName.name, diff --git a/packages/typra-emitter/src/languages/csharp/emitter.ts b/packages/typra-emitter/src/languages/csharp/emitter.ts index 6974f35..aff0d66 100644 --- a/packages/typra-emitter/src/languages/csharp/emitter.ts +++ b/packages/typra-emitter/src/languages/csharp/emitter.ts @@ -43,6 +43,7 @@ import { shouldGuardMissingRequiredField, shouldMaterializeCollectionDefault, shouldOmitAbsentOnSave, + shouldRejectAbsentNullableKey, } from "../../ir/field-emission-policy.js"; import { orderedEntryShorthandCases, @@ -802,7 +803,7 @@ function emitLoadMethod( const field = type.fields.find( (candidate) => candidate.name === assign.fieldName, ); - if (isRequiredNullable(assign)) { + if (shouldRejectAbsentNullableKey(assign)) { // Required nullable (#339): the key must be present; an explicit null is accepted. lines.push( ` if (!data.ContainsKey(${csStringLiteral(assign.sourceName)}))`, diff --git a/packages/typra-emitter/src/languages/go/emitter.ts b/packages/typra-emitter/src/languages/go/emitter.ts index 52dc529..58b05a7 100644 --- a/packages/typra-emitter/src/languages/go/emitter.ts +++ b/packages/typra-emitter/src/languages/go/emitter.ts @@ -47,6 +47,7 @@ import { shouldMaterializeCollectionDefault, shouldOmitAbsentOnSave, shouldPreserveOptionalAbsence, + shouldRejectAbsentNullableKey, } from "../../ir/field-emission-policy.js"; import { INTEGRAL_SCALAR_TYPES, @@ -163,7 +164,8 @@ export function emitGoFileContent( type.serialized && type.fields.some( (field) => - shouldGuardMissingRequiredField(field) || isRequiredNullable(field), + shouldGuardMissingRequiredField(field) || + shouldRejectAbsentNullableKey(field), ), ); const needsFmt = @@ -666,7 +668,7 @@ function emitLoadFunction( const field = type.fields.find( (candidate) => candidate.name === assign.fieldName, ); - if (isRequiredNullable(assign)) { + if (shouldRejectAbsentNullableKey(assign)) { // Required nullable (#339): the key must be present; an explicit nil is accepted. body.push( `\t\tif _, exists := m[${goLiteral(assign.sourceName)}]; !exists {`, diff --git a/packages/typra-emitter/src/languages/java/emitter.ts b/packages/typra-emitter/src/languages/java/emitter.ts index 5981ab3..d1ed841 100644 --- a/packages/typra-emitter/src/languages/java/emitter.ts +++ b/packages/typra-emitter/src/languages/java/emitter.ts @@ -20,6 +20,7 @@ import { shouldGuardMissingRequiredField, shouldMaterializeCollectionDefault, shouldOmitAbsentOnSave, + shouldRejectAbsentNullableKey, } from "../../ir/field-emission-policy.js"; import { orderedEntryShorthandCases, @@ -637,7 +638,7 @@ function emitLoadBaseInto( (helper) => helper.propertyName === assignment.fieldName && helper.hasNameProperty, ); - if (isRequiredNullable(assignment)) { + if (shouldRejectAbsentNullableKey(assignment)) { // Required nullable (#339): the key must be present; an explicit null is accepted. const wireName = escapeJava(assignment.sourceName); lines.push(` if (!map.containsKey("${wireName}")) {`); diff --git a/packages/typra-emitter/src/languages/python/emitter.ts b/packages/typra-emitter/src/languages/python/emitter.ts index fcbe1f8..ec21c2d 100644 --- a/packages/typra-emitter/src/languages/python/emitter.ts +++ b/packages/typra-emitter/src/languages/python/emitter.ts @@ -45,6 +45,7 @@ import { shouldGuardMissingRequiredField, shouldMaterializeCollectionDefault, shouldOmitAbsentOnSave, + shouldRejectAbsentNullableKey, } from "../../ir/field-emission-policy.js"; import { orderedEntryShorthandCases, @@ -1154,7 +1155,7 @@ function emitLoadMethod( const field = type.fields.find( (candidate) => candidate.name === a.fieldName, ); - if (isRequiredNullable(a)) { + if (shouldRejectAbsentNullableKey(a)) { // Required nullable (#339): the key must be present; an explicit None is accepted. lines.push(` if ${pythonLiteral(a.sourceName)} not in data:`); lines.push( diff --git a/packages/typra-emitter/src/languages/rust/emitter.ts b/packages/typra-emitter/src/languages/rust/emitter.ts index c0f7ffa..edab64f 100644 --- a/packages/typra-emitter/src/languages/rust/emitter.ts +++ b/packages/typra-emitter/src/languages/rust/emitter.ts @@ -67,6 +67,7 @@ import { shouldMaterializeCollectionDefault, shouldOmitAbsentOnSave, shouldPreserveOptionalAbsence, + shouldRejectAbsentNullableKey, } from "../../ir/field-emission-policy.js"; import { INTEGRAL_SCALAR_TYPES, @@ -1300,7 +1301,7 @@ function emitFieldInputValidation( const helper = type.collectionHelpers.find( (candidate) => candidate.propertyName === assignment.fieldName, ); - if (isRequiredNullable(assignment)) { + if (shouldRejectAbsentNullableKey(assignment)) { // Required nullable (#339): the key must be present; an explicit null is accepted. lines.push( `${indent}if value.as_object().map_or(false, |object| !object.contains_key(${rustStringLiteral(field)})) {`, diff --git a/packages/typra-emitter/src/languages/swift/emitter.ts b/packages/typra-emitter/src/languages/swift/emitter.ts index 96bdea4..4037d5d 100644 --- a/packages/typra-emitter/src/languages/swift/emitter.ts +++ b/packages/typra-emitter/src/languages/swift/emitter.ts @@ -17,6 +17,7 @@ import { shouldGuardMissingRequiredField, shouldMaterializeCollectionDefault, shouldOmitAbsentOnSave, + shouldRejectAbsentNullableKey, } from "../../ir/field-emission-policy.js"; import { ExprVisitor } from "../../ir/visitor.js"; import { flattenInheritance } from "../../ir/inheritance.js"; @@ -295,7 +296,7 @@ function emitPolymorphicEnum( // `.unknown` carrier: the key must be present (null allowed), as in every // other runtime's unknown-discriminator path. const nullableBaseKeys = type.load.assignments - .filter((assignment) => isRequiredNullable(assignment)) + .filter((assignment) => shouldRejectAbsentNullableKey(assignment)) .map((assignment) => assignment.sourceName); if (nullableBaseKeys.length === 0) { lines.push(" default: return .unknown(object)"); @@ -623,7 +624,7 @@ function emitLoad( helper.propertyName === assignment.fieldName && supportsNamedCollectionHelper(helper, polymorphicTypeNames), ); - if (isRequiredNullable(assignment)) { + if (shouldRejectAbsentNullableKey(assignment)) { // Required nullable (#339): the key must be present; an explicit null is accepted. lines.push( ` if object[${swiftStringLiteral(assignment.sourceName)}] == nil {`, diff --git a/packages/typra-emitter/src/languages/typescript/emitter.ts b/packages/typra-emitter/src/languages/typescript/emitter.ts index 81f5ac3..f40a0c9 100644 --- a/packages/typra-emitter/src/languages/typescript/emitter.ts +++ b/packages/typra-emitter/src/languages/typescript/emitter.ts @@ -51,6 +51,7 @@ import { shouldGuardMissingRequiredField, shouldMaterializeCollectionDefault, shouldOmitAbsentOnSave, + shouldRejectAbsentNullableKey, } from "../../ir/field-emission-policy.js"; import { orderedEntryShorthandCases, @@ -1165,7 +1166,7 @@ function emitLoadMethod( const field = type.fields.find( (candidate) => candidate.name === a.fieldName, ); - if (isRequiredNullable(a)) { + if (shouldRejectAbsentNullableKey(a)) { // Required nullable (#339): the key must be present; an explicit null is accepted. lines.push( ` if (data[${JSON.stringify(a.sourceName)}] === undefined) {`, diff --git a/packages/typra-emitter/test/nullable.test.ts b/packages/typra-emitter/test/nullable.test.ts index 9e51f24..a5e753b 100644 --- a/packages/typra-emitter/test/nullable.test.ts +++ b/packages/typra-emitter/test/nullable.test.ts @@ -1,6 +1,7 @@ // Issue #339: `x: T | null` is a REQUIRED key whose value may be null — load rejects an // absent key but accepts `null`, and save always writes the key (as `null` when unset) — // distinct from `x?: T` / `x?: T | null`, which are omitted when unset. +// `x: T | null = null` keeps the save rule but loads an absent key as null. import { after, before, describe, it } from "node:test"; import assert from "node:assert/strict"; @@ -198,7 +199,10 @@ describe("required nullable fields `x: T | null` (#339)", () => { it("Go uses pointers without omitempty and always writes the key", () => { const source = model("go", /^row\.go$/); - assert.match(source, /Caption \*string\s+`json:"caption" yaml:"caption"`/); + assert.match( + source, + /Caption\s+\*string\s+`json:"caption" yaml:"caption"`/, + ); assert.match(source, /`json:"hint,omitempty" yaml:"hint,omitempty"`/); for (const key of NULLABLE) { assert.match(source, new RegExp(`json:"${key}" yaml:"${key}"`)); @@ -580,4 +584,258 @@ describe("required nullable fields `x: T | null` (#339)", () => { } } }); + + // `x: T | null = null` (lenient load, strict save): an absent key loads as null, so no + // missing-key guard is emitted, yet save still always writes the key as null. + describe("required nullable with a `null` default", () => { + const RUNTIMES: Array<{ + dirs: string[]; + row: RegExp; + shape: RegExp; + guard: (key: string) => RegExp; + save: (key: string) => RegExp; + }> = [ + { + dirs: ["typescript", "typescript-zod"], + row: /^row\.ts$/, + shape: /^shape\.ts$/, + guard: (k) => new RegExp(`data\\["${k}"\\] === undefined\\) \\{`), + save: (k) => new RegExp(`result\\["${k}"\\] = null;`), + }, + { + dirs: ["python", "python-pydantic"], + row: /_Row\.py$/, + shape: /_Shape\.py$/, + guard: (k) => new RegExp(`"${k}" not in data`), + save: (k) => new RegExp(`result\\["${k}"\\] = None`), + }, + { + dirs: ["go"], + row: /^row\.go$/, + shape: /^shape\.go$/, + guard: (k) => new RegExp(`m\\["${k}"\\]; !exists`), + save: (k) => new RegExp(`result\\["${k}"\\] = nil`), + }, + { + dirs: ["csharp"], + row: /^Row\.cs$/, + shape: /^(Shape|Circle|Square)\.cs$/, + guard: (k) => new RegExp(`!data\\.ContainsKey\\("${k}"\\)`), + save: (k) => new RegExp(`result\\["${k}"\\] = null;`), + }, + { + dirs: ["java", "java-jackson"], + row: /(^|\/)Row\.java$/, + shape: /(^|\/)(Shape|Circle|Square)\.java$/, + guard: (k) => new RegExp(`!map\\.containsKey\\("${k}"\\)`), + save: (k) => new RegExp(`result\\.put\\("${k}", null\\)`), + }, + { + dirs: ["rust", "rust-serde"], + row: /(^|\/)row\.rs$/, + shape: /(^|\/)shape\.rs$/, + guard: (k) => new RegExp(`!object\\.contains_key\\("${k}"\\)`), + save: (k) => + new RegExp( + `result\\.insert\\("${k}"\\.to_string\\(\\), serde_json::Value::Null\\)`, + ), + }, + { + dirs: ["swift", "swift-codable"], + row: /^Sources\/.*\/row\.swift$/, + shape: /^Sources\/.*\/shape\.swift$/, + guard: (k) => new RegExp(`object\\["${k}"\\] == nil`), + save: (k) => new RegExp(`result\\["${k}"\\] = NSNull\\(\\)`), + }, + ]; + + for (const { dirs, row, shape, guard, save } of RUNTIMES) { + for (const dir of dirs) { + it(`${dir} skips the missing-key guard but always writes the key`, () => { + const rowSource = model(dir, row); + const shapeSource = model(dir, shape); + // The plain required nullable keeps its guard alongside. + assert.match(rowSource, guard("caption"), `${dir} caption guard`); + assert.match(shapeSource, guard("label"), `${dir} label guard`); + for (const key of ["legacy", "legacyDetail"]) { + assert.doesNotMatch(rowSource, guard(key), `${dir} ${key} guard`); + assert.match(rowSource, save(key), `${dir} ${key} save`); + } + // Inherited through the discriminated base, the variants, and (Swift) the + // unknown-discriminator carrier. + assert.doesNotMatch(shapeSource, guard("legacyLabel"), dir); + assert.match(shapeSource, save("legacyLabel"), dir); + }); + } + } + + it("keeps the TypeScript declaration and zod schema nullable", () => { + for (const dir of ["typescript", "typescript-zod"]) { + assert.match(model(dir, /^row\.ts$/), /legacy: string \| null = null;/); + } + // zod describes the canonical saved form, where the key is always present. + assert.match( + model("typescript-zod", /^row\.ts$/), + /legacy: z\.string\(\)\.nullable\(\),/, + ); + assert.match( + model("typescript-zod", /^shape\.ts$/), + /legacyLabel: z\.string\(\)\.nullable\(\),/, + ); + }); + + it("Go does not import an unused fmt when no key is guarded", () => { + assert.doesNotMatch(model("go", /^legacy_only\.go$/), /"fmt"/); + }); + + // Executed against a small dedicated spec rather than the fixture: the fixture's + // `mood: Mood | null` hits an unrelated Python literal-alias/field name collision + // (`mood: mood | None = None`) that makes its package un-importable. + it("Python loads an absent key as None and saves it as null", (t) => { + if (spawnSync("uv", ["--version"]).status !== 0) { + t.skip("uv toolchain not available"); + return; + } + const dir = mkdtempSync(path.join(process.cwd(), "tmp-null-default-py-")); + try { + const source = path.join(dir, "main.tsp"); + writeFileSync( + source, + [ + 'import "@typra/emitter";', + "namespace Typra.Fixtures.NullDefaultProbe;", + "@serializable", + "model Root {", + " caption: string | null;", + " legacy: string | null = null;", + " legacyDetail: Detail | null = null;", + " shape: Shape;", + "}", + "model Detail { label: string; }", + '@discriminator("kind")', + "model Shape {", + " kind: string;", + " legacyLabel: string | null = null;", + "}", + 'model Circle extends Shape { kind: "circle"; radius: int32; }', + "", + ].join("\n"), + ); + const output = compile( + dir, + source, + "Typra.Fixtures.NullDefaultProbe.Root", + [ + ["Python", "python"], + ["Python", "python-pydantic", "pydantic"], + ], + ); + assert.match(output, /Compilation completed successfully/, output); + const script = path.join(dir, "probe.py"); + writeFileSync( + script, + [ + "import importlib.util, json, os, sys", + "pkg = sys.argv[1]", + 'spec = importlib.util.spec_from_file_location("probe_pkg", os.path.join(pkg, "__init__.py"), submodule_search_locations=[pkg])', + "m = importlib.util.module_from_spec(spec)", + 'sys.modules["probe_pkg"] = m', + "spec.loader.exec_module(m)", + 'circle = {"kind": "circle", "radius": 1}', + "try:", + ' m.Root.load({"shape": circle})', + " missing = None", + "except Exception as e:", + " missing = str(e)", + "print(json.dumps({", + ' "absent": m.Root.load({"caption": None, "shape": circle}).save(),', + ' "kept": m.Root.load({"caption": "c", "legacy": "x", "legacyDetail": {"label": "d"}, "shape": {**circle, "legacyLabel": "l"}}).save(),', + ' "missingCaption": missing,', + "}))", + "", + ].join("\n"), + ); + for (const target of ["python", "python-pydantic"]) { + const result = spawnSync( + "uv", + [ + "run", + "--python", + "3.12", + "--with", + "pydantic", + "--with", + "pytest", + "--with", + "PyYAML", + "python", + script, + path.join(dir, "generated", target), + ], + { encoding: "utf8" }, + ); + assert.equal( + result.status, + 0, + `${target}: ${result.stdout}${result.stderr}`, + ); + const probe = JSON.parse(result.stdout.trim().split("\n").pop()!); + // Absent keys load as null and are still written as null. + assert.deepEqual( + probe.absent, + { + caption: null, + legacy: null, + legacyDetail: null, + shape: { kind: "circle", radius: 1, legacyLabel: null }, + }, + target, + ); + assert.deepEqual( + probe.kept, + { + caption: "c", + legacy: "x", + legacyDetail: { label: "d" }, + shape: { kind: "circle", radius: 1, legacyLabel: "l" }, + }, + target, + ); + // A plain required nullable still rejects an absent key. + assert.match(String(probe.missingCaption), /missing required field/); + } + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("still rejects a `null` default on an optional field", () => { + const dir = mkdtempSync(path.join(process.cwd(), "tmp-null-default-")); + try { + const source = path.join(dir, "main.tsp"); + writeFileSync( + source, + [ + 'import "@typra/emitter";', + "namespace Typra.Fixtures.NullDefault;", + "@serializable", + "model Root {", + " hint?: string | null = null;", + "}", + "", + ].join("\n"), + ); + const result = compile(dir, source, "Typra.Fixtures.NullDefault.Root", [ + ["TypeScript", "typescript"], + ]); + assert.match(result, /typra-emitter-unsupported-complex-default/); + assert.match( + result, + /only supported on a required nullable property \(`x: T \| null = null`\)/, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + }); }); From bcf66aa1aba4075f9fa5711f70cbd6cd604798af Mon Sep 17 00:00:00 2001 From: Seth Juarez Date: Fri, 16 Oct 2026 01:09:49 -0700 Subject: [PATCH 2/8] fix(swift): preserve nullable defaults in unknown discriminator saves Materialize absent required-nullable base keys without overwriting existing values or unknown payload fields. Keep optional fields and variant-only fields unchanged. Add executable absent/null/populated conformance across ten runtimes and dedicated Swift/Codable unknown-kind red-green regressions. Scope inherited Rust coverage to typed polymorphic collections; raw single-reference forwarding is unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 25c59b95-6ae9-4775-bd14-86a11b625d8d --- .../fixtures/integration/main.tsp | 35 +++++++++ .../scripts/fixtures/conformance.mjs | 76 +++++++++++++++++++ .../scripts/fixtures/targets/swift.mjs | 39 +++++++++- .../src/languages/swift/emitter.ts | 16 +++- packages/typra-emitter/test/nullable.test.ts | 10 +++ 5 files changed, 173 insertions(+), 3 deletions(-) diff --git a/packages/typra-emitter/fixtures/integration/main.tsp b/packages/typra-emitter/fixtures/integration/main.tsp index 578b4ce..f400c74 100644 --- a/packages/typra-emitter/fixtures/integration/main.tsp +++ b/packages/typra-emitter/fixtures/integration/main.tsp @@ -85,6 +85,41 @@ model FixtureRoot { @sample(#{ passthroughEvents: #[#{ kind: "click", event_id: "event-1", x: 3 }] }) @doc("Polymorphic models whose base preserves undeclared fields (#336)") passthroughEvents: FixturePassthroughEvent[]; + + @sample(#{ nullDefaults: #[#{ requiredText: null }] }) + @doc("Absent nullable defaults become explicit nulls on save; ordinary required keys remain required") + nullDefaults: FixtureNullDefaults[]; +} + +model FixtureNullDefaults { + requiredText: string | null; + + @encodedName("application/json", "legacy_text") + legacyText: string | null = null; + legacyCount: uint32 | null = null; + legacyEnabled: boolean | null = null; + legacyPayload: unknown | null = null; + legacyState: FixtureNullDefaultState | null = null; + legacyOwner: FixtureOwner | null = null; + legacyTags: string[] | null = null; + legacyOwners: FixtureOwner[] | null = null; + legacyLabels: Record | null = null; + shapes: FixtureNullDefaultShape[] | null = null; + note?: string; +} + +alias FixtureNullDefaultState = "ready" | "archived"; + +@discriminator("kind") +model FixtureNullDefaultShape { + kind: string; + label: string | null; + legacyLabel: string | null = null; +} + +model FixtureNullDefaultCircle extends FixtureNullDefaultShape { + kind: "circle"; + legacyRadius: uint32 | null = null; } @doc(""" diff --git a/packages/typra-emitter/scripts/fixtures/conformance.mjs b/packages/typra-emitter/scripts/fixtures/conformance.mjs index 5e37b99..e38a68f 100644 --- a/packages/typra-emitter/scripts/fixtures/conformance.mjs +++ b/packages/typra-emitter/scripts/fixtures/conformance.mjs @@ -206,6 +206,58 @@ export const fixtureRootSample = { hoverTarget: { id: "menu" }, }, ], + nullDefaults: [ + { requiredText: null }, + { + requiredText: null, + legacy_text: null, + legacyCount: null, + legacyEnabled: null, + legacyPayload: null, + legacyState: null, + legacyOwner: null, + legacyTags: null, + legacyOwners: null, + legacyLabels: null, + shapes: null, + }, + { + requiredText: "kept", + legacy_text: "kept", + legacyCount: 0, + legacyEnabled: false, + legacyPayload: { opaque: [1, null, { nested: true }] }, + legacyState: "ready", + legacyOwner: { id: "legacy-owner", displayName: "Kept Owner" }, + legacyTags: ["kept"], + legacyOwners: [{ id: "legacy-item", displayName: "Kept Item" }], + legacyLabels: { key: "kept" }, + shapes: [{ kind: "circle", label: null }], + note: "", + }, + { + requiredText: "", + legacy_text: "", + legacyCount: 4294967295, + legacyEnabled: true, + legacyPayload: [], + legacyState: "ready", + legacyOwner: null, + legacyTags: [], + legacyOwners: [], + legacyLabels: {}, + shapes: [{ + kind: "circle", + label: "kept", + legacyLabel: "kept", + legacyRadius: 0, + }], + }, + { + requiredText: null, + shapes: [{ kind: "future", label: null }], + }, + ], }; // One canonical conformance input, embedded into every target program. @@ -260,6 +312,30 @@ export const fixtureRootExpected = { }, emptyForm: {}, }, + // Only the input may omit these keys. Canonical saved output must contain them, + // including inherited defaults on both known and unknown discriminator values. + nullDefaults: fixtureRootSample.nullDefaults.map((entry) => ({ + legacy_text: null, + legacyCount: null, + legacyEnabled: null, + legacyPayload: null, + legacyState: null, + legacyOwner: null, + legacyTags: null, + legacyOwners: null, + legacyLabels: null, + shapes: null, + ...entry, + ...(entry.shapes + ? { + shapes: entry.shapes.map((shape) => ({ + legacyLabel: null, + ...(shape.kind === "circle" ? { legacyRadius: null } : {}), + ...shape, + })), + } + : {}), + })), }; let defaultCorpusCoverageChecked = false; diff --git a/packages/typra-emitter/scripts/fixtures/targets/swift.mjs b/packages/typra-emitter/scripts/fixtures/targets/swift.mjs index 93921b6..74fcf43 100644 --- a/packages/typra-emitter/scripts/fixtures/targets/swift.mjs +++ b/packages/typra-emitter/scripts/fixtures/targets/swift.mjs @@ -465,6 +465,40 @@ final class ConformanceValidateTests: XCTestCase { }` } + func testUnknownKindNullDefaults() throws { + let inputs: [[String: Any]] = [ + ["kind": "future", "label": NSNull(), "extra": ["nested": [1, NSNull(), "kept"]]], + ["kind": "future", "label": NSNull(), "legacyLabel": NSNull(), "extra": false], + ["kind": "future", "label": "label", "legacyLabel": "kept", "extra": ["key": "value"]], + ] + for input in inputs { + ${ + useCodable + ? "let shape = try JSONDecoder().decode(FixtureNullDefaultShape.self, from: JSONSerialization.data(withJSONObject: input))" + : "let shape = try FixtureNullDefaultShape.load(input)" + } + let saved = try shape.save() + var expected = input + if expected["legacyLabel"] == nil { expected["legacyLabel"] = NSNull() } + XCTAssertEqual(try TypraRuntime.jsonString(from: saved), try TypraRuntime.jsonString(from: expected)) + XCTAssertNotNil(saved["legacyLabel"], "absent null-default key must be written") + try assertCodableMatchesTypra(shape, "unknown-kind Codable encode must preserve null defaults and raw fields") + let reloaded = try FixtureNullDefaultShape.load(saved) + XCTAssertEqual(try TypraRuntime.jsonString(from: reloaded.save()), try TypraRuntime.jsonString(from: expected)) + } + for kind in ["future", "circle"] { + let missingRequired: [String: Any] = ["kind": kind] + XCTAssertThrowsError(${ + useCodable + ? "try JSONDecoder().decode(FixtureNullDefaultShape.self, from: JSONSerialization.data(withJSONObject: missingRequired))" + : "try FixtureNullDefaultShape.load(missingRequired)" + }) { error in + let diagnostic = String(describing: error) + XCTAssertTrue(diagnostic.contains("label") && diagnostic.contains("missing required field"), diagnostic) + } + } + } + func testEmitsCanonicalConformancePayload() throws { let propertyCases = try JSONSerialization.jsonObject(with: Data(${propertyCorpusJsonLiteral}.utf8)) as! [[String: Any]] let propertyOutputs = try propertyCases.map { entry -> [String: Any] in @@ -618,7 +652,7 @@ final class ConformanceValidateTests: XCTestCase { const payloadPath = path.join(buildDir, "conformance.json"); try { - execFileSync( + const output = execFileSync( "swift", [ "test", @@ -639,6 +673,7 @@ final class ConformanceValidateTests: XCTestCase { }, ); assertConformanceResult(targetDir, readFileSync(payloadPath, "utf8")); + return output; } catch (error) { const output = `${error.stdout?.toString() ?? ""}${error.stderr?.toString() ?? ""}`.trim(); @@ -654,7 +689,7 @@ final class ConformanceValidateTests: XCTestCase { } export function runSwiftCodableExecutableConformance() { - runSwiftExecutableConformance("swift-codable", true); + return runSwiftExecutableConformance("swift-codable", true); } export function runSwiftVectorConformanceCompile(context) { diff --git a/packages/typra-emitter/src/languages/swift/emitter.ts b/packages/typra-emitter/src/languages/swift/emitter.ts index 4037d5d..0d9f41f 100644 --- a/packages/typra-emitter/src/languages/swift/emitter.ts +++ b/packages/typra-emitter/src/languages/swift/emitter.ts @@ -340,7 +340,21 @@ function emitPolymorphicEnum( lines.push(" return result"); } if (usesUnknownFallback) { - lines.push(" case .unknown(let value): return value"); + const nullableKeys = type.save.assignments + .filter(isRequiredNullable) + .map((assignment) => assignment.targetName); + if (nullableKeys.length === 0) { + lines.push(" case .unknown(let value): return value"); + } else { + lines.push(" case .unknown(let value):"); + lines.push(" var result = value"); + for (const key of nullableKeys) { + lines.push(` if result[${swiftStringLiteral(key)}] == nil {`); + lines.push(` result[${swiftStringLiteral(key)}] = NSNull()`); + lines.push(" }"); + } + lines.push(" return result"); + } } lines.push(" }"); lines.push(" }"); diff --git a/packages/typra-emitter/test/nullable.test.ts b/packages/typra-emitter/test/nullable.test.ts index a5e753b..031603e 100644 --- a/packages/typra-emitter/test/nullable.test.ts +++ b/packages/typra-emitter/test/nullable.test.ts @@ -541,6 +541,16 @@ describe("required nullable fields `x: T | null` (#339)", () => { assert.match(variant, /result\["label"\] = NSNull\(\)/); } assert.match(circle, /result\["center"\] = NSNull\(\)/); + const unknownSave = section(shape, "case .unknown(let value):", "\n }"); + for (const key of ["label", "legacyLabel"]) { + assert.match( + unknownSave, + new RegExp(`if result\\["${key}"\\] == nil \\{\\s*result\\["${key}"\\] = NSNull\\(\\)`), + `${dir} unknown carrier saves absent ${key} as null`, + ); + } + assert.match(unknownSave, /return result/); + assert.doesNotMatch(unknownSave, /result\["center"\]/); } }); }); From 03c8eab2a33c01dca6521786081b336a6bc73af4 Mon Sep 17 00:00:00 2001 From: Seth Juarez Date: Fri, 16 Oct 2026 01:19:22 -0700 Subject: [PATCH 3/8] test(emitter): pin nullable-default shape sample to canonical output Keep the shape sample fully materialized for native serde object-equality round trips. The executable conformance input still omits defaulted keys and independently requires explicit nulls on save. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 25c59b95-6ae9-4775-bd14-86a11b625d8d --- packages/typra-emitter/fixtures/integration/main.tsp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/packages/typra-emitter/fixtures/integration/main.tsp b/packages/typra-emitter/fixtures/integration/main.tsp index f400c74..10b44ba 100644 --- a/packages/typra-emitter/fixtures/integration/main.tsp +++ b/packages/typra-emitter/fixtures/integration/main.tsp @@ -86,7 +86,8 @@ model FixtureRoot { @doc("Polymorphic models whose base preserves undeclared fields (#336)") passthroughEvents: FixturePassthroughEvent[]; - @sample(#{ nullDefaults: #[#{ requiredText: null }] }) + // Shape samples are canonical; executable conformance separately feeds omitted keys. + @sample(#{ nullDefaults: #[#{ requiredText: null, legacy_text: null, legacyCount: null, legacyEnabled: null, legacyPayload: null, legacyState: null, legacyOwner: null, legacyTags: null, legacyOwners: null, legacyLabels: null, shapes: null }] }) @doc("Absent nullable defaults become explicit nulls on save; ordinary required keys remain required") nullDefaults: FixtureNullDefaults[]; } From 21a0e6ae7567ca13ca50a01074fcac4ceb304de8 Mon Sep 17 00:00:00 2001 From: Seth Juarez Date: Fri, 16 Oct 2026 02:07:30 -0700 Subject: [PATCH 4/8] fix(emitter): declare shared string enums once in Go, Swift, and Rust Keep identical enum definitions in the first declaring file and import them elsewhere. Rust crate re-exports stay unchanged, but the owning source module may move with emission order. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 25c59b95-6ae9-4775-bd14-86a11b625d8d --- .../fixtures/features/enums/main.tsp | 4 +- .../fixtures/integration/main.tsp | 4 + .../scripts/fixtures/conformance.mjs | 3 + .../scripts/validate-fixtures.mjs | 27 +- packages/typra-emitter/src/ir/lower.ts | 49 ++++ .../typra-emitter/src/languages/go/driver.ts | 17 +- .../src/languages/rust/driver.ts | 19 +- .../src/languages/swift/driver.ts | 12 +- .../typra-emitter/test/shared-enums.test.ts | 260 ++++++++++++++++++ 9 files changed, 378 insertions(+), 17 deletions(-) create mode 100644 packages/typra-emitter/test/shared-enums.test.ts diff --git a/packages/typra-emitter/fixtures/features/enums/main.tsp b/packages/typra-emitter/fixtures/features/enums/main.tsp index 41af444..37f38d2 100644 --- a/packages/typra-emitter/fixtures/features/enums/main.tsp +++ b/packages/typra-emitter/fixtures/features/enums/main.tsp @@ -29,5 +29,7 @@ union MotionPlanKind { @serializable model MotionPlan { kind: MotionPlanKind = MotionPlanKind.subtlePush; -} + @sample(#{ status: "ready" }) + status?: Status; +} diff --git a/packages/typra-emitter/fixtures/integration/main.tsp b/packages/typra-emitter/fixtures/integration/main.tsp index 10b44ba..0e3fee8 100644 --- a/packages/typra-emitter/fixtures/integration/main.tsp +++ b/packages/typra-emitter/fixtures/integration/main.tsp @@ -107,6 +107,10 @@ model FixtureNullDefaults { legacyLabels: Record | null = null; shapes: FixtureNullDefaultShape[] | null = null; note?: string; + + // Shares FixtureStatus with FixtureRoot, so a target whose files share one namespace + // (Go package, Swift module, Rust re-exports) must still declare the enum only once. + status?: FixtureStatus; } alias FixtureNullDefaultState = "ready" | "archived"; diff --git a/packages/typra-emitter/scripts/fixtures/conformance.mjs b/packages/typra-emitter/scripts/fixtures/conformance.mjs index e38a68f..06fb837 100644 --- a/packages/typra-emitter/scripts/fixtures/conformance.mjs +++ b/packages/typra-emitter/scripts/fixtures/conformance.mjs @@ -234,6 +234,7 @@ export const fixtureRootSample = { legacyLabels: { key: "kept" }, shapes: [{ kind: "circle", label: null }], note: "", + status: "complete", }, { requiredText: "", @@ -326,6 +327,8 @@ export const fixtureRootExpected = { legacyLabels: null, shapes: null, ...entry, + // The shared FixtureStatus enum applies its parse alias here too. + ...(entry.status === "complete" ? { status: "ready" } : {}), ...(entry.shapes ? { shapes: entry.shapes.map((shape) => ({ diff --git a/packages/typra-emitter/scripts/validate-fixtures.mjs b/packages/typra-emitter/scripts/validate-fixtures.mjs index 98898d5..c53d7cd 100644 --- a/packages/typra-emitter/scripts/validate-fixtures.mjs +++ b/packages/typra-emitter/scripts/validate-fixtures.mjs @@ -2619,7 +2619,6 @@ function assertActualGeneratedSurface() { ); assertIncludes( path.join("generated", "fixtures", "rust", "fixture_root.rs"), - "pub enum FixtureStatus", "pub enum FixtureMode", "Self::from_str_ignore_case_opt(s)", "pub fn from_str_ignore_case_opt(s: &str) -> Option", @@ -2670,6 +2669,32 @@ function assertActualGeneratedSurface() { "public protocol EventSink", "func emit(event: Any) async throws", ); + // FixtureRoot and FixtureNullDefaults share FixtureStatus. Targets whose generated files + // share one namespace must declare it in exactly one file and import it elsewhere. + for (const [dir, declaration] of [ + [["rust"], "pub enum FixtureStatus {"], + [["rust-serde"], "pub enum FixtureStatus {"], + [["go"], "type FixtureStatus string"], + [["swift", "Sources", "TypraFixtures"], "public enum FixtureStatus:"], + [["swift-codable", "Sources", "TypraFixtures"], "public enum FixtureStatus:"], + ]) { + assertDeclaredOnce(path.join("generated", "fixtures", ...dir), declaration); + } +} + +function assertDeclaredOnce(relativeDir, declaration) { + const dir = requirePath(relativeDir); + const owners = walkFiles( + dir, + (file) => !path.relative(dir, file).split(path.sep).includes("tests"), + ).filter((file) => readFileSync(file, "utf8").includes(declaration)); + if (owners.length !== 1) { + fail( + `${relativeDir} must declare \`${declaration}\` in exactly one file; found ${owners.length}: ${owners + .map((file) => path.relative(dir, file)) + .join(", ")}`, + ); + } } function assertNoEmptyTargetDirs() { diff --git a/packages/typra-emitter/src/ir/lower.ts b/packages/typra-emitter/src/ir/lower.ts index a49c8b8..65d4a5d 100644 --- a/packages/typra-emitter/src/ir/lower.ts +++ b/packages/typra-emitter/src/ir/lower.ts @@ -879,6 +879,55 @@ function collectEnums(types: TypeDecl[]): EnumDef[] { return Array.from(seen.values()).sort((a, b) => a.name.localeCompare(b.name)); } +/** + * Declare each string-literal enum shared across files once. + * + * `collectEnums` dedupes within a file, which suffices where every file is its own + * namespace (Python, TypeScript). Targets whose files share one namespace (a Go + * package, a Swift module, Rust's glob re-exports) would redeclare the enum, so the + * first file that uses it keeps the declaration and later files that use an identical + * enum import it from there. Same-named enums with different definitions stay put. + */ +export function hoistSharedEnums(files: FileDecl[]): FileDecl[] { + const owners = new Map(); + const signature = (def: EnumDef) => + JSON.stringify([def.values, def.parseAliases, def.isOpen]); + return files.map((file) => { + const enums: EnumDef[] = []; + const imports = file.imports.map((imp) => ({ + ...imp, + names: [...imp.names], + })); + for (const def of file.enums) { + const owner = owners.get(def.name); + if (!owner) { + owners.set(def.name, { file, def }); + enums.push(def); + continue; + } + if (signature(owner.def) !== signature(def)) { + enums.push(def); + continue; + } + const module = owner.file.typeName.name; + const group = owner.file.group ?? ""; + const existing = imports.find( + (imp) => imp.module === module && imp.group === group, + ); + if (existing) { + if (!existing.names.includes(def.name)) { + existing.names.push(def.name); + existing.names.sort(); + } + } else { + imports.push({ module, names: [def.name], group }); + imports.sort((a, b) => a.module.localeCompare(b.module)); + } + } + return { ...file, enums, imports }; + }); +} + // ============================================================================ // Import resolution // ============================================================================ diff --git a/packages/typra-emitter/src/languages/go/driver.ts b/packages/typra-emitter/src/languages/go/driver.ts index 8012a21..59817af 100644 --- a/packages/typra-emitter/src/languages/go/driver.ts +++ b/packages/typra-emitter/src/languages/go/driver.ts @@ -17,6 +17,7 @@ import { toSnakeCase } from "../../ir/utilities.js"; import { TypeRegistry } from "../../ir/expansion.js"; import { GoExprVisitor } from "./visitor.js"; import { + hoistSharedEnums, lowerFile, lowerType, collectPolymorphicTypeNames, @@ -123,6 +124,15 @@ export const generateGo = async ( const declarationUniverse = nodes.map((n) => lowerType(n, registry, polymorphicTypeNames, serializationClosure), ); + // One Go package: a string-literal enum shared across files is declared once. + const rootNodes = nodes.filter((n) => !n.base); + const fileDecls = new Map( + hoistSharedEnums( + rootNodes.map((n) => + lowerFile(n, registry, polymorphicTypeNames, serializationClosure), + ), + ).map((fileDecl, index) => [rootNodes[index], fileDecl]), + ); // Emit context file (LoadContext/SaveContext utilities) const contextContent = emitGoContext({ @@ -140,12 +150,7 @@ export const generateGo = async ( for (const n of nodes) { // Skip child types - they're rendered with their parent if (!n.base) { - const fileDecl = lowerFile( - n, - registry, - polymorphicTypeNames, - serializationClosure, - ); + const fileDecl = fileDecls.get(n)!; // Go stays flat: pass group as a header comment only, no subfolder emission const fileContent = emitGoFileContent( fileDecl.types, diff --git a/packages/typra-emitter/src/languages/rust/driver.ts b/packages/typra-emitter/src/languages/rust/driver.ts index 26bdf39..d54d0be 100644 --- a/packages/typra-emitter/src/languages/rust/driver.ts +++ b/packages/typra-emitter/src/languages/rust/driver.ts @@ -38,6 +38,7 @@ import { isTypedSeamEntry, } from "../../ir/vector.js"; import { + hoistSharedEnums, lowerFile, collectPolymorphicTypeNames, computeSerializationClosure, @@ -240,18 +241,24 @@ export const generateRust = async ( } } + // Rust re-exports every module with `pub use module::*`, so a string-literal enum + // shared across files is declared once and imported by the other modules. + const rootNodes = nodes.filter((n) => !n.base); + const fileDecls = new Map( + hoistSharedEnums( + rootNodes.map((n) => + lowerFile(n, registry, polymorphicTypeNames, serializationClosure), + ), + ).map((fileDecl, index) => [rootNodes[index], fileDecl]), + ); + // Render each base type and its children as a single file, into group subfolder const groupModuleNames = new Map(); // group → module names const testGroupModuleNames = new Map(); // group → test module names for (const n of nodes) { if (!n.base) { const group = n.group || ""; - const fileDecl = lowerFile( - n, - registry, - polymorphicTypeNames, - serializationClosure, - ); + const fileDecl = fileDecls.get(n)!; const fileContent = emitRustFileDecl( fileDecl, visitor, diff --git a/packages/typra-emitter/src/languages/swift/driver.ts b/packages/typra-emitter/src/languages/swift/driver.ts index ef0a134..849ab2f 100644 --- a/packages/typra-emitter/src/languages/swift/driver.ts +++ b/packages/typra-emitter/src/languages/swift/driver.ts @@ -6,6 +6,7 @@ import { BaseTestContext, enumerateTypes, TypeNode } from "../../ir/ast.js"; import { GeneratorOptions, filterNodes } from "../../emitter.js"; import { TypeRegistry } from "../../ir/expansion.js"; import { + hoistSharedEnums, lowerFile, collectPolymorphicTypeNames, computeSerializationClosure, @@ -111,10 +112,15 @@ export const generateSwift = async ( // Serialization is opt-in via `@serializable`: compute the closure once and // thread it so only its members emit load/save. const serializationClosure = computeSerializationClosure(nodes, registry); + // One Swift module: a string-literal enum shared across files is declared once. const fileDecls = new Map( - rootNodes.map((n) => [ - `${n.typeName.namespace}.${n.typeName.name}`, - lowerFile(n, registry, polymorphicTypeNames, serializationClosure), + hoistSharedEnums( + rootNodes.map((n) => + lowerFile(n, registry, polymorphicTypeNames, serializationClosure), + ), + ).map((fileDecl, index) => [ + `${rootNodes[index].typeName.namespace}.${rootNodes[index].typeName.name}`, + fileDecl, ]), ); const declarationUniverse = Array.from(fileDecls.values()).flatMap( diff --git a/packages/typra-emitter/test/shared-enums.test.ts b/packages/typra-emitter/test/shared-enums.test.ts new file mode 100644 index 0000000..24166a3 --- /dev/null +++ b/packages/typra-emitter/test/shared-enums.test.ts @@ -0,0 +1,260 @@ +// A string-literal enum used by models in more than one generated file must be declared +// once in targets whose files share a namespace (a Go package, a Swift module, Rust's +// glob re-exports from mod.rs); otherwise the generated code does not compile. + +import { after, before, describe, it } from "node:test"; +import assert from "node:assert/strict"; +import { spawnSync } from "node:child_process"; +import { createRequire } from "node:module"; +import { + existsSync, + mkdtempSync, + readdirSync, + readFileSync, + rmSync, + statSync, + writeFileSync, +} from "node:fs"; +import path from "node:path"; +import { hoistSharedEnums } from "../src/ir/lower.js"; +import type { EnumDef, FileDecl } from "../src/ir/declarations.js"; + +const require = createRequire(import.meta.url); + +const TARGETS: Array<[type: string, dir: string, native?: string]> = [ + ["Go", "go"], + ["Rust", "rust"], + ["Rust", "rust-serde", "serde"], + ["Swift", "swift"], + ["Swift", "swift-codable", "codable"], +]; + +const SPEC = `import "@typra/emitter"; + +namespace Typra.Fixtures.SharedEnums; + +@serializable +model Root { + first: First; + second: Second; +} + +@parseAlias("ready", #["done"]) +union Status { + draft: "draft"; + ready: "ready"; +} + +alias Tone = "calm" | "loud"; + +model First { + status: Status; + tone: Tone; +} + +model Second { + status?: Status; + tone: Tone; +} +`; + +function tspCli(): string { + const compilerEntry = require.resolve("@typespec/compiler"); + const compilerRoot = path.resolve(path.dirname(compilerEntry), "../.."); + return path.join(compilerRoot, "cmd", "tsp.js"); +} + +function compile(root: string): string { + const source = path.join(root, "main.tsp"); + writeFileSync(source, SPEC); + const config = path.join(root, "tspconfig.yaml"); + const lines = [ + "emit:", + ' - "@typra/emitter"', + "options:", + ' "@typra/emitter":', + ` emitter-output-dir: ${JSON.stringify(path.join(root, "generated"))}`, + ' root-object: "Typra.Fixtures.SharedEnums.Root"', + " deterministic-output: true", + " emit-targets:", + ]; + for (const [type, dir, native] of TARGETS) { + lines.push(` - type: ${type}`); + lines.push( + ` output-dir: ${JSON.stringify(path.join(root, "generated", dir))}`, + ); + lines.push( + ` test-dir: ${JSON.stringify(path.join(root, "generated", dir, "tests"))}`, + ); + lines.push(" format: false"); + if (native) lines.push(` native-serialization: ${native}`); + } + writeFileSync(config, [...lines, ""].join("\n")); + const result = spawnSync( + process.execPath, + [tspCli(), "compile", source, "--config", config], + { cwd: process.cwd(), encoding: "utf8" }, + ); + assert.equal(result.status, 0, `${result.stdout}${result.stderr}`); + return `${result.stdout}${result.stderr}`; +} + +function readTree(dir: string): Map { + const files = new Map(); + const walk = (current: string) => { + for (const entry of readdirSync(current)) { + const full = path.join(current, entry); + if (statSync(full).isDirectory()) walk(full); + else + files.set( + path.relative(dir, full).split(path.sep).join("/"), + readFileSync(full, "utf8"), + ); + } + }; + if (existsSync(dir)) walk(dir); + return files; +} + +describe("string-literal enums shared across generated files", () => { + let root = ""; + let output = ""; + const generated = new Map>(); + + before(() => { + root = mkdtempSync(path.join(process.cwd(), "tmp-shared-enums-")); + output = compile(root); + for (const [, dir] of TARGETS) { + generated.set(dir, readTree(path.join(root, "generated", dir))); + } + }); + + after(() => { + if (root) rmSync(root, { recursive: true, force: true }); + }); + + // Library sources only; generated tests live under tests/. + const declaringFiles = (dir: string, declaration: RegExp) => + [...generated.get(dir)!.entries()] + .filter(([name]) => !name.startsWith("tests/")) + .filter(([, content]) => declaration.test(content)) + .map(([name]) => name); + + it("compiles without diagnostics", () => { + assert.doesNotMatch(output, /error|warning/i, output); + }); + + const DECLARATIONS: Array<[dir: string, enumName: string, pattern: RegExp]> = + []; + for (const enumName of ["Status", "Tone"]) { + DECLARATIONS.push( + ["go", enumName, new RegExp(`^type ${enumName} string$`, "m")], + ["rust", enumName, new RegExp(`^pub enum ${enumName} \\{`, "m")], + ["rust-serde", enumName, new RegExp(`^pub enum ${enumName} \\{`, "m")], + ["swift", enumName, new RegExp(`^public enum ${enumName}\\b`, "m")], + [ + "swift-codable", + enumName, + new RegExp(`^public enum ${enumName}\\b`, "m"), + ], + ); + } + + for (const [dir, enumName, pattern] of DECLARATIONS) { + it(`${dir} declares ${enumName} exactly once`, () => { + assert.deepEqual( + declaringFiles(dir, pattern).length, + 1, + `${enumName} declared in: ${declaringFiles(dir, pattern).join(", ")}`, + ); + }); + } + + for (const dir of ["rust", "rust-serde"]) { + it(`${dir} imports the shared enums from the declaring module`, () => { + const [owner] = declaringFiles(dir, /^pub enum Status \{/m); + const ownerModule = path.posix.basename(owner, ".rs"); + const files = generated.get(dir)!; + const user = [...files.entries()].find( + ([name, content]) => + !name.startsWith("tests/") && + name !== owner && + name.endsWith(".rs") && + /\bstatus: Option/.test(content), + ); + assert.ok(user, "no non-owner module uses Status"); + assert.match( + user[1], + new RegExp(`^use super::${ownerModule}::\\{Status, Tone\\};$`, "m"), + ); + }); + } +}); + +describe("hoistSharedEnums", () => { + const enumDef = (name: string, values: string[]): EnumDef => ({ + name, + values, + parseAliases: {}, + isOpen: false, + }); + const file = ( + name: string, + enums: EnumDef[], + imports: FileDecl["imports"] = [], + ): FileDecl => ({ + typeName: { name, namespace: "Typra.Test" }, + types: [], + imports, + containsAbstract: false, + enums, + group: "", + }); + + it("keeps a same-named enum whose definition differs", () => { + const [first, second] = hoistSharedEnums([ + file("First", [enumDef("Level", ["low", "high"])]), + file("Second", [enumDef("Level", ["a", "b"])]), + ]); + assert.deepEqual( + first.enums.map((def) => def.values), + [["low", "high"]], + ); + assert.deepEqual( + second.enums.map((def) => def.values), + [["a", "b"]], + ); + assert.deepEqual(second.imports, []); + }); + + it("merges a hoisted enum into an existing import of the owning module", () => { + const [, second] = hoistSharedEnums([ + file("First", [enumDef("Level", ["low", "high"])]), + file( + "Second", + [enumDef("Level", ["low", "high"])], + [ + { module: "Alpha", names: ["Alpha"], group: "" }, + { module: "First", names: ["First"], group: "" }, + ], + ), + ]); + assert.deepEqual(second.enums, []); + assert.deepEqual(second.imports, [ + { module: "Alpha", names: ["Alpha"], group: "" }, + { module: "First", names: ["First", "Level"], group: "" }, + ]); + }); + + it("treats differing parse aliases as a different definition", () => { + const aliased = { + ...enumDef("Level", ["low", "high"]), + parseAliases: { low: ["lo"] }, + }; + const [, second] = hoistSharedEnums([ + file("First", [enumDef("Level", ["low", "high"])]), + file("Second", [aliased]), + ]); + assert.deepEqual(second.enums, [aliased]); + }); +}); From 59d8e984d699d672bb716a0d7b169816f9c74d21 Mon Sep 17 00:00:00 2001 From: Seth Juarez Date: Fri, 16 Oct 2026 02:09:43 -0700 Subject: [PATCH 5/8] fix(emitter): preserve enum alias names through nullable unions Recover the alias from the declaring variants, including nested aliases, instead of naming the enum after its property. Pin the Python import failure and alias-sharing behavior in generated output and executable conformance. Generated symbols for Alias | null now use the declared alias name (for example Mood instead of mood); consumers directly referencing the old property-derived symbol must update it. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 25c59b95-6ae9-4775-bd14-86a11b625d8d --- .../fixtures/features/nullable/main.tsp | 2 + .../fixtures/integration/main.tsp | 1 + .../scripts/fixtures/conformance.mjs | 2 +- .../scripts/validate-fixtures.mjs | 5 + packages/typra-emitter/src/ir/ast.ts | 40 +++++--- packages/typra-emitter/test/nullable.test.ts | 98 ++++++++++++++++++- 6 files changed, 132 insertions(+), 16 deletions(-) diff --git a/packages/typra-emitter/fixtures/features/nullable/main.tsp b/packages/typra-emitter/fixtures/features/nullable/main.tsp index cc6697a..c3bcd9b 100644 --- a/packages/typra-emitter/fixtures/features/nullable/main.tsp +++ b/packages/typra-emitter/fixtures/features/nullable/main.tsp @@ -58,6 +58,7 @@ model Square extends Shape { } alias Mood = "calm" | "busy"; +alias Combo = "other" | Mood; model Detail { label: string; @@ -70,6 +71,7 @@ model Row { count: int32 | null; mood: Mood | null; + combo?: Combo | null; detail: Detail | null; tags: string[] | null; details: Detail[] | null; diff --git a/packages/typra-emitter/fixtures/integration/main.tsp b/packages/typra-emitter/fixtures/integration/main.tsp index 0e3fee8..4532677 100644 --- a/packages/typra-emitter/fixtures/integration/main.tsp +++ b/packages/typra-emitter/fixtures/integration/main.tsp @@ -120,6 +120,7 @@ model FixtureNullDefaultShape { kind: string; label: string | null; legacyLabel: string | null = null; + state?: FixtureNullDefaultState; } model FixtureNullDefaultCircle extends FixtureNullDefaultShape { diff --git a/packages/typra-emitter/scripts/fixtures/conformance.mjs b/packages/typra-emitter/scripts/fixtures/conformance.mjs index 06fb837..f86012e 100644 --- a/packages/typra-emitter/scripts/fixtures/conformance.mjs +++ b/packages/typra-emitter/scripts/fixtures/conformance.mjs @@ -232,7 +232,7 @@ export const fixtureRootSample = { legacyTags: ["kept"], legacyOwners: [{ id: "legacy-item", displayName: "Kept Item" }], legacyLabels: { key: "kept" }, - shapes: [{ kind: "circle", label: null }], + shapes: [{ kind: "circle", label: null, state: "archived" }], note: "", status: "complete", }, diff --git a/packages/typra-emitter/scripts/validate-fixtures.mjs b/packages/typra-emitter/scripts/validate-fixtures.mjs index c53d7cd..10c345b 100644 --- a/packages/typra-emitter/scripts/validate-fixtures.mjs +++ b/packages/typra-emitter/scripts/validate-fixtures.mjs @@ -2677,6 +2677,11 @@ function assertActualGeneratedSurface() { [["go"], "type FixtureStatus string"], [["swift", "Sources", "TypraFixtures"], "public enum FixtureStatus:"], [["swift-codable", "Sources", "TypraFixtures"], "public enum FixtureStatus:"], + [["rust"], "pub enum FixtureNullDefaultState {"], + [["rust-serde"], "pub enum FixtureNullDefaultState {"], + [["go"], "type FixtureNullDefaultState string"], + [["swift", "Sources", "TypraFixtures"], "public enum FixtureNullDefaultState:"], + [["swift-codable", "Sources", "TypraFixtures"], "public enum FixtureNullDefaultState:"], ]) { assertDeclaredOnce(path.join("generated", "fixtures", ...dir), declaration); } diff --git a/packages/typra-emitter/src/ir/ast.ts b/packages/typra-emitter/src/ir/ast.ts index 17dbaee..91ffbca 100644 --- a/packages/typra-emitter/src/ir/ast.ts +++ b/packages/typra-emitter/src/ir/ast.ts @@ -1452,19 +1452,24 @@ export const resolveUnionProperty = ( getStateValue(program, StateKeys.parseAliases, union), prop.allowedValues, ); - // Resolve enum type name from union.name (named unions) or alias name (alias statements) - let enumName: string | undefined = union.name; - if (!enumName && union.node) { - const node = union.node as any; - // For aliases: union.node is a UnionExpression whose parent is the AliasStatement - if (node.parent?.id?.sv && typeof node.parent.id.sv === "string") { - // Guard against parent.id being a file path (happens for named union declarations) - const parentId: string = node.parent.id.sv; - if (!parentId.includes("/") && !parentId.includes("\\")) { - enumName = parentId; + // `Alias | null` flattens variants but preserves their declaring union. + // Match the entire non-null variant set so nested aliases keep their own name. + const nonNullVariants = Array.from(union.variants.values()).filter( + (v) => !isNullType(v.type), + ); + const aliasUnion = [...new Set(nonNullVariants.map((v) => v.union))].find( + (candidate) => { + if ( + candidate === union || + candidate.variants.size !== nonNullVariants.length + ) { + return false; } - } - } + const candidateVariants = new Set(candidate.variants.values()); + return nonNullVariants.every((v) => candidateVariants.has(v)); + }, + ); + const enumName = literalUnionName(aliasUnion ?? union); if (enumName) { prop.enumName = enumName; } @@ -1485,6 +1490,17 @@ export const resolveUnionProperty = ( return prop; }; +function literalUnionName(union: Union): string | undefined { + if (union.name) return union.name; + const parent = union.node?.parent; + if (parent && "id" in parent && typeof parent.id?.sv === "string") { + const name = parent.id.sv; + // A script's id is a path, not an enum name. + if (!name.includes("/") && !name.includes("\\")) return name; + } + return undefined; +} + function normalizeParseAliases( program: Program, union: Union, diff --git a/packages/typra-emitter/test/nullable.test.ts b/packages/typra-emitter/test/nullable.test.ts index 031603e..5242c0b 100644 --- a/packages/typra-emitter/test/nullable.test.ts +++ b/packages/typra-emitter/test/nullable.test.ts @@ -153,6 +153,100 @@ describe("required nullable fields `x: T | null` (#339)", () => { assert.doesNotMatch(output, /unsupported-union-types|error/i, output); }); + it("keeps the alias name for an `Alias | null` enum in every target", () => { + for (const dir of ["python", "python-pydantic"]) { + const source = model(dir, /^_Row\.py$/); + assert.match(source, /^Mood = Literal\["calm", "busy"\]$/m, dir); + assert.match(source, /^ {4}mood: Mood \| None\b/m, dir); + assert.doesNotMatch(source, /^mood = /m, dir); + assert.match(source, /^Combo = Literal\["other", "calm", "busy"\]$/m, dir); + assert.match(source, /^ {4}combo: Combo \| None\b/m, dir); + assert.doesNotMatch(source, /^combo = /m, dir); + } + for (const dir of ["typescript", "typescript-zod"]) { + const source = model(dir, /^row\.ts$/); + assert.match(source, /export type Mood = "calm" \| "busy";/, dir); + assert.match(source, /export type Combo = "other" \| "calm" \| "busy";/, dir); + } + for (const name of ["Mood", "Combo"]) { + assert.match(model("go", /^row\.go$/), new RegExp(`^type ${name} string$`, "m")); + for (const dir of ["rust", "rust-serde"]) { + assert.match( + model(dir, /\/row\.rs$|^row\.rs$/), + new RegExp(`^pub enum ${name} \\{$`, "m"), + dir, + ); + } + for (const dir of ["swift", "swift-codable"]) { + assert.match( + model(dir, /^Sources\/.*\/row\.swift$/), + new RegExp(`public var ${name.toLowerCase()}: ${name}\\?`), + dir, + ); + } + for (const dir of ["csharp", "java", "java-jackson"]) { + assert.ok( + [...generated.get(dir)!.keys()].some((file) => + new RegExp(`(^|[\\\\/])${name}\\.(cs|java)$`).test(file), + ), + `${dir} ${name}`, + ); + } + } + }); + + it("Python imports and round-trips nullable aliases, including nested aliases", (t) => { + if (spawnSync("uv", ["--version"]).status !== 0) { + t.skip("uv toolchain not available"); + return; + } + const script = path.join(root, "mood_probe.py"); + writeFileSync( + script, + [ + "import importlib.util, json, os, sys", + "pkg = sys.argv[1]", + 'spec = importlib.util.spec_from_file_location("mood_pkg", os.path.join(pkg, "__init__.py"), submodule_search_locations=[pkg])', + "m = importlib.util.module_from_spec(spec)", + 'sys.modules["mood_pkg"] = m', + "spec.loader.exec_module(m)", + 'row = {key: None for key in ["caption", "count", "detail", "tags", "details", "labels"]}', + "print(json.dumps({", + ' "calm": m.Row.load({**row, "mood": "calm"}).save()["mood"],', + ' "null": m.Row.load({**row, "mood": None}).save()["mood"],', + ' "combo": m.Row.load({**row, "mood": None, "combo": "other"}).save()["combo"],', + "}))", + "", + ].join("\n"), + ); + for (const target of ["python", "python-pydantic"]) { + const result = spawnSync( + "uv", + [ + "run", + "--python", + "3.12", + "--with", + "pydantic", + "--with", + "pytest", + "--with", + "PyYAML", + "python", + script, + path.join(root, "generated", target), + ], + { encoding: "utf8" }, + ); + assert.equal(result.status, 0, `${target}: ${result.stdout}${result.stderr}`); + assert.deepEqual( + JSON.parse(result.stdout.trim().split("\n").pop()!), + { calm: "calm", null: null, combo: "other" }, + target, + ); + } + }); + it("TypeScript requires the key, accepts null, and always writes it", () => { for (const dir of ["typescript", "typescript-zod"]) { const source = model(dir, /^row\.ts$/); @@ -698,9 +792,7 @@ describe("required nullable fields `x: T | null` (#339)", () => { assert.doesNotMatch(model("go", /^legacy_only\.go$/), /"fmt"/); }); - // Executed against a small dedicated spec rather than the fixture: the fixture's - // `mood: Mood | null` hits an unrelated Python literal-alias/field name collision - // (`mood: mood | None = None`) that makes its package un-importable. + // Executed against a small dedicated spec that isolates the null-default contract. it("Python loads an absent key as None and saves it as null", (t) => { if (spawnSync("uv", ["--version"]).status !== 0) { t.skip("uv toolchain not available"); From 1ce76ee697671d5f88ee290a9dedacb5b82b1cef Mon Sep 17 00:00:00 2001 From: Seth Juarez Date: Fri, 16 Oct 2026 02:16:48 -0700 Subject: [PATCH 6/8] fix(csharp): guard nullable string record values before conversion Generate a non-null guard with the record key path instead of dereferencing object?. Preserve non-null and empty strings; reject null entries rather than putting null in IDictionary. Reproduced CS8602 for plain and nullable records. Both generated C# build and executable conformance pass after repair; the full emitter suite passes 931 tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 25c59b95-6ae9-4775-bd14-86a11b625d8d --- .../src/content/docs/mappings/collections.mdx | 6 ++++++ packages/typra-emitter/fixtures/README.md | 2 +- .../fixtures/features/collections/main.tsp | 3 +++ .../fixtures/integration/main.tsp | 8 +++++-- .../scripts/fixtures/conformance.mjs | 5 +++++ .../scripts/fixtures/targets/csharp.mjs | 2 ++ .../src/languages/csharp/emitter.ts | 3 ++- .../typra-emitter/test/csharp-emitter.test.ts | 21 ++++++++++++++++--- 8 files changed, 43 insertions(+), 7 deletions(-) diff --git a/docs/src/content/docs/mappings/collections.mdx b/docs/src/content/docs/mappings/collections.mdx index d68f06c..d48a3ab 100644 --- a/docs/src/content/docs/mappings/collections.mdx +++ b/docs/src/content/docs/mappings/collections.mdx @@ -21,9 +21,15 @@ model FixtureRoot { | `string[]` | Array or list of scalar values. | | `FixtureContent[]` | Array or list of nested model values, including polymorphic dispatch when the item type is discriminated. | | `Record` | Dictionary-shaped metadata with unknown values. | +| `Record` | Dictionary with string values. | Generated tests exercise scalar arrays, nested object arrays, dictionaries, and polymorphic collections from fixture samples. +In C#, `Record` emits `IDictionary`. Loading a null entry +throws an error that includes its key path. `Record | null = null` allows +the whole dictionary to be absent or null, but its entries still cannot be null. +Use `Record` for values that may include null. + ## Unknown-field passthrough A model whose string indexer is `unknown` round-trips fields that it does not declare, so a document diff --git a/packages/typra-emitter/fixtures/README.md b/packages/typra-emitter/fixtures/README.md index c76afb9..efe7fe4 100644 --- a/packages/typra-emitter/fixtures/README.md +++ b/packages/typra-emitter/fixtures/README.md @@ -27,7 +27,7 @@ behavior is genuinely runtime-specific. | --- | --- | --- | | Model shapes | `features/model-shapes/main.tsp` | Required, optional, scalar-defaulted fields, typed records, unknown records, and named references. | | Scalars | `features/scalars/main.tsp` | String, boolean, integer widths, numeric, float32, float64, and unknown values. | -| Collections | `features/collections/main.tsp` | Scalar arrays, model arrays, unknown maps, model maps, and `Record \| Named[]` dual-form collections. | +| Collections | `features/collections/main.tsp` | Scalar arrays, model arrays, unknown maps, string maps (including nullable defaults), model maps, and `Record \| Named[]` dual-form collections. | | Polymorphism | `features/polymorphism/main.tsp` | Closed discriminated unions, abstract open bases, and named open discriminator bases. | | Enums | `features/enums/main.tsp` | Closed string unions, open string unions, and parse-only aliases. | | Samples | `features/samples/main.tsp` | Inline `@sample` payloads and imported constant sample payloads with nested items. | diff --git a/packages/typra-emitter/fixtures/features/collections/main.tsp b/packages/typra-emitter/fixtures/features/collections/main.tsp index 2fad18d..a6b0864 100644 --- a/packages/typra-emitter/fixtures/features/collections/main.tsp +++ b/packages/typra-emitter/fixtures/features/collections/main.tsp @@ -12,6 +12,9 @@ model Root { scalarList: string[]; objectList: CollectionItem[]; unknownMap: Record; + @sample(#{ stringMap: #{ label: "kept", empty: "" } }) + stringMap: Record; + nullableStringMap: Record | null = null; objectMap: Record; dualForm: Record | Named[]; } diff --git a/packages/typra-emitter/fixtures/integration/main.tsp b/packages/typra-emitter/fixtures/integration/main.tsp index 4532677..200e4c1 100644 --- a/packages/typra-emitter/fixtures/integration/main.tsp +++ b/packages/typra-emitter/fixtures/integration/main.tsp @@ -23,7 +23,7 @@ model FixtureRoot { @doc("Dictionary-shaped metadata") metadata?: Record; - @sample(#{ typedRecords: #{ counts: #{ alpha: 1, beta: 2 }, owners: #{ primary: #{ id: "owner-typed-1", displayName: "Typed Owner" } } } }) + @sample(#{ typedRecords: #{ counts: #{ alpha: 1, beta: 2 }, owners: #{ primary: #{ id: "owner-typed-1", displayName: "Typed Owner" } }, names: #{ label: "kept", empty: "" } } }) @doc("Records with scalar and model value types") typedRecords: FixtureTypedRecords; @@ -87,7 +87,7 @@ model FixtureRoot { passthroughEvents: FixturePassthroughEvent[]; // Shape samples are canonical; executable conformance separately feeds omitted keys. - @sample(#{ nullDefaults: #[#{ requiredText: null, legacy_text: null, legacyCount: null, legacyEnabled: null, legacyPayload: null, legacyState: null, legacyOwner: null, legacyTags: null, legacyOwners: null, legacyLabels: null, shapes: null }] }) + @sample(#{ nullDefaults: #[#{ requiredText: null, legacy_text: null, legacyCount: null, legacyEnabled: null, legacyPayload: null, legacyState: null, legacyOwner: null, legacyTags: null, legacyOwners: null, legacyLabels: null, legacyNames: null, shapes: null }] }) @doc("Absent nullable defaults become explicit nulls on save; ordinary required keys remain required") nullDefaults: FixtureNullDefaults[]; } @@ -105,6 +105,7 @@ model FixtureNullDefaults { legacyTags: string[] | null = null; legacyOwners: FixtureOwner[] | null = null; legacyLabels: Record | null = null; + legacyNames: Record | null = null; shapes: FixtureNullDefaultShape[] | null = null; note?: string; @@ -163,6 +164,9 @@ model FixturePassthroughClick extends FixturePassthroughEvent { @doc("Typed record values must preserve their declared value type.") @serializable model FixtureTypedRecords { + @sample(#{ names: #{ label: "kept", empty: "" } }) + names: Record; + @sample(#{ counts: #{ alpha: 1, beta: 2 } }) @doc("A record whose values are declared scalar integers") counts: Record; diff --git a/packages/typra-emitter/scripts/fixtures/conformance.mjs b/packages/typra-emitter/scripts/fixtures/conformance.mjs index f86012e..cf060af 100644 --- a/packages/typra-emitter/scripts/fixtures/conformance.mjs +++ b/packages/typra-emitter/scripts/fixtures/conformance.mjs @@ -14,6 +14,7 @@ export const fixtureRootSample = { version: 1, }, typedRecords: { + names: { label: "kept", empty: "" }, counts: { alpha: 1, beta: 2, @@ -219,6 +220,7 @@ export const fixtureRootSample = { legacyTags: null, legacyOwners: null, legacyLabels: null, + legacyNames: null, shapes: null, }, { @@ -232,6 +234,7 @@ export const fixtureRootSample = { legacyTags: ["kept"], legacyOwners: [{ id: "legacy-item", displayName: "Kept Item" }], legacyLabels: { key: "kept" }, + legacyNames: { key: "kept", empty: "" }, shapes: [{ kind: "circle", label: null, state: "archived" }], note: "", status: "complete", @@ -247,6 +250,7 @@ export const fixtureRootSample = { legacyTags: [], legacyOwners: [], legacyLabels: {}, + legacyNames: {}, shapes: [{ kind: "circle", label: "kept", @@ -325,6 +329,7 @@ export const fixtureRootExpected = { legacyTags: null, legacyOwners: null, legacyLabels: null, + legacyNames: null, shapes: null, ...entry, // The shared FixtureStatus enum applies its parse alias here too. diff --git a/packages/typra-emitter/scripts/fixtures/targets/csharp.mjs b/packages/typra-emitter/scripts/fixtures/targets/csharp.mjs index 97c84c3..989c51c 100644 --- a/packages/typra-emitter/scripts/fixtures/targets/csharp.mjs +++ b/packages/typra-emitter/scripts/fixtures/targets/csharp.mjs @@ -457,6 +457,8 @@ export function runCSharpExecutableConformance() { 'if (reloadedUnknownRecords.RequiredValues["null"] is not null || reloadedUnknownRecords.OptionalValues?["null"] is not null) throw new InvalidOperationException("Record null values must survive load/save/reload");', "unknownRecords.OptionalValues = null;", 'if (unknownRecords.OptionalValues is not null) throw new InvalidOperationException("optional Record must accept an absent dictionary");', + 'try { FixtureTypedRecords.FromJson("""{"names":{"bad":null},"counts":{},"owners":{}}"""); throw new InvalidOperationException("Record accepted a null entry"); } catch (ArgumentException error) { if (!error.Message.Contains("names.bad") || !error.Message.Contains("expected non-null string")) throw; }', + 'try { FixtureNullDefaults.FromJson("""{"requiredText":null,"legacyNames":{"bad":null}}"""); throw new InvalidOperationException("nullable Record accepted a null entry"); } catch (ArgumentException error) { if (!error.Message.Contains("legacyNames.bad") || !error.Message.Contains("expected non-null string")) throw; }', 'var wire = WireOptions.Load(new Dictionary { ["maxOutputTokens"] = 256, ["temperature"] = 0.7 });', 'var imageContent = FixtureContent.Load(new Dictionary { ["kind"] = "image", ["url"] = "https://example.com/fixture.png" });', 'var knownContent = FixtureContent.Load(new Dictionary { ["kind"] = "text", ["value"] = "hello" }).Save();', diff --git a/packages/typra-emitter/src/languages/csharp/emitter.ts b/packages/typra-emitter/src/languages/csharp/emitter.ts index aff0d66..99126a5 100644 --- a/packages/typra-emitter/src/languages/csharp/emitter.ts +++ b/packages/typra-emitter/src/languages/csharp/emitter.ts @@ -1527,7 +1527,8 @@ function csharpDictValueLoadExpr( contextExpr: string, ): string { const csType = CSHARP_TYPE_MAP[valueType]; - if (csType === "string") return `${valueExpr}.ToString()!`; + if (csType === "string") + return `(${valueExpr} ?? throw new ArgumentException(${contextExpr}.Path + ": expected non-null string")).ToString()!`; if (csType && NON_NULLABLE_VALUE_TYPES.has(csType)) { return `Convert.To${CONVERT_MAP[csType]}(${valueExpr})`; } diff --git a/packages/typra-emitter/test/csharp-emitter.test.ts b/packages/typra-emitter/test/csharp-emitter.test.ts index 5598230..95eddba 100644 --- a/packages/typra-emitter/test/csharp-emitter.test.ts +++ b/packages/typra-emitter/test/csharp-emitter.test.ts @@ -114,6 +114,13 @@ describe("C# emitter guarded scalar loads", () => { typeName: { namespace: "Test", name: "Record" }, category: { kind: "dict" as const, valueType: "FixtureOwner" }, }; + const stringField = { + ...field, + name: "names", + typeName: { namespace: "", name: "Record" }, + category: { kind: "dict" as const, valueType: "string" }, + }; + const fields = [field, optionalField, typedField, stringField]; const model: TypeDecl = { typeName: { namespace: "Test", name: "Envelope" }, serialized: true, @@ -121,11 +128,11 @@ describe("C# emitter guarded scalar loads", () => { isAbstract: false, isProtocol: false, description: "", - fields: [field, optionalField, typedField], + fields, coercionProperty: null, load: { coercions: [], - assignments: [field, optionalField, typedField].map((item) => ({ + assignments: fields.map((item) => ({ sourceName: item.name, fieldName: item.name, category: item.category, @@ -141,7 +148,7 @@ describe("C# emitter guarded scalar loads", () => { hasContextHooks: true, }, save: { - assignments: [field, optionalField, typedField].map((item) => ({ + assignments: fields.map((item) => ({ targetName: item.name, fieldName: item.name, category: item.category, @@ -181,6 +188,14 @@ describe("C# emitter guarded scalar loads", () => { /public IDictionary Owners \{ get; set; \} = new Dictionary\(\);/, ); assert.doesNotMatch(source, /IDictionary/); + assert.match(source, /public IDictionary Names/); + assert.ok( + source.includes( + '(kvp.Value ?? throw new ArgumentException(context!.At("names").At(kvp.Key).Path + ": expected non-null string")).ToString()!', + ), + "string record values must be checked before dereferencing", + ); + assert.doesNotMatch(source, /kvp\.Value\.ToString\(\)/); }); it("emits nullable Record values in protocol parameters", () => { From 758ae3918e3726e2a1f80ed271b2adc0c9e82726 Mon Sep 17 00:00:00 2001 From: Seth Juarez Date: Fri, 16 Oct 2026 02:17:20 -0700 Subject: [PATCH 7/8] docs(emitter): clarify enum compatibility and Rust raw references Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 25c59b95-6ae9-4775-bd14-86a11b625d8d --- .../content/docs/mappings/unions-polymorphism.mdx | 15 +++++++++++++++ packages/typra-emitter/fixtures/README.md | 2 +- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/docs/src/content/docs/mappings/unions-polymorphism.mdx b/docs/src/content/docs/mappings/unions-polymorphism.mdx index 42ce79a..4a9188a 100644 --- a/docs/src/content/docs/mappings/unions-polymorphism.mdx +++ b/docs/src/content/docs/mappings/unions-polymorphism.mdx @@ -21,6 +21,16 @@ union FixtureStatus { `@parseAlias` is parse-only. Loading accepts `complete`, but saving emits the canonical TypeSpec value `ready`. +String literal aliases keep their declared name when combined with `null`. +For `alias Mood = "calm" | "busy";`, a field `mood: Mood | null` uses the generated +type `Mood`, not `mood`. Regenerating code produced by 3.0.0 can change these +property-derived type names; update any direct references to the old names. + +Go and Swift declare an enum shared by several models once in their package or +module. Rust also declares it once and imports it into the other model files. +Use Rust's generated module re-exports rather than depending on which model file +owns the enum. + ## Open string unions ```typespec @@ -55,6 +65,11 @@ model ImageContent extends FixtureContent { Typra uses the discriminator field to load and save the right child shape. In the fixture contract, `FixtureRoot.content` and `FixtureRoot.contentItems` exercise both a single polymorphic value and a polymorphic collection. +Rust stores a single polymorphic reference as raw `serde_json::Value`. Saving its +parent preserves that JSON without filling in nested defaults, including absent +`T | null = null` fields. Direct model loading and typed polymorphic collections +do apply those defaults. This limitation is unchanged. + ## Generated shape diff --git a/packages/typra-emitter/fixtures/README.md b/packages/typra-emitter/fixtures/README.md index efe7fe4..feb9f28 100644 --- a/packages/typra-emitter/fixtures/README.md +++ b/packages/typra-emitter/fixtures/README.md @@ -29,7 +29,7 @@ behavior is genuinely runtime-specific. | Scalars | `features/scalars/main.tsp` | String, boolean, integer widths, numeric, float32, float64, and unknown values. | | Collections | `features/collections/main.tsp` | Scalar arrays, model arrays, unknown maps, string maps (including nullable defaults), model maps, and `Record \| Named[]` dual-form collections. | | Polymorphism | `features/polymorphism/main.tsp` | Closed discriminated unions, abstract open bases, and named open discriminator bases. | -| Enums | `features/enums/main.tsp` | Closed string unions, open string unions, and parse-only aliases. | +| Enums | `features/enums/main.tsp` | Closed string unions, open string unions, parse-only aliases, and a union shared across model files. | | Samples | `features/samples/main.tsp` | Inline `@sample` payloads and imported constant sample payloads with nested items. | | Vectors | `features/vectors/main.tsp` | Inline `@vector` payloads, imported vector arrays, success expectations, and error expectations. | | Coercions | `features/coercions/main.tsp` | Scalar-to-model coercions, factories, method stubs, and entry shorthand for named collections. | From 113dc4a3deffddc86f77cb67b02f0d4df12eba87 Mon Sep 17 00:00:00 2001 From: Seth Juarez Date: Fri, 16 Oct 2026 02:30:03 -0700 Subject: [PATCH 8/8] test(emitter): pin nested nullable alias in fixture IR golden The full validation run passed runtime gates but found the newly added Combo field missing from the nullable fixture expected flags. Record its optional-nullable shape. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 25c59b95-6ae9-4775-bd14-86a11b625d8d --- packages/typra-emitter/scripts/validate-fixtures.mjs | 1 + 1 file changed, 1 insertion(+) diff --git a/packages/typra-emitter/scripts/validate-fixtures.mjs b/packages/typra-emitter/scripts/validate-fixtures.mjs index 10c345b..9f8b9ca 100644 --- a/packages/typra-emitter/scripts/validate-fixtures.mjs +++ b/packages/typra-emitter/scripts/validate-fixtures.mjs @@ -855,6 +855,7 @@ function assertFocusedFeatureFixtures() { caption: "required+nullable", count: "required+nullable", mood: "required+nullable", + combo: "optional+nullable", detail: "required+nullable", tags: "required+nullable", details: "required+nullable",