Fix OpenAI strict-mode schema for /api/memory/extract + /api/decide#5
Draft
wdwd720 wants to merge 1 commit into
Draft
Conversation
OpenAI's strict structured outputs require every object schema to:
1. set type="object", properties, additionalProperties:false
2. include EVERY property key in "required"
3. apply the same recursively to nested objects (incl. array items)
The MEMORY_EXTRACTION_SCHEMA's nested item schema only listed 5 of its
12 keys in required, so OpenAI rejected the request with
Invalid schema for response_format 'hearer_response':
In context=('properties','items','items'), 'required' is required to
be supplied
and /api/memory/extract silently fell back to mode='rule_based'.
Changes:
- server/src/llm/llmSchemas.ts:
- new strictObject() helper that auto-derives "required" from the
properties keys so it's impossible to forget a field;
- rebuilt MEMORY_EXTRACTION_SCHEMA with strictObject(), so the inner
item schema now lists kind/label/triggerDescription/actionType/
actionLabel/person/deadlineText/locationLabel/priority/confidence/
save/privacyNote in required (optional fields stay required but
nullable via type:["string","null"] / enum:[...,null]);
- rebuilt CUE_REASONING_SCHEMA the same way;
- new exported assertStrictOpenAiSchema(schema, path?) walks a schema
and throws on the first violation (missing properties, missing
required, missing additionalProperties:false, required vs properties
mismatch, recurses into array items).
- server/src/llm/openAiCompatibleProvider.ts:
- chatJson now takes a schemaName so each schema gets its own name in
the OpenAI request ("hearer_memory_extraction" /
"hearer_cue_reasoning");
- on a non-2xx response we parse OpenAI's error.message + error.code,
print [hearer-llm] provider HTTP <status> on schema='<name>': <msg>
to the server log, and propagate the same compact message in the
LlmError. We never log the API key (it's in the Authorization
header) and we never log the user prompt.
- server/src/__tests__/openAiSchemaStrictness.test.ts (new, 5 tests):
- MEMORY_EXTRACTION_SCHEMA passes assertStrictOpenAiSchema;
- CUE_REASONING_SCHEMA passes assertStrictOpenAiSchema;
- the nested memory item schema's required matches its properties keys
(lists every one of the 12 fields);
- the assertion throws when required is missing a property;
- the assertion throws when additionalProperties is missing;
- the assertion recurses into array items.
- README.md updated with the smoke curl + the expected
mode='llm', fallbackUsed=false response.
Tests: 71/71 passing across 19 suites (was 65/18). Build passes.
Co-authored-by: wdwd720 <wdwd720@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
OpenAI's strict structured outputs require every object schema to list every property key in
required(optional fields stay required but nullable). Our nesteditems[]schema inMEMORY_EXTRACTION_SCHEMAonly listed 5 of its 12 keys inrequired, so the API rejected the request with:Result:
/api/memory/extractsilently fell back tomode: "rule_based", even withenabled=true configured=true provider=openai_compatible.The fix
server/src/llm/llmSchemas.tsstrictObject(properties)helper. It auto-derivesrequiredfromObject.keys(properties)so it's impossible to forget a field. Setstype: "object",additionalProperties: false,required, andproperties.MEMORY_EXTRACTION_SCHEMArebuilt withstrictObject(). The inner item schema now lists all 12 keys (kind, label, triggerDescription, actionType, actionLabel, person, deadlineText, locationLabel, priority, confidence, save, privacyNote) inrequired. Optional fields stay required but use nullable union types (type: ["string", "null"], orenum: [..., null]forpriority).CUE_REASONING_SCHEMArebuilt the same way for safety.assertStrictOpenAiSchema(schema, path?)walks a schema and throws on the first violation (missing properties / required / additionalProperties, mismatch between required and properties keys, recurses into array items).server/src/llm/openAiCompatibleProvider.tschatJsonnow takes aschemaName, so each request gets its own name in the OpenAI payload:"hearer_memory_extraction"/"hearer_cue_reasoning".On a non-2xx response we parse OpenAI's
error.message+error.codeand surface a compact line in the server log:We never log the API key (it's in the Authorization header) and we never log the user transcript.
Tests
server/src/__tests__/openAiSchemaStrictness.test.ts(new):MEMORY_EXTRACTION_SCHEMApassesassertStrictOpenAiSchema.CUE_REASONING_SCHEMApassesassertStrictOpenAiSchema.requiredmatches itspropertieskeys (every one of the 12 fields).requiredis missing a property.additionalPropertiesis missing.README
Smoke commands added with the expected
mode: "llm",fallbackUsed: falseresponse.Results
npm run build→ success.npm test→ 71 / 71 passing across 19 suites (was 65 / 18). All prior tests still pass.After merging this stack