feat: identity-keyed codegen pipeline (manifest-pinned names, multi-doc merge, classified releases) - #32
Merged
Merged
Conversation
Identity-keyed codegen: committed api-surface manifest, multi-doc spec merge, canonicalization, and a classified release gate. Eliminates the class of breakage where JD spec reorders/splits rebind generated method names or silently drop documents.
Entrypoints swallowed thrown errors (main().catch(console.error) exits 0) and fix-specs swallowed per-file failures, so CI could ship stale specs on a green run. Generators embedded wall-clock timestamps, making output nondeterministic. Fail loudly, drop the timestamps, sort the types-index read order, and ignore upcoming scratch files.
Relocate inferMethodName, the order-sensitive collision loop, and the string helpers out of the generate-sdk entrypoint. generateMethod now consumes a precomputed name. Regenerated output is byte-identical; the extracted module exists only to seed the upcoming api-surface manifest and is scheduled for deletion.
Pure library for the committed operation-identity registry: loader with loud validation, deterministic serializer with an embedded runbook, shared op extraction, order-independent name resolution, and a deterministic proposer for new operations (no positional counters). normalizePathPattern moves here so manifest identity and HATEOAS matching share one implementation.
… name
crop-types declares GET /cropTypes/{name} and GET /cropTypes/{id} as
distinct operations, which collide under normalize-all-params identity.
Entries sharing a normalized key are now legal when their raw paths
differ; matching inside such ambiguous groups is by exact path, so a
param rename there surfaces as a breaking diagnostic instead of a
silent mismatch. Single-sibling groups keep absorbing param renames.
Replay the legacy order-sensitive naming over the committed fixed specs (still pre-incident path order) to pin every current public method name to its operation identity. A permanent test loads the manifest and locks the known-critical bindings so neither hand edits nor auto-appends can silently move the published surface.
The generator now resolves every public method name from the committed operation-identity registry instead of inferring it from spec document order. Unknown upstream operations get deterministic proposed names and are appended to the manifest; a manifest operation missing upstream fails the run with a per-entry diagnostic before anything is written. Every run emits sync-report.json (gitignored) for the workflow gate. Regeneration from committed specs is byte-identical, so the published surface is unchanged; the legacy order-sensitive naming, its tests, and the one-time seed script are deleted.
Sorts the paths map and every components category map so semantically null upstream reorders produce byte-identical spec files at the fetch boundary. Stringification matches fix-specs' emission options plus aliasDuplicateObjects: false, so deduplicated merged components can never surface as YAML anchors.
The portal returns multiple documents for 7 of 28 slugs; the fetch keeps element zero and silently drops the rest. Validate every element and structurally merge a slug's documents: repo-owned primary table (never portal array order), method-level path union with deep-equal dedupe, component conflict renames with subtree ref rewriting to a fixpoint, server agreement checks, and source-document stamping. check-api-health now checks every returned document.
fetch-specs validates all returned documents, structurally merges multi-document slugs through the repo-owned primary table, and writes canonicalized YAML so upstream reorders produce byte-identical raw files. The single-document validator is retired; summary.json records the merged document inventory per slug.
Rewrite specs/raw and everything downstream through the canonical paths/components ordering that fetch-specs now emits, so future live fetches diff cleanly and upstream reorders vanish. Every path in every spec changes position while scripts/api-surface.yaml is byte-identical, which is this branch's reorder-immunity proof executed against the real spec corpus.
The sync now runs the full pipeline (redact-specs and generate-api-servers were missing), reads sync-report.json after generation, and acts on the classification: additive changes release a minor version with the new operations listed in the changelog, benign churn keeps patch semantics, and breaking runs fail at the Generate SDK step with manifest remediation guidance in the job summary.
… docs Live portal data showed JD's secondary documents carry servers blocks that differ only as environment instances or documentation defects of the one platform family (api vs partnerapi hosts, missing /platform segments, placeholder hosts, omitted variables). Declaring documents that all belong to the platform family now resolve to the primary's block, which fix-specs normalizes to the templated form downstream, matching what the SDK already ships for these specs. Placeholder-only blocks are treated as non-declaring with a logged warning. Genuinely different server families still refuse to merge.
First live fetch through the multi-document pipeline. The 7 multi-doc slugs now merge every portal document: 15 secondary documents the old fetch silently dropped are merged in, adding 56 operations JD published under those documents (field-operations measurementTypes; the products input families chemicals, fertilizers, dry-blends, tank-mixes, active-ingredients, companies, documents; flags categories and category preferences; map-layers and file-resources; file transfers; webhook delivery; machine-location breadcrumbs). New operations enter scripts/api-surface.yaml with deterministic proposed names; the run classifies additive with zero missing operations. Every previously published method keeps its exact name and binding, verified by a before/after extraction of all 146 generated (class, method, operation) triples: 146 preserved, 0 changed, 56 added. src/api-servers.generated.ts and src/deere.ts are unchanged, so the platform-family servers reconciliation routes nothing differently. equipment.yaml carries an unrelated upstream JD edit (two response schemas changed from a typed array-item ref to a bare array) that weakens two EquipmentApi return types to unknown; no binding changes and the full suite (lint, typecheck, build, 432 tests, 12 fuzz) passes.
JD's 2026-07 equipment doc edit dropped the values.items refs from two responses while leaving the target schemas defined, collapsing EquipmentApi.get and getEquipment returns to unknown. A guarded fix-specs transform restores the refs while both conditions hold and no-ops once JD repairs the doc or removes the schemas. Two unreleased products action names gain their family prefix for symmetry with the fertilizer twins via the manifest's hand-edit point.
…final review Validate the sync classification against its closed set, neutralize newlines in portal-derived strings emitted to workflow outputs, and route classification through step env instead of shell interpolation. Fail generate-sdk when any spec fails to parse (a dropped spec was invisible to the missing-operation detector) and generate-types when any per-file generation fails. Append the execution amendments to the committed design plan so the durable record matches the implementation.
Main's legacy sync went green on 2026-07-02 (upstream reverted the field-operations reorder) and released v2.3.1 carrying the equipment type regression this branch counters. Resolution: canonicalized branch artifacts win for all spec and generated files (main's fresh equipment raw spec is canonically byte-identical to ours, verified), the 2.3.1 changelog entry slots beneath 2.4.0, and the full pipeline regenerated over the merged tree, restoring the typed equipment returns the textual auto-merge had lost. All gates green on the result.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The daily Sync John Deere API workflow was red from 2026-06-24 through 2026-07-01. Root cause: generated method names derived from spec document order. When JD re-published the field-operations spec with reordered paths and a second document,
FieldOperationsApi.getsilently rebound to the shapefile endpoint and the build broke (the near-miss was worse: for callers not passingembed, the rebinding compiles and calls the wrong endpoint). Separately, the fetch kept only portal array element[0], silently dropping 15 of 43 spec documents.What
scripts/api-surface.yaml): every public method name pinned to(HTTP method, normalized path). Upstream reorders are provably inert (the branch canonicalizes every committed spec, changing every path's position, with a byte-identical manifest and generated surface). Removals fail the sync loudly with per-operation remediation; new operations get deterministic proposed names appended automatically.EquipmentApi.get/getEquipmentreturn types to the previously shipped surface.Verification
437 tests (including property tests for order-invariance, merge permutation laws, and the fixpoint rename cascade), lint, both typechecks, build, fuzz: all green. Byte-identity gates at every regeneration; idempotence proven by matching git tree hashes; the reorder-immunity claim executed against the full real spec corpus; 146/146 binding preservation verified by before/after extraction; final whole-branch review passed with its three findings fixed and re-reviewed.
Post-merge follow-ups
npm version minor && git push --follow-tags(CHANGELOG 2.4.0 entry is in this PR; do not patch-bump).detect_changes({scope: "compare", base_ref: "main"})as the deferred repo-mandated check.New operations (the names below become permanent public API on merge)