|
| 1 | +# Goal 0046 — Schema evolution: rename/retype/delete without breaking history |
| 2 | + |
| 3 | +Owner-raised 2026-08-13 from a real incident on a commercial |
| 4 | +decision-automation platform used in regulated enterprises: its |
| 5 | +attribute schemas map to physical database columns, so renames are |
| 6 | +forbidden, deletes are undocumented soft-deletes, a mis-chosen field |
| 7 | +type is permanent, and — because workflows reference the single LIVE |
| 8 | +decision definition rather than a version — retiring one decision |
| 9 | +required manually clearing references across ~30 historical workflow |
| 10 | +versions. Research pass delivered same day (agent-run, primary |
| 11 | +sources: Camunda DMN versioning, Confluent Schema Registry |
| 12 | +soft/hard-delete + compatibility modes, Avro/Protobuf evolution |
| 13 | +rules, event-sourcing upcasters, expand/contract migrations; full |
| 14 | +report in the session transcript, verdicts summarized here). |
| 15 | + |
| 16 | +## Verified Mill baseline (from code, not assumed) |
| 17 | + |
| 18 | +- SAFE ALREADY: workflow-owned `Attributes` freeze into each |
| 19 | + published `WorkflowVersion` (ADR-0021 `SnapshotHead`); run history |
| 20 | + stores attribute values schema-on-read (`map[string]any` in the |
| 21 | + run's own JSON), so live-schema edits can never corrupt old runs. |
| 22 | +- THE GAP: Configure entities are live-referenced. A |
| 23 | + `decision-outcome` node stores only `decisionId` and resolves |
| 24 | + against the CURRENT `Decision.Outputs` on every execution — even |
| 25 | + when the workflow itself is version-pinned. `DeleteDecision` has no |
| 26 | + reference-integrity check (dangling IDs fail at run time). |
| 27 | + `typedfield.Field.Key` is simultaneously identity and name (a |
| 28 | + rename is structurally delete+add); `Field.Label` exists but isn't |
| 29 | + the guaranteed-renamable surface. Same live-reference class: |
| 30 | + `requestId`/`listId`/`mcpServerId` (ADR-0009 RefKinds). |
| 31 | + `.claude/rules/node-standard.md` already names the sibling gap for |
| 32 | + `NodeType.ConfigFields` (latent, not built). |
| 33 | + |
| 34 | +## Design direction (research-backed sketch — the ADR this goal opens |
| 35 | +with decides it; not pre-resolved here) |
| 36 | + |
| 37 | +1. Identity/label split: `Field.Key` immutable after first save |
| 38 | + (enforcement precedent: `Decision.Category`'s existing |
| 39 | + server-side immutability), `Label` freely renamable — Protobuf |
| 40 | + field-number / Avro-alias insight, near-free since both fields |
| 41 | + exist. |
| 42 | +2. Retype: never in place (no adopted system allows it); "add new |
| 43 | + field + deprecate old" as the guided path, optionally a narrow |
| 44 | + safe-promotion set. |
| 45 | +3. Delete: tombstone-not-drop for schema fields, reusing |
| 46 | + `internal/services/seeding`'s existing tombstone mechanism; hard |
| 47 | + delete of a referenced Configure entity blocked or made soft. |
| 48 | +4. Shared-entity versioning: give `Decision` the same |
| 49 | + Versions/Publish shape `Workflow` already has (ADR-0021 applied |
| 50 | + to a second entity type), with an optional pinned-version config |
| 51 | + on `decision-outcome` (precedent: `child-workflow`'s `version`). |
| 52 | +5. Stamp the resolved definition version onto each run record |
| 53 | + (precedent: `runInput.Version`) — the audit-replay guarantee |
| 54 | + ("what did this decision look like when this run executed") |
| 55 | + preserved exactly, without freezing the live definition. |
| 56 | + |
| 57 | +## Acceptance (checkable) |
| 58 | + |
| 59 | +- [ ] An ADR decides the rename/retype/delete/versioning semantics |
| 60 | + (the sketch above is its input, not its conclusion), including |
| 61 | + which parts apply to ALL RefKind entities vs. Decision first. |
| 62 | +- [ ] The decided semantics implemented with the standing proof |
| 63 | + discipline (unit + seeded-example + e2e per testing.md's |
| 64 | + layering), including a regression proving a version-pinned |
| 65 | + workflow resolves the pinned definition after the live one |
| 66 | + changes. |
| 67 | +- [ ] Reference-integrity at delete time: deleting a still-referenced |
| 68 | + Configure entity is blocked or soft per the ADR — never a |
| 69 | + run-time-only failure. |
| 70 | +- [ ] SPEC.md updated (the typed-system section gains the evolution |
| 71 | + semantics; any OPEN it resolves is surfaced first). |
0 commit comments