Merge pull request #11 from alicoding/goal/0011-lists

Lists maturation: typed datasets + List Search (goal 0011)

Author: alicoding

docs/SPEC.md

@@ -1446,6 +1446,25 @@ resource-inventory prescription. Capability map + build plan: goal
 0011; evidence gaps (lifecycle/versioning, fuzzy semantics,
 first-match schema behavior) recorded there, not guessed at.
 
+**Update — goal 0011 delivered the core of this review's model,
+`LOCKED`/built; a few items stay named-and-deferred, not guessed
+at.** Typed column schema (reusing ADR-0029's canonical
+`typedfield.Field`, never a fifth vocabulary), system-managed audit
+columns (`Row.CreatedAt`/`UpdatedAt`/`Status`, platform-owned,
+excluding `CreatedBy`/`UpdatedBy` — Mill is single-user forever,
+§3.7), a schema-generated row editor, `list-search` as the workflow
+step (multiple match parameters, exact/fuzzy via an adopted matching
+library — `github.com/hbollon/go-edlib`, never invented — a typed
+Object output, Expired excluded from matching by default with a
+per-step opt-in), and in-place migration of pre-existing key/value
+Lists (`list.MigrateLegacyEntries`) are all built — see §3.3's List
+row for the full writeup. Still open, deliberately deferred: CSV/
+JSON/JSON-Schema row+schema import, a first-match-only toggle's exact
+schema behavior, a per-column Jaro-Winkler override, and full
+per-execution dataset-version snapshotting (today's `list_id` on the
+output Object is the goal's own named minimum evidence bar, not the
+full snapshot this review calls for).
+
 ### 3.2.3 Home/landing-dashboard reference review — design input, `OPEN`
 
 **Fifth owner-supplied reference review (2026-08-10, five screenshots,
@@ -1632,7 +1651,7 @@ Plan step for this as a standing rule.
 | **Parallel Steps** | Fan out to multiple steps concurrently, then join | Graph/fan-in semantics: build. Concurrency execution: DBOS's `Queue`/`WithWorkerConcurrency` (§7) is a plausible real backing mechanism once designed, not hand-rolled goroutine management | ADR-0005 names it, deferred |
 | **Child Workflow** | One workflow invokes another as a step | Graph/node semantics: build. Execution: **adopt** — DBOS (already adopted, §7) has real, native parent/child primitives (`RunWorkflow` called from inside a running workflow auto-tracks `ParentWorkflowID`; a workflow ID is DBOS's own idempotency key), corrected from ADR-0005's original "no library has an opinion" verdict | `LOCKED` — [ADR-0010](adr/0010-child-workflow.md), built |
 | **Integration / Connector node** | Call an external HTTP API, auth'd | Wire protocol: adopt (stdlib `net/http`, via `internal/adapters/httpconnector`). Connector config/credential model: build (`internal/domain/connector`) + adopt (`zalando/go-keyring` via `internal/adapters/credential`) | `LOCKED` (execution) — `internal/domain/connector`'s `Connector{ID, Label, Type, BaseURL, AuthType, Headers}` + a new `integration-http` `NodeType` (`KindProcess`) execute real HTTP calls, resolving `AuthType`/secret into the right header (`X-Api-Key` or `Authorization: Bearer`) via `composition.SetConnectorLookup`'s injected seam (mirrors `TriggerService`'s `Syncer` pattern — the domain package doesn't own connector storage). §4 stays `OPEN` on the Configure-surface UI to author a Connector; see §3.5's own row |
-| **List** (a reusable lookup/reference dataset) | Look up an Attributes value against a named, Configure-authored table, write the match back into Attributes | Build (core domain — no library has an opinion on Mill's own List model; the lookup itself is a plain map read) | `LOCKED` (execution) — `internal/domain/list.List{ID, Label, Entries}` + a new `list-lookup` `NodeType` (`KindProcess`) resolve a `listId` via `composition.SetListLookup` (same injected-seam pattern as Integration/Connector's `SetConnectorLookup`) and write the matched entry into `ExecContext.Attributes[outputKey]`. Not in ADR-0005's original taxonomy at all (a real gap flagged in §3.5) — added here as the first thing built against it. §3.5 stays `OPEN` on the Configure-surface UI to author a List |
+| **List** (a reusable typed tabular dataset) | Look up an Attributes value against a named, Configure-authored table, write the match back into Attributes — either a single exact key (`list-lookup`) or multiple exact/fuzzy match parameters against typed columns (`list-search`) | Build (core domain — no library has an opinion on Mill's own List model; matching itself is a plain map read or, for fuzzy, an adopted library) | `LOCKED` (execution), grown from a flat key/value map to typed columns + rows by goal 0011: `internal/domain/list.List{ID, Label, Description, Columns []typedfield.Field, Rows []Row}` — `Row{ID, Values, CreatedAt, UpdatedAt, Status}` (`Status` is `Active`/`Expired`, a platform-owned audit field, never a user-declared Column; no `CreatedBy`/`UpdatedBy` — Mill is single-user forever, §3.7). `list-lookup` (`KindProcess`) keeps working completely unchanged against a typed List via `list.DeriveEntries` (a flat key/value view over the first two Columns). `list-search` (`KindProcess`) is the richer successor: multiple match parameters (JSON-encoded in one `matchParams` ConfigField, the `inputBindings`/`argumentsJSON` precedent), each a column + a literal-or-`attr:<name>` value + exact/fuzzy match type, AND'd together; fuzzy matching adopts `github.com/hbollon/go-edlib` (MIT) behind `internal/adapters/fuzzymatch`, Damerau-Levenshtein by default (industry research: the most explainable algorithm, and Elasticsearch's/OpenRefine's own default); exact match is always plain string equality, never routed through the fuzzy library. Expired rows are excluded from matching by default, uniform across exact and fuzzy (industry research: the soft-delete/OFAC-sanctions-screening/Informatica-MDM convention), with a per-step `includeExpired` opt-in. Output is a typed Object Attribute (`{results, matched, first_match, match_count, list_id}`) — `list_id` is the goal's own minimum execution-evidence bar (full per-run dataset-version snapshotting stays deferred). Both nodes resolve a `listId` via `composition.SetListLookup` (unchanged seam, now returning `Entries`+`Columns`+`Rows`). Configure's Lists tab (`ConfigureLists.tsx`) authors the typed schema (a flat column editor mirroring `ConfigureAttributes.tsx`) and rows (a schema-generated row editor, type-aware inputs); pre-existing key/value Lists migrate in place on first load (`list.MigrateLegacyEntries`, synthesized `key`/`value` Columns) — was previously `OPEN` on the Configure-surface UI, now closed. CSV/JSON row import, per-column Jaro-Winkler override, and full per-run dataset snapshot/versioning are named, deliberately deferred future work. |
 | **MCP tool call** (§3.6's extension point — call a tool on a Configure-authored MCP server) | Call one tool on a locally-configured MCP server over stdio, replace the payload with its text result | Wire protocol: adopt (`modelcontextprotocol/go-sdk`'s client role, via `internal/adapters/mcpclient`). Server config/CRUD: build, same shape as Connector | `LOCKED` (execution + authoring, end-to-end) — `internal/domain/mcpserver.MCPServer{ID, Label, Command, Args}` + a new `mcp-tool-call` `NodeType` (`KindProcess`) resolve an `mcpServerId` via `composition.SetMCPServerLookup` and call `toolName` with `argumentsJSON`. Verified against a real spawned subprocess (an official MCP reference server via `npx`), not just unit tests — see §3.6 for the full writeup. This is the "add a new capability without a core code change" answer §3.6 set out to find |
 | **AI completion (local Ollama / BYO endpoint)** | Send a configured prompt + the running payload to a user-configured LLM endpoint, write the completion back into the payload — one deterministic call per step, never a loop (§1.1's owner-confirmed invariant, 2026-08-11) | Transport: adopt/reuse (Ollama and OpenAI-compatible endpoints are plain HTTP — candidate is the existing `httpconnector` path or a small dedicated adapter; research pass owed before building). Node/config model: build (the stamped Configure-entity recipe, same shape as MCP Server) | `OPEN` — invariant locked, nothing built; the named next capability after capture (ADR-0030). Local-Ollama variant is zero-egress and works at the bank |
 | **Durable step execution / retry / resume** | Survive the process dying mid-workflow, checkpoint per step, retry transient failures | Adopt (DBOS-Go) | `LOCKED` — ADR-0004 `accepted`, `internal/adapters/execution` + `executionservice.go` built and e2e-verified; a real regression test (`TestResumeAfterFailure_DoesNotReExecuteCheckpointedStep`) proves a checkpointed step doesn't re-execute on resume against a real DBOS SQLite runtime. Since [ADR-0008](adr/0008-single-execution-path.md), this is the *only* execution path — every run is durable, not an opt-in alternative to a plain in-memory Run |
@@ -1972,7 +1991,7 @@ true and isn't what was asked for.
 | **Input / Attributes** | **Configure** | **1:1** — scoped to the one workflow that declares it, per §3.2's original cardinality note | `LOCKED` end-to-end — `ConfigureView.tsx`'s Attributes tab (`ConfigureAttributes.tsx`) picks a workflow and edits its declared schema (key/label/type rows, `FieldOptions` excluded — see §3.3's rule-builder Update note for why), calling `ConfigureService.UpdateWorkflowAttributes` |
 | **Branch** (routing — UI-renamed from "Decision: route" by [ADR-0027](adr/0027-decision-terminal-outcome.md); code IDs `KindDecision`/`decision-route` unchanged) | Canvas — conditions live on edges, authored via the rule builder | **1:1** — a workflow's routing logic is that workflow's own; §3.2's "cardinality unconfirmed" flag is now resolved by the split below, not by promoting routing to Configure | `LOCKED` end-to-end — see §3.3's rule-builder writeup (`react-querybuilder` + `ruleTranslate.ts`, one-way translation only) |
 | **Decision** (a reusable, typed **terminal outcome** — a genuinely new concept, not the routing node matured; the reference platform's own semantics: "rulesets route; Decisions terminate") | **Configure**, a Decisions tab — category (approve/deny/manual-review/action-needed/uncategorized, **immutable** after create, server-enforced with Duplicate as the migration path), typed output schema, optional webhook-by-HTTPRequest-reference | **1:many** — one configured Decision referenced by many workflows' terminal nodes via the ADR-0009 picker (`RefKind: "decision"`, quick-create included) | `LOCKED`, built end-to-end — [ADR-0027](adr/0027-decision-terminal-outcome.md) `accepted` 2026-08-10, three owner calls decided directly (Branch rename; webhook reuses the HTTPRequest capability by reference, never a second outbound-HTTP surface; manual-review parks into the existing Review queue). See §3.3's row for the build details |
-| **List** (a reusable lookup/reference dataset) | **Configure** | **1:many** recommended, same shape as Integration — a shared lookup table is the kind of thing multiple workflows would plausibly reference | `LOCKED` end-to-end — `ConfigureView.tsx`'s Lists tab (`ConfigureLists.tsx`) is a real page: create/edit/delete a List and its key/value entries, calling `ConfigureService`'s `Lists`/`CreateList`/`UpdateList`/`DeleteList` |
+| **List** (a reusable, typed tabular dataset) | **Configure** | **1:many** — a shared dataset multiple workflows plausibly reference | `LOCKED` end-to-end, grown from key/value to typed by goal 0011 — `ConfigureView.tsx`'s Lists tab (`ConfigureLists.tsx`) authors a Column schema (a flat key/label/type editor, `ConfigureAttributes.tsx`'s own style) and Rows (a schema-generated, type-aware row editor), calling `ConfigureService`'s `Lists`/`CreateList`/`UpdateList`/`DeleteList`/`AddListRow`/`UpdateListRow`/`DeleteListRow`. See §3.3's List row for the full execution-side writeup |
 
 **What Configure is *not*: a plugin system for user-defined node kinds.**
 Worth being explicit about, since "define a dedicated thing in Configure"

docs/goals/BACKLOG.md

@@ -44,6 +44,10 @@ this pipeline and on this code)**
    backend-side `isAway`), alert-style authorization request (notify.Start),
    cross-device forward (`composition.SendJSONWebhook`,
    `ForwardPendingApproval`) — see ADR-0032's Update note
+4. [ ] [0026 — Request lifecycle honesty](0026-request-lifecycle-honesty.md)
+   — withdrawal verb (`cancel_write`, the MCP Tasks `tasks/cancel`
+   precedent ADR-0032 already mirrors) + staleness/expiry presentation
+   (owner-observed 2026-08-11: a 4h-old stale ask reads as breakage)
 
 **Ratified 2026-08-10 (owner): three groups, A→B→C. 0001 stays standing
 live-review material, interleaved during owner reviews, not a lane.**
@@ -63,7 +67,7 @@ live-review material, interleaved during owner reviews, not a lane.**
 **Unscheduled (reorder into a group when prioritized)**
 7. [x] [0012 — Authoring hot-exit](archive/0012-authoring-hot-exit.md) — canvas half delivered 2026-08-10 (scratch persistence + restored-unsaved banner + dirty dots; Configure forms recorded-remaining in the archived file)
 8. [x] [0013 — Canonical type system](archive/0013-canonical-type-system.md) — COMPLETE 2026-08-10 (typedfield leaf pkg; all 4 vocabularies converged incl. openapispec Phase 3; the #1 kernel investment)
-9. [ ] [0011 — Lists maturation](0011-lists-maturation.md) (typed datasets + List Search per SPEC §3.2.2's reference review; evidence-gap research first)
+9. [x] [0011 — Lists maturation](archive/0011-lists-maturation.md) — DELIVERED 2026-08-12 (harvested from a parallel owner session + reconciled onto main: typed Columns/Rows against ADR-0029's canonical typedfield, system-managed audit columns w/ Expired-excluded-by-default, `list-search` node w/ go-edlib fuzzy matching, in-place legacy-List migration; CSV import + full per-run dataset snapshot named-deferred)
 10. [x] [0014 — Home dashboard / value mirror](archive/0014-home-dashboard.md) — delivered 2026-08-10 (Recharts, industry-decided metric semantics, editable minutes-saved, default landing)
 11. [ ] [0015 — Summon quick-invoke](0015-summon-quick-invoke.md) — CORE delivered 2026-08-11 (⌘K palette: commands with inline shortcuts, workflow run, tab jump/close; delegated build); PHASE 2 delivered same day (ADR-0033: the summon hotkey opens a dedicated floating Quick Panel — frameless, floats over fullscreen, Esc/blur dismiss, focus-yield; supersedes "summon opens the main window"). Remainder open: frecency/pins (needs the 0014 usage substrate), Configure entities, pending-review count, ⌘?/⌘/ alias (needs multi-binding registry support)
 12. [x] [0022 — Workflow view mode](archive/0022-workflow-view-mode.md) — delivered 2026-08-11 (row click → read-only canvas w/ Run+step-debug; Edit explicit in-place mode switch; breakpoint dot moved onto the node card, both modes; fixed a latent bug where a policy deny could hide a breakpoint's existence)

docs/goals/0011-lists-maturation.md …s/goals/archive/0011-lists-maturation.mddocs/goals/0011-lists-maturation.md renamed to docs/goals/archive/0011-lists-maturation.md docs/goals/0011-lists-maturation.md renamed to docs/goals/archive/0011-lists-maturation.md

@@ -83,6 +83,61 @@ workflow) proven per the layered-coverage model; the
 List-as-database boundary documented; evidence gaps resolved by
 research or explicitly deferred with reasons.
 
+## Delivered (2026-08-12)
+
+Harvested from a parallel owner session's in-progress worktree
+(`wt-lists`) and landed on `goal/0011-lists`, reconciled against main
+(typedfield's Phase 1/2 convergence, entity-level `CreatedAt`/
+`UpdatedAt`, ADR-0028 validation, confirmed-delete/`InventoryList`
+conventions, and SPEC §3.2.4 all landed on main after the worktree
+branched).
+
+- Item 1 (typed columns + rows): `internal/domain/list.List.Columns
+  []typedfield.Field` / `Rows []Row`, built directly against
+  ADR-0029's canonical vocabulary from day one — no parallel schema
+  system.
+- Item 2 (system-managed columns): `Row{ID, Values, CreatedAt,
+  UpdatedAt, Status}` — `Status` (`Active`/`Expired`) is a
+  platform-owned struct field, never a user-declared Column;
+  `CreatedBy`/`UpdatedBy` deliberately NOT modeled (Mill is
+  single-user forever, §3.7) — the goal's own open call, resolved.
+  Expired rows excluded from matching by default, uniform across
+  exact and fuzzy, with a per-step `includeExpired` opt-in (the
+  industry-research verdict this file recorded).
+- Item 3 (schema/row import): NOT built — CSV/JSON row+schema import
+  stays named, deliberately deferred future work (recorded in
+  SPEC.md §3.2.2's Update note).
+- Item 4 (`list-search`): built as a new `NodeType` alongside
+  `list-lookup` (kept, unchanged, via `list.DeriveEntries`'s
+  first-two-columns view) — multiple match parameters (column +
+  literal-or-`attr:<name>` value + exact/fuzzy, AND'd), fuzzy via
+  `github.com/hbollon/go-edlib` (MIT) behind
+  `internal/adapters/fuzzymatch`, Damerau-Levenshtein default; exact
+  match stays plain equality, never routed through the fuzzy lib.
+  Output is the fixed-by-construction typed Object `{results,
+  matched, first_match, match_count, list_id}`.
+- Item 5 (execution evidence): minimum bar only — `list_id` recorded
+  inline on every `list-search` output. Full per-run dataset-version
+  snapshotting stays deferred, named in SPEC.md, not silently dropped.
+- Item 6 (migration): `list.MigrateLegacyEntries` converts a
+  pre-0011 flat key/value List into synthesized `key`/`value` typed
+  Columns + Rows in place, on first load, idempotently.
+
+Proof: `internal/domain/list`, `internal/domain/typedfield`,
+`internal/domain/composition`, `internal/adapters/fuzzymatch`,
+`internal/services/configuresvc`, `internal/services/executionsvc`
+Go suites green (race + cover); two seeded workflows
+(`example-list-lookup-workflow`, `example-list-search-workflow`)
+against the shared seeded "Example: Country codes" List (typed
+code/name columns, one deliberately Expired row), each proven via a
+real-DBOS Go test AND a `seed-completeness.spec.ts` e2e case,
+registered in `seedproof_test.go`; a dedicated
+`configure-lists.spec.ts` e2e exercises the Configure Column/Row
+editors and the `list-search` node's Inspector
+(`ListSearchParamsEditor.tsx`) live through the canvas; `configure-
+export-import.spec.ts` round-trips a List's typed columns/rows.
+SPEC.md §3.2.2 (Update note), §3.3 (List row), and §3.5 (Configure
+table) all updated in the same change.
 
 ## Design section (research pass delivered 2026-08-12 — full report in session record)
 
@@ -111,9 +166,14 @@ Key verdicts, primary-sourced:
   consistency); SQLite-via-DBOS is the named future trigger the day a
   real four-digit-row List exists — not before.
 
-**GATING DECISION, owner-owned:** `.claude/worktrees/wt-lists` holds a
-near-complete uncommitted implementation of this goal from a parallel
-session, on a branch diverged behind main (pre-§3.2.4, pre-goal-0018).
-Rebase-and-land vs treat-as-scratch must be decided by the owner before
-any build starts — another session's live workspace is never touched
-from this one.
+**GATING DECISION, owner-owned — resolved.** `.claude/worktrees/wt-lists`
+held a near-complete uncommitted implementation of this goal from a
+parallel session, on a branch diverged behind main (pre-§3.2.4,
+pre-goal-0018). Rebase-and-land was the path taken (this file's own
+"Delivered" section above): the worktree's uncommitted work was
+harvested by diff (never checked out or mutated directly) onto a fresh
+`goal/0011-lists` branch off current `main`, then reconciled against
+everything that landed on `main` after the worktree branched — every
+verdict in this Design section (fuzzy library, Row shape, output
+stability, storage, the snapshot reframing) was independently
+cross-checked against the delivered build and matched.

frontend/bindings/github.com/alicoding/mill/internal/domain/list/index.ts

@@ -1,6 +1,11 @@
 // Cynhyrchwyd y ffeil hon yn awtomatig. PEIDIWCH Â MODIWL
 // This file is automatically generated. DO NOT EDIT
 
+export {
+    RowStatus
+} from "./models.js";
+
 export type {
-    List
+    List,
+    Row
 } from "./models.js";