Skip to content

feat(memory): add approval-gated turn distillation - #1266

Merged
XingYu-Zhong merged 11 commits into
KunAgent:developfrom
SunwardL:codex/memory-distillation-p1b
Sep 6, 2026
Merged

feat(memory): add approval-gated turn distillation#1266
XingYu-Zhong merged 11 commits into
KunAgent:developfrom
SunwardL:codex/memory-distillation-p1b

Conversation

@SunwardL

@SunwardL SunwardL commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

When Memory distillation is explicitly enabled, completed user turns can produce durable Memory suggestions. Each candidate requires independent approval before writing. The feature defaults off and targets workspace scope.

Changes

  • Reuse the initiating Kun model route with no tools or retries, up to 24,000 input characters, 2,048 output tokens, eight candidates, and 15 seconds.
  • Bind workspace, evidence, observation time, and comparison targets on the host. Skip failed, aborted, empty, and internal turns; approved records remain reference-only.
  • Persist independent candidates with replay fingerprints, seven-day expiry, and durable apply receipts. Recover interrupted approved writes without duplicate Memory records.
  • Add localized candidate controls and wire settings through defaults, strict IPC validation, and runtime hot apply.
  • Rebase onto develop at 7324725c3 (including provider-save/updater lifecycle fixes and desktop startup readiness), retaining its packaging and release checks alongside the exact-child handoff regression suite.

Maintainer concurrency fixes

  • Move candidate-ledger operations onto the shared Service Manager data plane. Both runtime flavors observe one durable state; runtime reconnects cannot overwrite a denial or fail another live extraction.
  • Revalidate target timestamp, complete record fingerprint, lifecycle, and exact duplicates inside the canonical mutation queue. Update/supersede now return conflict instead of overwriting intervening edits, including edits in the same millisecond.
  • Cover file storage, the real Manager HTTP API, and SQLite projection. Old unapplied target proposals lacking fingerprints fail closed; matching committed receipts remain recoverable.
  • Track in-flight approval work during shutdown and drain Manager Memory writes before closing stores.
  • Resolve comparison versions from canonical records for the retrieved IDs, so prompt-budget truncation cannot produce a false approval conflict.

Upgrade acceptance integration

  • Bind PR installer/feed bytes to the same workflow run, merge commit, repository, and SHA-512 digests. Keep release-tag validation for release candidates; public release validation rejects PR-only reports.
  • Reuse upstream desktop startup readiness and diagnostic collection. Accept UTF-8 BOM in installer JSON and retain native OS credential policy across baseline, automatic relaunch, and inspection.
  • Open Providers settings through the real UI on released 0.3.7 before in-app upgrade. That released version registers its settings drain operations only while this view is mounted. No binary patch or forged acknowledgement is used; this covers the available legacy path, not every 0.3.7 view.
  • Require successful native GUI and original launcher exits using unique capture identities, even if Playwright transport shutdown hangs. Preserve each scenario's owned installer recovery directory so fixed per-account transaction paths cannot leak across separate test installations.

Tests

  • Rebased onto develop at 7324725c3.
  • Runtime/Manager HTTP/real SQLite focused tests: 139 passed. Memory/settings/IPC focused tests: 119 passed. Interactive actual Memory settings component with fixture data verified independent approval/denial, busy controls, empty state, and default-off behavior.
  • Release/workflow tests: 83 passed. Updater pending/install tests: 39 passed. Updater/provider-save tests: 41 passed in isolated rerun after one import timeout under combined load.
  • Typecheck, production build, lint (0 errors; existing warnings), file-line gate, diff check, and strict OpenSpec validation passed.
  • Earlier full desktop run: 8,895 passed, 78 skipped. Full local runtime run: 5,671 passed, with two timing failures in Graph LoopGate/TUI PTY; both failed suites passed in isolation (13/13). The initial full local npm test is not claimed green.
  • Previous native run 33999671265 passed Quality gates, all four platform packaging/handoff jobs, and the complete normal Windows GUI upgrade scenario. Busy acceptance exposed cross-scenario installer recovery state; ownership-preserving isolation and regression tests are included here.
  • Current head 7bb8dc580b74c46ae7cc685626b1f8fdffd55455: CI 34002494302 passed. Quality gates, all four platform packaging/handoff jobs, native Windows GUI upgrade acceptance (normal, busy, rollback, manual), and the final PR gate passed.

Validation

  • I agree that this contribution is submitted under the Contributor License Agreement.
  • Focused runtime, Manager HTTP, SQLite, renderer, settings, and IPC tests passed.
  • Typecheck, build, lint, file-lines, and strict OpenSpec validation passed.
  • Interactive renderer approval smoke verified with fixture data.
  • Current-head full CI and cross-platform packaging passed.

OpenSpec archival remains pending until merge and acceptance are complete.

@SunwardL
SunwardL force-pushed the codex/memory-distillation-p1b branch from 6e766e8 to 58f5880 Compare September 4, 2026 02:42
@SunwardL

SunwardL commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Update: rebased onto the latest develop and resolved the P1-A/P1-B stack conflict

This PR was originally stacked on the pre-merge P1-A commit afdf1a4. P1-A was later squash-merged as ba36b1c with additional maintainer changes, so GitHub could not recognize the old P1-A patch as shared history and reported conflicts.

The branch has now been rebuilt as a single P1-B commit (58f5880) on top of upstream/develop@7085c506.

What changed during the conflict resolution:

  • Kept both the upstream write-document guard wiring and the P1-B Memory distillation services in runtime composition.
  • Adapted P1-B to the final P1-A host-owned evidence contract:
    • model extraction returns only bounded sourceIds;
    • the runtime constructs and resolves the complete current-turn evidence;
    • observedAt, source kind/trust, thread/turn IDs, excerpts, scope, and authority remain host-owned;
    • model-supplied observation time, full evidence, or unknown source IDs fail closed before approval or persistence.
  • Updated newly merged MemoryStore test fixtures for the required distillation setting.
  • Added coordinator coverage for forged observation time and unauthorized source IDs.

Validation completed after the rebase:

  • npm run typecheck
  • 166 focused P1-A/P1-B and Memory regression tests
  • Electron-ABI SQLite/FTS5 Memory tests
  • npm run build:kun
  • openspec validate add-kun-memory-distillation --strict --no-interactive
  • changed-file ESLint, file-line gate, and git diff --check

@SunwardL
SunwardL marked this pull request as ready for review September 4, 2026 03:11
@SunwardL

SunwardL commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Incremental reliability hardening (6fc699a3)

This follow-up keeps the work in P1-B because it tightens the guarantees of the existing TurnEnd -> candidate -> approval -> Memory write lifecycle; it does not add the broader P2 retrieval or knowledge-graph features discussed in the alignment note.

Why this was needed

  • The prior Quality Gates run exposed a deterministic Linux failure because a coordinator test asserted a Windows-only workspace path.
  • A delayed approval could otherwise act on a Memory record that had changed since extraction, or create an exact duplicate that appeared while the candidate was waiting.
  • A process interruption between the canonical Memory write and the pending-ledger commit needed an explicit recovery path.
  • Pending state was mutated in memory before durable persistence, so a failed write could leave a ghost state in the live cache.
  • Background extraction needed to participate in runtime shutdown instead of outliving shared stores.
  • Internal runtime-generated turns must not be represented as explicit-user evidence.

What changed

  • Skip all five current internal messageSource values and derive source IDs from thread ID, turn ID, source kind, and content hash.
  • Capture the target updatedAt for update/supersede proposals; revalidate target lifecycle/version and exact normalized duplicates immediately before allow.
  • Add terminal conflicted outcomes and return HTTP 409 for stale approvals without writing or overwriting Memory.
  • Persist an apply receipt before mutation, use deterministic expected Memory IDs, and reconcile applying candidates against canonical Memory on restart.
  • Make every pending-ledger mutation copy-on-write so the in-memory cache advances only after a successful durable write.
  • Track and abort in-flight extraction during shutdown, wait for it to settle, and reject newly scheduled work once shutdown begins.
  • Replace Windows-specific workspace fixtures with platform-neutral temporary paths so the test has the same semantics on Linux and Windows.
  • Split apply/recovery logic into memory-distillation-apply.ts; the coordinator remains below the 700-line gate.

Local verification

  • 178 focused P1-A/P1-B and Memory tests passed, including Electron-ABI SQLite/FTS coverage.
  • npm run typecheck passed.
  • npm run build:kun passed.
  • npm run build passed.
  • npm run lint passed with 0 errors (30 existing React Hook warnings remain).
  • npm run check:file-lines passed.
  • openspec validate add-kun-memory-distillation --strict --no-interactive passed.
  • git diff --check passed.

OpenSpec progress is now 34/37. The remaining items are intentionally left open for the PR's full CI/UI evidence and post-merge archive. Persistent extraction retry queues, model-role routing, batching/budgets, citation offsets, embeddings, graph expansion, and Crystal generation remain out of scope for P1-B.

@SunwardL

SunwardL commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Quality Gates follow-up

The latest Quality Gates failure was caused by an incomplete settings contract in this PR, not by the previously observed Windows TUI job.

memoryDistillationEnabled had already been added to the shared Kun settings type, defaults, merge logic, renderer, and runtime configuration, but the strict settings:set IPC Zod schema still rejected it as an unknown key. This made the full SettingsView auto-apply snapshot fail in register-app-ipc-handlers.workspace.test.ts.

Commit dff4e3d4 adds the missing boolean field to kunRuntimePatchSchema, keeping the strict IPC whitelist aligned with the existing settings contract.

Local verification:

  • failing workspace IPC test: 12/12 passed after the fix
  • related settings tests: 77/77 passed
  • npm run typecheck: passed
  • changed-file ESLint: passed
  • git diff --check: passed

A new PR Checks run has started for the updated head SHA.

@SunwardL
SunwardL force-pushed the codex/memory-distillation-p1b branch from dff4e3d to c1a2779 Compare September 5, 2026 06:29
@SunwardL

SunwardL commented Sep 5, 2026

Copy link
Copy Markdown
Contributor Author

Rebased this PR onto upstream/develop at d71cc52c, including merged #1278. The updated head is c1a2779d35bbc24ce0f4e6b8e893c06f9489e4f2; it contains all 34 upstream commits since the prior base and retains the three P1-B commits plus one focused integration fix.

The rebase applied without textual conflicts. Combined upstream and P1-B changes brought kun-runtime-services.ts to 702 lines, above the 700-line gate. The follow-up directly delegates the two candidate operations to the existing request helper, reducing the file to 691 lines, and adds five request-routing regressions. Candidate scope, encoded ids, decision bodies, and error propagation remain covered.

Local verification: Kun focused tests 83/83; desktop/settings/IPC focused tests 150/150; typecheck, build, full lint (0 errors, 30 warnings), production audit, file-lines, diff check, and OpenSpec strict all passed. Release-workflow tests passed 8/8 with Git Bash in the child-process PATH.

The PR description now reflects the current implementation and removes the obsolete P1-A draft dependency. Full tests and platform packaging are running on PR Checks 33949941868. Manual UI/video evidence remains pending and is not marked complete.

@SunwardL

SunwardL commented Sep 5, 2026

Copy link
Copy Markdown
Contributor Author

Windows update-handoff smoke follow-up

Pushed 985a4aed177f78bad5c36db5e985e6b22d511e9d (fix(ci): track exact predecessor exits in handoff smoke).

The preceding run passed Quality gates, Linux, macOS, Windows TUI, Windows NSIS creation, and installer smoke. Its failure was specifically Candidate left predecessor PID 5868 alive in the packaged update-handoff smoke, not the ECONNRESET seen in the separate upstream build.

The smoke retained the original spawned ChildProcess objects but tested predecessor exit using process.kill(pid, 0). That only establishes that a PID currently exists, not that the original process is still alive. The auto-start path also did not wait for all predecessor Runtime exits before its final assertion.

This patch:

  • Waits, within the existing timeout, for the original Manager, production Runtime, and development Runtime child exit states. Live original children still fail; child.killed alone does not count as exit.
  • Verifies the in-app preflight's predecessors have exited before launching the candidate GUI; the external and auto-start-off paths use the same exact-child check.
  • Prints the preflight acceptance report and includes each predecessor's role, PID, and recorded exit state on timeout. No full discovery records or credentials are added to these diagnostics.
  • Adds eight regressions, including deterministic PID-reuse simulation, live-owner rejection, delayed/signal exits, and a real Windows child process that exits normally without the checker killing it.
  • Adds the handoff test file to test:release-workflows, so the regular npm test Quality gate now runs these checks.

Local validation: release/handoff tests 28/28, runtime handoff/identity/process-safety tests 37/37, typecheck, production build, full lint (0 errors, 30 existing warnings), file-lines, syntax checks, and diff check passed.

A separately explored broad Extension desktop smoke test still fails in the existing workflow-contract assertion (process-and-release.cjs:225, missing job .steps). The test and referenced workflows are identical to upstream/develop; this unrelated baseline issue was not modified or marked passed.

The old CI logs do not prove whether PID 5868 was actually reused or genuinely remained alive. This fixes the demonstrable PID-only assertion flaw and makes a genuine residual child failure explicit. Production Runtime and Memory behavior are unchanged. The new PR Checks run 33953100945 is in progress and must validate the packaged Windows result; full-suite/platform success is not claimed yet.

@XingYu-Zhong
XingYu-Zhong force-pushed the codex/memory-distillation-p1b branch from 985a4ae to 0d386bc Compare September 5, 2026 16:40
XingYu-Zhong
XingYu-Zhong previously approved these changes Sep 5, 2026

@XingYu-Zhong XingYu-Zhong left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reproduced and fixed the two approval concurrency defects. Candidate state now goes through the shared Manager; canonical commits atomically revalidate timestamps, record fingerprints, lifecycle, and duplicates. File, Manager HTTP, and SQLite regressions pass, including same-millisecond edits, concurrent duplicate approvals, and lost-reply recovery. Full desktop tests and new-head CI Quality gates pass. Interactive renderer acceptance used isolated fixture data. Merge remains contingent on the remaining platform and PR gate checks.

SunwardL and others added 6 commits September 6, 2026 02:38
Wait for the original Manager and Runtime child processes instead of probing reusable numeric PIDs. Verify in-app preflight exits before launching the candidate, retain bounded failure diagnostics, and run handoff regressions in the regular PR test path.
Route pending candidate state through the shared Manager instead of runtime-local caches. Revalidate target fingerprints, lifecycle, and exact duplicates inside the canonical Memory mutation queue; preserve manual edits and reconcile lost commit replies without duplicate writes. Cover the file, Manager HTTP, and SQLite paths.
@XingYu-Zhong
XingYu-Zhong force-pushed the codex/memory-distillation-p1b branch from 05be97f to 3b2067a Compare September 5, 2026 18:42
Verify PR candidate provenance against the tested merge commit, workflow run, installer bytes, and update metadata. Keep draft-release tag checks as the default and reject PR-only evidence during public release verification.
@XingYu-Zhong
XingYu-Zhong force-pushed the codex/memory-distillation-p1b branch from 3b2067a to f224774 Compare September 5, 2026 19:45
…tion

Accept Windows PowerShell UTF-8 BOMs in pending-state readers and the native acceptance harness. Keep native GUI launches on the same OS credential policy as automatic installer relaunches, preserving existing DPAPI keys rather than forcing an incompatible file-key fallback.
@XingYu-Zhong
XingYu-Zhong force-pushed the codex/memory-distillation-p1b branch from 40a5970 to 33bf9ec Compare September 5, 2026 21:53
Require capture-specific native GUI exit evidence and the original launcher exit code instead of waiting indefinitely for Playwright close transport completion. Reject missing or abnormal exits and PID reuse, and allow independent Manager cleanup to proceed after verified GUI shutdown.
@XingYu-Zhong
XingYu-Zhong merged commit f3c0fd0 into KunAgent:develop Sep 6, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants