Skip to content

feat(web): sign in from Tailscale Serve identity headers (phase 1) #457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bbingz merged 1 commit into from
Oct 8, 2026
Merged

feat(web): sign in from Tailscale Serve identity headers (phase 1) #457

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,107 @@ All notable changes to this project will be documented in this file.
Format based on [Keep a Changelog](https://keepachangelog.com/).


## HQ T+10.5h check, old local Service CPU spin, Web identity-auth design (2026-10-08)

Read-only status pass on HQ at 23:28 UTC (07:28 CST) plus one new design doc.
No host was changed and nothing was deployed.

**HQ new stack.** service-index PID 88127 up 10 h 24 min, 12-14% CPU at a
10-second `top` sample, 32 MB, stderr 0 bytes, checkpoints ok; receiver PID
89758 up 10 h 23 min, `/v1/health` 200 on 127.0.0.1:18787 and on
`https://macmini-hq.tail1cb16.ts.net:8443` (`tailscale serve` proxy, tailnet
only), stderr still only the two index-warm notices. Central DB: arrivals
41,529 (41,517 at T+7h), ledger index_ready 8,567 / parsed 32,390 /
quarantined 579, sessions 45,412, newest ingest 23:30:24 UTC; `origin=local`
rows 6,191 (grok 5,959, pi 232) with the newest `indexed_at` still
2026-10-07 11:20 UTC, so the role gate holds. `session_index_jobs`: embedding
pending 12,151 (expected, no provider), fts 12,558 all completed. Disk 687 GB
free. Daily collector (SSH read-only): PID 30548 up 5 d 10 h, 0% CPU at the
sample, 25 MB, no log line since 2026-10-02. M1 replica `/v1/health` 200,
`tailscale ping` 4 ms although `tailscale status` labels it offline.

**Old HQ local stack (retires at runbook R8).** The old local-role
`EngramService` PID 30392 (`~/.engram-service/current` =
`releases/cabeeb24-build1569`, started by the system boot daemon, up 32 days,
lifetime CPU average 80%) ran at 82-94% CPU on one thread across a 10-second
sample. A 3-second `sample` shows the hot thread inside an FTS5 cursor scan
(`fts5SeekCursor`, `sqlite3BtreeNext`) on the GRDB reader connection; the
caller is unresolved because the binary is stripped. Its own WAL checkpoint
has logged "Timed out waiting for the EngramService write lock" 4,525 times
(19 in the last 300 lines of `~/.engram/logs/engram-service.boot.out.log`).
It still indexes HQ claude-code/codex into the old 1.3 GB DB (81 sessions in
24 h, 59,876 total). The old hub `100.125.101.60:8787` answers 200 on plain
HTTP, kept alive by the Daily watchdog every 2 minutes
(`~/.engram/logs/hq-boot-ensure.log`). Recommendation recorded for the owner:
bring R8 (watchdog off, old service and hub booted out) into the next
authorized window rather than fixing the old build.

**Untracked `.grok/workflows/engram-session-status.rhai` explained.** A Grok
Build workflow script dated 2026-09-27 (read-only session replay plus cutover
inventory); its 15 siblings in the same directory are tracked. Commit or
delete; not a product file.

**Web login design.** The owner judged the shared-credential Web login too
crude and chose Tailscale identity. New design doc
`docs/superpowers/specs/2026-10-08-web-tailscale-identity-auth-design.md`:
`ENGRAM_REMOTE_WEB_AUTH=tailscale-serve` makes the receiver mint the existing
hardened session from the `Tailscale-User-Login` header that `tailscale
serve` asserts (Tailscale KB 1312, fetched 2026-10-08: added on proxied
requests, client copies stripped, absent on Funnel and tagged devices),
matched against `ENGRAM_REMOTE_WEB_VIEWERS` / `ENGRAM_REMOTE_WEB_EDITORS`
exact-login allowlists; credential keys are forbidden in that mode and the
bind host must be loopback; cookie, exact-Host, marker-header and Origin
rules are unchanged; the UI re-mints silently so the 900-second lifetime is
invisible. Credential mode stays the default and is byte-for-byte unchanged.
Phase 2 (separate PR) carries the login through the Web write envelope as
the audit actor instead of `mcp`. Not implemented yet.

**Serve identity-header pre-check (owner-authorized, PASS).** A loopback
header-echo server on `127.0.0.1:18799` was published with `tailscale serve
--bg --https=8444`, requested from HQ and from the Daily Mac with spoofed
`Tailscale-User-Login: attacker@example` and `Tailscale-User-Name: Mallory`.
Both echoes carried exactly one `Tailscale-User-Login: zzbhlx@gmail.com`,
the Serve-asserted name and profile picture, `Tailscale-Headers-Info`, and
`X-Forwarded-For` with the requesting node's tailnet address (so Serve does
add forwarded headers despite the KB summary); the spoofed values were
removed. A direct loopback request bypassing Serve delivered the spoofed
header unchanged (design risk R1). Mapping removed with `tailscale serve
--https=8444 off`; `tailscale serve status` back to the two pre-existing
entries, 8443 Web origin 200 throughout. Recorded in the design doc's test
plan.

**Phase 1 implemented (branch `feat/web-tailscale-identity-auth-20261008`,
uncommitted at the time of writing; not merged, not deployed).**
`EngramRemoteWebConfig` gains `AuthMode` (`credential` with the two digests,
`tailscaleServe` with exact-login viewer/editor sets), `ENGRAM_REMOTE_WEB_AUTH`
parsing, five new `ConfigError` cases and `isWellFormedLogin`;
`EngramRemoteServerConfig` rejects identity mode on a non-loopback
`ENGRAM_REMOTE_HOST`; `WebAuthSessionStore` adds `WebIdentity`,
`login(identity:)` (no attempt window, same capacity and collision guards),
`LoginResult.forbidden`, a per-session login and `actor(sessionToken:)`, and
each store answers `.unavailable` to the other mode's entry point;
`WebRequestBoundary.tailscaleIdentity(in:)` reads exactly one well-formed
`Tailscale-User-Login` and only in identity mode; `POST /web/api/auth` in
identity mode accepts only `{}` plus exact Origin and answers 401/403/204;
the identity-mode `GET /web/api/auth` body adds `login`; credential-mode
responses are unchanged. `WebUIRoutes.mount(tailscaleServeIdentity:)` serves
the same HTML with `<body data-auth-mode="tailscale-serve">` and the
credential form shipped `hidden`; the script signs in silently on a 401
probe, shows "Signed in as <login>", hides the form and logout, re-mints
once after an expired read with a five-second floor, and shows the
not-permitted copy on 403. Ledger entry "Web Reader and Editor Authority"
rewritten for both modes; `cutover-web-editor-hardening-1` annotated.
Tests: 13 new Swift tests across `WebConfigTests`, `WebAuthSessionTests`,
`WebAuthRouteTests`, `WebUIRoutesTests`, `WebServerIntegrationTests`
(including the forged-header-in-credential-mode guard and the
viewer-session-cannot-write check); full `EngramRemoteServerCore` 519
passed, 0 failed after the final edit; `tests/scripts/collector-web-ui.test.ts`
gained a harness `authMode` option and 4 behavioral tests, 180 passed; the
fake DOM has no `document.body`, which caught an unguarded read in the
first draft and was the reason the identity check is defensive.
`invariants-ledger.test.ts` 12 passed; `npm run lint` exit 0. Phase 2 (audit
actor through the Web write envelope) is not started.

## P2 and P3 merged, dependency PRs landed, HQ T+7h check (2026-10-07)

Closes the 2026-10-07 session. All eight PRs opened or revived today are
Expand Down
9 changes: 9 additions & 0 deletions MEMO.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

## Changelog Memo

### 2026-10-08

- [验证] HQ T+10.5h 只读检查：service-index 与 receiver 健康、stderr 为零、摄取继续（arrivals 41,529、index_ready 8,567）、`origin=local` 行自重部署后无新增；Daily 采集器运行 5 天 10 小时、瞬时 CPU 0%；M1 副本健康 200。详见 `CHANGELOG.md`。
- [排查] HQ 旧 local Service（build1569，系统 boot 守护进程拉起，已跑 32 天）单线程 82–94% CPU，热点在 FTS5 游标扫描，自身 WAL checkpoint 已超时 4,525 次；旧 hub :8787 仍由 Daily 看门狗维持。建议把 runbook R8 提前到下一个授权窗口，而不是修旧构建。
- [排查] 未跟踪的 `.grok/workflows/engram-session-status.rhai` 是 9 月 27 日的 Grok Build 工作流脚本，同目录 15 个同类文件已跟踪；提交或删除即可。
- [设计] Web 登录改用 Tailscale 身份：新设计稿 `docs/superpowers/specs/2026-10-08-web-tailscale-identity-auth-design.md`（`ENGRAM_REMOTE_WEB_AUTH=tailscale-serve`，按精确登录名的 viewer/editor 允许表，口令模式保持默认且不变，第二阶段把登录名作为审计 actor）。
- [变更] 第一阶段已在分支 `feat/web-tailscale-identity-auth-20261008` 实现（未提交、未合并、未部署）：配置模式开关与允许表、身份模式只接受 `{}` 登录体、回环绑定强制、会话记录登录名、页面静默换会话并隐藏口令表单；账本条目改写。Swift `EngramRemoteServerCore` 519 过 0 失败，TS Web UI 回归 180 过，lint 通过。详见 `CHANGELOG.md`。
- [验证] 授权后在 HQ 做了 serve 身份头回显检查：从 HQ 与 MacBook 经 8444 临时映射请求，均收到唯一的 `Tailscale-User-Login: zzbhlx@gmail.com`，伪造头被剥掉替换；绕过 serve 直连回环则伪造头原样到达（设计风险 R1）。映射已撤，serve 配置恢复原状，8443 全程 200。

### 2026-10-07

- [合并] 今日 8 个 PR 全部合并到 main（`e3df52ab`）：#451 codeql-action、#438 @types/node 26.6.4、#453 文档、#452 vitest 4.1.11、#455 P2 跨机精确重复隔离、#454 P3 安装工具链（审查出的 6 项缺陷已先修复）、#440 biome 2.5.15、#439 openai 7.28.0。P2 尚未部署，随 R7 的 service-index 构建上线。详见 `CHANGELOG.md`。
Expand Down
2 changes: 1 addition & 1 deletion docs/followups.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ decisions are in `docs/roadmap.md`, engineering gaps in `docs/TODO.md`.
| `collector-start-cost-1` | medium/perf | `macos/EngramCollectorCore/CollectorEventCoordinator.swift` | Every process start forces a full walk of every root, and a new process tries each stale event checkpoint once before falling back to a walk. A busy root can also repeat walks if its event queue overflows during recovery. Likely contributors to the missed CPU target; not measured after the 2026-10-02 fixes. |
| `collector-epoch-change-scope-1` | low | `macos/EngramCollectorCore/CollectorRuntime.swift` | A changed FSEvents epoch for one root stops the whole runtime, including uploads for other roots. Decide whether it should suspend only that root, as an identity change does. |
| `cutover-install-tooling-1` | medium | `scripts/plan-headless-install.mjs`; `macos/*/Packaging/` templates | A new Mac cannot be set up as a collector from the repo alone: there is no settings-block template, identity initialization step, credentials provisioning, or apply/upgrade/rollback mode, and no repo script creates the HQ index-role job. The dry-run plan also reports every role's job as disabled with RunAtLoad/KeepAlive off, while the remote-server plist template sets RunAtLoad and KeepAlive to true. |
| `cutover-web-editor-hardening-1` | medium/security, needs triage | `macos/EngramRemoteServer/Core/WebAuthSessionStore.swift`; `WebWriteRoutes.swift` | Web login throttling is one global five-attempts window with no per-client partition, so anyone who can reach the origin can delay the editor's login. Unverified risk: an editor can change AI/embedding base URLs while stored keys stay in place. Project moves from Web are audited with actor `mcp`. `docs/invariants.md` now records Web authority; these behaviors are not guarantees there. |
| `cutover-web-editor-hardening-1` | medium/security, needs triage | `macos/EngramRemoteServer/Core/WebAuthSessionStore.swift`; `WebWriteRoutes.swift` | Web login throttling is one global five-attempts window with no per-client partition, so anyone who can reach the origin can delay the editor's login. Unverified risk: an editor can change AI/embedding base URLs while stored keys stay in place. Project moves from Web are audited with actor `mcp`. `docs/invariants.md` now records Web authority; these behaviors are not guarantees there. Since 2026-10-08 (`docs/superpowers/specs/2026-10-08-web-tailscale-identity-auth-design.md`) a deployment can set `ENGRAM_REMOTE_WEB_AUTH=tailscale-serve`, where no credential or throttle exists and the session records the tailnet login; the audit actor stays `mcp` until that design's phase 2 lands. |
| `cutover-role-surface-1` | low | `macos/EngramMCP/Core/MCPToolRegistry.swift` | In `collector`/`replica`/invalid roles MCP still advertises the full tool catalog while every call returns the unavailable message. |

## HQ live ingest residuals (2026-08-24)
Expand Down
6 changes: 3 additions & 3 deletions docs/invariants.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,9 +165,9 @@ Invariants are properties that must survive every change; each entry names where

## Web Reader and Editor Authority

- **Statement** - Web is off unless explicitly configured; when configured, RemoteServer mounts the authenticated read routes and the write routes together. The viewer credential must differ from every server bearer credential, and the optional editor credential must differ from the viewer and server credentials and is kept only as a digest. Every authenticated session can read, but only a session minted from the editor credential can write, and that authority ends at logout or expiry. Write routes pass the same exact-Host and `X-Engram-Web: 1` API validation as reads and additionally require an exact Origin; a viewer session or a missing Origin is rejected with 403 before the write surface is called. The read client sends only allowlisted typed commands and attaches no capability token. The write client rejects unlisted commands, attaches the local service capability token only to the Unix-socket request and never to HTTP JSON, and opens no IPC when the token is missing. Login throttling is one global five-attempt window with no per-client partition. This entry does not limit what an editor may change through the allowlisted writes, and it does not isolate a compromised RemoteServer process.
- **Enforced by** - `macos/EngramRemoteServer/Core/EngramRemoteServerApp.swift`, `macos/EngramRemoteServer/Core/EngramRemoteWebConfig.swift`, `macos/EngramRemoteServer/Core/WebAuthSessionStore.swift`, `macos/EngramRemoteServer/Core/WebRequestBoundary.swift`, `macos/EngramRemoteServer/Core/WebWriteRoutes.swift`, `macos/Shared/Service/EngramServiceWebWriteClient.swift`.
- **Verified by** - `macos/EngramRemoteServerCoreTests/WebServerIntegrationTests.swift` (testWebConfigurationDefaultsOffAndExplicitValuesRoundTrip, testDefaultOffReturns404WithZeroFactoryCallsAndZeroRealSocketAccepts), `macos/EngramRemoteServerCoreTests/WebConfigTests.swift` (testViewerCredentialMustDifferFromEveryProvidedBearer, testOptionalEditorCredentialIsDistinctAndStoredOnlyAsDigest), `macos/EngramRemoteServerCoreTests/WebAuthSessionTests.swift` (testEditorAuthorityIsBoundToItsOwnSessionAndRevokedOnLogoutOrExpiry, testViewerOnlyConfigurationNeverGrantsWriteAuthority, testGlobalFiveAttemptWindowCountsBothFailedAndSuccessfulLogins), `macos/EngramRemoteServerCoreTests/WebAliasHTTPTests.swift` (testViewerPostIs403AndDoesNotCallWriteSurface, testEditorAddPreservesPathShapedAliasText), `macos/EngramRemoteServerCoreTests/WebProjectMigrationHTTPTests.swift` (testViewerCsrfAndDisabledEditorNeverReachProjectWriter), `macos/EngramRemoteServerCoreTests/WebRelationshipHTTPTests.swift` (testViewerAndMissingOriginCannotReachAnyRelationshipWriter), `macos/EngramRemoteServerCoreTests/WebAuthRouteTests.swift` (testAuthorityMismatchDuplicateHostAndForwardedSpoofingFailClosed, testAPIMarkerMustBeSingleExactOneForEveryAPIRequest), `macos/EngramRemoteServerCoreTests/WebReadClientTests.swift` (testMessagesSendsOnlyHardcodedCommandTypedDataUUIDAndNoCapability, testAllowlistRejectsExhaustiveLiveHandlerInventoryBeforeAnyIPC), `macos/EngramRemoteServerCoreTests/WebWriteClientTests.swift` (testAllowlistRejectsUnknownCommandsWithoutIPC, testAddAliasAttachesLocalTokenAndNeverPutsItOnHTTPJSON, testMissingTokenDoesNotOpenIPC).
- **Statement** - Web is off unless explicitly configured; when configured, RemoteServer mounts the authenticated read routes and the write routes together. `ENGRAM_REMOTE_WEB_AUTH` selects where a session's authority comes from. In credential mode (the default) the viewer credential must differ from every server bearer credential, and the optional editor credential must differ from the viewer and server credentials and is kept only as a digest; login throttling is one global five-attempt window with no per-client partition. In `tailscale-serve` mode no shared Web credential may be configured, the receiver must bind a loopback address, and a session is minted only from the single `Tailscale-User-Login` header the local Tailscale Serve proxy asserts, matched exactly against the explicit viewer and editor allowlists; credential mode never reads that header. In both modes a session is minted only by a `POST /web/api/auth` that passes the exact-Origin check, every authenticated session can read, only a session minted with editor authority can write, and that authority ends at logout or expiry. Write routes pass the same exact-Host and `X-Engram-Web: 1` API validation as reads and additionally require an exact Origin; a viewer session or a missing Origin is rejected with 403 before the write surface is called. The read client sends only allowlisted typed commands and attaches no capability token. The write client rejects unlisted commands, attaches the local service capability token only to the Unix-socket request and never to HTTP JSON, and opens no IPC when the token is missing. This entry does not limit what an editor may change through the allowlisted writes, and it does not isolate a compromised RemoteServer process or any process that can open the receiver's loopback port.
- **Enforced by** - `macos/EngramRemoteServer/Core/EngramRemoteServerApp.swift`, `macos/EngramRemoteServer/Core/EngramRemoteServerConfig.swift`, `macos/EngramRemoteServer/Core/EngramRemoteWebConfig.swift`, `macos/EngramRemoteServer/Core/WebAuthSessionStore.swift`, `macos/EngramRemoteServer/Core/WebAuthRoutes.swift`, `macos/EngramRemoteServer/Core/WebRequestBoundary.swift`, `macos/EngramRemoteServer/Core/WebWriteRoutes.swift`, `macos/Shared/Service/EngramServiceWebWriteClient.swift`.
- **Verified by** - `macos/EngramRemoteServerCoreTests/WebServerIntegrationTests.swift` (testWebConfigurationDefaultsOffAndExplicitValuesRoundTrip, testDefaultOffReturns404WithZeroFactoryCallsAndZeroRealSocketAccepts, testEnvironmentIdentityModeRequiresLoopbackBindAndNoCredential), `macos/EngramRemoteServerCoreTests/WebConfigTests.swift` (testViewerCredentialMustDifferFromEveryProvidedBearer, testOptionalEditorCredentialIsDistinctAndStoredOnlyAsDigest, testTailscaleServeModeForbidsSharedCredentialsAndRequiresViewers, testIdentityAllowlistEntriesMustBeExactPrintableLoginsWithoutDuplicates), `macos/EngramRemoteServerCoreTests/WebAuthSessionTests.swift` (testEditorAuthorityIsBoundToItsOwnSessionAndRevokedOnLogoutOrExpiry, testViewerOnlyConfigurationNeverGrantsWriteAuthority, testGlobalFiveAttemptWindowCountsBothFailedAndSuccessfulLogins, testIdentityLoginMintsViewerOrEditorByExactLoginAndRecordsActor, testEachStoreAnswersOnlyItsOwnModeAndCredentialSessionsHaveNoActor), `macos/EngramRemoteServerCoreTests/WebAliasHTTPTests.swift` (testViewerPostIs403AndDoesNotCallWriteSurface, testEditorAddPreservesPathShapedAliasText), `macos/EngramRemoteServerCoreTests/WebProjectMigrationHTTPTests.swift` (testViewerCsrfAndDisabledEditorNeverReachProjectWriter), `macos/EngramRemoteServerCoreTests/WebRelationshipHTTPTests.swift` (testViewerAndMissingOriginCannotReachAnyRelationshipWriter), `macos/EngramRemoteServerCoreTests/WebAuthRouteTests.swift` (testAuthorityMismatchDuplicateHostAndForwardedSpoofingFailClosed, testAPIMarkerMustBeSingleExactOneForEveryAPIRequest, testIdentityLoginMintsFromTheServeHeaderWithEmptyBodyAndExactOrigin, testIdentityLoginRejectsCredentialBodiesMissingDuplicateAndUnlistedLogins, testCredentialModeIgnoresForgedServeHeadersAndKeepsItsStatusBody, testIdentityViewerSessionCannotReachWriteRoutesEvenWithAnEditorHeader), `macos/EngramRemoteServerCoreTests/WebReadClientTests.swift` (testMessagesSendsOnlyHardcodedCommandTypedDataUUIDAndNoCapability, testAllowlistRejectsExhaustiveLiveHandlerInventoryBeforeAnyIPC), `macos/EngramRemoteServerCoreTests/WebWriteClientTests.swift` (testAllowlistRejectsUnknownCommandsWithoutIPC, testAddAliasAttachesLocalTokenAndNeverPutsItOnHTTPJSON, testMissingTokenDoesNotOpenIPC).
- **Gate** - `none`.

## Optional AI Maintenance Does Not Gate Required Index Readiness
Expand Down
Loading
Loading