feat(host): make the host/client transport an interface, not a MessagePort#28
Open
jamesyong-42 wants to merge 8 commits into
Open
feat(host): make the host/client transport an interface, not a MessagePort#28jamesyong-42 wants to merge 8 commits into
jamesyong-42 wants to merge 8 commits into
Conversation
CLAUDE.md records the architecture, the exact build/test commands, and the gotchas that cost time to rediscover — tests importing from dist/, the mandatory --exclude mille-binding, the dangling SPEC/PLAN citations, and why panic = "unwind" is load-bearing. planning/REMOTE_WORKSPACE_PLAN.md sequences draft/mille-truffle-spec.md into shippable PRs, validated against Truffle v0.7.6 (09367f6) rather than the spec's assumptions. Two findings there change the security design: mesh.net listeners are tailnet-authenticated but app-unauthenticated, so mille's authorize() is the only application gate and the Tailscale grant is required rather than advisory; and authorization must key on the Tailscale node id, because the RFC 017 device_id arrives in the peer's own hello envelope and is self-declared. Claude-Session: https://claude.ai/code/session_01BxUdtf4tecfv4uLAqRa1CR
…ePort Both sides only ever needed an ordered, reliable, message-oriented pipe with a close notification, but that requirement was never named — postMessage calls and port lifecycle were spread through host.ts and client-port.ts. They now talk through ExplorerChannel (SPEC §9), so a framed Node Duplex, and through it a remote workspace, can be dropped in without either side learning anything about the transport. MessagePort is the compatibility path, not a special case: attachPort(port) wraps attachChannel(channel, context) and connectFileExplorer(port) wraps connectFileExplorerChannel(channel). PortFileExplorer accepts either shape, duck-typed, so `new PortFileExplorer(port)` is unchanged. The wire format is untouched and no consumer needs to migrate. Two defects fell out of the consolidation. adaptPort was duplicated verbatim in the host and the client — it is now one function. And its Node MessagePort branch has always carried a no-op removeEventListener with a TODO(7.10), which meant a detached session could still observe messages; the channel now gates delivery on its own state, so a closed channel delivers nothing even when the underlying listener cannot be detached. The contract test proves it by posting straight at the raw port after close. A closed channel also rejects the client's in-flight calls and emits a connection event instead of leaving callers awaiting forever, while the last mirror snapshot stays readable (§18.3). Session context and policy types ship with the attach signature so the Truffle server can be written against it, but nothing enforces them yet — the permission tables are PR 3. channel-contract.test.mjs covers CH-001 through CH-008 per SPEC §24.1, including 10,000 messages arriving in send order and a dead transport closing with TRANSPORT_ERROR. Full suite green: mille 359 (354 pass, 5 pre-existing skips), mille-ui 470, playground 29, workspace typecheck clean. Claude-Session: https://claude.ai/code/session_01BxUdtf4tecfv4uLAqRa1CR
The git remote is vibecook-dev/mille, so the spec's target was right and the plan had it backwards. The jamesyong-42/mille references in the package manifests and README still resolve via GitHub's post-transfer redirect — they are stale, not broken. Cargo.toml:14 is the one that actually 404s. Claude-Session: https://claude.ai/code/session_01BxUdtf4tecfv4uLAqRa1CR
PR 1 named the transport an interface; this implements the one that is not
a MessagePort. `@vibecook/mille/node` exports framed-stream host and client
channels that run the existing protocol over any Node Duplex — which is what
makes a remote workspace possible, since a Truffle mesh socket is a Duplex
with a tailnet-verified peer identity attached.
The wire format is a 20-byte header (MLLE, big-endian, versioned major and
minor) plus UTF-8 JSON metadata and raw attachments. Snapshots and deltas
already carry bincode buffers, so JSON-encoding them would inflate every
payload; instead each binary view becomes a {$mille:'bin',i} placeholder and
its bytes are appended verbatim. Views honour byteOffset/byteLength, so a
10-byte subarray of a 1000-byte buffer ships 10 bytes — getting that wrong
would silently hand the peer the wrong data, so it has its own test.
The decoder validates the header before allocating: a frame claiming 4 GiB
of attachments is rejected having buffered only the 20 bytes that arrived.
Fragmentation is arbitrary in both directions, and a property test
re-partitions a stream at random cut points to prove it. A fuzzer mutates
every byte of a valid frame and asserts the only failure mode is a clean
FrameProtocolError — never a TypeError or an allocation from a trusted
length.
Writes queue in the channel rather than in the stream, one frame handed over
at a time and nothing further after write() returns false (SPEC §20.1). So
bufferedBytes measures what we hold, not what the OS took, and the hard
limit closes the channel instead of growing without bound.
Two things the tests taught me, both fixed here rather than worked around.
resolveLimits rejected lowering one bound without restating its dependent,
which is backwards — a remote policy may only ever reduce a maximum, so an
unspecified inner bound now clamps to follow the outer one and only an
explicitly incoherent pair throws. And the channel used to drop its 'error'
listener before tearing the stream down, so a dying Duplex emitted an error
with nothing attached and Node threw it as an uncaught exception; the
listener is now swapped for a logging sink, never removed.
entry-boundaries.test.mjs records two facts worth pinning: node:stream is a
type-only import, so the framed channel has no runtime Node dependency at
all and will accept any duck-typed Duplex; and the package root is Node-only
by design via the native loader, so ./port and ./react are the browser-safe
entries, not the barrel.
Suite green: mille 399 (394 pass, 5 pre-existing skips), mille-ui 470,
playground 29, workspace typecheck clean.
Claude-Session: https://claude.ai/code/session_01BxUdtf4tecfv4uLAqRa1CR
PR 1 shipped ExplorerSessionPolicy as a type that gated nothing. This is the enforcement, and it is what makes attaching an untrusted peer defensible: every mutation and call is authorized before native dispatch, so a permissive client, a future transport, or a bug in either cannot route around it. The matrix lives in two lookup tables in src/channel/policy.ts rather than in conditionals threaded through the dispatch switch. That is deliberate — "which of the forty entry points did we forget to gate" should be answerable by reading one file, and the policy test walks the whole table so a missing entry fails rather than silently allowing. Read-only sessions get EROFS on writes. The host-global controls — undo, projection settings, workspace roots, workspace resync, copyFromPath, and client-pushed decorations — are denied to remote sessions by default and opened only by explicit per-flag grant, because each changes what every other session sees. attachPort still means local-admin, so in-process consumers are unaffected. Three things fell out that are worth naming on their own. Transfer progress was broadcast to every session. OP_PROGRESS detail carries source and destination paths, so any attached window could watch what every other one was copying. Progress now routes to the session that owns the operation, and non-operation warnings stay global. Operation ids are claimed per session. A second session naming an in-flight id is refused EEXIST; cancelling another session's operation is refused indistinguishably from cancelling one that does not exist, so an unprivileged peer cannot probe for live ids. Claims release on success, failure, completion and session close. Capability masking needed a reader. A read-only session is told Readonly and not ReadWrite/Trash/AtomicWrite so a UI can disable write affordances instead of failing at EROFS — but PortFileExplorer had no way to ask, which would have made the masking decorative. capabilities() now exists. Also: entry resync is rate-limited to 10/min per session on a sliding window, per-session so a noisy peer cannot starve a neighbour; readFile/writeFile carry Uint8Array instead of expanding every byte into a JSON number, with both sides accepting either form so old and new peers interoperate; and ErrorCode gains EFBIG for the size limit PR 4 will enforce. Suite green: mille 413 (408 pass, 5 pre-existing skips), mille-ui 470, playground 29, workspace typecheck clean. Claude-Session: https://claude.ai/code/session_01BxUdtf4tecfv4uLAqRa1CR
Ran the remote-workspace thesis against a real tailnet before writing any of the Truffle package: two ephemeral mesh nodes, a genuine FileExplorerHost behind mesh.net.createServer, a genuine PortFileExplorer over mesh.net.connect. Eight checks pass, including PR 3's policy holding across the real transport and a 512 KiB payload round trip in 22 ms. The identity capture is the part worth keeping. remotePeer.deviceId is null on a raw TCP accept, so authorizing on the RFC 017 ULID — which this plan briefly recommended — would have meant authorizing on null. The Tailscale StableID is what actually arrives. That is now measured rather than inferred from source comments. Does not discharge AC-002: two stacks in one process exercise the sidecar, WireGuard and framing, but not NAT traversal, DERP or wide-area latency. Claude-Session: https://claude.ai/code/session_01BxUdtf4tecfv4uLAqRa1CR
Adds @vibecook/mille-truffle (server half). serveMille binds a Truffle mesh.net listener, runs the open handshake, authorizes the peer, and attaches the accepted socket to a FileExplorerHost — so a client on another machine drives the same explorer API it uses locally, with no per-file network RPC. Exports are named and server-defined. A client asks for "work", never for a path, which is why traversal is not a request-time concern: roots are canonicalized once at startup and a misconfigured export fails the service at boot rather than surfacing as a confusing denial on someone's first connect. The ordering inside an accepted connection is the security property. The verified Tailscale peer id is readable synchronously at accept, and no host, no engine and no filesystem handle exists until authorization has passed — there is a test asserting hostCount stays zero for every denial path. An unknown export and a forbidden one return byte-identical rejections, or the service enumerates its own exports. A socket with no verified identity is refused, an authorize callback that throws is a denial, and read-write on a read-only export is rejected rather than quietly downgraded, because a silent downgrade hides a misconfiguration from whoever wrote it. One host is shared per distinct export configuration, fingerprinted on roots and explorer options and explicitly not on identity: who you are decides whether you may attach, never which engine you attach to. It stays warm for five minutes after its last session so a reconnect keeps the same EntryIds. Truffle is reached only through a structural MeshLike interface, so the suite runs on a fake mesh with no tailnet and no native addon. That fake cannot prove the interface matches a real MeshNode, so serveMille was also run against a live tailnet — two ephemeral nodes, a denied export refused with no host created, an authorized peer browsing and mutating, and PR 3's policy still returning EACCES for host-global calls across the wire. Ten checks. That run caught a defect the fake could not. mille's default initialWalk is 'full', which is a documented no-op meaning "the consumer calls populateFromRoots itself". An in-process embedder does; a remote peer cannot, so every served workspace would have sat empty forever. Served exports now default to 'roots-only', which is also the right shape for a large remote tree, and a regression test drives a real client to assert the tree is not empty. Also fixes ExplorerSessionContext, shipped in PR 1 with optional fields declared `?:` but without `| undefined`. Under exactOptionalPropertyTypes those differ, and every real caller builds the context from values that are already `string | undefined`. The package is private for now — `pnpm -r publish` would otherwise pick it up, and it ships experimentally with the release wiring in PR 6. Suite green: mille 413, mille-truffle 19, mille-ui 470, playground 29, workspace typecheck clean. Claude-Session: https://claude.ai/code/session_01BxUdtf4tecfv4uLAqRa1CR
All three JS jobs enumerate the packages they build rather than using `pnpm -r build`, because that would trigger mille's release napi build. The new package was not on the list, so its dist never existed and every job died with ERR_MODULE_NOT_FOUND on `mille-truffle/dist/index.js`. Reproduced locally by removing the dist directory — same error, three occurrences — and confirmed green again after rebuilding. The enumeration is a standing trap: the next package added will fail the same way. Left as-is here rather than switching to `pnpm -r build`, which would cost every JS job a full LTO release compile of the native binary. Claude-Session: https://claude.ai/code/session_01BxUdtf4tecfv4uLAqRa1CR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ExplorerChannel(SPEC §9) so the host and client depend on an ordered, reliable, message-oriented pipe rather than onMessagePortspecifically. MessagePort becomes the compatibility path:attachPortwrapsattachChannel,connectFileExplorerwrapsconnectFileExplorerChannel, andPortFileExploreraccepts either shape. Wire format unchanged; no consumer migration.adaptPortwas duplicated verbatim inhost.tsandclient-port.ts, and its NodeMessagePortbranch carried a no-opremoveEventListener(TODO(7.10)) that let a detached session keep observing messages. The channel now gates delivery on its own state.This is PR 1 of the remote-workspace sequence in
planning/REMOTE_WORKSPACE_PLAN.md. No network code, no behavior change.Test Plan
packages/mille/test/channel-contract.test.mjscovers CH-001…CH-008 per SPEC §24.1, including 10,000 messages arriving in send order, close idempotency, post-close delivery gating (asserted by posting straight at the raw port), a throwing listener not killing the channel, and a dead transport closing withTRANSPORT_ERROR.pnpm -r --if-present test— mille 359 (354 pass, 5 pre-existing skips, 0 fail), mille-ui 470/470, playground 29/29.pnpm -r --if-present typecheckclean.prettier --check. Notesrc/react.tsandsrc/undo.tsfail it on untouched code, so rootformat:checkis red independent of this branch — left alone rather than bulking the diff.🤖 Generated with Claude Code