Epic: Node.js fs module — 100% Node.js compatibility (sync + async + architecture)

[nickna]

Goal

Bring SharpTS's Node.js fs shim to 100% Node.js compatibility across sync, callback, and promise APIs, with the interpreter and compiled paths behaving identically — so we can confidently declare fs done and move on to other Node modules. The async surface has the largest gaps and an interp↔compiled divergence, but several already-"working" sync APIs have correctness bugs too.

Four findings from the audit (2026-06-26):

1. Compiled mode has no callback-async fs. FsModuleEmitter's dispatch switch only handles sync methods, streams, watchers, and the constants/promises property-gets — the callback forms (fs.readFile(p, cb), fs.stat(p, cb), …) fall through to _ => false and don't compile. The interpreter implements all of them. This is a parity bug.
2. Compiled fs/promises is fake-async. It calls the sync impl and wraps the result in Task.FromResult (RuntimeEmitter.FsAsync.cs, BeginFsAsyncTryCatch), so it blocks the calling thread and never overlaps I/O. The interpreter runs real Task.Run background I/O and marshals the callback back through the event loop.
3. Whole async families are simply absent in both modes: promises.open/FileHandle, callback rm/rmSync, cp/cpSync/promises.cp, async chown/lchown, callback fd ops, promises.watch, promises.opendir, and the long tail (fsync/fdatasync, f*/l* variants, readv/writev, statfs, glob).
4. Sync-side correctness gaps in APIs that already "work" (verified in FsModuleInterpreter.cs): readFileSync/writeFileSync/appendFileSync ignore encodings and corrupt binary Buffer data (.ToString() on write); mkdirSync ignores its options (no EEXIST, wrong recursive default, no created-path return); accessSync/access ignore mode; fs.constants is missing the permission bits and several O_*/UV_FS_* flags; ReadStream/WriteStream need a full option/event parity pass.

The async machinery to implement all of this already exists and is proven by timers in both modes: event loop + pending-op refcount + guest-callback invocation ($EventLoop.Schedule / InvokeValue compiled; ScheduleTimer / InvokeGuestCallback interp). Nothing here is blocked on missing infrastructure.

Approach decision (affects every child)

Two ways to land this; the first child is the decision point:

• (A) Dual C# (status quo): add the async forms in both the interpreter (Runtime/BuiltIns/Modules/Interpreter/Fs*.cs) and the compiled emitter (Compilation/RuntimeEmitter.Fs*.cs). More code, maintained twice, no new architecture.
• (B) primitive:fs + TypeScript facade (recommended): keep only raw syscalls in C# behind primitive:fs (interpreter impl + BCL-only emitter) and write stdlib/node/fs.ts + stdlib/node/fs/promises.ts as the Node-shape facade — the pattern already used for os, readline, path, etc. The callback↔promise↔sync wrapping, FileHandle, rm/cp, error normalization, and the Stats object are written once in TS and both execution modes inherit them. This structurally closes the compiled callback gap for free and lets the type surface be inferred from the .ts source (which can express overloads the hand-written C# type dict cannot).

Children are written approach-agnostically where possible; each notes how it maps to A vs B.

Children (rough priority order)

• fs: adopt primitive:fs + TypeScript facade architecture (decision) #969 — Adopt primitive:fs + TypeScript facade architecture (decision). Foundational. Choosing the facade makes most children below small TS wrappers and closes the compiled callback gap structurally.
• fs: compiled callback-async parity (readFile/writeFile/stat/… with callbacks) #970 — Compiled callback-async parity. fs.readFile(p, cb) etc. run interpreted but don't compile. Highest correctness value (real interp↔compiled divergence).
• fs: compiled fs/promises — real thread-pool backgrounding #971 — Compiled fs/promises: real thread-pool backgrounding. Today it's sync + Task.FromResult (blocks). The consistency follow-up — do it alongside fs: compiled callback-async parity (readFile/writeFile/stat/… with callbacks) #970 so the async wrapper is written once.
• fs/promises: open + FileHandle class #972 — fs/promises.open + FileHandle. Entirely missing; largest single new surface (fd-holding object forwarding to fd ops).
• fs: rm/rmSync + cp/cpSync/promises.cp #973 — rm/rmSync + cp/cpSync/promises.cp. Modern recursive remove/copy, missing across sync/callback/promise (only promises.rm exists).
• fs: async chown/lchown + callback fd ops (open/read/write/close/fstat/ftruncate) #974 — async chown/lchown + callback fd ops (open/read/write/close/fstat/ftruncate). Sync forms exist; async/callback forms don't.
• fs/promises: watch (async iterator) + opendir #975 — fs/promises.watch (async iterator) + opendir. The one genuinely non-mechanical item (async-iterator bridge over FSWatcher).
• fs: long-tail async ops (fsync/fdatasync, f*/l* variants, readv/writev, statfs, glob) #976 — Long-tail async ops (fsync/fdatasync, f*/l* variants, readv/writev, statfs, glob).
• fs: complete & unify the Stats/Dirent objects (sync↔async consistency) #977 — Complete & unify Stats/Dirent (sync↔async consistency). Supporting correctness; async stat/FileHandle.stat/Dir all need one canonical Stats shape.

Sync-side correctness & completeness (finding 4) — needed for a true 100%:

• fs: binary/encoding I/O correctness (readFile/writeFile/appendFile honor encodings + Buffer data) #978 — Binary/encoding I/O correctness. readFile/writeFile/appendFile honor encodings and handle Buffer data — fixes today's binary-write data corruption (bug).
• fs: sync option semantics + complete fs.constants (mkdir options, access mode, O_*/S_* bits) #979 — Sync option semantics + complete fs.constants. mkdirSync options (EEXIST/created-path/recursive default), access mode enforcement, full O_*/S_*/UV_FS_* constants, sync Dir/callback opendir (bug).
• fs: ReadStream/WriteStream full Node option + event parity (audit & close gaps) #980 — Stream parity. ReadStream/WriteStream full Node option + event parity audit (incl. fd/FileHandle construction, pipe/backpressure, AbortSignal).

Constraints / non-negotiables for any child PR

• Green on dotnet test, SharpTS.Test262, and SharpTS.TypeScriptConformance (no baseline regression).
• No SharpTS.dll hard-dependency in standalone compiled output — fs is currently BCL-only/standalone; any primitive:fs emitter must stay BCL-only (see CLAUDE.md "Standalone DLL Constraint").
• Interpreter and compiled output must agree (same observable behavior); add dual-mode tests for every new API.
• Node target: 24.15.0 (match the existing stdlib facades).

Reference (key seams)

• Interp callback-async (the working reference): FsModuleInterpreter.cs + FsAsyncHelpers.cs → Ref() → Task.Run → ScheduleTimer(0, …) → InvokeGuestCallback → Unref().
• Compiled async wrapper: RuntimeEmitter.FsAsync.cs; dispatch in FsModuleEmitter.cs / FsPromisesModuleEmitter.cs.
• Event loop: RuntimeEmitter.TSEventLoop.cs ($EventLoop), RuntimeEmitter.VirtualTimers.cs (InvokeValue).
• Type surface: TypeSystem/BuiltInModuleTypes.cs GetFsModuleTypes/GetFsPromisesTypes (or, under approach B, inferred from the .ts facade).