dasLLAMA: the .dlim arc - explicit config identity, streaming/offline/cross-box conversion, server surface - #3586
Merged
Merged
Conversation
…flushed its printer Copilot caught this on #3581 after it merged. jit_run_main_guarded built a TextPrinter and streamed "EXCEPTION: <msg>" into it, then returned without calling output(). TextWriter's destructor only frees largeBuffer; TextPrinter::output() is the sole thing in the class that printf+fflush's. So the print half of e2b8186 was dead code: a standalone -exe panic set the right exit status and said nothing. Why e2b8186's own check missed it: it validated by observation and shipped no fixture, so nothing automated ever asserted the text. The exit code was correct throughout, which is the half that was easy to see. Proven on this build, not argued: a `panic("boom from the exe entry")` fixture compiled with -jit -exe and run beside its runtime DLLs now prints EXCEPTION: boom from the exe entry and exits 1. The message is verbatim the literal in this function, which is what attributes the output to this line rather than to some other path on the panic rail. Still missing, and named here so it is not lost again: a regression test. An -exe fixture in CI is awkward (it needs the JIT plus a working linker, and clang-cl link discovery is already flaky on Windows locally), so the honest home for this is a tests-cpp case driving jit_run_main_guarded with a panicking context and capturing stdout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT
…ak commit 150 -> 82 GB Three pieces, measured on GLM-4.5-Air Q4_K_M (78.6 GB image): - fio dwrite_* — cache-bypassing strictly-ascending sequential writer (open/append/band/commit/stat/close). Win32: FILE_FLAG_NO_BUFFERING + preallocate (ascending writes keep NTFS VDL zero-fill off the bill); POSIX: fallocate + fadvise(DONTNEED); macOS: F_NOCACHE + F_PREALLOCATE. The handle owns a sector-aligned bounce band, so callers append any pointer at any size; dwrite_band/commit hand the band out for zero-copy producers; dwrite_stat splits stage vs syscall time. save_image rides it: buffered long_fwrite was worth 275 MB/s on big images, the direct writer sustains 1664-1720 MB/s per plane (rested- drive GLM aggregate 760 MB/s; per-plane MB/s now logged >= 256 MB). Meta serializes first so the exact file size preallocates up front; the header patches through a normal handle after close. - win32 mmap shim: PAGE_WRITECOPY -> PAGE_READONLY. Copy-on-write sections charge commit for every page that COULD be privatized, so a 73 GB gguf mapping billed 73 GB of private bytes on top of the model (measured 149.9 GB peak for the GLM conversion; 81.9 GB after). Now matches POSIX PROT_READ and faults loudly on an accidental write. - utils/dasllama-convert — offline GGUF -> .dlim (clargs; -m/-o/-q/ -f planar|metal|vulkan/--force/--keep-stale-tmp). Emits the identity- hashed name the load path looks for, and sweeps stale *.dlim.<ticks>.tmp siblings by default (killed runs leave full-size ones; 177.5 GB of them found on this box). Output verified byte-identical (SHA256) to the previous writer at 2.4 GB and 82 GB; the load path maps the tool's image directly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQW3fFpEpq8YtqNZTVh7jJ
…matter, config rides the image meta The .dlim bake config is now ONE explicit struct (dasllama_config.das) instead of scattered host probes: cpu section (backend + layout numbers + native/wscale OUTCOMES), vulkan section (RESOLVED coopmat mode / vram_mb / all walk-shaping bits — the old tag folded raw env strings and missed the dense bits + the cls default split), metal section. Tiers register section sources at [init]; image_identity(tag) = dlim_identity(config, IMAGE_VERSION, tag), no side effects (the backend select lives in the cpu source). vk_moe_init's coopmat ladder and budget resolution extracted into shared resolvers (resolve_coopmat_mode / resolve_weight_cap — auto vram now rounds down to 256 MB so the identity doesn't churn with heap wiggle) so the config can never drift from what the device runs; the caps probe is phys-level only, the 518ae164a stream-reserve carve stays in vk_moe_init. IMAGE_VERSION 6 -> 7: the meta blob now leads with [identity string][config JSON], so an identity-mismatched load logs "baked for '<X>', this run is '<Y>'" instead of a bare hash miss (bounds checks moved ahead of the identity check to make that read safe; a wrong-version image still declines on the header alone). dasllama-convert: --dump-config writes this box's config JSON (+ identity); --config x.json bakes against a supplied one — host-independent knobs applied, then FAIL-CLOSED verified against the resulting identity (an unavailable backend errors with want/have strings; true cross-box repack-by-numbers is a later arc slice). dlim_config_from_json tolerates a UTF-8 BOM. Also fixed: quant_from_string took `var q : QuantMode` — a workhorse-type COPY, so the parsed quant never reached the caller; LINT003 flagged the dead write and was right. Now QuantMode&. tests/run.das now works on Windows at all: the env prefix was POSIX-only (cmd.exe has no VAR=val prefix — now `set "VAR=..."&&` like llvm_tune's tuner_cmd), the log dir fell back to a nonexistent /tmp (now TMPDIR -> TEMP -> /tmp), and the default binary ./bin/daslang picked up a STALE top-level exe on the MSVC layout (now the binary running the script). TEMP registered in env_registry (the tripwire caught the new read); utils/dasllama-convert added to the registry scan roots. Gates: test_model_image mechanics arm 9/9 (synthetic carrier + header-patch decline matrix), smol arm 8/8 (real SmolLM2 regenerated at v7 with the embedded config, mapped back), test_env_registry 14/14, lint clean on every touched file, das-fmt clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQW3fFpEpq8YtqNZTVh7jJ
…drop the 256 MB rounding Boris's catch: no hardware works in 256 MB steps — the rounding existed only to stabilize a number that fluctuates by construction. Heap SIZE is the hardware constant, so auto resolves identically every run with no rounding at all; WEIGHT_CAP (4.2 GB) dominates on any card >= ~5 GB anyway, so auto is a constant 4200 MB on real boxes. Transient VRAM pressure is the memprio shield's job now, not the budget's; explicit DASLLAMA_GPU_VRAM_MB is untouched. query_heap_info also fills size without VK_EXT_memory_budget (size needs no extension; only budget/usage do — heaps rank by live budget when the ext reports it, by raw size otherwise). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQW3fFpEpq8YtqNZTVh7jJ
…, byte-identical to live The vulkan flavor now bakes OFFLINE: `dasllama-convert -f vulkan` runs the real GPU upload walk with a DRY tier — the SAME accept/refuse arithmetic (budget, MAX_STACKS, arena shard cap, rdec guards) against DlimVulkanConfig's numbers, with every device call gated off (no instance, no device, no VRAM). vk_moe_init grows a dry branch fed by vulkan_dry_bake_arm (seam: set_moe_gpu_dry_bake_arm in dasllama_math, so the converter stays vulkan-agnostic); the arm fail-closes when the config disagrees with the process' env wants (prints both tags). maxStorageBufferRange moved to init-time capture (g_gpu.msr — config-supplied when dry). GATE PASSED: offline bake is BYTE-IDENTICAL to the in-process live bake — Qwen3-4B Q4_K_M at budget 6000, both 4856 MB, same SHA256 (69cf07f2…f0f1). The dry image then MAPPED on a live GPU-armed load and the walk consumed the baked plan exactly (no divergence warnings). VkPlanEntry grows `role` (VkBakeRole: cls/dense/shexp/dn/attn/qkv/expert/expert_stream/arena) — the walk declares its stage via vulkan_bake_role, the slice path verifies it per entry, and P3's trim will key on it. Collect moved AFTER the upload/placement verdict (was BEFORE, so refused uploads left phantom plan entries whose validity depended on the device reproducing its budget verdicts — the old B4 hazard). This fixes the old `-f vulkan`, which did a FULL live VRAM upload and then wrote an image with empty vkblob/vkplan. Also: the doubled "vulkan|vulkan|" identity prefix (both load_model_cached branches + the converter prepended what dlim_vulkan_tag already carries). Gates: parity SHA256 (above); slice-load parity probe; image mechanics+smol arms 9/9; lint+format clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQW3fFpEpq8YtqNZTVh7jJ
…ded "q8" image_identity / image_path_for / save_image / load_image (+ the model wrappers) take a quant string, defaulted "q8" — so every existing rail (load_model_cached's q8-gated branch, the tower/whisper rails whose tags already encode their mode) behaves exactly as before, while the converter stamps the REAL -q/-config quant into the identity, filename, and embedded config. Before this, a `-q q4_0` conversion wrote a file whose name and identity claimed q8 — colliding with the actual q8 image's path. Non-q8 images remain converter-only artifacts (the load path still images q8 only); they just stop lying about themselves. Gates: image mechanics+smol arms 9/9 (existing q8/tag defaults byte-stable), converter compiles+help, lint+format clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQW3fFpEpq8YtqNZTVh7jJ
…ht families (-44%) `dasllama-convert -f vulkan --trim` bakes a vulkan image WITHOUT the big CPU weight planes: after the (dry) bake, trim_model_planes packs the emb/cls region into new Model planes embq/embs (q8: mblob-style 34B interleaved blocks; kq: the region's plane pair copied verbatim, repack layout preserved, rebased to superblock 0), sets planes_trimmed (meta 66+3), and FREES qblob/qscales*/wblob/bf16blob/k*q/k*s/q40*/q51*/mx*/q4* — generate-then-free, so save_image writes them as empty planes with zero writer changes. fblob (norms/rope) stays. STRICT v1 gate: every baked-plan role must be arena (pure resident-driver dense, no MTP) or the trim declines. embed_row serves all three access modes from the trimmed planes (q8 via dequant_embq_row; kq grp<mr> and disk-order arms over embq/embs rebased); the decode path is otherwise all-GPU on the resident driver. Fail-closed: the three gather wrappers PANIC on any live gather against a trimmed model (covers slot re-bind, stale-plan divergence, and every CPU re-gather path), and mm_cls panics if the classifier is not GPU-served. The trim is part of the flavor identity: DlimVulkanConfig.trim -> " T" in the tag, driven by the new DASLLAMA_TRIM knob at serve (registered; ENVIRONMENT.md regenerated) and by --trim in the converter (the dry-arm wants-check treats trim as save-shaping, not walk-shaping). Gate so far: Qwen3-4B Q4_K_M trimmed image 2738 MB vs 4856 untrimmed (-44%); maps + slices + arms the resident driver under DASLLAMA_TRIM=1 with zero divergence warnings; env-registry 14/14; image mechanics+smol arms 9/9; lint+format clean. OPEN: decode IDS parity trimmed vs untrimmed (the embq read path needs a real generation to prove — next slice). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQW3fFpEpq8YtqNZTVh7jJ
…mmed CPU fallback fails closed The C decode gate PASSED at the strongest level: Qwen3-4B Q8_0, resident-driver rail (f32-KV session), 24 greedy tokens — trimmed image vs untrimmed image IDS are token-for-token IDENTICAL (the embq read path serves every embed). The trimmed Q8 image is 4475 MB vs ~8.5 GB untrimmed. blob_only_panic grows a planes_trimmed arm — all six CPU fall-through sites (decode/prefill/ batch-decode stacks + their cls steps) now fail closed with an actionable message (the commonest trigger: a non-f32-KV session makes the resident overrides decline, e.g. create_session's f16 default). dasllama-server's bind_gpu_slot re-binds a vulkan-flavor slot by SLICING its baked planes (same rail as its load) — on a trimmed image a live re-gather would panic, and on any flavor image the slice replaces redundant re-gather work. KNOWN PRE-EXISTING (not this arc): Qwen3-4B Q4_K_M on the resident rail generates all-zero IDS (logits degenerate) — reproduced on an UNTRIMMED image and matching the banked _vk_prefill_parity degeneracy note; Q4_K_M therefore cannot gate end-to-end decode either way. Rode along: the 5 pre-existing LINT017s in openai_server.das (int64(length) -> long_length). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQW3fFpEpq8YtqNZTVh7jJ
…s and docs New fio builtin `prefetch_map(base, bytes)`: PrefetchVirtualMemory on Windows, madvise(MADV_WILLNEED) on POSIX — ask the OS to fault a mapped range in AHEAD of a parallel pass (on-demand faults inside the transcode lanes serialize them; the banked cold-conversion gap is 9x on Q4_K source). Advisory by contract: false = the OS declined, reads still work. The gguf loader arms it over the whole source mapping (both the single-file fmap and the split-shard rail) behind the new DASLLAMA_PREFETCH knob (default on, =0 restores on-demand faulting for the A/B; registered, ENVIRONMENT.md regenerated) and logs one "source prefetch armed (N MB)" proof-of-engagement line. Ships with its contract test (tests/fio/fio_prefetch.das — acceptance over a real fmap, content intact after the advise, degenerate-argument decline; 3/3) and docs: handmade RST for prefetch_map plus the six dwrite_* functions (which shipped bare in 8cb3a8b), and a "Direct IO and mapping advice" group in das2rst so none of them render as Uncategorized. The judged cold-conversion A/B waits for the quiet box; the engagement smoke passes warm. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQW3fFpEpq8YtqNZTVh7jJ
…plan mode fill_plan_begin/fill_plan_take + a FillJob record of load_big's call args (name/woff/n/ src_off/fmt). Armed, load_big appends a job and returns instead of transcoding — the ~45-call weights walk runs UNCHANGED as the plan pass, and the fill pass will re-derive each branch by replaying load_big's own conditions, so plan and fill cannot drift. Off by default; zero behavior change (mechanics unchanged, compiles+lints clean). The fill executor (per-plane- family sorted jobs, job-sized temps, fused per-region repack + wscale, dwrite_band consumers, scale side-buffers) is the next slice per the banked design. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQW3fFpEpq8YtqNZTVh7jJ
…s through load_big Under fill_plan_mode the big q-halves (qblob/k*q/q40q/q51q) are never sized — the streamed fill writes them straight to the image — while the scale halves (6-16%) stay in RAM so the wscale/save paths see them normally. Models with direct-write loaders (mx4 experts, grouped deltanet, bf16 PLE, non-q8 modes) disarm the plan BEFORE sizing and convert eagerly with an empty job list — fail-open to the correct path. The tied-Q8 classifier/embedding double-copy now routes through load_big (its q8 arm is the same transcode with the same offsets) so the plan records both copies. PROVEN byte-clean: a fresh Q8_0 conversion's entire plane region [64, meta_off) — all 4.69 GB — is byte-identical to the pre-refactor image (the file-level SHA delta is the embedded config JSON recording GPU-armed vs unarmed box state, by design). Image mechanics+smol arms 9/9. Still inert — nothing arms fill_plan_mode yet. Next slice: the fill executor (repack-region enumeration seam on the walkers, per-job temp transcode replay, the save-walk hook + wscale epilogue after qblob, the mapping-alive streaming entry, converter --stream). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQW3fFpEpq8YtqNZTVh7jJ
…r, big planes never in RAM `dasllama-convert --stream` converts without materializing the big q-planes: the plan-mode walk records FillJobs, load_gguf_streaming keeps the source mapping alive while save_model_image_streaming's walk streams each jobbed field through fill_stream_plane — per job: transcode into a job-sized temp (load_big's own arms at temp offset 0), scales land in the RAM plane, the job's repack regions apply with the q pointer BIASED into the temp (the repack walkers' collector halves feed the SAME regions — register_stream_layout in dasllama_layout), and the temp appends to the direct writer. qblob's tail runs the wscale epilogue — legal because every q-half precedes its scale-half in Model field order. Unstream- able models (mx4/dn-grouped/bf16-ple/non-q8/shards) fail open to the eager path through the same save call (empty jobs ≡ eager walk). GATES PASSED — whole-file SHA identity vs eager conversion, x64-gen mr8 repack armed: Qwen3-4B Q4_K_M (kq transcode + grp8 repack + kq emb): 2323ebec…34e7 both Qwen3-4B Q8_0 (q8 + repack + tied-cls + wscale-f16): 96b97b33…af01 both One real divergence found and fixed en route: the plan-mode wscale pre-mark set wscale_f16 unconditionally while the eager convert early-outs on EMPTY qscales (pure-kq models) — the mark now mirrors the convert's outcome. Image mechanics+smol arms 9/9; lint+format clean. Remaining for the phase receipts: GLM streamed peak-commit (busy-box-valid), then the judged GLM wall-clock A/B on the cold box. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQW3fFpEpq8YtqNZTVh7jJ
…ak 81.9 -> 16.34 GB load_gguf_streaming grows the split-gguf arm (every shard mapping alive across plan-load AND the blk-save; per-shard prefetch; the transcode kernels resolve shard bases through the meta), and the blk signature settles on a plain array view served by temp_array in both arms. THE PHASE'S RECEIPT (busy-box-valid — it is a memory number, not a timing): converting the 73 GB split GLM-4.5-Air Q4_K_M with --stream peaked at 16.34 GB private (sampled at 500 ms) vs the banked 81.9 GB eager peak — 5x less, the model never in RAM. The wscale epilogue ran on the real mixed q8+kq model (1.09 B scales converted mid-save); the produced image maps and parses cleanly (NextN offsets, mr8 marks, flags all sane). Q4_K_M re-gated SHA-identical after the signature change (2323ebec…). Honest wall-clock note: streamed GLM took 640 s on a busy box vs eager's ~145 s — the serial transcode-then-append per job leaves the device idle during transcode and vice versa. The async-writer overlap (double-buffered temps) is the known next slice; its judged GLM A/B is the cold-box measurement this arc parks on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQW3fFpEpq8YtqNZTVh7jJ
…boundary-exact slice replay The import rail: on a vulkan-flavor load, streamed expert mirrors IMPORT the mapped image window (VK_EXT_external_memory_host) instead of pinned copies — SliceWindow table recorded while slicing, moe_gpu_slice_window/unmap_notify seams, Model finalizer notifies the tier before fmap_close. DASLLAMA_VK_IMPORT=0 forces pinned. Driver verdict on NVIDIA/Windows (probe-verified, _vk_import_probe.das): anon WRITABLE imports succeed, READ-ONLY file-backed pages are refused with ERROR_OUT_OF_DEVICE_MEMORY regardless of memory type — so the import declines once (named reason), disables for the process, and every stack falls back to pinned copies. The rail self-enables wherever a driver accepts the mapping. Fixes found by the 35B gate (budget-bounded model, first slice replay WITH a refusal boundary): - Boundary-exact slice replay: the bake records only ACCEPTED gathers, but the walk still attempts the one the budget refuses at each stage boundary — the slicer now asks the tier's shared would-accept arithmetic (vk_moe_resident_verdict via the moe_gpu_would_accept seam) and serves a predicted refusal EMPTY, holding the cursor: the walk re-refuses, rolls back, re-syncs. Before this, ANY model that didn't fully fit declined to live gathers mid-walk. - carved_budget(): the stream-slot reserve derives at READ time from the loader's reported need — the init-time carve was timing-dependent (the expert_q8n probe could init the device before the need was known), which made dry-bake and live budgets diverge. - make_imported_buf is fail-soft end to end (create/props/alloc/bind all checked) and prefers the host_cached non-device-local memory type — the lowest usable bit can be the ReBAR heap, where multi-GB imports exhaust VRAM. - make_host_buf grows a soft mode: a pinned-mirror alloc failure now reads as the streamed walk's budget stop (roll back, stop streaming) instead of a panic. - moe_gpu_upload_stack tolerates empty planes (the refusal-boundary serve). Gate receipts (Qwen3.6-35B Q4_K_M, 6000 MB cap): import-ON and import-OFF runs produce IDENTICAL 24-token IDS; slice replay consumes the plan exactly (no regather warning, refusal replays at 4787159040 bytes both sides); resident [35..40) + streamed [0..35) matches the bake. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
…ds overlap transcode and device time DasDirectWriter grows a second band and a lazily-spawned writer thread: a full band hands off to the thread while the caller stages into the other, so the converter's transcode work and the device write overlap instead of serializing. One job in flight, handed off in order — the strictly-ascending cursor NTFS's valid-data-length watermark demands is preserved. The aligned-bulk direct path stays synchronous (the caller may retire its pointer the moment append returns) after draining the in-flight band. Async failures land in the writer's ok state at the next append/commit/close; band2 allocation failure falls back to the old synchronous flush. das API and call sites unchanged. Gates (busy box): streamed and eager conversions reproduce the banked SHA256 byte-for-byte — Q4_K_M 2323ebec…34e7 (eager AND streamed), Q8_0 96b97b33…af01; fio suite 194 tests 0 failed. Indicative only (unjudged): streamed planes write 377 -> 564 MB/s (Q4KM) / 712 MB/s (Q8), eager 1500 -> 1627 MB/s. The judged GLM wall-clock A/B waits for the quiet box. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
…act, band/commit, stat counters tests/fio/fio_dwrite.das: odd-size appends (sub-sector through multi-band) round-trip byte-for-byte and close truncates the preallocation to exactly the appended bytes; the band/commit producer path crosses band boundaries (the async flush + swap point) intact; append and band/commit interleave in order; bounce/direct stat accounting; an impossible path declines with null. 5/5, and the suite exercises the new double-buffered writer thread on every boundary flush. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
…ek + B5 weight-bytes fix image_peek reads what tooling needs WITHOUT loading: header fields plus the v7 meta head's baked identity and embedded config JSON (pre-v7 layouts are unknowable and report empty). --list inventories every .dlim beside -m (a GGUF or a directory) with size, version, verdict, identity; verdicts compare against the identities THIS box would produce now across flavors and quants — CURRENT / OTHER (loadable, possibly another box's bake) / STALE vN / BROKEN. --clean GCs only what can never load again (BROKEN + version-stale), never a loadable image; dry-run by default, --apply acts, and an applied clean also sweeps the stale-tmp siblings. Live receipt (Qwen3-4B Q4KM, 9 sibling images): 1 CURRENT, 4 OTHER v7 (trimmed vulkan bakes + portable-backend configs — kept), 4 STALE v6 flagged for --apply. B5: model_weights_bytes now counts q51q/q51s, mblob, vkblob, and the trim planes embq/embs — a flavor image no longer reports its bake bytes as 0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
…g round-trip The mechanics suite (CI-safe, no models) grows: image_peek on the synthetic carrier reads version + baked identity + embedded config without loading, and the embedded config re-derives the exact identity the image was saved under; DlimConfiguration JSON round-trips with the identity intact, garbage declines. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
… 2x2 The win32 mmap shim now honors PROT_WRITE (PAGE_READWRITE / FILE_MAP_WRITE — a real shared writable view, writes go back to the file, still no commit charge), and fmap_open_rw is the writable twin of fmap_open on both platforms. Same ownership contract; fmap_close unmaps either kind. Tests are a flagged follow-up. The consumer that motivated it: VK_EXT_external_memory_host imports. The probe 2x2 (_vk_import_probe.das) is now closed on NVIDIA/Windows: anonymous writable memory imports fine, file-backed memory is refused with ERROR_OUT_OF_DEVICE_MEMORY in BOTH protections — the blocker is the file backing, not the page protection. So the imported-mirror rail cannot engage on this driver under any mapping mode, and the pinned-copy fallback stands; the rail self-enables on drivers that accept file-backed host pointers. make_imported_buf's comment records the proven rule. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
…UT, not the job sum The GLM judged A/B caught a phase-D correctness bug the 4B gates could not see: a plane layout can hold allocated-but-never-written holes — glm4moe's dense-lead layer gets shexp slices in the wsh spans (layout_offsets allocates all 47 layers) but has no shexp tensors on disk, so no FillJob covers them. The streamed fill concatenated jobs, so everything past the first hole landed SHIFTED and the qblob section came up 17,301,504 bytes short (3 planes x one 4096x1408 slice) — a corrupt image that its own header could not detect. Fix: the sizing pass records each streamed q-plane's LAYOUT byte total (g_stream_plane_total); stream_plane_bytes serves that total; fill_stream_plane tracks a byte cursor and zero-fills the gaps between jobs and the tail — exactly the zeros the eager path writes from its zero-initialized RAM planes. Gates: GLM streamed vs eager now cmp-IDENTICAL across the whole 82.4 GB file (peak 16.34 GB held); 4B Q4KM/Q8_0 streamed SHAs unchanged (2323ebec…, 96b97b33…). Judged GLM receipts (quiet box): streamed 739 s / 16.36 GB peak vs eager 880 s / 81.87 GB peak — the writer split shows 99.9% of the eager save inside write syscalls, so conversion is device-bound at the drive's sustained rate and streaming wins both axes. (Rates degraded 112 -> 98 -> 73 MB/s across the three consecutive 78 GB writes — SLC-cache exhaustion; cross-run rate comparisons are drive-state-confounded, the byte parity is not.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
tests/fio/fio_fmap_rw.das: writes through the mapping persist to the file after unmap (verified via a read-only remap), mapped content matches the write, and the decline shapes (missing file, empty file) return null. 3/3. The handmade RST fills the das2rst stub — the symbol rides the File manipulation group beside fmap_open; doc/source/stdlib/generated is CI-regenerated and stays untracked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
…ession coverage test_model_image_vulkan.das (run.das --suite image-vulkan, arm `vulkan`): the runner arms DASLLAMA_GPU + a 2000 MB budget so the probed config carries a vulkan section, the DRY tier collects the bake with zero device calls (safe on GPU-less boxes), and the test asserts a non-empty role-stamped plan whose planes sit inside the blob, then saves the flavor image under the tier's bake tag and maps it back with the plan and blob intact. The offline vulkan bake previously had zero automated coverage — only the manual parity probe. run.das grows the per-suite env hook. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
…utomatic GC, flavor locks Library first: dlim_inventory / dlim_clean / dlim_image_verdict / dlim_current_identities move into dasllama_image (the converter's --list/--clean refactor onto them), so the server and the CLI share one implementation. Server: GET /v1/images lists every slot's .dlim siblings (size, version, verdict, identity) plus the mapped flavor; POST /bake spawns dasllama-convert as a child on the bench worker's proven single-spawn shape (quiesced-only, one child at a time, tune-relaunch retry, the converter's receipt lines surface in the log) — planar slots stream, the GPU owner bakes the vulkan flavor against the server's LIVE config via a --config handoff; GET /bake serves state/log. The dlim GC (only never-loadable images) runs at server start and on every bake completion. ModelSlot carries its source GGUF path; /v1/stats slots grow `flavor`. Page (existing forge style): each model card gets a .dlim line (serving flavor, current/stale counts, total GB, per-file tooltip) and a bake button with completion polling; config GPU knobs lock while the active slot serves a flavor image — a vulkan .dlim cannot switch to cpu mode, the knobs are baked into the file. Gates: server suite 19/19 (new /v1/images + /bake decline arms) with a live tinyllama server; live end-to-end smoke: startup GC reclaimed 7292 MB from 4 stale images, POST /bake spawned the converter, streamed 1299 MB in 18.5 s to the identity path, completion drained with receipts, clean shutdown. env-registry 14/14 (no new knobs); lint + format clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
…byte-exact M1 images The gen repack primitives were already number-parametrized (generic grp<mr> shuffles); only their number SOURCES were stamp-bound. The repack fns now read math's active_* layout getters — value-identical to the stamped companions during normal serving (activation caches from them) — and the getters grow a bake override: pin_backend_for_bake pins a registered name normally, while a FOREIGN name (arm64-gen on a PC) arms the override with the target config's numbers and aliases the identity's reported backend (dlim_backend_name). The q51 family gets a registered bake pointer (x64-gen keeps disk-order q51 slots) reached through active_repack_q51 at both load-path call sites. The converter's --config path uses the new pin; the identity verify stays fail-closed and passes naturally when the override took. Gates (the plan's verification #3, run against a live M1 over ssh): - M1 dump-config -> the PC bakes SmolLM2 Q8_0 with it: identity v7|q8|arm64-gen|s16|q8 mr4 b0 g4|kq 8/8/4/8|q51 mr1|nat 1101 — and the file is BYTE-IDENTICAL to the M1's own local bake (sha256 506d36a9…6ecd), proving the mr4 repack a PC never serves. - K-quant family: Llama-3.2-1B Q4_K_M cross-baked and M1-baked images identical (fa4b678c…b8c0) — covers the differing k6 mr4 and the wscale16 epilogue. - The M1 MAPS the image (MAPPED:true) and serves it token-exact vs its gguf load (16/16 greedy IDS parity). - PC regression: the 4B streamed bake reproduces the banked SHA (2323ebec…34e7) — the getter swap is byte-invisible for same-box bakes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
…ortable The Metal drivers' MODEL-SHAPE gates move verbatim into a new portable module (no das_metal require): the decline enum, MetalNeed + metal_needs/missing/record, kq_fmt/kq_load GPU-support, kv_share_ok, dn_metal_ok, moe_metal_ok + moe_site_ok, DECODE/BATCH_NEEDS_OK, decode_shape_decline (public now), and the mint-time metal_model_servable_impl, whose [init] registration now fires on EVERY build — the piece that build-locked metal-flavor baking to Apple. dasllama_metal_common re-exports the module so the Apple driver family keeps its names; the umbrella requires it unconditionally. The runtime gates (device, PSO, planar, mirrors) stay in the metal modules. PC compile + full dlim chain green; the Apple-side build/lint validation runs on the M1 next (the metal modules cannot compile on Windows). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
… contract) The blob transform reads ROW-MAJOR planes — under a repack backend the servable gate correctly declines backend_repack, which is why bare -f metal failed on BOTH boxes. Portable's layout numbers are box-independent, so the metal identity converges across boxes with no --config: a PC now mints v7|q8|portable|s16|q8 mr4 b0 g4|kq 4/4/4/4|q51 mr1|nat 1101|metal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
…n32 PTY cold-start timeout tests/metal/test_metal_tensor_ops.das: macOS-26 CI runners pass the SDK/MSL-4.0 probe but their PARAVIRTUAL device cannot lower cooperative tensors — every pipeline build fails with the "deferred-static-alloca-size" signature (both open PRs' darwin26 lanes red on it). A signature-gated pre-probe now feints exactly that case before any allocation; every other compile error still fails loudly, so real emitter regressions stay visible on Metal-4 hardware. tests-cpp/small/test_terminal.cpp: the ConPTY probe timed out at 30s waiting for DASHERD_READY on a loaded windows-32 runner — Windows PowerShell's cold start (pwsh absent) can exceed it. 120s; the REQUIREs stay, real transport breakage still fails. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
Contributor
There was a problem hiding this comment.
Pull request overview
This PR advances the dasLLAMA “.dlim” arc by making prepared model images explicit, portable artifacts (config-identified, streamable/offline-convertible, cross-box) and exposing inventory/bake operations through the dasllama-server admin surface. It also adds new fio primitives to support the conversion pipeline and includes a couple of CI stability fixes plus a JIT exception visibility fix.
Changes:
- Add
DlimConfigurationas an explicit, serializable identity for.dlimimages; expand conversion tooling (newdasllama-convert) for streaming/offline/cross-box workflows. - Extend dasllama-server with
/v1/imagesinventory and/bakespawning, plus UI wiring incontrol.html. - Add fio APIs (
prefetch_map,fmap_open_rw,dwrite_*) with tests/docs, and apply targeted CI/test fixes (Metal runner skip signature, Windows PTY timeout).
Reviewed changes
Copilot reviewed 41 out of 41 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| utils/dasllama-server/test_openai_server.das | Adds coverage for /v1/images and /bake endpoints. |
| utils/dasllama-server/openai_server.das | Implements image inventory endpoint and bake child-spawn workflow; tracks slot source paths. |
| utils/dasllama-server/main.das | Passes GGUF source path into server slots and runs .dlim GC on startup. |
| utils/dasllama-server/control.html | Adds UI for .dlim inventory display and bake triggering/polling; locks GPU controls when serving a flavor. |
| utils/dasllama-convert/main.das | New CLI tool to convert GGUF → .dlim (streaming/offline bake/list/clean/config). |
| tests/metal/test_metal_tensor_ops.das | Skips Metal-4 cooperative tensor ops on macOS-26 paravirtual runner signature. |
| tests/fio/fio_prefetch.das | New contract tests for prefetch_map. |
| tests/fio/fio_fmap_rw.das | New contract tests for writable shared mapping fmap_open_rw. |
| tests/fio/fio_dwrite.das | New mechanics tests for direct sequential writer (dwrite_*). |
| tests-cpp/small/test_terminal.cpp | Increases Windows ConPTY integration timeout to reduce CI flakes. |
| src/builtin/module_jit.cpp | Ensures guarded JIT exception printer flushes output. |
| src/builtin/module_builtin_fio.cpp | Adds prefetch_map, fmap_open_rw, and dwrite_*; fixes Windows mmap mapping mode to avoid commit charge. |
| modules/dasLLAMA/tests/test_model_image.das | Extends image tests to validate v7 meta peek + config round-trip. |
| modules/dasLLAMA/tests/test_model_image_vulkan.das | New test for offline/dry Vulkan bake plan collection + .dlim round-trip. |
| modules/dasLLAMA/tests/test_env_registry.das | Includes new converter directory in env registry scan roots. |
| modules/dasLLAMA/tests/run.das | Adds image-vulkan suite; improves TMPDIR/TEMP handling; uses current executable by default; Windows env-prefix handling. |
| modules/dasLLAMA/tests/CLAUDE.md | Documents new image-vulkan suite intent and arming. |
| modules/dasLLAMA/ENVIRONMENT.md | Documents new env knobs (DASLLAMA_PREFETCH, DASLLAMA_VK_IMPORT, DASLLAMA_TRIM, TEMP). |
| modules/dasLLAMA/dasllama/dasllama_transformer.das | Ensures portable Metal shape-gate registration is always pulled in. |
| modules/dasLLAMA/dasllama/dasllama_metal_shapes.das | New portable Metal servability gates (no das_metal dependency) for cross-box baking. |
| modules/dasLLAMA/dasllama/dasllama_metal_llama.das | Removes duplicated shape-gate logic now extracted into dasllama_metal_shapes. |
| modules/dasLLAMA/dasllama/dasllama_metal_common.das | Re-exports extracted shape gates to keep driver family naming consistent. |
| modules/dasLLAMA/dasllama/dasllama_math.das | Adds dry-bake/slice-window/unmap-notify/would-accept hooks and cross-box bake layout overrides. |
| modules/dasLLAMA/dasllama/dasllama_math_gen.das | Routes repack slots through override-aware layout getters; registers q51 repack for cross-box bake. |
| modules/dasLLAMA/dasllama/dasllama_layout.das | Refactors repack region collection and registers streaming layout hooks. |
| modules/dasLLAMA/dasllama/dasllama_env.das | Registers new env knobs and TEMP fallback in env registry. |
| modules/dasLLAMA/dasllama/dasllama_config.das | New DlimConfiguration and identity/tag/JSON helpers for .dlim configuration. |
| modules/dasLLAMA/.das_module | Exports new modules (dasllama_config, dasllama_metal_shapes). |
| include/daScript/simulate/aot_builtin_fio.h | Exposes new fio builtins for AOT. |
| doc/source/stdlib/handmade/function-fio-prefetch_map-0xc63f5d4e91b2e653.rst | Adds stdlib docs for prefetch_map. |
| doc/source/stdlib/handmade/function-fio-fmap_open_rw-0x2abb9e9d0dfc252e.rst | Adds stdlib docs for fmap_open_rw. |
| doc/source/stdlib/handmade/function-fio-dwrite_stat-0xbf270ab2cb2df9d6.rst | Adds stdlib docs for dwrite_stat. |
| doc/source/stdlib/handmade/function-fio-dwrite_open-0x46f123a0dc4b0ce0.rst | Adds stdlib docs for dwrite_open. |
| doc/source/stdlib/handmade/function-fio-dwrite_commit-0x122d2094a7b99d77.rst | Adds stdlib docs for dwrite_commit. |
| doc/source/stdlib/handmade/function-fio-dwrite_close-0xca8d8214309caf7.rst | Adds stdlib docs for dwrite_close. |
| doc/source/stdlib/handmade/function-fio-dwrite_band-0x160228ade96e97be.rst | Adds stdlib docs for dwrite_band. |
| doc/source/stdlib/handmade/function-fio-dwrite_append-0x6658e660a37b258d.rst | Adds stdlib docs for dwrite_append. |
| doc/reflections/das2rst.das | Adds fio doc grouping for new direct-IO/mapping advisory APIs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…nts, Copilot fixes - DAS_NO_FILEIO: the 7 dwrite/prefetch builtins get their GENERATE_IO_STUB entries (build_eastl links registrations without fileio bodies) - fio_dwrite test: the direct-vs-bounce stat split is platform-dependent (win32 demands sector alignment, POSIX align=1 takes band-sized runs direct) - assert conservation (direct + bounce == total) plus sub-sector staging instead of the win32-only split - test_model_image_vulkan: STYLE030 nolints - the body is static_if'd on the vulkan module existing, so no-vulkan lint builds see every body-only require as unused - POST /bake: a malformed JSON body is now 400 instead of silently baking the default slot - shutdown: leave the bench/bake channel to a still-running worker (channel_remove throws on outstanding refs) instead of aborting cleanup - dwrite header comment: per-platform cache semantics stated honestly (NO_BUFFERING / F_NOCACHE / buffered + fadvise DONTNEED) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 41 out of 41 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
utils/dasllama-convert/main.das:353
apply_box_profile_runtime()is called a second time inside the streaming conversion branch, after--configmay have pinned a backend viapin_backend_for_bake(...). Becauseapply_box_profile_runtimecan re-pin the kernel backend from the tune sidecar, this can silently override the already-verified config/pin and break the tool’s “fail-closed” config identity guarantee for streamed planar bakes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
…reaming branch It re-read the tune sidecar AFTER the --config override and its fail-closed identity verify, so a sidecar carrying runtime knobs (backend pin, wscale_f16, kquant natives) could silently revert a pinned cross-box streamed bake. The first call (before --config) already applies the sidecar; the config override must stay last. Surfaced by a Copilot low-confidence remark - verified real. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 42 out of 42 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
tests/fio/fio_dwrite.das:150
- This test assumes
"Z:/no/such/..."is an impossible path, but on POSIX it’s just a relative path with missing intermediate directories (and could theoretically exist), making the intent less portable/robust. Use a path that is guaranteed to be invalid across platforms (e.g. empty string) to assert the “declines without crashing” contract deterministically.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The .dlim arc: the prepared-model image becomes an explicit, offline, streamable, cross-box artifact — plus the server surface over it, and two master-level CI lane fixes.
The format (one IMAGE_VERSION bump, 6 -> 7)
DlimConfiguration(dasllama_config.das): the config that keys an image is now an explicit struct (cpu backend + layout numbers, vulkan section with resolved caps/wants, metal section) instead of probed process state. The identity is a pure formatter of it; the identity string and the config JSON are embedded in the image meta head, so a mismatched load prints what the image was baked for instead of a hash miss.dasllama-convertgrows--config/--dump-config(fail-closed identity verify),--stream,-f vulkanoffline bake,--trim,--list/--clean(the dlim GC: only never-loadable images are removed; dry-run by default,--applyacts).Conversion
load_bigrecords FillJobs; the fill pass streams source -> transcode -> image without materializing the big q-planes. Byte-identical to eager (SHA-gated on Q4_K_M and Q8_0); GLM-4.5-Air peak commit 81.9 GB -> 16.34 GB. Streamed planes are sized and gap-filled from the LAYOUT (a job-sum concatenation shifted everything past the first allocated-but-never-written hole — caught by the GLM byte gate, cmp-identical across the full 82.4 GB after the fix).prefetch_map(fio): advisory readahead over a mapped range; cold-conversion weight reads -66% (device-saturated).fmap_open_rw: a writable SHARED mapping twin offmap_open(win32 mmap shim honors PROT_WRITE).The offline vulkan bake (plan-as-data)
VkPlanEntrycarries a stagerole; the slice replay verifies role + shape per gather and is boundary-exact: a would-refuse attempt (the bake never records refused gathers) is served empty via the sharedmoe_gpu_would_acceptarithmetic, so budget-bounded models replay their refusal instead of falling back to live gathers.--trim): the vulkan flavor drops the CPU weight families a resident plan never reads (-44% on Q4KM); embedding rows split into their own planes; all CPU fall-backs fail closed.Cross-box (bake an M1 image from a PC)
active_*layout getters (value-identical to the stamps in normal serving); a foreign-backend bake overrides them with the target config's numbers and aliases the reported backend. Gates: PC-baked SmolLM2 q8 and Llama-3.2-1B Q4_K_M images are byte-identical to the M1's own bakes (sha256 both families, incl. the differing k6 mr and wscale16); the M1 maps and serves them token-exact.dasllama_metal_shapes: the metal servable predicate's pure-model shape gates extracted portable (registration fires on every build);-f metalpins the portable backend (the donor contract — bare-f metalwas broken on both platforms). A Windows box now mints metal-flavor images byte-identical to the M1's; the M1 metal GPU-serve arm passes post-extraction.Server surface
GET /v1/images(per-slot inventory with verdicts),POST /bake(spawns the converter on the bench worker's proven pattern; the GPU owner bakes the vulkan flavor against the live config), automatic GC at start and on bake completion, config GPU knobs lock while a flavor image serves. Live-gated end to end.Tests / docs
tests/fio/fio_dwrite.das,fio_fmap_rw.das,fio_prefetch.das;test_model_imagepeek/config arms;test_model_image_vulkan.das(the offline dry bake's first regression coverage, viarun.das --suite image-vulkan). Handmade RST for the seven new fio symbols + the das2rst group.jit_run_main_guardednever flushed its printer).CI fixes (master-level, the two red PRs' lanes)
test_metal_tensor_opsnow feints on that exact compile-error signature (any other error still fails loudly on Metal-4 hardware).🤖 Generated with Claude Code
https://claude.ai/code/session_01LcZKEkh1e2z8zPrLX1pKR2