W7900 update - #6
Merged
Merged
Conversation
Data (docs/results/w7900-gfx1100.md): W7900 (gfx1100) Study 2 throughput — baseline at c (=-np) 1/4/16/32 and DFlash at c=1/4, with concurrency scaling, measured VRAM, and a gfx1151 comparison. All numbers verbatim from cell JSONs. Scripts (scripts/w7900-repro/): two-method reproduction — Method 1 (Docker image, llama.cpp baked in) and Method 2 (bare metal, user-provided gfx1100 llama-server). Both reuse the committed harness unmodified and render a 12-cell matrix (baseline c=1/4/16/32 + DFlash c=1/4). Models download from the official Hugging Face; no proxy/mirror is hard-coded. Status: W7900 -> Validated in README.md, docs/hardware-validation.md and docs/results/README.md. .gitignore: exclude the local models symlink and the repro _out/ so model weights never become part of the project.
…to main README Method-1 Dockerfile now `FROM flagos/flagtree-amd-tle:rocm7.2.4` (like ../../deploy/Dockerfile) instead of the prebuilt ACR image: installs huggingface_hub + aiohttp, pins the official Hugging Face, and COPYs the prebuilt gfx1100 llama-server from bin/ (staged by the user — a custom muse-glimmer + DFlash build, not upstream, so not compiled here). IMAGE default -> muse-glimmer-llamacpp:repro; the private registry URL is dropped from the package. bin/llama-server is gitignored; bin/README.md documents staging. README.md: add the W7900 (gfx1100) Study 2 throughput table next to the gfx1151 (AI MAX 395) one — baseline at c=1/4/16/32 and DFlash at c=1/4 — and update the "validated here" line and results preamble to include the Radeon dGPU.
Study 1 (Meta-aligned greedy anchor): add measured W7900 rows — 17GB 33.19 -> 63.98 tok/s, dynamic 30.23 -> 58.26 tok/s (both ~1.93x, draft acceptance ~0.24), measured with the repo harness (study1.conf: greedy temp=0, batch 1, 3 reps). Study 2: merge the two per-GPU tables into one table with a GPU column (gfx1151 c=1/4/16; W7900 c=1/4/16/32, gfx1151 has no c=32).
…context) Two W7900-tuned llama-server presets, grounded in the measurements: - High throughput: `-np 16 -c 1048576` (16 slots x 65536/slot), DFlash off. - Long context: `-np 1 -c 131072` (full 128K) + DFlash, or `-np 2 -c 262144`. Measured on the live W7900 (`-fa on`): high-throughput 30 GiB (17GB) / 33 GiB (dynamic); long-context (`-np 2 -c 262144`) 22 GiB — all fit the 48 GiB card with margin. llama.cpp pads each slot's context to a multiple of 256, so use round sizes (16x65536 = 1048576, not 1048560; a bare 65535/slot rounds up to 65536).
Resolve conflicts keeping the two tracks distinct, per design: - gfx1151 stays on ROCm 7.14 — adopt upstream's updated Study 1/2 numbers, intro/preamble reframing, and the CI-checked generated hardware blocks. - W7900 (gfx1100) is a separate community-validated Radeon dGPU track on our ROCm 7.2.4 — layer its Study 1/2 rows onto upstream's tables, add the serving presets, mark it "🧪 Community validated" in the generated matrices, and add configs/public-claims.json + the evidence manifest so scripts/check_claim_consistency.py passes. No experiments were re-run.
…d-start blocker) gguf-quickstart.sh clones llama.cpp with --filter=blob:none --no-checkout, which leaves an empty worktree and no index; git diff then reads every tracked path as staged-deleted, so the dirty-tree guard refused to change commits on every cold start and every rerun. The clone is created by the script itself and cannot hold user work, so fetch and detach at the selected ref immediately after cloning, before any guard runs. Factor the guard decision into llama_has_tracked_changes() (scripts/lib/llama_build.sh) which treats the script's own index-less empty-clone state as clean while still refusing genuinely dirty populated checkouts on later runs; the ref-integrity check is unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
Both dirty-checkout refusals in gguf-quickstart.sh dead-ended users with a bare one-liner. Route them through llama_refuse_dirty_checkout() which names the actual situation (switching commits vs reusing the checkout), excerpts the first 10 lines of git status --porcelain, offers a stash (keep) and a checkout -- . (discard) recovery verb, and points at the new docs/troubleshooting.md#dirty-llama-cpp-checkout entry. Co-Authored-By: Claude <noreply@anthropic.com>
The installer's only pipeline - the cosmetic hipcc version tail piped into head -1 - SIGPIPEd under set -o pipefail once head closed the pipe, turning a fully successful, size+SHA256-verified 1.6 GiB install into exit 141. Capture the output first and print line 1 without any pipe. Tests drive the real full-install path end to end (download -> verify -> extract -> mv -> version tail) via a new ROCM714_MANIFEST test seam with a stub tarball, stub curl and a five-line hipcc, asserting exit 0 and a single version line; a second pair pins the mechanism itself (old pattern 141, new pattern 0 under identical pipefail options). Co-Authored-By: Claude <noreply@anthropic.com>
The 1.6 GiB tarball stayed in $TMPDIR (tmpfs, i.e. RAM, on UMA hosts) after success with no message and no README mention. Delete it once size+SHA256 verification and extraction have succeeded and say so on one output line; failure paths keep the archive so a bad download can be inspected or retried. Co-Authored-By: Claude <noreply@anthropic.com>
A tight disk surfaced only as a raw curl/tar 'No space left on device' mid-install. Check each target filesystem against its own worst case before downloading: the archive filesystem needs the manifest's size_bytes (~1.6 GiB), the prefix filesystem needs the extracted tree - 8.3 GiB measured on the validated install, rounded to a 9 GiB constant since the manifest carries no extracted size. Refusal output names the target path, required vs available GiB, and the TMPDIR/ROCM714_PREFIX escape hatch. README states the ~11 GiB transient disk peak next to the existing ~20 GiB figure. Co-Authored-By: Claude <noreply@anthropic.com>
python3 (the manifest reader) and curl (the downloader) were used unguarded; on a host missing either, the installer died inside read_field with a raw bash 'command not found'. Verify both exist before first use and exit 1 naming the tool plus a per-distro install hint (apt/dnf/pacman). Co-Authored-By: Claude <noreply@anthropic.com>
Three failure exits said only what failed. Size mismatch now prints expected vs actual bytes, SHA256 failure prints expected vs computed hash, and both give the delete-the-partial-archive-and-rerun command. A missing post-extract bin/hipcc names the incomplete extraction and points at disk space plus the move-aside rerun. Co-Authored-By: Claude <noreply@anthropic.com>
--profile gguf verified none of the README-declared host tools yet printed 'OK: gguf environment ready'; a user missing cmake sailed through step 3 and met 'ERROR: required command not found: cmake' in step 4, and a host without python3 killed the checker itself with a raw bash error at the first read_manifest call. - itemize git/cmake/curl/python3 with per-tool pass lines feeding the verdict; a missing tool FAILs the check with the tool name plus the same per-distro install hints (apt/dnf/pacman) as install-rocm-7.14.sh - guard python3 at the top of the script, before the first manifest read, so the checker prints a proper FAIL with the install hint instead of dying with 'command not found' Co-Authored-By: Claude <noreply@anthropic.com>
'GPU-visible pool: 80 GiB' printed bare, so a user could not self-assess against the project's own floors, and the README's '~20 GiB available' disk figure had no labeled counterpart. - the gguf passing line now carries the manifest-derived default-GGUF floor (15.6 GiB, from the same value the hard-fail below it uses) and the 80 GiB validated envelope, explicitly labeled a warning boundary - the vllm/reference passing line carries the profile's 60 GiB floor - both lines label the unit as GPU-visible memory so disk and GPU-visible cannot be confused Co-Authored-By: Claude <noreply@anthropic.com>
…tion Three checker failure exits said what failed but not what was expected or what to do next: - ROCm-version mismatch: now prints the expected 7.14.x/7.2.x alongside the observed version and the next action (run scripts/install-rocm-7.14.sh or set ROCM_PREFIX) - gfx1151 absent: lists the GPU id(s) rocminfo actually reported and points non-gfx1151 platforms at docs/hardware-validation.md - pool below the default GGUF: states available and required GiB in the same units and hints the smaller-quant escape hatch (GGUF_FILE), with the BIOS/UMA lever as the alternative Co-Authored-By: Claude <noreply@anthropic.com>
The port-availability gate was the last check before exec, so a busy port
cost the full reuse chain (ROCm resolve, tool checks, checkout guards,
build fingerprint compare, 7.4 s artifact re-hash; minutes on a
rebuild-needed tree) before refusing. Hoist it to the top of the script,
right after set -euo pipefail/sourcing/cd and before the plan header,
ROCm resolution and the host-tool loop. The probe now catches the bind
OSError and sys.exit(1)s, so the exemplary ERROR line is no longer
preceded by a raw Python traceback (F-11). PORT env knob preserved; the
gate stays silent when the port is free. The probe falls through to the
required-tools guard when python3 itself is missing instead of
misdiagnosing that as a busy port.
Ref: Fix-A review cosmetic - the checkout-reuse log line said "no fetch
needed" seconds after the fresh-clone branch fetched; reworded to state
the checkout's actual condition ("at $LLAMA_CPP_REF; working tree clean").
Evidence: /home/amd/Desktop/muse-newuser/FINDINGS.md F-11, F-16.
Co-Authored-By: Claude <noreply@anthropic.com>
Add an upfront available-space check mirroring the installer's df -Pk pattern (scripts/install-rocm-7.14.sh): the filesystem holding MODEL_DEST is checked against the manifest-derived floor (sum of size_bytes for each selected artifact not yet present locally; WITH_DFLASH/WITH_MMPROJ add the drafter/projector), and the build filesystem against a 1.5 GiB checkout/build allowance (206 MiB validated worktree + 1022 MiB build-714 tree measured, rounded up - the task sketch said ~1 GiB, but the measured 1.2 GiB would make that floor false-pass). When both directories share a mount the floors are combined and checked once, fixing the shared-mount undercount. Refusal prints expected-vs-got GiB, the filesystem and the directory it holds, and a remedy naming the MODEL_DEST escape hatch. Artifacts already on disk are skipped for the floor: re-requiring their space would lock warm reruns out of serving on any disk that filled up after the download. The quarantine+refetch path stays covered because a quarantined artifact is absent on the rerun, so the preflight re-engages before the refetch can start. Evidence: /home/amd/Desktop/muse-newuser/FINDINGS.md F-03. Co-Authored-By: Claude <noreply@anthropic.com>
…_size-1)
Every DFlash call site requested n-max 16, which upstream unconditionally
clamps to block_size - 1 = 15 with a permanently-emitted warning line at
each server start, and the two label sites misstated the cap ("n_max=32
caps at 16 because 16 is the DFlash block_size"; "16 ... equals the DFlash
drafter's block_size"). Perf delta 15-vs-16 is nil (the sweep is flat past
the elbow), so request the effective maximum directly:
- gguf-quickstart.sh: SPEC_DRAFT_N_MAX=15 constant consumed by SERVER_ARGS
- gguf_bench_args.py: SPEC_DRAFT_N_MAX = 15 + corrected physics comment
- check_dflash_equiv.sh: harness requests 15
- docs/troubleshooting.md dflash-silent-noop: cap is block_size - 1 = 15;
recorded sweep's 16/32 cells effectively ran at 15
docs/results/ is frozen recorded data and keeps its historical n_max=16
sweep labels by design. The clamp warning - one of F-09's noise lines -
disappears at the source.
Co-Authored-By: Claude <noreply@anthropic.com>
…spec args WITH_DFLASH=1 used to be invisible until the drafter's fetch line appeared mid-run: the plan header was byte-identical to a non-DFlash run, the ~1.52 GiB drafter download was an unannounced cost, and --spec-type draft-dflash appeared nowhere on screen (the server never echoes its argv). WITH_MMPROJ=1 had the same shape, folded in under the same "flag acknowledged" principle: - plan header: a dflash block (drafter filename + manifest size to fetch, or "already present; will be verified" when on disk, plus "spec decoding : draft-dflash") and a one-line mmproj entry - serve start: the effective speculative arguments on one line just before exec, e.g. "speculative decoding: draft-dflash (draft: models/dflash-kquant.gguf, n-max 15)" Sizes come from configs/artifact-manifest.json, read the same way the disk preflight already reads it (artifact_size_bytes). Co-Authored-By: Claude <noreply@anthropic.com>
…start
During a fully successful DFlash load, upstream llama.cpp prints
E "failed to initialize the context ..." (with an easy-to-miss
"this warning is normal" caveat) and W "[spec] failed to measure draft
model memory" exactly in the window where the user is checking whether
DFlash engaged - indistinguishable, on screen, from the silent-noop
failure mode the README warns about. The lines cannot be silenced
without touching llama.cpp, so the quickstart now prints one
plain-English note from its own tooling just before exec, naming the
definitive confirmation ('adding speculative implementation') that
follows. F-18 already removed the third noise line (the clamp warning)
at the source.
Co-Authored-By: Claude <noreply@anthropic.com>
The Quick start ended at a bare server-URL comment while the quickstart
execs a foreground llama-server, and no request example existed anywhere
in README/docs, so verifying the 18-minute install presupposed
OpenAI-API knowledge (F-06/F-07, both observed LIVE in the audit).
- F-06: comment guidance to leave the terminal running (Ctrl-C stops
the server) and use a second terminal.
- F-07: new "Verify it works" block with the two audit-verified
requests: GET /health -> {"status":"ok"} and a POST
/v1/chat/completions curl whose expected output is shown (content
"OK" after ~50-70 hidden reasoning tokens, finish_reason stop).
- F-08: reasoning-first note next to the example: small max_tokens
spends the whole budget on reasoning_content and returns empty
content with finish_reason "length" (HTTP 200); use max_tokens >=
512 or omit it. Links the existing troubleshooting#reasoning-length
entry.
tests/test_docs_ux.py pins the promise, mirroring the existing
doc-content test pattern (test_dflash_nmax.py, test_llama_checkout_guard.py).
Co-Authored-By: Claude <noreply@anthropic.com>
No remediation command for any declared host prerequisite existed anywhere in README/docs (F-13, STATIC): a bare-OS user meeting 'required command not found' had no documented install path at any click depth, and the checker's fail()/missing_tool_fail() pointers led to a troubleshooting file with no matching entry. - README Requirements: fenced block with per-distro one-liners for git/cmake/curl/python3 (apt-get/dnf/pacman), using the exact verbs install-rocm-7.14.sh and 00-check-env.sh already print in their failure text (F-12 wording, kept in lockstep). - docs/troubleshooting.md: new #missing-tool entry in the established symptom -> cause -> fix shape, discoverable from both the ERROR (installer/quickstart) and FAIL (checker) wording, plus a jump-table row. The scripts themselves are unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
F-17 (STATIC): MODEL_DEST was honored by quickstart.sh, gguf-quickstart.sh and 02-fetch-model.sh but documented nowhere, so every clone re-downloaded the 15.6 GiB model by default with no dedup escape hatch visible. The quickstart's disk-preflight remedy text (Fix-F03) also points at the README for this knob. Document it in the Reproducibility contract overrides area: an absolute MODEL_DEST reuses the hash-verified model across clones (it moves the file, not the validated revision - manifest hash checks still apply). F-10 fold: the optional-features intro now sizes the two extra downloads (mmproj ~1.3 GiB, DFlash drafter ~1.5 GiB, from configs/artifact-manifest.json) alongside the existing ROCm ~1.6 GiB and GGUF 15.6 GiB figures. Co-Authored-By: Claude <noreply@anthropic.com>
…pointer Three ledger fold-ins from the audit fix phase: - Fix-A fold (docs/troubleshooting.md#dirty-llama-cpp-checkout): a checkout interrupted mid-switch (Ctrl-C during a commit change) can leave a tree that 'checkout -- .' does not settle; document the reset --hard <pinned-ref> and remove-and-rerun alternatives alongside the existing stash/discard verbs. - F-18 fold (docs/adaptation.md): the DFlash requirement line said --spec-draft-n-max 16 while every call site now requests 15 (upstream clamps at block_size - 1 = 15); one-word consistency fix, matching docs/troubleshooting.md and test_dflash_nmax.py. - F-14 fold (docs/hardware-validation.md): short BIOS/UMA sizing section - the 80 GiB pool is the checker's warning envelope, not a requirement; hosts below it should compare against the thresholds 00-check-env.sh prints (GPU-visible, not disk) and the README requirements; distinguishes the uma-bug kernel issue from sizing. tests/test_docs_ux.py grows matching assertions for all three. Co-Authored-By: Claude <noreply@anthropic.com>
Task review approved Fix-F with one Important finding plus two cheap minors; this commit rewords the docs (scripts untouched) so they describe what the failure text actually prints. - Important (F-13): README Requirements and the troubleshooting #missing-tool Fix both claimed the failure text prints "these same one-liners" (the all-four-tools blocks). Reality: gguf-quickstart.sh prints only the bare ERROR line, while install-rocm-7.14.sh and 00-check-env.sh print the per-distro command naming the single missing tool. README now says the installer and environment checker print the matching per-distro install command for the missing tool; the troubleshooting Fix drops the "same one-liners" claim and spells out which scripts print a hint (the quickstart stops with the bare error line). The fenced all-four blocks stay as the install-all lead-in. - Minor 1 (F-08): README reasoning-first blockquote says "at least ~50-70 tokens" — reasoning can run to hundreds at the default reasoning_strength=high, so the range is a floor, not a typical value. - Minor 2: docs/hardware-validation.md now uses the checker's verbatim term "warning boundary, not a minimum" (00-check-env.sh line) instead of "warning envelope". tests/test_docs_ux.py: pin the corrected F-13 claim (and assert the "same one-liners" phrasing stays gone) and the checker-verbatim boundary term; update the module docstring to match. Co-Authored-By: Claude <noreply@anthropic.com>
Unreleased entries for the user-visible fixes on this branch: cold-start blocker, installer exit 141, disk preflights, port-probe traceback and PORT value guard, checker failure text, dirty-checkout recovery guidance; archive cleanup with pre-placed ROCM714_ARCHIVE semantics, DFlash n-max 15, port gate hoist; README verify block, tool one-liners, MODEL_DEST. Co-Authored-By: Claude <noreply@anthropic.com>
When the archive staging dir and the prefix parent sit on one filesystem, the true peak is the archive and the extracted tree coexisting during tar. Mirror gguf-quickstart.sh's fs_of_dir walk-up + mount compare and check the combined archive+extracted floor once against that mount; two separate checks against the same mount each passed at 10 GiB while the 14 GiB peak still ENOSPC'd mid-extract. The harness df stub now reports per-mount names so same-mount and separate-mount scenarios are both staged. Co-Authored-By: Claude <noreply@anthropic.com>
A garbage PORT (e.g. abc) raised ValueError inside the port probe: the traceback leaked and the gate then misreported the port as already in use (live-verified). The probe now parses the port first and exits with its own code for an unparseable value, and the gate reports 'PORT=<v> is not a usable port number' for it; the busy-port message and every other nonzero probe exit keep the existing exact wording. Co-Authored-By: Claude <noreply@anthropic.com>
Pin the documented idempotency contract: with a valid prefix/bin/hipcc present, a rerun exits 0 fast, prints the nothing-to-do line, and never invokes curl (poison-curl stub proves it while still satisfying the top tool guard). The final-review found this path untested. Co-Authored-By: Claude <noreply@anthropic.com>
Only delete the archive when this script downloaded it (DOWNLOADED flag set at the curl step): a user-pre-placed ROCM714_ARCHIVE is verified, used, and left in place after success. A pre-placed archive whose size matches the manifest expectation also skips the re-download - the size+SHA256 gate still runs next, and any other size is re-downloaded over and cleaned up as before. The disk preflight still reserves the archive floor deliberately (conservative one-time check, not a warm-rerun gate). Co-Authored-By: Claude <noreply@anthropic.com>
Maintainer ruling: master advanced via PRs AIwork4me#4/AIwork4me#5 (v0.1.0 publish) after the plan was written; the new-user-today sha governs the audit. Co-Authored-By: Claude <noreply@anthropic.com>
docs: new-user one-pass audit spec and plan (audit trail)
Promotes the Unreleased section (18 one-pass audit fixes, PR AIwork4me#7) to [0.1.1]; bumps CITATION.cff and pyproject to 0.1.1. Co-Authored-By: Claude <noreply@anthropic.com>
uv.lock root-package version follows the pyproject bump so uv sync --locked and CI resolve on the release commit. Co-Authored-By: Claude <noreply@anthropic.com>
release: v0.1.1 — one-pass audit fixes
aiohttp's content iterator yields arbitrary byte chunks; at np=16 most chunks carried several coalesced SSE events, so the per-chunk parser failed json.loads on them and lost the usage event — the np=16 17gb baseline cell of 2026-08-15 recorded 96 tokens where the server log showed ~173k decoded (corrupt record quarantined to /tmp/np16-incident/, never published). Buffer and split on newlines so parsing is independent of chunk boundaries; EOF-tail parsing keeps the legacy behavior for a final unterminated event. Regression tests lock both failure shapes (whole-stream-as-one-chunk and 7-byte splits) and fail on the old code.
~188k was the total (13,846 prompt + 174,045 generated) from the 2026-08-15 np=16 incident server log; the generated-only figure is ~174k, matching the CHANGELOG entry.
The np=16 17gb baseline cell on ROCm 7.14.0, run 2026-08-15 with the newline-framing SSE client: agg 36.97 tok/s (7.2.1: 34.47, +7.3%), TTFT p50 2.25 s (2.12), TPOT median +4.9%, total tokens 32,529 (29,801). VmPeak is +16.1% vs 7.2.1, so the 'lower VmPeak' statements stay scoped to the original 17 cells. The pre-fix attempt's corrupt low-count record (96 tokens vs ~174k server-side) was quarantined, never published. Scope moved from 17/21 to 18/21: validation manifest, its schema (deferred_cells relaxed to 0-4, completed_cells pinned to 18), README generated validation-tracks block, TPOT claim generator (now renders the np=16 pair), benchmark/methodology/results READMEs, and the cell-count test all updated together. Three np=16 cells stay deferred; DFlash c=16 remains pathological per the existing warning.
Verifier-found stragglers outside the claims gate: schemas/README.md (two spots) and handoff.md; also scope the VmPeak statement in the 7.14 result summary to the original pass and refresh comparison.md so it matches the generator byte-for-byte.
Second np16 baseline cell (study2 dynamic, fixed client): agg 32.01 tok/s vs 7.2.1's 31.05 (+3.1%), TTFT p50 +7.1%, TPOT +26.1%, VmPeak -1.6%. Provenance: an unrelated llama-bench sweep briefly shared the GPU during this cell's first repetition (12:03-12:07); the affected rep is the visible min (18.2 vs median 32.0 tok/s) - disclosed in the scoped result. The 17gb np16 DFlash full-fidelity attempt (MAX_TOKENS=512, REPS=5) decayed from ~74 to ~35 prompt-tokens/min and was aborted by the operator at ~2h (~45% done); the bounded 16x48 probe (16.9 tok/s, 19.3% acceptance) is committed as probe-study2-17gb-np16-df1-max48-reps1.json. Both np16 DFlash cells stay deferred; the do-not-combine warning now covers both runtimes. Scope 18/21 -> 19/21: manifest, schema (const 19, deferred = 2 DFlash cells), README generated block, TPOT claim (np16 pairs mean +15.5%), benchmark/methodology/results READMEs, schemas README, handoff, cell-count test, renders and checksums.
Exclusive-GPU llama-bench pp512/tg128 sweeps over -fa off/on x -ub 256/512/1024 on both weights (raw JSON committed beside the matrix): fa=on wins decode on both (+2.3..+2.8% 17gb, +1.7% dynamic; tg128 10.95 / 9.50 t/s best), -ub is decode-insensitive. Documented as descriptive evidence in the scoped result; validated matrix flags and defaults unchanged (llama-bench cannot exercise DFlash in this build, so fa x DFlash interaction is explicitly untested).
Verifier-found staleness: public-claims.json still said 18/21 with three deferred cells. The scope string is now derived-checked against the validation manifest (completed/planned cell counts and the deferred-cell count), so it cannot silently drift again. Also refreshes the np=16 group comment and two last-digit figures in the fa-sweep table (+2.3..+2.9%, 309.2 t/s).
validate_forward_tracks() must stay generic for synthetic/future track sets; the GGUF scope gate now applies only when both the GGUF track and the real manifest are present.
…oalescing fix(bench): SSE newline-framing; np16 baselines measured (19/21); c16+DFlash pathology; fa micro-sweep
Controlled diagnostic probes on the 7.14 stack (2026-08-15) isolated the deeper mechanism behind the c=16 + DFlash pathology and it is now reported upstream as ggml-org/llama.cpp#27117, reproduced on master 0177dcc: - the drafter's predictions are corrupted per slot, from the first speculative tick, once roughly 8+ sequences draft concurrently; - the trigger is the concurrently-drafting sequence count - not the server's -np configuration (an -np 16 server with 4 in-flight requests reproduces -np 4 bit-for-bit) and not the batch token count; - --spec-draft-n-max 1 sidesteps the corruption (0.83-0.92 acceptance at -np 16, ~2.2x the no-spec baseline in the probes); - identical behavior on ROCm 7.2.1 and 7.14.0. README known-good/bad table, troubleshooting, methodology section 6, the benchmark warning block, the scoped 7.14 result and the changelog now link the issue. The probes are diagnostic records carried by the upstream issue, not matrix cells; the two deferred np=16 DFlash cells stay deferred pending an upstream fix. Claim-consistency gate: ok.
…ash-c16-issue docs: link upstream llama.cpp#27117 for the c16+DFlash root cause
Restructure the README for first-visit readability (357 -> 250 lines): - 'What you get' bullets and the performance highlights now open the page; the quick start follows immediately. - Detailed explanations moved to a new docs/getting-started.md (prerequisites with per-distro tool one-liners, installer internals, download sizes, the reproducibility contract and env overrides), with the README linking it; a documentation map was added. - The adaptation summary table was dropped in favor of the fuller adaptation.md delta table; the memory-terminology paragraph was redundant with METHODOLOGY section 5, which the map points at. No claim changed: the three README generated blocks, the TPOT sentence, the known-good/bad table (including the upstream llama.cpp#27117 link) and the claims-hygiene caveats are preserved verbatim or linked. The docs-UX tests that pinned the moved content were updated to its new location; the hardware-validation README anchor now targets #requirements. Gates: claim-consistency ok, pytest green.
…e-first-visit-ux docs(readme): lead with benefits, performance and quick start
Add files via upload
Update README.md
…on package Conflict resolution: master's restructured README (AIwork4me#12-AIwork4me#14) kept; W7900 Study-2 rows integrated into the unified performance tables. The PR's Study-1 W7900 rows (33.19/63.98, 30.23/58.26 tok/s) were NOT carried: they have no committed cell JSONs, manifest entry or results-doc section, which violates the evidence-first policy the PR's own checklist states.
…link, link-check gap Three issues found while verifying PR AIwork4me#6 on a W7900 host: 1. 00_prepare.sh: the documented mirror invocation (HF_ENDPOINT=https://... bash 00_prepare.sh) crashed with 'No such file or directory' — bash treats a command word built from ${HF_ENDPOINT:+HF_ENDPOINT=...} as a command name, not an assignment. export HF_ENDPOINT instead; huggingface_hub reads it from the env. Reproduced broken, then verified the fix downloads + size-checks all four GGUFs end to end. 2. w7900-repro/README.md linked ../../deploy/Dockerfile, which does not exist in this repository; point at the package's own Dockerfile. 3. test_markdown_links.py did not cover the new documents, which is how (2) slipped through; add w7900-gfx1100.md, the manifest and the repro README to the checked set.
- Commit 9 raw cells from an independent reproduction on a second W7900 host (ROCm 7.2.1 system stack, upstream llama.cpp 0b1bad14f, kernel 6.8.0-79-generic). Every cell self-labels rocm_version/kernel/build in its manifest; SHA256SUMS pins the bundle. - Coverage: Study 1 (17gb baseline + DFlash) and Study 2 (7 of 12 cells; run order is the driver's fixed sequence, stopped early to prioritize the full ROCm 7.14.0 pass). All overlap with the PR AIwork4me#6 submission agrees within +/-3%. - This also backs the Study-1 W7900 numbers PR AIwork4me#6 quoted in the README without committed raw evidence; the README rows now cite these cells. - w7900-gfx1100.md gains an ROCm-version provenance table (7.2.4 submission / 7.2.1 reproduction / 7.14.0 recommended-default pass). The cells-rocm-7.14.0 bundle referenced by the provenance table lands in the follow-up commit with the full 7.14 matrix.
Second W7900 pass on the project-recommended ROCm 7.14.0, installed
side-by-side from the official therock-dist-linux-gfx110X-all tarball,
same validated llama.cpp pin (0b1bad14f) and committed harness:
- Study 2: full 12-cell matrix, {17gb,dynamic} x {c=1,4,16,32} + DFlash
at c=1/4. Study 1: both 17gb cells. 14/14 planned cells completed.
- Every cell self-labels rocm_version 7.14.0; SHA256SUMS pins the bundle;
all cells validate against benchmark-cell-v1.schema (schema extended
with optional driver provenance keys host/gfx/image/gpu_index).
- README headline W7900 rows now show the 7.14 numbers; 7.2.1/7.2.4
passes remain as corroboration (every overlap within +/-3%, c=32 -7%).
Negative findings preserved per project policy:
- The repo-pinned gfx1151 tarball cannot serve W7900 beyond c=1: rocBLAS
carries no gfx1100 Tensile data, multi-slot decode core-dumps
(rocBLAS error: TensileLibrary.dat ... GPU arch: gfx1100). gfx110X-all
is required; the four c=1 cells first measured under the gfx1151
tarball were discarded and re-measured.
- An apparent c=32 pathology (>56 min, no completion) was diagnosed as a
polluted process tree (orphan llama-server sharing GPU/port); a clean
isolated retry completed normally (276.16 tok/s, wall 339 s). One
dynamic c=1 cell contaminated by a concurrent diagnostic server
(15.47 tok/s vs true ~30.5) was deleted and re-measured cleanly.
Owner
|
Merged after on-hardware verification. Summary of the maintainer review (all measurements on a second Radeon PRO W7900, 48 GiB, gfx1100): Corroborated — every overlapping number reproduced within ±3% on ROCm 7.2.1 (9 committed cells), and the full 12-cell Study-2 matrix + Study 1 re-run completely on ROCm 7.14.0 (14/14 cells, official Fixed during review —
Findings preserved — the repo-pinned gfx1151 ROCm 7.14 tarball core-dumps multi-slot decode on W7900 (rocBLAS ships no gfx1100 Tensile data): W7900 users need Conflict resolution kept master's restructured README (#12–#14) and folded the W7900 data into the unified tables. Thank you @zihaomu — excellent submission; the repro package made the independent verification straightforward. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Update w7900 result
c = concurrency
-np)Platform and scope
Verification
Evidence and claims