Skip to content

Commit 4ffca22

Browse files
skills: note that DSpark/DFlash 2 are MLX drafter modes, not a third engine.
Keep the two-family decision rule. A better drafter can raise short-context decode and still does not cancel the long-context KV-verify crossover. The one-engine memory rule also applies on a 64 GB machine.
1 parent dcae97a commit 4ffca22

2 files changed

Lines changed: 13 additions & 3 deletions

File tree

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,8 @@ The global baseline supports Codex and Claude Code and contains these 38 skills:
2222
skills from `rorkai/app-store-connect-cli-skills`
2323

2424
Install the global baseline from this repository and it becomes available to
25-
agents on the machine. `local-model-serving` guides engine selection and the
25+
agents on the machine. `local-model-serving` guides engine selection (MLX
26+
spec-decode vs llama.cpp; drafters are MLX modes, not a third engine) and the
2627
one-engine-at-a-time memory rule for local LLM work; see its `SKILL.md`.
2728

2829
Install or reconcile it from a checkout of the desired repository revision:

local-model-serving/SKILL.md

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,13 +26,22 @@ Both are OpenAI-compatible (`/v1/chat/completions`), so the same client code
2626
works against either — only the base URL and the thinking-off parameter
2727
differ.
2828

29+
Drafters (DSpark, DFlash 2, and similar speculative-decoding heads) are
30+
**modes of the MLX family**, not a third engine. Do not install NVIDIA-only
31+
stacks (SGLang, vLLM) or a third MLX GUI to A/B a drafter. A better drafter
32+
can raise short-context decode; it does not remove the long-context
33+
crossover, because every verify round still walks the full KV. Use the
34+
drafter the local serving policy or helper names. llama.cpp may be LM Studio
35+
or `llama-server`; that is still the same family.
36+
2937
## First: The Memory Rule (crash prevention)
3038

3139
**Never run both engines with a model resident at the same time.** On a
3240
128 GB machine, two resident 28 GB models plus the OS drove the system into
3341
~20-46 GB of swap and crushed both engines' throughput (a long run was
34-
aborted, another finished at half speed). Each engine fits alone; both do
35-
not.
42+
aborted, another finished at half speed). The same exclusivity rule applies
43+
on a 64 GB machine that can fit only one ~28 GB model. Each engine fits
44+
alone; both do not.
3645

3746
- Stop the other engine before starting one:
3847
`lms unload <identifier>` / `lms unload all`, or

0 commit comments

Comments
 (0)