E2: Dense/MoE layer mixing + DeepSeek V3 forward wiring

[tlkahn]

Dense/MoE layer mixing + DeepSeek V3 forward wiring

Wire MODEL_DEEPSEEK_V3 end-to-end: layer stack, MLA, MoE/dense mix, generate path.

Spec: doc/plans/engine-inference-core.md § E2

Part of: #105 (E2) · Epic: #56
Depends on: E1 MoE block + gate; MLA attention/cache; weight sanitization; family split

Tasks

• First first_k_dense_replace layers use standard SwiGLU MLP; remaining layers use MoE
• Honor moe_layer_freq (MoE every N-th layer among the MoE-eligible suffix)
• Full MODEL_DEEPSEEK_V3 forward: embed -> layers (MLA + dense/MoE) -> norm -> lm_head
• Engine support-gate allow for V3 once path is real; accurate reject until then
• Smoke generate on a small V3-class 4-bit safetensors checkpoint

Notes

• V3.2 indexer and V4 characterization are separate follow-on issues
• Parity gate issue owns the formal mlx-lm comparison