Skip to content

E2: DeepSeek parity gates (mlx-lm oracle) #116

Description

@tlkahn

DeepSeek parity gates (mlx-lm oracle)

Temp-0 parity for DeepSeek families. Oracle is mlx-lm, not mlx-serve/ds4.

Spec: doc/plans/engine-inference-core.md § E2

Part of: #105 (E2) · Epic: #56
Depends on: V3 forward at minimum; V3.2/V4 gates as those paths land

Tasks

  • Extend parity script with skip-if-absent DeepSeek entries
  • Record canonical checkpoint ids in the script
  • V3-class gate green
  • V3.2-class gate green if a public small checkpoint is available
  • DeepSeek-V4-Flash-4bit gate once V4 is characterized/implemented
  • Prefer chat-mode parity for instruct checkpoints (MLXD_PARITY_MODE=chat) where applicable

Notes

  • From CLAUDE.md parity pitfalls: raw completion on instruct models can loop; do not blame forward path before chat-templated comparison

Metadata

Metadata

Assignees

Labels

engine-inferenceEngine inference core (issue #51) feature areaenhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions