Dear DeepSeek team,
We have been working on PathOracle, a lightweight prefill acceleration method that predicts hidden states at later anchor layers and skips intermediate Transformer blocks.
The key point is complementarity:
- DSpark accelerates the decode phase through speculative decoding.
- PathOracle accelerates the prefill phase through hidden-state prediction and layer skipping.
Together, the two methods could cover more of the end-to-end inference pipeline.
What PathOracle Does
PathOracle runs the early Transformer blocks, predicts the hidden state that would have been produced after skipped middle blocks, and then resumes the original late blocks:
embeddings -> early blocks -> PathOracle -> late blocks -> logits
The oracle is a compact external Transformer trained with MSE plus cosine similarity loss.
Experimental Evidence
All experiments are CPU-tested and reproducible in the open-source framework.
| Model |
Skip Pattern |
Best Oracle |
PPL Ratio |
Final Cosine |
| distilgpt2 |
run 0-1, skip 2-3, run 4-5 |
dim 192, 2 blocks |
1.66x |
0.9052 |
| GPT-2 Small |
run 0-1, skip 2-9, run 10-11 |
dim 256, 2 blocks |
1.92x |
0.8871 |
For GPT-2 Small, PathOracle skips 8 of 12 Transformer blocks and keeps the PPL ratio below 2.0x.
DeepSeek-V4 Migration Plan
We also prepared a concrete migration design for DeepSeek-V4-style MoE models:
- multi-anchor layer grouping instead of one long jump;
- compact per-segment or shared hidden-state oracles;
- target prefill block-compute reduction around 48-52%;
- compatibility with DSpark because PathOracle operates before decode starts.
The exact layer grouping should be validated against the final DeepSeek-V4 checkpoint configuration.
Open-Source Materials
Request
We would appreciate feedback from the DeepSeek inference team on whether PathOracle could be explored as a prefill acceleration component alongside DSpark.
We would also be happy to provide additional implementation details, experiment logs, or a more targeted DeepSeek-V4 prototype plan.
Thank you for the excellent work on DeepSeek and DSpark.
Dear DeepSeek team,
We have been working on PathOracle, a lightweight prefill acceleration method that predicts hidden states at later anchor layers and skips intermediate Transformer blocks.
The key point is complementarity:
Together, the two methods could cover more of the end-to-end inference pipeline.
What PathOracle Does
PathOracle runs the early Transformer blocks, predicts the hidden state that would have been produced after skipped middle blocks, and then resumes the original late blocks:
The oracle is a compact external Transformer trained with MSE plus cosine similarity loss.
Experimental Evidence
All experiments are CPU-tested and reproducible in the open-source framework.
For GPT-2 Small, PathOracle skips 8 of 12 Transformer blocks and keeps the PPL ratio below 2.0x.
DeepSeek-V4 Migration Plan
We also prepared a concrete migration design for DeepSeek-V4-style MoE models:
The exact layer grouping should be validated against the final DeepSeek-V4 checkpoint configuration.
Open-Source Materials
Request
We would appreciate feedback from the DeepSeek inference team on whether PathOracle could be explored as a prefill acceleration component alongside DSpark.
We would also be happy to provide additional implementation details, experiment logs, or a more targeted DeepSeek-V4 prototype plan.
Thank you for the excellent work on DeepSeek and DSpark.