1.15.3
EAGLE: Feature-Level Drafting
What is it?
EAGLE uses feature-level drafting to improve speculative decoding. It extracts hidden states from the target model and passes them to a lightweight autoregressive decoder. Unlike independent prediction heads, this decoder can condition its next prediction on previous hidden-state information, capturing dependencies between proposed tokens.
Key points
1. EAGLE uses hidden states from the target model.
2. A lightweight decoder generates draft tokens.
3. The decoder operates autoregressively.
4. Each draft prediction can use information from previous predictions.
5. This captures inter-token dependencies that independent Medusa heads miss.
6. The source reports an acceptance rate of 85–95% for EAGLE, compared with 60–80% for Medusa.
Using hidden states to improve draft predictions
The target model produces hidden states from its final layer. EAGLE passes these representations into a small decoder. The decoder predicts draft tokens one after another, using previous hidden-state information to improve the next proposal.
Real-world application
EAGLE-style drafting can be useful in inference systems where higher draft acceptance rates improve the overall generation speed.
Key takeaway
EAGLE improves drafting by using target-model features and autoregressive prediction.