7.5.2.5
Dynamic Sampling and DAPO Implementation
Understand resampling of zero-variance groups and the key TRL configuration options for DAPO.
What is it?
DAPO dynamically re-samples prompts when every response in a group receives the same reward. Such groups have zero advantage after normalization and provide no useful gradient signal. The source also provides a TRL configuration for enabling DAPO components.
Key points
Dynamic sampling re-samples all-correct or all-incorrect groups. TRL settings shown: epsilon=0.2; epsilon_high=0.28; loss_type="dapo"; mask_truncated_completions=True; max_completion_length=1024; num_generations=8. DAPO loss internally handles zero-variance group filtering.
DAPO Configuration in TRL
A GRPOConfig can enable DAPO using loss_type="dapo", epsilon=0.2, epsilon_high=0.28 and mask_truncated_completions=True. With num_generations=8, the trainer generates groups of eight completions. GRPOTrainer is then initialized with the model, reward function, configuration and training dataset before calling trainer.train().
Real-world application
Consider DAPO for long-form reasoning tasks, frequent length-limit hits, collapsing reward variance, loss spikes or entropy collapse. The source recommends it as a drop-in improvement over base GRPO for most tasks.
Key takeaway
DAPO combines adaptive clipping, token-level aggregation, completion-length handling and dynamic sampling to improve GRPO training stability.