7.5.5.3
Compute Savings with Two Rollouts
Understand the generation-compute reduction and reported end-to-end speedup of 2-GRPO.
What is it?
Using two generations instead of sixteen reduces generation compute by eight times. Since generation is typically 60–80% of training wall-clock time, the source reports approximately four to six times end-to-end speedup.
Key points
G = 2 versus G = 16: 8× less generation compute. Generation accounts for approximately 60–80% of wall-clock time. Reported total training speedup: approximately 4–6×. The source reports no accuracy loss on GSM8K, MATH and code benchmarks.
Reducing Generation Cost
If a training setup generates sixteen completions for every prompt, changing to two completions reduces the number of generated responses by a factor of eight. The overall training speedup is smaller because generation is only part of the total training time.
Real-world application
Consider 2-GRPO when generation cost is the main bottleneck. Measure end-to-end training time and benchmark accuracy in your own setup.
Key takeaway
Fewer rollouts can substantially reduce generation cost, but actual end-to-end speedup depends on the training workload.