7.2.3
Calculating Group Mean and Standard Deviation
Understand how GRPO calculates the average reward and standard deviation within a response group.
What is it?
GRPO calculates the mean and standard deviation of rewards within each response group. The group mean acts as an empirical baseline for comparing individual responses.
Key points
Group mean: μG = (1/G) Σj rj.
Group standard deviation: σG = std({rj}).
The group mean approximates the expected reward for the prompt.
The standard deviation measures reward variation within the group.
Calculating Group Reward Statistics
The source illustrates five responses: three receive reward 1 and two receive reward 0. The group mean is (1 + 1 + 1 + 0 + 0) / 5 = 0.6. This mean becomes the baseline for evaluating each response.
Real-world application
Calculate reward statistics separately for each prompt group. These statistics are used in the next step to normalise the response advantages.
Key takeaway
The group mean provides the baseline, while standard deviation measures reward variation.