7.5.11.2
Independent Reward Normalisation
Learn how GDPO normalises each reward independently using its own group mean and standard deviation.
What is it?
GDPO normalises every reward independently before combining them. Each reward uses its own group mean and standard deviation, so differences in reward scale do not automatically allow one objective to dominate.
Key points
Each reward is normalised separately.|Each reward has its own mean.|Each reward has its own standard deviation.|The normalised rewards are combined afterwards.
Normalising each reward independently
Suppose a completion receives a correctness reward and a format reward. GDPO calculates a separate normalised score for each reward using that reward’s group statistics before aggregation.
Real-world application
Use independent normalisation when reward functions measure different objectives or operate on different scales.
Key takeaway
GDPO normalises each reward independently before aggregation.