7.5.3.2
Sequence-Level Importance Weighting
Learn how GSPO calculates a length-normalized sequence importance weight using the geometric mean of per-token ratios.
What is it?
GSPO defines a sequence-level importance weight using the geometric mean of the per-token importance ratios. This is equivalent to taking the sequence probability ratio and raising it to the power of one divided by the sequence length. The result is a length-normalized sequence probability ratio.
Key points
Sequence weight: sᵢ(θ) = [πθ(oᵢ|q) / πold(oᵢ|q)]^(1/|oᵢ|). It can also be expressed as the exponential of the average per-token log-probability ratio. The geometric mean provides a length-normalized sequence-level measure.
Understanding the Geometric Mean
Suppose a response contains several token-level importance ratios. Instead of treating each ratio as a separate clipping unit, GSPO combines them using their geometric mean. The resulting single value represents the normalized probability change for the sequence.
Real-world application
Use sequence-level importance weights when the training objective needs to control policy changes across complete generated responses.
Key takeaway
GSPO converts token-level probability ratios into one length-normalized importance weight for each sequence.