OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / GRPO - Group Relative Policy Optimization / GRPO Variants and Extensions
Lesson 286 of 415 6 min read
7.5.3.2

Sequence-Level Importance Weighting

Learn how GSPO calculates a length-normalized sequence importance weight using the geometric mean of per-token ratios.

What is it?

GSPO defines a sequence-level importance weight using the geometric mean of the per-token importance ratios. This is equivalent to taking the sequence probability ratio and raising it to the power of one divided by the sequence length. The result is a length-normalized sequence probability ratio.

Key points

Sequence weight: sᵢ(θ) = [πθ(oᵢ|q) / πold(oᵢ|q)]^(1/|oᵢ|). It can also be expressed as the exponential of the average per-token log-probability ratio. The geometric mean provides a length-normalized sequence-level measure.

Understanding the Geometric Mean

Suppose a response contains several token-level importance ratios. Instead of treating each ratio as a separate clipping unit, GSPO combines them using their geometric mean. The resulting single value represents the normalized probability change for the sequence.

Real-world application

Use sequence-level importance weights when the training objective needs to control policy changes across complete generated responses.
Key takeaway

GSPO converts token-level probability ratios into one length-normalized importance weight for each sequence.