OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / GRPO - Group Relative Policy Optimization / GRPO Variants and Extensions
Lesson 302 of 415 6 min read
7.5.8.3

The VESPO Gradient Weighting Kernel

Understand the smooth weighting function and the roles of the sharpness parameter k and staleness parameter lambda.

What is it?

VESPO transforms the trajectory importance weight into a gradient weight using a smooth kernel. The parameter k controls weighting sharpness, while lambda controls exponential decay for stale, low-weight trajectories.

Key points

The kernel is smooth.|k controls the sharpness of the weighting.|lambda controls exponential decay.|The kernel transforms W(tau) into the final gradient weight.

The roles of k and lambda

The source gives the gradient weight as g(tau) = W(tau)^k × exp(lambda(1 − W(tau))). The exponent k controls sharpness, while lambda controls how strongly stale trajectories are down-weighted.

Real-world application

Understand the two parameters before tuning VESPO in an off-policy training configuration.
Key takeaway

k controls sharpness; lambda controls staleness decay.