OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / GRPO - Group Relative Policy Optimization / GRPO Variants and Extensions
Lesson 290 of 415 6 min read
7.5.7.2

Truncated Importance Sampling (TIS)

Learn how TIS corrects probability mismatch using a capped importance-sampling ratio.

What is it?

Truncated Importance Sampling corrects probability mismatch by multiplying the gradient by an importance ratio capped at a maximum value C. The cap prevents extreme corrections from destabilising training.

Key points

TIS compares training and vLLM probabilities.|The correction ratio is πtrain divided by πvLLM.|The ratio is capped at C.|The cap limits extreme correction weights.

Capping an importance ratio with TIS

The source defines the TIS weight as min(C, πtrain(oi|q) / πvLLM(oi|q)). If the calculated ratio exceeds C, the correction weight is limited to C rather than allowing an extreme value.

Real-world application

Use TIS to correct probability mismatch while limiting unusually large importance weights.
Key takeaway

TIS truncates the importance ratio to control extreme gradient corrections.