OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / GRPO - Group Relative Policy Optimization / GRPO Variants and Extensions
Lesson 301 of 415 5 min read
7.5.7.3

Masked Importance Sampling (MIS)

Understand how MIS removes gradient contributions when the importance ratio exceeds a specified threshold.

What is it?

Masked Importance Sampling removes the gradient contribution for a sequence when its importance ratio exceeds a threshold C. It is more conservative than TIS because it discards the contribution instead of capping its weight.

Key points

MIS compares training and vLLM probabilities.|It checks whether the ratio is at most C.|A ratio within the threshold receives a mask of 1.|A ratio above the threshold receives a mask of 0.

Masking an excessive importance ratio

The source defines the MIS weight as an indicator that πtrain(oi|q) / πvLLM(oi|q) is less than or equal to C. When the ratio exceeds C, the sequence receives no gradient contribution.

Real-world application

Consider MIS when large or unpredictable probability mismatches make even truncated correction weights risky.
Key takeaway

MIS excludes sequences whose importance ratio exceeds the chosen threshold.