7.5.7.3
Masked Importance Sampling (MIS)
Understand how MIS removes gradient contributions when the importance ratio exceeds a specified threshold.
What is it?
Masked Importance Sampling removes the gradient contribution for a sequence when its importance ratio exceeds a threshold C. It is more conservative than TIS because it discards the contribution instead of capping its weight.
Key points
MIS compares training and vLLM probabilities.|It checks whether the ratio is at most C.|A ratio within the threshold receives a mask of 1.|A ratio above the threshold receives a mask of 0.
Masking an excessive importance ratio
The source defines the MIS weight as an indicator that πtrain(oi|q) / πvLLM(oi|q) is less than or equal to C. When the ratio exceeds C, the sequence receives no gradient contribution.
Real-world application
Consider MIS when large or unpredictable probability mismatches make even truncated correction weights risky.
Key takeaway
MIS excludes sequences whose importance ratio exceeds the chosen threshold.