7.5.7.2
Truncated Importance Sampling (TIS)
Learn how TIS corrects probability mismatch using a capped importance-sampling ratio.
What is it?
Truncated Importance Sampling corrects probability mismatch by multiplying the gradient by an importance ratio capped at a maximum value C. The cap prevents extreme corrections from destabilising training.
Key points
TIS compares training and vLLM probabilities.|The correction ratio is πtrain divided by πvLLM.|The ratio is capped at C.|The cap limits extreme correction weights.
Capping an importance ratio with TIS
The source defines the TIS weight as min(C, πtrain(oi|q) / πvLLM(oi|q)). If the calculated ratio exceeds C, the correction weight is limited to C rather than allowing an extreme value.
Real-world application
Use TIS to correct probability mismatch while limiting unusually large importance weights.
Key takeaway
TIS truncates the importance ratio to control extreme gradient corrections.