7.5.7.6
When to Use TIS or MIS
Choose between TIS and MIS based on the size and predictability of the probability mismatch.
What is it?
The source recommends considering importance-sampling correction whenever vLLM is used for generation. TIS is preferred for small mismatches, while MIS is preferred when mismatches are large or unpredictable.
Key points
Consider correction when using vLLM.|TIS is preferred for small mismatches.|MIS is preferred for large or unpredictable mismatches.|Sequence-level correction is theoretically preferred.
Selecting TIS or MIS
If the same model is used for generation and training but different precision causes a small mismatch, TIS is the preferred choice in the source. If the mismatch is large or unpredictable, MIS offers a more conservative approach.
Real-world application
Choose TIS or MIS after assessing the size and predictability of the probability mismatch.
Key takeaway
Use TIS for small mismatches and MIS for large or unpredictable mismatches.