7.5.7.4
Sequence-Level vs Token-Level Importance Sampling
Compare sequence-level geometric-mean correction with token-level importance sampling, including their variance and bias trade-offs.
What is it?
TIS and MIS can operate at sequence level or token level. Sequence-level importance sampling uses the geometric mean of token ratios and is theoretically preferred but has higher variance. Token-level correction has lower variance but is biased relative to the sequence correction.
Key points
Sequence-level correction uses the geometric mean across tokens.|Sequence-level correction is theoretically correct but higher variance.|Token-level correction calculates a separate ratio for each token.|Token-level correction is biased but lower variance.
Choosing the importance-sampling level
For a generated sequence containing multiple tokens, sequence-level correction combines token ratios using their geometric mean. Token-level correction evaluates each token separately. The two approaches trade theoretical correctness against variance.
Real-world application
Choose the correction level based on whether theoretical sequence-level correction or lower-variance token-level correction is more important.
Key takeaway
Sequence-level correction is theoretically preferred; token-level correction is a practical lower-variance compromise.