7.5.9.4
Binary and Top-K Divergence Masks
Understand the binary and top-K token masks used to approximate divergence-based constraints.
What is it?
DPPO approximates divergence constraints with token-level binary or top-K masks. The source describes binary masks based on TV or KL contributions and top-K masks that retain tokens with the largest contributions.
Key points
binary_tv masks tokens using absolute probability difference.|binary_kl masks tokens using a KL contribution.|topk_tv retains tokens with the highest TV contribution.|topk_kl retains tokens with the highest KL contribution.
Four token-level divergence masks
The source lists four approaches: binary_tv, binary_kl, topk_tv, and topk_kl. Binary approaches mask tokens that exceed a divergence threshold. Top-K approaches retain tokens with the largest contributions under the selected divergence measure.
Real-world application
Use these mask types as a conceptual guide when implementing divergence-based token selection.
Key takeaway
DPPO approximates direct divergence constraints using binary or top-K token masks.