1.12.6
Min-p Sampling
What is it?
Min-p sampling keeps tokens whose probability is at least a chosen fraction of the highest-probability token. This creates a relative probability threshold that adapts to how confident the model is.
Key points
The threshold is pmin multiplied by the highest token probability. Tokens below the threshold are removed. When the model is confident, the candidate set can become smaller. When the model is uncertain, the candidate set can expand. It uses one intuitive relative threshold.
Min-p Sampling Example
Suppose the highest-probability token has probability 0.8 and pmin is 0.1. Only tokens with probability of at least 0.08 remain. If the highest probability is 0.05, the threshold becomes 0.005, allowing a wider candidate set.
Real-world application
Useful as an alternative to top-p when sampling should adapt to the relative confidence of the model.
Key takeaway
Min-p filters tokens relative to the most likely token rather than using a fixed candidate count or cumulative probability.