OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
Foundations / LLM Architecture and Optimization Methods / Text Generation: Decoding Methods
Lesson 71 of 415 5 min read
1.12.5

Top-p (Nucleus) Sampling

What is it?

Top-p sampling, also called nucleus sampling, selects the smallest set of tokens whose cumulative probability reaches a chosen threshold p. The candidate set changes according to the model distribution at each step.

Key points

Candidates are ordered by probability. Tokens are included until cumulative probability reaches p. The retained probabilities are renormalized. The number of eligible tokens can change at every step. The method adapts to the model confidence.

Top-p Sampling Example

If p is set to 0.9, the model retains the smallest group of highest-probability tokens whose combined probability is at least 90%. It then samples the next token from that group.

Real-world application

Useful for open-ended generation where the number of plausible next tokens changes depending on the context.
Key takeaway

Top-p uses cumulative probability rather than a fixed number of candidate tokens.