OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
Foundations / LLM Architecture and Optimization Methods / Diversity in LLM Training
Lesson 64 of 415 6 min read
1.11.1

Sampling Diversity

What is it?

Sampling diversity controls how varied the generated responses can be. The model produces probabilities for possible next tokens, and sampling settings influence which tokens can be selected.

Key points

Temperature controls probability distribution sharpness. Higher temperature makes the distribution more uniform and increases variation. Top-k restricts sampling to the k highest-probability tokens. Top-p selects from the smallest group whose cumulative probability reaches p. Min-p keeps tokens above a threshold relative to the highest-probability token. Frequency and presence penalties discourage repeated tokens.

Temperature and Top-p

Suppose the model predicts the next token using probabilities. With a low temperature, the most likely token dominates. With a higher temperature, less likely alternatives become more competitive. Top-p then limits sampling to a probability-based group of candidates.

Real-world application

Use sampling controls when generating multiple candidate responses, training with model-generated samples, or balancing creativity and consistency in language-model outputs.
Key takeaway

Temperature and sampling filters influence how diverse generated text becomes. Higher diversity is not always better; settings should match the task.