OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
Foundations / LLM Architecture and Optimization Methods / Text Generation: Decoding Methods
Lesson 69 of 415 6 min read
1.12.3

Diverse Beam Search

What is it?

Diverse beam search modifies standard beam search to produce more distinct candidate sequences. It divides beams into groups and penalizes candidates that repeat choices already made by earlier groups.

Key points

Beams are partitioned into groups. A diversity penalty discourages overlap between groups. The penalty strength controls how much diversity is encouraged. It can produce distinct candidates for reranking. Excessive diversity penalties may reduce individual candidate quality.

Generating Different Candidate Answers

Imagine four beams divided into two groups. If the first group generates candidates using similar opening words, the second group receives a penalty for repeating those choices. It is therefore encouraged to explore different continuations.

Real-world application

Useful in candidate-generation pipelines where several distinct outputs are generated and another model or scoring system selects the best one.
Key takeaway

Diverse beam search trades some individual sequence likelihood for greater variety among candidate outputs.