1.12.2
Beam Search
What is it?
Beam search maintains multiple candidate sequences instead of selecting only one token at each step. It expands the candidates and keeps the highest-scoring sequences according to their cumulative log probabilities.
Key points
Beam width B determines how many candidate sequences are retained. Each candidate is expanded using likely next tokens. Lower-scoring candidates are pruned. Length normalization can reduce the preference for shorter sequences. Larger beam widths require more computation.
Beam Search with Width 2
Suppose the model starts with two possible continuations. At each generation step, it expands both candidates and retains the two highest-scoring partial sequences. Other candidates are discarded. This process continues until the sequences finish.
Real-world application
Useful in translation and summarization tasks where a high-likelihood, relatively focused output is desirable.
Key takeaway
Beam search explores multiple candidate sequences, but it can still produce generic or similar outputs.