1.12.1
Greedy Decoding
What is it?
Greedy decoding selects the token with the highest probability at every generation step. It is the simplest decoding strategy and produces deterministic output.
Key points
Selects argmax probability at each step. Fast and requires no sampling hyperparameters. Produces the same output for the same model and input. Can generate repetitive or generic text. May miss a better overall sequence that begins with a lower-probability token.
Greedy Decoding Example
Prompt: The capital of France is... If the model assigns Paris a probability of 0.92 and all other tokens have lower probabilities, greedy decoding selects Paris. It repeats this highest-probability selection at every subsequent step.
Real-world application
Useful for deterministic tasks such as code generation and factual question answering when predictable output is preferred.
Key takeaway
Greedy decoding always chooses the most probable next token, not necessarily the best complete sequence.