1.12.7
Temperature Scaling
What is it?
Temperature scaling adjusts the sharpness of the model probability distribution before sampling. Lower temperatures concentrate probability on the most likely tokens, while higher temperatures make the distribution flatter.
Key points
Temperature T below 1 sharpens the distribution. T equal to 1 leaves the original distribution unchanged. T above 1 flattens the distribution and increases randomness. As T approaches zero, generation approaches greedy decoding. Very high temperature approaches uniform sampling.
Low and High Temperature
For the same prompt, temperature 0.3 tends to produce focused and conventional responses. Temperature 1.0 or higher allows more varied and creative continuations. The model itself is unchanged; only the token-selection distribution changes.
Real-world application
Use lower temperatures for focused factual or code-related tasks and higher temperatures when creative variation is required.
Key takeaway
Temperature controls randomness by changing the probability distribution used during token selection.