7.5.6.4
SAPO Temperature Behaviour
Explore how temperature changes the gating function, from hard clipping at very low temperature to an unclipped policy gradient at very high temperature.
What is it?
Temperature controls how softly SAPO gates policy updates. Higher temperature produces a softer gate and more exploration, while lower temperature approaches hard PPO clipping.
Key points
Higher temperature means a softer gate.|Lower temperature approaches hard clipping.|As temperature approaches zero, SAPO recovers hard PPO clipping.|As temperature approaches infinity, it recovers the unclipped policy gradient.
Understanding the temperature limits
The source describes two limits: as tau approaches zero, SAPO recovers hard PPO clipping; as tau approaches infinity, it recovers the unclipped policy gradient. These limits help explain how temperature changes the gating behaviour.
Real-world application
Adjust temperature settings to control the balance between restrictive clipping behaviour and more open policy updates.
Key takeaway
Temperature determines how softly SAPO controls policy updates.