7.5.6.2
SAPO: Smooth Temperature-Controlled Gating
Learn how SAPO replaces the hard clipping objective with a smooth surrogate controlled by asymmetric temperature parameters.
What is it?
SAPO replaces the hard-clipped PPO objective with a smooth surrogate controlled by temperature parameters. Its gating function uses the sigmoid function to make policy updates smoother.
Key points
SAPO means Soft Adaptive Policy Optimization.|It replaces hard clipping with a smooth gate.|The gate uses the sigmoid function.|The objective depends on the probability ratio and advantage.
From hard clipping to a smooth gate
Instead of applying a hard clipping boundary, SAPO uses a sigmoid-based gate. The gate changes gradually as the probability ratio moves, rather than switching abruptly at a fixed boundary.
Real-world application
Use SAPO when exploring a smoother alternative to PPO-style clipping in GRPO training.
Key takeaway
SAPO controls policy updates through a smooth, temperature-based gate.