6.7.3
Beta 0.05: Aggressive
Understand when beta 0.05 is suitable for good preference data and noticeable improvement over supervised fine-tuning.
What is it?
The source classifies β = 0.05 as aggressive. It is suitable when preference data is good and noticeable improvement over supervised fine-tuning is desired.
Key points
1. β = 0.05.
2. Regime: Aggressive.
3. Requires good preference data.
4. Suitable when noticeable improvement over SFT is desired.
5. Allows more policy movement than the standard β = 0.1 setting.
Beta 0.05 – Aggressive
Scenario: A supervised fine-tuned model already performs reasonably well, but good preference data is available to produce a noticeable improvement.
The source recommends β = 0.05 for this aggressive alignment setting.
Real-world application
Use this regime when preference comparisons are reliable and the goal is meaningful improvement over the SFT model.
Key takeaway
β = 0.05 suits good data and a desire for noticeable improvement over SFT.