6.7.2
Beta 0.01: Very Aggressive
Understand when beta 0.01 is appropriate: extremely clean preference data and a need for large distributional shift.
What is it?
The source classifies β = 0.01 as very aggressive. It recommends this setting only when preference data is extremely clean and a large distributional shift is required.
Key points
1. β = 0.01.
2. Regime: Very aggressive.
3. Use only with extremely clean preference data.
4. Appropriate when a large distributional shift is needed.
5. This setting permits substantial policy movement.
Beta 0.01 – Very Aggressive
Scenario: A model must undergo a large behavioural change, and the preference dataset has been carefully reviewed and is extremely clean.
The source identifies β = 0.01 as a very aggressive setting for this situation.
Real-world application
Consider this setting only when the preference data is exceptionally reliable and substantial behaviour change is necessary.
Key takeaway
β = 0.01 is reserved for extremely clean data and large distributional shifts.