6.7.5
Beta 0.2 and 0.5: Conservative Settings
Compare beta 0.2 for noisy data or an already-aligned model with beta 0.5 for capability-preserving safety fine-tuning.
What is it?
The source recommends β = 0.2 for noisy preference data or when the model is already close to the desired behaviour. It recommends β = 0.5 for very conservative safety fine-tuning where existing capabilities must be preserved.
Key points
1. β = 0.2: Conservative.
2. Use with noisy data or when the model is already close to the desired behaviour.
3. β = 0.5: Very conservative.
4. Use for safety fine-tuning where capabilities must not be broken.
5. Higher beta settings constrain policy movement more strongly.
Beta 0.2 and 0.5 – Conservative Settings
Scenario A – β = 0.2:
Preference data contains some noise, or the model is already close to the desired behaviour. A conservative setting is appropriate.
Scenario B – β = 0.5:
Safety fine-tuning must improve behaviour while preserving existing capabilities. The source identifies this as a very conservative setting.
Real-world application
Consider β = 0.2 when preference data is noisy and β = 0.5 when preserving existing capabilities is especially important.
Key takeaway
Conservative beta settings reduce policy movement when data quality or capability preservation requires caution.