6.2.4
The Bradley-Terry Preference Model
Understand how the Bradley-Terry model represents pairwise preference probability.
What is it?
The Bradley-Terry model represents the probability that one response is preferred over another using the difference between their reward scores.
Key points
1. y_w is the chosen response.
2. y_l is the rejected response.
3. Compare their reward scores.
4. Apply the sigmoid function.
5. A larger reward difference increases preference probability.
Bradley-Terry Model
P(y_w ≻ y_l) = σ(r(y_w) − r(y_l))
The sigmoid converts the reward difference into a preference probability.
Real-world application
Use pairwise response comparisons to model human preferences.
Key takeaway
Bradley-Terry translates reward differences into preference probabilities.