OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / DPO - Direct Preference Optimization / Mathematical Derivation
Lesson 211 of 415 5 min read
6.2.4

The Bradley-Terry Preference Model

Understand how the Bradley-Terry model represents pairwise preference probability.

What is it?

The Bradley-Terry model represents the probability that one response is preferred over another using the difference between their reward scores.

Key points

1. y_w is the chosen response. 2. y_l is the rejected response. 3. Compare their reward scores. 4. Apply the sigmoid function. 5. A larger reward difference increases preference probability.

Bradley-Terry Model

P(y_w ≻ y_l) = σ(r(y_w) − r(y_l)) The sigmoid converts the reward difference into a preference probability.

Real-world application

Use pairwise response comparisons to model human preferences.
Key takeaway

Bradley-Terry translates reward differences into preference probabilities.