OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / Reward Model Training / Listwise Rank-Based Rewards
Lesson 376 of 415 6 min read
9.7.2

The Plackett-Luce Model and Ranking Likelihood

Understand the Plackett-Luce model, its sequential ranking probability and its relationship to Bradley-Terry.

What is it?

The Plackett-Luce model extends Bradley-Terry to complete rankings. It models a ranking as a sequence of selections, where the best remaining response is selected at each step according to a softmax over remaining rewards.

Key points

Given K responses and ranking π, π(1) represents the best response. At each step, the probability of selecting the next ranked response is a softmax over the remaining responses. The Plackett-Luce likelihood represents the probability of the complete ranking. For K = 2, Plackett-Luce reduces exactly to Bradley-Terry. The source describes Plackett-Luce as a strict generalisation of Bradley-Terry.

Selecting the Best Remaining Response

Imagine four responses ranked from best to worst. The Plackett-Luce model first estimates the probability that the top-ranked response is selected from all four. It then evaluates the next response among the three remaining options, continuing until the ranking is complete.

Real-world application

Use the Plackett-Luce likelihood to train a reward model from complete rankings. For a two-response comparison, its probability model is equivalent to Bradley-Terry.
Key takeaway

Plackett-Luce models complete rankings through sequential choices among remaining responses.