OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / Preference Optimization Variants / Best-of-N Sampling (Rejection Sampling)
Lesson 347 of 415 6 min read
8.5.2

The Best-of-N Algorithm: Generate, Score and Select

Learn the candidate generation, reward scoring, selection and optional fine-tuning steps.

What is it?

Best-of-N generates N responses for each prompt, scores every response with a reward model and selects the highest-scoring candidate. Selected responses can optionally become supervised fine-tuning data for a later training iteration.

Key points

Generate N responses; score each response using a reward model; select the maximum-scoring response; optionally use selected responses as SFT data; typical N range is 4–64.

The Best-of-N Selection Algorithm

For a prompt x, generate candidates y1 through yN. Calculate each reward score rφ(x, yi). Select y* = arg max rφ(x, yi). The selected answer is the response with the highest reward score.

Real-world application

Build a candidate-generation and reward-scoring pipeline for tasks where response quality can be evaluated reliably.
Key takeaway

Best-of-N is a generate, score and select process, with optional iterative fine-tuning.