OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / Preference Optimization Variants / Best-of-N Sampling (Rejection Sampling)
Lesson 346 of 415 5 min read
8.5.1

Why Best-of-N Sampling Needs No RL Training

Understand how Best-of-N improves response quality by selecting among generated candidates without updating model weights.

What is it?

Best-of-N improves response quality without changing model weights during the RL phase. The method generates multiple candidate responses and selects the best one using a reward model. It can also serve as the starting point for Rejection Sampling Fine-Tuning.

Key points

No model-weight updates during inference-time selection; generate multiple candidates; score candidates with a reward model; select the highest-scoring response; optional fine-tuning on selected responses.

Selecting a Better Answer Without Training

For a prompt, generate 16 candidate answers. A reward model scores each answer. The system returns the candidate with the highest reward score. The original model weights remain unchanged.

Real-world application

Use Best-of-N when you need stronger outputs at inference time without running a new RL training process.
Key takeaway

Best-of-N improves output selection by generating alternatives and choosing the highest-reward candidate.