OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / DPO - Direct Preference Optimization / DPO Batch Size Configuration and Scaling
Lesson 243 of 415 6 min read
6.8.2

Mathematical Decomposition

Understand how global batch size is calculated from micro-batch size, number of GPUs and gradient accumulation steps, including the DPO pairing multiplier.

What is it?

DPO global batch size is determined by the per-device micro-batch size, the number of GPUs and the number of gradient accumulation steps. Each preference pair contains a prompt, a chosen response and a rejected response. The source counts the two responses as the sequence load, giving T_sequences = 2 × B_micro.

Key points

B_global = B_micro × N_GPUs × K_accum B_micro: Preference pairs processed per device in one forward pass N_GPUs: Number of parallel processing devices K_accum: Gradient accumulation steps before a weight update T_sequences = 2 × B_micro For models above 7B parameters on 80GB GPUs with 4096–8192 token contexts, the source gives B_micro = 1–2.

Calculating the Global Batch Size

For one GPU, B_micro = 2 and K_accum = 32 gives B_global = 2 × 1 × 32 = 64. For an 8-GPU node, B_micro = 2 and K_accum = 4 gives B_global = 2 × 8 × 4 = 64. Both configurations reach the same global batch target.

Real-world application

Calculate the global batch size before training. Account for the paired chosen and rejected responses when estimating the micro-batch memory requirement.
Key takeaway

Global batch size depends on micro-batch size, GPU count and gradient accumulation; DPO processes two response sequences per preference pair.