6.8.3
Distributed Scaling Configurations
Compare single-GPU and 8-GPU configurations for reaching a global DPO batch size of 64.
What is it?
Distributed DPO training can reach the same global batch size using different combinations of GPUs and gradient accumulation. The source compares a single-GPU setup with an 8-GPU node, both targeting a global batch size of 64.
Key points
Both configurations use B_global = 64
Single GPU: B_micro = 2, N_GPUs = 1, K_accum = 32
8-GPU node: B_micro = 2, N_GPUs = 8, K_accum = 4
The source describes single-GPU throughput as sequential/slow and 8-GPU throughput as high parallel throughput.
Single GPU vs 8-GPU Node
A single GPU with a micro-batch of 2 and 32 accumulation steps reaches 64 preference pairs per global batch. An 8-GPU node with the same micro-batch and only 4 accumulation steps also reaches 64. The second configuration processes work in parallel across devices.
Real-world application
Use the decomposition formula to compare hardware configurations and estimate the accumulation steps needed to reach the desired global batch.
Key takeaway
Distributed GPUs can reduce the gradient accumulation steps required to reach the same global batch size.