OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / System Architecture and Infrastructure at Scale / The 4-Model Memory Challenge
Lesson 407 of 415 7 min read
11.1.2

Policy Model Memory Requirements

Break down policy weights, FP32 master weights, Adam optimizer states and gradients for a 70B BF16 model.

What is it?

The policy model itself requires memory for its BF16 weights, FP32 master weights, Adam optimiser states and gradients. For a 70B BF16 model, the source lists 140 GB for policy weights, 280 GB for FP32 master weights, 560 GB for Adam states and 140 GB for gradients.

Key points

Policy weights (BF16): 140 GB. FP32 master weights: 280 GB. Adam optimiser states (m + v, FP32): 560 GB. Gradients (BF16): 140 GB. These components are part of the policy training memory budget.

Policy Training Memory for a 70B Model

Policy memory components from the source: BF16 policy weights = 140 GB FP32 master weights = 280 GB Adam states = 560 GB BF16 gradients = 140 GB Total policy training state = 1,120 GB.

Real-world application

Use the component-level breakdown when estimating memory for policy training. Optimiser states and master weights can consume substantially more memory than the model weights alone.
Key takeaway

The policy model’s training state is much larger than its BF16 weights alone.