OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / GRPO - Group Relative Policy Optimization / Algorithm
Lesson 259 of 415 5 min read
7.2.1

Generating Multiple Responses per Prompt

Understand how GRPO samples G completions from the current policy for each input prompt.

What is it?

For each prompt, GRPO samples G completions from the current policy. These responses form a group that can be compared with one another during training.

Key points

For each prompt x, sample G completions. The responses are sampled from πθ(·|x). G represents the group size. Multiple responses provide a basis for relative reward comparison.

Generating a Group of Responses

For a mathematics prompt, GRPO may generate five candidate solutions. Instead of evaluating only one response, it evaluates all five so their rewards can be compared within the same prompt group.

Real-world application

Configure the number of generations in GRPO training. The source TRL example uses num_generations = 8 and temperature = 1.0 to encourage diversity within the group.
Key takeaway

GRPO begins by generating multiple responses to the same prompt.