OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / GRPO - Group Relative Policy Optimization / Group Size Analysis
Lesson 271 of 415 5 min read
7.4.3

Why Groups Need Both Successes and Failures

Understand why a group containing only correct or only incorrect responses produces zero advantages and no learning signal.

What is it?

A useful GRPO group must contain both successful and unsuccessful responses. If every response receives the same reward, all group-relative advantages become zero and the policy receives no learning signal.

Key points

All responses correct: ri = 1 for every response; all advantages are zero. All responses incorrect: the same zero-advantage problem occurs. Groups need variation in reward outcomes. Prompt difficulty must match the model capability.

Why a Group Needs Mixed Outcomes

Suppose a group contains eight responses and every answer is correct. Each response receives the same reward, so subtracting the group mean produces zero advantage. The same happens when every response is incorrect. Neither group provides a useful relative learning signal.

Real-world application

When designing a GRPO dataset, check whether prompts produce a mixture of successful and unsuccessful responses for the current model.
Key takeaway

GRPO needs reward variation within a group; identical outcomes produce no learning signal.