OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / GRPO - Group Relative Policy Optimization / Group Size Analysis
Lesson 272 of 415 5 min read
7.4.4

The Goldilocks Rule: Prompt Difficulty and Re-filtering

Learn the recommended 20–80% prompt pass-rate range and why prompts should be re-filtered every 500 training steps.

What is it?

The Goldilocks rule recommends filtering prompts so that the current model solves approximately 20–80% of them. This gives groups a reasonable chance of containing both successes and failures. The source recommends repeating this filtering every 500 training steps as the model improves.

Key points

Target prompt pass rate: 20–80%. Filter prompts according to current model capability. Re-filter every 500 training steps. As the model improves, previously difficult prompts may become easy. Maintain a useful mixture of successful and unsuccessful responses.

Keeping Prompt Difficulty at the Right Level

A prompt initially solved by only 10% of responses may be too difficult. Another solved by 100% of responses provides no reward variation. Filtering toward a 20–80% pass rate helps maintain useful learning signals as training progresses.

Real-world application

Measure prompt pass rates during training and refresh the training prompt set every 500 steps, as recommended by the source.
Key takeaway

The Goldilocks rule keeps prompts challenging enough to produce both successful and unsuccessful responses.