OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / GRPO - Group Relative Policy Optimization / Algorithm
Lesson 260 of 415 5 min read
7.2.2

Scoring Responses with a Reward Function

Learn how each generated response receives a reward score based on its quality or correctness.

What is it?

After generating responses, GRPO assigns a reward to each completion using a reward function. The reward can measure correctness or other desired response properties.

Key points

Each response yi receives reward ri = R(x, yi). The reward function evaluates a response in the context of its prompt. The source demonstrates binary correctness rewards: 1.0 for correct and 0.0 for incorrect. Multiple reward functions can be combined.

Scoring Mathematical Responses

Suppose five responses are generated for a maths question. A correctness reward function compares each extracted answer with the expected answer. Correct answers receive 1.0 and incorrect answers receive 0.0. A separate format reward can add 0.5 for correctly formatted LaTeX.

Real-world application

Design reward functions that evaluate the intended task. The source TRL example combines a correctness reward function and a format reward function through reward_funcs.
Key takeaway

GRPO uses reward functions to score every generated response before calculating group-relative advantages.