OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / GRPO - Group Relative Policy Optimization / TRL Implementation
Lesson 267 of 415 7 min read
7.3.4

Creating Reward Functions

Learn how to implement binary correctness rewards and additional formatting rewards for generated responses.

What is it?

GRPO uses reward functions to evaluate generated completions. The source demonstrates a binary correctness reward for mathematics and a separate formatting reward. Multiple reward functions can be combined.

Key points

reward_fn receives completions and prompts. It extracts the generated answer and compares it with the expected answer. Correct answer: reward 1.0. Incorrect answer: reward 0.0. format_reward_fn gives 0.5 for the demonstrated LaTeX boxed format. Multiple reward functions are passed through reward_funcs.

Building Multiple Reward Functions

The mathematics reward function extracts an answer from each completion and compares it with the ground truth. It returns 1.0 when correct and 0.0 otherwise. A second function adds a 0.5 reward when the completion contains the demonstrated LaTeX boxed format.

Real-world application

Create reward functions that reflect the task requirements. The source example combines correctness and formatting rewards when constructing the GRPOTrainer.
Key takeaway

Reward functions define what successful model responses look like during GRPO training.