7.3.4
Creating Reward Functions
Learn how to implement binary correctness rewards and additional formatting rewards for generated responses.
What is it?
GRPO uses reward functions to evaluate generated completions. The source demonstrates a binary correctness reward for mathematics and a separate formatting reward. Multiple reward functions can be combined.
Key points
reward_fn receives completions and prompts.
It extracts the generated answer and compares it with the expected answer.
Correct answer: reward 1.0.
Incorrect answer: reward 0.0.
format_reward_fn gives 0.5 for the demonstrated LaTeX boxed format.
Multiple reward functions are passed through reward_funcs.
Building Multiple Reward Functions
The mathematics reward function extracts an answer from each completion and compares it with the ground truth. It returns 1.0 when correct and 0.0 otherwise. A second function adds a 0.5 reward when the completion contains the demonstrated LaTeX boxed format.
Real-world application
Create reward functions that reflect the task requirements. The source example combines correctness and formatting rewards when constructing the GRPOTrainer.
Key takeaway
Reward functions define what successful model responses look like during GRPO training.