4.1.3
Capability Enhancement via Verifiable Rewards: RLVR
Understand how verifiable outcomes train models for reasoning, mathematics and code generation.
What is it?
RL from Verifiable Rewards (RLVR) uses objectively checkable outcomes rather than human preference judgments. It is particularly useful for reasoning, mathematics and code generation, where answers can be checked for correctness.
Key points
1. RLVR means RL from Verifiable Rewards.
2. Rewards come from verifiable outcomes.
3. Mathematics: Was the answer correct?
4. Code generation: Did the code pass the tests?
5. DeepSeek-R1 demonstrated GRPO with rule-based rewards.
6. Its rewards included format correctness and answer accuracy.
7. The approach developed sophisticated chain-of-thought reasoning without human preference data.
8. The source describes RLVR as the dominant approach for building reasoning models and agentic systems.
Example: Training a mathematics model
A language model receives a mathematics problem and generates a solution.
A verifier checks the final answer against the correct result.
If the answer is correct, the model receives a positive reward. If it is incorrect, it receives a lower or negative reward according to the reward design.
The model uses these signals to improve its problem-solving behaviour over repeated training attempts.
Real-world application
RLVR is useful for mathematics, programming, reasoning and agentic tasks where correctness or task completion can be verified automatically.
Key takeaway
RLVR trains new capabilities using verifiable outcomes such as mathematical correctness and successful code execution.