OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / GRPO - Group Relative Policy Optimization / GRPO Variants and Extensions
Lesson 315 of 415 6 min read
7.5.10.4

Configuring CISPO in TRL

Learn the TRL settings for CISPO, including batch reward scaling, truncated-completion masking, and asymmetric clipping parameters.

What is it?

The source configures CISPO in TRL through the cispo loss type, batch reward scaling, truncated-completion masking, and asymmetric clipping parameters.

Key points

Set loss_type to cispo.|Set scale_rewards to batch.|Enable mask_truncated_completions.|The example uses epsilon = 0.2.|The example uses epsilon_high = 5.0 and num_generations = 8.

Configuring CISPO in TRL

from trl import GRPOConfig, GRPOTrainer config = GRPOConfig( loss_type="cispo", scale_rewards="batch", mask_truncated_completions=True, epsilon=0.2, epsilon_high=5.0, num_generations=8, ) trainer = GRPOTrainer( model=model, reward_funcs=[reward_fn], args=config, train_dataset=dataset, )

Real-world application

Use the source configuration as a starting reference and verify parameter support in the TRL version used by your project.
Key takeaway

CISPO configuration combines batch scaling, completion masking, and asymmetric clipping.