7.5.10.4
Configuring CISPO in TRL
Learn the TRL settings for CISPO, including batch reward scaling, truncated-completion masking, and asymmetric clipping parameters.
What is it?
The source configures CISPO in TRL through the cispo loss type, batch reward scaling, truncated-completion masking, and asymmetric clipping parameters.
Key points
Set loss_type to cispo.|Set scale_rewards to batch.|Enable mask_truncated_completions.|The example uses epsilon = 0.2.|The example uses epsilon_high = 5.0 and num_generations = 8.
Configuring CISPO in TRL
from trl import GRPOConfig, GRPOTrainer
config = GRPOConfig(
loss_type="cispo",
scale_rewards="batch",
mask_truncated_completions=True,
epsilon=0.2,
epsilon_high=5.0,
num_generations=8,
)
trainer = GRPOTrainer(
model=model,
reward_funcs=[reward_fn],
args=config,
train_dataset=dataset,
)
Real-world application
Use the source configuration as a starting reference and verify parameter support in the TRL version used by your project.
Key takeaway
CISPO configuration combines batch scaling, completion masking, and asymmetric clipping.