OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / GRPO - Group Relative Policy Optimization / GRPO Variants and Extensions
Lesson 323 of 415 6 min read
7.5.7.5

Configuring TIS and MIS in TRL

Learn the TRL configuration options for enabling vLLM importance-sampling correction and selecting TIS or MIS.

What is it?

TRL provides configuration options for enabling vLLM importance-sampling correction and selecting TIS or MIS. The source demonstrates sequence-level truncation for TIS and sequence-level masking for MIS.

Key points

Enable use_vllm.|Enable vllm_importance_sampling_correction.|Use sequence_truncate for TIS.|Use sequence_mask for MIS.|The source examples use caps of 5.0 for TIS and 3.0 for MIS.

Configuring TIS and MIS in TRL

from trl import GRPOConfig config_tis = GRPOConfig( use_vllm=True, vllm_importance_sampling_correction=True, vllm_importance_sampling_mode="sequence_truncate", vllm_importance_sampling_cap=5.0, ) config_mis = GRPOConfig( use_vllm=True, vllm_importance_sampling_correction=True, vllm_importance_sampling_mode="sequence_mask", vllm_importance_sampling_cap=3.0, )

Real-world application

Apply the relevant configuration when using vLLM with a GRPO training setup in a compatible TRL version.
Key takeaway

TRL exposes separate modes for truncated and masked importance-sampling correction.