OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / SFT Best Practices and Techniques / Completion-Only Masking
Lesson 389 of 415 6 min read
10.3.2

Completion-Only Masking in TRL

Configure DataCollatorForCompletionOnlyLM to apply loss only to tokens after the assistant response template.

What is it?

TRL provides DataCollatorForCompletionOnlyLM to mask prompt tokens during supervised fine-tuning. The response template identifies where the assistant response begins, allowing the collator to apply loss only to subsequent tokens.

Key points

Load the model-compatible tokenizer. Define the assistant response template. Create DataCollatorForCompletionOnlyLM with the response template and tokenizer. The collator masks tokens before the response template. The response tokens remain available for loss calculation.

Creating a Completion-Only Collator

from transformers import AutoTokenizer from trl import DataCollatorForCompletionOnlyLM tokenizer = AutoTokenizer.from_pretrained( "meta-llama/Llama-3.1-8B-Instruct" ) response_template = "<|start_header_id|>assistant<|end_header_id|>" collator = DataCollatorForCompletionOnlyLM( response_template=response_template, tokenizer=tokenizer )

Real-world application

Configure the response template to match the assistant header used by the selected model. Pass the collator to the training pipeline so prompt tokens are excluded from loss.
Key takeaway

The response template tells the collator where assistant-response loss should begin.