OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / SFT Best Practices and Techniques / Completion-Only Masking
Lesson 390 of 415 7 min read
10.3.3

Configuring the SFTTrainer with a Masking Collator

Connect the completion-only data collator, tokenizer, formatting function and training configuration.

What is it?

The completion-only collator is connected to SFTTrainer through its data_collator argument. The trainer can also use a formatting function to prepare role-based examples before applying the masking logic.

Key points

Import SFTConfig and SFTTrainer. Load the tokenizer for the target model. Create the completion-only collator. Configure the maximum sequence length and output directory. Pass data_collator and formatting_func to SFTTrainer. Start training with trainer.train().

Using Completion Masking with SFTTrainer

from transformers import AutoTokenizer from trl import SFTConfig, SFTTrainer, DataCollatorForCompletionOnlyLM tokenizer = AutoTokenizer.from_pretrained( "meta-llama/Llama-3.1-8B-Instruct" ) response_template = "<|start_header_id|>assistant<|end_header_id|>" collator = DataCollatorForCompletionOnlyLM( response_template=response_template, tokenizer=tokenizer ) config = SFTConfig( max_seq_length=2048, output_dir="sft_model" ) trainer = SFTTrainer( model=model, tokenizer=tokenizer, args=config, train_dataset=dataset, data_collator=collator, formatting_func=formatting_func ) trainer.train()

Real-world application

Integrate the collator into an SFT pipeline when the dataset contains prompts and assistant responses. Keep the formatting function and response template consistent with the model chat format.
Key takeaway

The data_collator argument applies completion-only masking during SFTTrainer data preparation.