10.3.3
Configuring the SFTTrainer with a Masking Collator
Connect the completion-only data collator, tokenizer, formatting function and training configuration.
What is it?
The completion-only collator is connected to SFTTrainer through its data_collator argument. The trainer can also use a formatting function to prepare role-based examples before applying the masking logic.
Key points
Import SFTConfig and SFTTrainer.
Load the tokenizer for the target model.
Create the completion-only collator.
Configure the maximum sequence length and output directory.
Pass data_collator and formatting_func to SFTTrainer.
Start training with trainer.train().
Using Completion Masking with SFTTrainer
from transformers import AutoTokenizer
from trl import SFTConfig, SFTTrainer, DataCollatorForCompletionOnlyLM
tokenizer = AutoTokenizer.from_pretrained(
"meta-llama/Llama-3.1-8B-Instruct"
)
response_template = "<|start_header_id|>assistant<|end_header_id|>"
collator = DataCollatorForCompletionOnlyLM(
response_template=response_template,
tokenizer=tokenizer
)
config = SFTConfig(
max_seq_length=2048,
output_dir="sft_model"
)
trainer = SFTTrainer(
model=model,
tokenizer=tokenizer,
args=config,
train_dataset=dataset,
data_collator=collator,
formatting_func=formatting_func
)
trainer.train()
Real-world application
Integrate the collator into an SFT pipeline when the dataset contains prompts and assistant responses. Keep the formatting function and response template consistent with the model chat format.
Key takeaway
The data_collator argument applies completion-only masking during SFTTrainer data preparation.