10.1.4
Configuring Sequence Packing in TRL
Configure packing in SFTConfig and train with SFTTrainer.
What is it?
TRL supports sequence packing through the packing parameter in SFTConfig. The source demonstrates enabling packing with SFTTrainer and setting a maximum sequence length of 4,096 tokens.
Key points
Import SFTConfig and SFTTrainer from TRL.
Set max_seq_length=4096.
Enable packing=True.
The source example uses per_device_train_batch_size=4.
It uses gradient_accumulation_steps=4, learning_rate=2e-5 and num_train_epochs=3.
Pass the model, configuration and training dataset to SFTTrainer, then call trainer.train().
Enabling Packing in TRL
from trl import SFTConfig, SFTTrainer
config = SFTConfig(
max_seq_length=4096,
packing=True,
output_dir="sft_model",
per_device_train_batch_size=4,
gradient_accumulation_steps=4,
learning_rate=2e-5,
num_train_epochs=3,
)
trainer = SFTTrainer(
model=model,
args=config,
train_dataset=dataset,
)
trainer.train()
Real-world application
Enable packing=True in SFTConfig when using a compatible TRL SFTTrainer setup. Confirm that the attention masking behaviour is correct for the selected model and dataset.
Key takeaway
TRL makes sequence packing configurable through SFTConfig and SFTTrainer.