10.4.4
Implementing Data Mixing in TRL
Combine datasets using concatenate_datasets and interleave_datasets before passing the mixed dataset to SFTTrainer.
What is it?
The Datasets library provides concatenate_datasets and interleave_datasets for combining training datasets. The resulting mixed dataset can be passed directly to TRL SFTTrainer.
Key points
Import concatenate_datasets and interleave_datasets.
Use concatenate_datasets to combine datasets and shuffle the result for proportional mixing.
Use interleave_datasets with probabilities to control sampling proportions.
Set a random seed for reproducibility.
Pass the mixed dataset to SFTTrainer.
Data Mixing with HuggingFace Datasets and TRL
from datasets import concatenate_datasets, interleave_datasets
from trl import SFTConfig, SFTTrainer
# Proportional mixing
mixed_dataset = concatenate_datasets([
dataset_math,
dataset_code,
dataset_general
]).shuffle(seed=42)
# Controlled mixing with sampling probabilities
mixed_dataset = interleave_datasets(
[dataset_math, dataset_code, dataset_general],
probabilities=[0.4, 0.4, 0.2],
seed=42
)
config = SFTConfig(output_dir="sft_model")
trainer = SFTTrainer(
model=model,
args=config,
train_dataset=mixed_dataset
)
Real-world application
Prepare each task dataset separately, select the mixing strategy, and create a reproducible mixed dataset before passing it to SFTTrainer.
Key takeaway
Dataset mixing happens before training: prepare the mixed dataset and then supply it to SFTTrainer.