OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / SFT Best Practices and Techniques / Data Mixing Strategies for Multi-Task SFT
Lesson 395 of 415 8 min read
10.4.4

Implementing Data Mixing in TRL

Combine datasets using concatenate_datasets and interleave_datasets before passing the mixed dataset to SFTTrainer.

What is it?

The Datasets library provides concatenate_datasets and interleave_datasets for combining training datasets. The resulting mixed dataset can be passed directly to TRL SFTTrainer.

Key points

Import concatenate_datasets and interleave_datasets. Use concatenate_datasets to combine datasets and shuffle the result for proportional mixing. Use interleave_datasets with probabilities to control sampling proportions. Set a random seed for reproducibility. Pass the mixed dataset to SFTTrainer.

Data Mixing with HuggingFace Datasets and TRL

from datasets import concatenate_datasets, interleave_datasets from trl import SFTConfig, SFTTrainer # Proportional mixing mixed_dataset = concatenate_datasets([ dataset_math, dataset_code, dataset_general ]).shuffle(seed=42) # Controlled mixing with sampling probabilities mixed_dataset = interleave_datasets( [dataset_math, dataset_code, dataset_general], probabilities=[0.4, 0.4, 0.2], seed=42 ) config = SFTConfig(output_dir="sft_model") trainer = SFTTrainer( model=model, args=config, train_dataset=mixed_dataset )

Real-world application

Prepare each task dataset separately, select the mixing strategy, and create a reproducible mixed dataset before passing it to SFTTrainer.
Key takeaway

Dataset mixing happens before training: prepare the mixed dataset and then supply it to SFTTrainer.