OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / SFT Best Practices and Techniques / Data Mixing Strategies for Multi-Task SFT
Lesson 392 of 415 6 min read
10.4.1

The Multi-Task Challenge and Task Interference

Understand how combining multiple tasks can improve generalisation while conflicting gradients can degrade individual task performance.

What is it?

Training on multiple tasks can improve generalisation, but gradients from different tasks may conflict. This task interference can degrade performance on individual tasks. Data mixing strategies control how much each dataset contributes to the training signal.

Key points

Multi-task SFT combines datasets from different tasks. Training across tasks can improve generalisation. Different tasks can produce conflicting gradients. Task interference can reduce performance on individual tasks. Data mixing controls the relative contribution of each task.

Math, Code and General Instruction Data

Imagine an SFT dataset containing: Math examples: numerical reasoning and problem solving. Code examples: programming and debugging. General examples: instruction following and conversation. If one task dominates the training signal, the model may become less effective on the others. A mixing strategy controls how frequently examples from each dataset are sampled.

Real-world application

Before combining datasets, identify the tasks represented and monitor model performance on each task independently. Use a deliberate sampling strategy rather than allowing dataset size alone to determine every training example.
Key takeaway

Multi-task SFT needs a deliberate balance because task gradients can interfere with one another.