10.4.1
The Multi-Task Challenge and Task Interference
Understand how combining multiple tasks can improve generalisation while conflicting gradients can degrade individual task performance.
What is it?
Training on multiple tasks can improve generalisation, but gradients from different tasks may conflict. This task interference can degrade performance on individual tasks. Data mixing strategies control how much each dataset contributes to the training signal.
Key points
Multi-task SFT combines datasets from different tasks.
Training across tasks can improve generalisation.
Different tasks can produce conflicting gradients.
Task interference can reduce performance on individual tasks.
Data mixing controls the relative contribution of each task.
Math, Code and General Instruction Data
Imagine an SFT dataset containing:
Math examples: numerical reasoning and problem solving.
Code examples: programming and debugging.
General examples: instruction following and conversation.
If one task dominates the training signal, the model may become less effective on the others. A mixing strategy controls how frequently examples from each dataset are sampled.
Real-world application
Before combining datasets, identify the tasks represented and monitor model performance on each task independently. Use a deliberate sampling strategy rather than allowing dataset size alone to determine every training example.
Key takeaway
Multi-task SFT needs a deliberate balance because task gradients can interfere with one another.