OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / SFT Best Practices and Techniques / Data Mixing Strategies for Multi-Task SFT
Lesson 394 of 415 7 min read
10.4.3

Temperature and Quality-Weighted Mixing

Use temperature scaling to adjust dataset proportions and quality scores to influence sampling probabilities.

What is it?

Temperature mixing adjusts dataset sampling proportions using a temperature parameter. Quality-weighted mixing adjusts them using an estimated quality score. These approaches provide more control than sampling only according to dataset size.

Key points

Temperature mixing uses pk proportional to Nk raised to the power 1/T. T = 1 gives proportional mixing. As T approaches infinity, sampling approaches uniform mixing. T < 1 gives more weight to larger datasets. T > 1 gives more weight to smaller datasets. Quality-weighted mixing uses pk proportional to Nk multiplied by qk, where qk is the dataset quality score. Quality can be estimated using reference-model perplexity or human quality ratings.

Temperature and Quality-Weighted Sampling

Temperature mixing: T = 1: preserve proportional dataset sampling. T > 1: increase the relative contribution of smaller datasets. T approaching infinity: move towards uniform sampling. T < 1: increase the dominance of larger datasets. Quality-weighted mixing: pk ∝ Nk × qk Here, Nk is dataset size and qk is its estimated quality score.

Real-world application

Use temperature mixing when dataset sizes are highly imbalanced. Consider quality-weighted mixing when datasets differ in reliability or annotation quality. Estimate quality consistently before assigning weights.
Key takeaway

Temperature changes the influence of dataset size, while quality weighting incorporates an estimate of dataset quality.