10.3.1
Why Mask the Prompt?
Understand why instruction fine-tuning should calculate loss on assistant responses rather than system prompts and user messages.
What is it?
In instruction fine-tuning, the model should learn to generate the assistant response, not predict the system prompt or user question. Computing loss on prompt tokens uses training signal on text that the model is meant to condition on. Completion-only masking sets the loss to zero for non-assistant tokens.
Key points
The prompt provides context for the response.
The assistant response is the target to learn.
Prompt-token loss can waste gradient signal and encourage prompt memorisation.
Completion-only masking excludes non-assistant tokens from the loss calculation.
Prompt Tokens vs Completion Tokens
System: You are a helpful assistant.
User: What is sequence packing?
Assistant: Sequence packing combines multiple short examples into a longer sequence to reduce padding.
With completion-only masking, the system and user text provide context, while the assistant response contributes to the training loss.
Real-world application
Use completion-only masking when training instruction-following models on prompt-response datasets. Ensure the target loss is applied to the assistant response rather than the prompt.
Key takeaway
Train the model to produce the answer, while using the prompt as context.