1.4.2
Conditional Generation Head (SFT / Instruction Following)
What is it?
Conditional generation uses an input context to guide the output produced by a model. The Transformer processes the provided instructions or input, and its generation head produces token scores conditioned on that context. In instruction-following models, supervised fine-tuning teaches the model to generate responses that follow examples of desired behavior.
Key points
1. The input provides context for generation.
2. The Transformer processes the prompt and its instructions.
3. The language modeling head produces next-token logits.
4. The output distribution depends on the input context.
5. Supervised fine-tuning uses examples of desired input-output behavior.
6. Instruction following is learned through training, not simply by changing the output head.
Example: Generating a response from an instruction
Instruction:
"Explain machine learning in two sentences."
The model processes the instruction and generates a response conditioned on it.
Possible response:
"Machine learning is a branch of artificial intelligence that learns patterns from data. It uses those patterns to make predictions or decisions."
The generation head predicts the response tokens one step at a time.
Real-world application
Conditional generation is used in instruction-following assistants, summarization, translation, question answering and content generation.
Key takeaway
Conditional generation produces output based on an input context. Instruction-following behavior is developed through appropriate training and prompting.