OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
Foundations / LLM Architecture and Optimization Methods / Prediction Heads: What Transformers Output
Lesson 26 of 415 5 min read
1.4.4

Head Selection Summary

What is it?

Different prediction heads support different model outputs. A language modeling head produces vocabulary logits for token prediction. A value head produces numerical estimates used in reinforcement learning. Conditional generation uses the language modeling output while conditioning generation on an input prompt. Selecting the appropriate head depends on the training objective and task.

Key points

1. Language modeling head: predicts scores for vocabulary tokens. 2. Conditional generation: generates output based on an input context. 3. Value head: estimates a numerical value for a state or representation. 4. Language generation typically uses vocabulary logits. 5. Reinforcement learning workflows may use both policy and value outputs. 6. The required output determines the appropriate model configuration.

Example: Choosing the output head

Task: Generate an answer to a question. Required output: A sequence of text tokens. Relevant component: Language modeling head. Task: Estimate the expected return of a state during PPO training. Required output: A numerical value. Relevant component: Value head. The two outputs serve different roles in the learning process.

Real-world application

Understanding prediction heads helps developers configure language models, build reinforcement learning pipelines and distinguish generation outputs from value estimates.
Key takeaway

Choose the prediction head according to the output required: vocabulary scores for generation or numerical values for value estimation.