1.4.5
Hugging Face Implementation
What is it?
Hugging Face Transformers provides model classes that expose different outputs according to their configuration and task. A causal language model typically returns vocabulary logits, while a model used for sequence classification returns task-specific scores. Reinforcement learning implementations may attach a separate value head to a language model.
Key points
1. Hugging Face provides AutoModelForCausalLM for causal language modeling.
2. Its output commonly includes logits over the vocabulary.
3. AutoModelForSequenceClassification provides classification outputs.
4. A value head is not automatically present in every causal language model.
5. Reinforcement learning libraries may provide models with an additional value head.
6. Check the model class and output structure before using its results.
Example: Getting language model logits with Python
Python example:
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "gpt2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
inputs = tokenizer("Artificial intelligence is", return_tensors="pt")
outputs = model(**inputs)
logits = outputs.logits
print(logits.shape)
The logits tensor contains scores for vocabulary tokens at each input position. This example demonstrates language modeling output, not a value-head prediction.
Real-world application
This knowledge is useful when integrating pretrained models, inspecting model outputs, building text-generation applications and preparing models for reinforcement learning.
Key takeaway
Hugging Face model classes expose task-specific outputs. Understand the model class and output tensor before building an application around it.