6.4.1
Loading the Language Model and Tokenizer
Understand how to load a causal language model and tokenizer using Transformers.
What is it?
The TRL implementation loads a causal language model and its tokenizer using Hugging Face Transformers. The source example uses Llama 3.1 8B Instruct with bfloat16 and Flash Attention 2.
Key points
1. AutoModelForCausalLM loads the model.
2. AutoTokenizer loads the tokenizer.
3. Model: meta-llama/Llama-3.1-8B-Instruct.
4. torch_dtype = torch.bfloat16.
5. attn_implementation = flash_attention_2.
6. The model requires appropriate hardware and access permissions.
Loading the Model
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Llama-3.1-8B-Instruct",
torch_dtype=torch.bfloat16,
attn_implementation="flash_attention_2"
)
tokenizer = AutoTokenizer.from_pretrained(
"meta-llama/Llama-3.1-8B-Instruct"
)
Real-world application
Load the same model checkpoint and tokenizer before preparing DPO training.
Key takeaway
Model and tokenizer loading establish the foundation for DPO training.