7.3.1
Loading the Model and Tokenizer
Understand how to load a causal language model and its tokenizer for GRPO training.
What is it?
GRPO training begins by loading a causal language model and its tokenizer. The source example uses Qwen2.5-7B-Instruct with bfloat16 precision and Flash Attention 2.
Key points
Import GRPOConfig and GRPOTrainer from trl.
Load AutoModelForCausalLM from transformers.
Load the matching AutoTokenizer.
The source model is Qwen/Qwen2.5-7B-Instruct.
Model settings: torch_dtype=torch.bfloat16 and attn_implementation="flash_attention_2".
Loading a GRPO Model
The source loads the model using AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct", torch_dtype=torch.bfloat16, attn_implementation="flash_attention_2"). It then loads the matching tokenizer with AutoTokenizer.from_pretrained().
Real-world application
Load the model and tokenizer before creating the GRPO configuration. Use a matching tokenizer for the selected model.
Key takeaway
GRPO requires a language model and its tokenizer before training can be configured.