OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / SFT Best Practices and Techniques / Chat Templates and Formatting
Lesson 385 of 415 6 min read
10.2.2

ChatML and Llama 3 Conversation Formats

Compare ChatML and Llama 3 special-token formats for representing conversations.

What is it?

ChatML and Llama 3 use special tokens to represent conversation roles and message boundaries. Both formats separate system, user and assistant messages, but their token conventions differ.

Key points

ChatML uses <|im_start|> to begin a message and <|im_end|> to end it. Llama 3 uses <|begin_of_text|> to begin the sequence. Llama 3 uses <|start_header_id|> and <|end_header_id|> to delimit role headers. Llama 3 uses <|eot_id|> to mark the end of a message. The model tokenizer and its expected template determine which format to use.

ChatML and Llama 3 Formats

ChatML: <|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>user Explain AI agents.<|im_end|> <|im_start|>assistant AI agents use models and tools to perform tasks.<|im_end|> Llama 3: <|begin_of_text|><|start_header_id|>system<|end_header_id|> You are a helpful assistant.<|eot_id|><|start_header_id|>user<|end_header_id|> Explain AI agents.<|eot_id|><|start_header_id|>assistant<|end_header_id|> AI agents use models and tools to perform tasks.<|eot_id|>

Real-world application

Inspect the tokenizer associated with a pretrained model before formatting an SFT dataset. Do not assume that ChatML and Llama 3 templates are interchangeable.
Key takeaway

Different models can use different chat templates. Use the format expected by the selected model.