OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / SFT Best Practices and Techniques / Chat Templates and Formatting
Lesson 387 of 415 8 min read
10.2.4

Formatting SFT Datasets with TRL

Build a formatting function and connect it to SFTConfig and SFTTrainer.

What is it?

TRL SFTTrainer can use a formatting_func to convert each dataset example into a formatted conversation. The function builds role-based messages and applies the tokenizer chat template before training.

Key points

Import SFTConfig and SFTTrainer from TRL. Load the tokenizer for the target model. Create a formatting_func that accepts a dataset example. Build system, user and assistant messages. Return tokenizer.apply_chat_template() output. Pass formatting_func to SFTTrainer.

Formatting an SFT Dataset with TRL

from transformers import AutoTokenizer from trl import SFTConfig, SFTTrainer tokenizer = AutoTokenizer.from_pretrained( "meta-llama/Llama-3.1-8B-Instruct" ) def formatting_func(example): messages = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": example["instruction"]}, {"role": "assistant", "content": example["response"]} ] return tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=False ) config = SFTConfig( max_seq_length=2048, output_dir="sft_model" ) trainer = SFTTrainer( model=model, tokenizer=tokenizer, args=config, train_dataset=dataset, formatting_func=formatting_func ) trainer.train()

Real-world application

Use formatting_func when the source dataset stores instructions and responses in separate columns. This keeps dataset conversion consistent and connects directly to the TRL supervised fine-tuning workflow.
Key takeaway

A formatting function connects structured training records to the model-specific chat template used by SFTTrainer.