6.4.3
Configuring LoRA Adapters
Understand LoRA rank, alpha, dropout and target modules for parameter-efficient training.
What is it?
LoRA enables parameter-efficient DPO training by adapting selected model projection layers instead of updating all model parameters.
Key points
1. r = 64.
2. lora_alpha = 16.
3. lora_dropout = 0.05.
4. Target modules include q_proj, k_proj, v_proj and o_proj.
5. Additional targets: gate_proj, up_proj and down_proj.
LoRA Configuration
from peft import LoraConfig
lora_config = LoraConfig(
r=64,
lora_alpha=16,
lora_dropout=0.05,
target_modules=[
"q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj"
]
)
Real-world application
Use LoRA to reduce trainable parameter count and make fine-tuning more resource-efficient.
Key takeaway
LoRA adapts selected model layers while avoiding full-model fine-tuning.