OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
RL Methods for LLMs / SFT Best Practices and Techniques / When SFT Hurts – Catastrophic Forgetting and Alignment Tax
Lesson 400 of 415 7 min read
10.5.5

Diagnosing Capability Regression

Use base-model comparisons, prompt tests, perplexity checks and few-shot recovery to identify the source of performance degradation.

What is it?

Capability regression can be diagnosed using base-model comparisons, prompt engineering, perplexity and few-shot recovery. These tests help distinguish erased knowledge from knowledge that remains but is difficult to trigger under the aligned policy.

Key points

Base-model comparison: test the failing tasks on the original checkpoint and fine-tuned model. Prompt engineering: recovery through careful prompting suggests alignment tax. Perplexity: a spike on the pre-training validation set suggests catastrophic forgetting; stable perplexity suggests alignment tax. Few-shot recovery: recovery after in-context examples suggests alignment tax. Failure to recover even with many examples suggests catastrophic forgetting.

A Practical Capability Regression Diagnosis

1. Run the same failing task on the base model and fine-tuned model. 2. Try a carefully designed prompt to see whether the fine-tuned model can recover the capability. 3. Compare perplexity on the pre-training validation set. 4. Provide a few in-context examples and test again. Interpretation: - Base model succeeds but fine-tuned model fails completely: investigate catastrophic forgetting. - Prompting or few-shot examples restore performance: alignment tax is more likely. - Perplexity spike: evidence of forgetting. - Stable perplexity: evidence consistent with alignment tax.

Real-world application

Build a regression test suite that compares base and fine-tuned checkpoints. Include representative prompts, pre-training validation perplexity and few-shot recovery tests before deciding whether to adjust SFT or alignment training.
Key takeaway

Diagnose whether knowledge was erased or suppressed before attempting to fix capability regression.