1.17.3
Key Safety Mechanisms
What is it?
LLM safety mechanisms use several complementary techniques to reduce harmful or unreliable behaviour. The source describes data filtering, safety supervised fine-tuning, Constitutional AI, safety reward models, runtime guardrails and red-teaming.
Key points
1. Data filtering removes toxic, biased and personally identifiable information-containing text.
2. Safety SFT trains models using examples of appropriate refusals.
3. Constitutional AI uses principles to critique and revise model outputs.
4. Safety reward models score responses for safety.
5. Guardrails use input and output classifiers during serving.
6. Red-teaming uses adversarial tests to discover weaknesses.
7. Multiple independent mechanisms provide layered protection.
Applying multiple safety mechanisms
Consider a customer-support AI assistant. Input classifiers can detect unsafe requests, the model can be trained to refuse inappropriate instructions, output classifiers can inspect responses, and red-team testing can identify bypass attempts before deployment.
Real-world application
For an AI agent, combine model-level safety training with runtime input and output checks, tool restrictions, monitoring and appropriate human review.
Key takeaway
Reliable safety requires multiple complementary controls across training and runtime.