OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
Foundations / LLM Architecture and Optimization Methods / LLM Safety and Responsible AI
Lesson 94 of 415 5 min read
1.17.1

Threat Taxonomy

What is it?

LLM safety begins by identifying the threats that can arise when language models generate content or interact with users and systems. The source identifies six major categories: harmful content, bias and discrimination, privacy violations, jailbreaking, misinformation and dual-use risks.

Key points

1. Harmful content: Generating toxic, violent or illegal instructions. 2. Bias and discrimination: Perpetuating stereotypes or treating groups unfairly. 3. Privacy violations: Exposing personal information or memorized training data. 4. Jailbreaking: Using adversarial prompts to bypass safety guardrails. 5. Misinformation: Generating convincing but false claims. 6. Dual-use: Using legitimate capabilities for harmful purposes.

Identifying safety threats in an AI assistant

An AI assistant may generate harmful instructions, reproduce biased language, expose private information, follow a jailbreak prompt, produce false claims or provide technical capabilities that can be misused. Each case represents a different safety threat and requires an appropriate control.

Real-world application

Before deploying an AI assistant, identify the types of information it handles, the actions it can perform and the ways its capabilities could be misused. Use this threat classification to plan safeguards.
Key takeaway

AI safety starts with understanding the different ways a model can cause harm or be misused.