OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
Foundations / LLM Architecture and Optimization Methods / Prompt Engineering
Lesson 78 of 415 6 min read
1.13.1

In-Context Learning (ICL)

What is it?

In-Context Learning (ICL) is the ability of a language model to learn a task at inference time from examples included in the prompt. The model identifies the pattern in the examples and applies it to new inputs without updating its weights.

Key points

ICL uses examples inside the prompt. No gradient updates or fine-tuning are required. The model infers the task from input-output patterns. The source discusses implicit Bayesian inference, induction heads and task vectors as explanations for how ICL works. ICL generally improves with model scale.

Learning a Pattern from Examples

Prompt examples: Input: happy, Output: positive. Input: terrible, Output: negative. New input: excellent. The model identifies the demonstrated sentiment classification pattern and predicts positive.

Real-world application

Use ICL when a model needs to follow a particular task format or pattern without changing the model through fine-tuning.
Key takeaway

In-context learning allows a model to adapt its behaviour from examples provided directly in the prompt.