OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
Foundations / Introduction to Reinforcement Learning / The Markov Decision Process (MDP)
Lesson 103 of 415 7 min read
3.1.5

Markov Property and Agent-Environment Interaction

Understand the Markov property and the step-by-step interaction between an agent and its environment.

What is it?

The Markov property states that the future state depends on the current state and action rather than the complete history of previous states and actions.

Key points

1. Current state contains relevant information about the past. 2. Next-state distribution depends on current state and action. 3. Agent observes sₜ. 4. Agent selects aₜ using policy π(a|s). 5. Environment transitions to sₜ₊₁. 6. Agent receives reward rₜ and repeats.

Example: Agent-environment interaction

1. Observe: Agent receives state sₜ. 2. Act: Agent selects action aₜ. 3. Transition: Environment moves to sₜ₊₁. 4. Reward: Agent receives rₜ. 5. Repeat: Agent observes the new state and chooses another action.

Real-world application

The Markov property helps developers design state representations for reinforcement learning environments.
Key takeaway

The Markov property represents relevant history through the current state.