3.1.1
What Is a Markov Decision Process?
Understand the MDP framework and why it is used to model sequential decisions.
What is it?
An MDP is a mathematical framework for sequential decision-making. An agent observes an environment, selects actions and receives rewards as the environment changes.
Key points
1. MDP describes sequential decisions.
2. Agent interacts with environment.
3. Agent observes the current state.
4. Agent selects an action.
5. Environment changes and provides a reward.
6. The process repeats to maximize cumulative reward.
Example: A robot navigating a room
A robot needs to reach a charging station.
Current state: Near the door.
Action: Move forward.
Next state: Middle of the room.
Reward: The environment evaluates the outcome.
The robot repeats this process until it reaches its destination.
Real-world application
MDPs are used in robotics, autonomous systems, resource allocation and game-playing agents.
Key takeaway
An MDP models how an agent makes decisions and receives rewards while interacting with an environment.