3.1.2
The Five Components of an MDP
Explore the five elements: state space, action space, transition function, reward function, and discount factor.
What is it?
An MDP is formally represented by the five-component tuple (S, A, P, R, γ). These components describe states, actions, transitions, rewards and the discount factor.
Key points
1. S: State space.
2. A: Action space.
3. P: Transition function.
4. R: Reward function.
5. γ: Discount factor.
6. Together, these components define the MDP.
Example: Five components in a navigation problem
S: All possible grid positions.
A: Move up, down, left or right.
P: Probability of reaching a next position.
R: Reward for reaching the destination and penalties for obstacles.
γ: Importance assigned to future rewards.
Real-world application
These components help developers formally define reinforcement learning environments.
Key takeaway
An MDP is defined by states, actions, transitions, rewards and the discount factor.