3.3.1
Understanding the Taxonomy of RL Methods
Introduction to the different dimensions used to classify reinforcement learning algorithms.
What is it?
Reinforcement learning algorithms can be classified along several dimensions. This taxonomy helps identify how an algorithm learns, represents decisions and uses experience.
Key points
1. RL methods can be classified along multiple axes.
2. Model-Free vs Model-Based describes how the environment is represented.
3. Value-Based vs Policy-Based describes what the algorithm learns.
4. On-Policy vs Off-Policy describes which experience is used for learning.
5. Actor-Critic combines policy and value learning.
6. Different problems require different approaches.
Example: Choosing an RL approach
Consider an autonomous robot learning to navigate a warehouse.
One algorithm may learn action values and select the best action.
Another may directly optimize a policy.
A model-based approach may simulate future movements before acting.
An off-policy approach may reuse previously collected experience.
These differences represent the major dimensions of RL taxonomy.
Real-world application
The taxonomy helps researchers and developers select suitable algorithms based on action spaces, environment knowledge and available training data.
Key takeaway
RL taxonomy organizes algorithms according to how they learn, make decisions and use experience.