OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
Foundations / Introduction to Reinforcement Learning / Core Concepts and Definitions
Lesson 106 of 415 5 min read
3.2.3

The State-Value Function

Learn how V under a policy represents the expected return from a given state.

What is it?

The state-value function estimates the expected return when an agent starts in state s and follows policy π. It is represented by Vᵖⁱ(s) = Eᵖⁱ[Gₜ | sₜ = s].

Key points

1. Vᵖⁱ(s) represents the value of state s. 2. It estimates expected cumulative discounted reward. 3. The agent follows policy π. 4. Higher value indicates higher expected return. 5. Value depends on the policy and environment. 6. It supports value-based and policy-optimization methods.

Example: Evaluating a warehouse location

A robot operates in a warehouse. State A: Near the packing station. State B: Far from the packing station. Under a particular policy, State A may have a higher expected return because it is closer to completing deliveries.

Real-world application

State-value estimates help algorithms evaluate situations and improve policies.
Key takeaway

The state-value function estimates expected return from a state under a policy.