Google

The cumulative future reward, often called the return, represents the total reward an agent accumulates from a specific time step $$t$$ until the final time step $$T$$. It is a key metric in reinforcement learning for assessing the long-term value of actions or states. The return is calculated by summing the rewards from time step $$t$$ onwards, as shown by the formula: $$\sum_{k=t}^{T} r_k$$, where $$r_k$$ represents the reward received at time step $$k$$.

Cumulative Future Reward (Return)

An agent interacts with an environment over five time steps and receives the following sequence of rewards, starting from time step 1: `[-1, +3, +10, -5, +2]`. What is the cumulative future reward (also known as the return) calculated from time step 3?

Based on the principle of maximizing the total reward accumulated from the current time step onwards, which path should the agent prefer? Justify your answer by calculating the cumulative future reward for each path.

Evaluating Agent Action Sequences

An agent is at time step `t`. It must choose between two actions, Action A and Action B. If it chooses Action A, the sequence of rewards it will receive from time step `t` until the end of the episode is `[+1, +1, +10]`. If it chooses Action B, the sequence of rewards it will receive is `[+5, -2, +5]`. To maximize its total accumulated reward from this point forward, which action should the agent choose and why?

Learn Before

Related