Quiz Space

Reinforcement Learning · Quiz 2 · 23 Nov 2025 · September 2025 term

Question 14: Recall from the lectures, the forward-view\ -return is d…

Question 14

+3 marksOne correct option

Recall from the lectures, the forward-view
-return is defined as:

where
denotes the
-step return starting at time
.Consider using the TD(
) algorithm for an episodic task.
Based on the above data, answer the given subquestions.

As
in an episodic TD(
) task, the
-return
approaches which of the following?

  1. A

    The Monte Carlo return.

  2. B

    The one-step TD target.

  3. C

    The average of all
    -step returns.

  4. D

    Zero, since weights vanish as
    .

Show answer

Correct answer

  • A

    The Monte Carlo return.

See the whole paperTake it as a mock test

Question 14 of 17 in the IIT Madras BS Reinforcement Learning (Reinforcement Learning) Quiz 2 paper sat on 23 Nov 2025, in the September 2025 term (IIT M DEGREE AN EXAM QDB2 23 Nov 2025 NEW). It carries 3 marks.

More questions from this paper

  1. Q1Consider the following assertion reason pair:\ Assertion: In Monte Carlo (MC), the value function converges to the cert…
  2. Q2Which of the following is the correct way to represent policy for policy search methods? Assume represents a real-value…
  3. Q3Recall the incremental update rule for REINFORCE: Consider the following binary-bandit problem: Which of the following …
  4. Q4In Q-Learning, the update rule for the action-value function is based on bootstrapping from the current estimate. Which…
  5. Q5Consider an episodic task where the\ -step returns follow a decaying exponential pattern: Assume the episode is suffici…
  6. Q6What is the primary reason for using a separate target network with periodically updated weights in DQN?
  7. Q7What are the effects of periodically updating the weights of the target network in DQN?
  8. Q8Find the Q values for all three actions\ for the state\ . Express your answer as a vector q, where
  9. Q9Compute the TD target for this transition using SARSA with\ . Use a greedy policy derived from the Q-values at\ to sele…
  10. Q10Perform one Step of semi-gradient TD using this transition. What would be\ if\ ?
  11. Q11Consider the following algorithm for learning action-value estimates in an episodic MDP: Based on the above data, answe…
  12. Q12Consider the following algorithm for learning action-value estimates in an episodic MDP: Based on the above data, answe…
  13. Q13Recall from the lectures, the forward-view\ -return is defined as: where\ denotes the\ -step return starting at time\ .…
  14. Q15Using every-visit Monte Carlo, estimate\ and\ . What is the value of\ ? (use\ )
  15. Q16Using every-visit Monte Carlo, estimate\ and\ . What is the value of\ ? (use )
  16. Q17Batch Monte Carlo (MC) methods estimate the value function by finding the least-squares fit to the sampled returns gene…