Quiz Space

Reinforcement Learning · Quiz 2 · 23 Nov 2025 · September 2025 term

Question 13: Recall from the lectures, the forward-view\ -return is d…

Question 13

+3 marksOne correct option

Recall from the lectures, the forward-view
-return is defined as:

where
denotes the
-step return starting at time
.Consider using the TD(
) algorithm for an episodic task.
Based on the above data, answer the given subquestions.

What is the effect of setting
in TD(
) on the weighting of
-step returns?

  1. A

    All
    -step returns are ignored.

  2. B

    Only the 1-step return contributes, with weight 1.

  3. C

    All
    -step returns contribute equally.

  4. D

    The weighting distribution remains unchanged.

Show answer

Correct answer

  • B

    Only the 1-step return contributes, with weight 1.

See the whole paperTake it as a mock test

Question 13 of 17 in the IIT Madras BS Reinforcement Learning (Reinforcement Learning) Quiz 2 paper sat on 23 Nov 2025, in the September 2025 term (IIT M DEGREE AN EXAM QDB2 23 Nov 2025 NEW). It carries 3 marks.

More questions from this paper

  1. Q1Consider the following assertion reason pair:\ Assertion: In Monte Carlo (MC), the value function converges to the cert…
  2. Q2Which of the following is the correct way to represent policy for policy search methods? Assume represents a real-value…
  3. Q3Recall the incremental update rule for REINFORCE: Consider the following binary-bandit problem: Which of the following …
  4. Q4In Q-Learning, the update rule for the action-value function is based on bootstrapping from the current estimate. Which…
  5. Q5Consider an episodic task where the\ -step returns follow a decaying exponential pattern: Assume the episode is suffici…
  6. Q6What is the primary reason for using a separate target network with periodically updated weights in DQN?
  7. Q7What are the effects of periodically updating the weights of the target network in DQN?
  8. Q8Find the Q values for all three actions\ for the state\ . Express your answer as a vector q, where
  9. Q9Compute the TD target for this transition using SARSA with\ . Use a greedy policy derived from the Q-values at\ to sele…
  10. Q10Perform one Step of semi-gradient TD using this transition. What would be\ if\ ?
  11. Q11Consider the following algorithm for learning action-value estimates in an episodic MDP: Based on the above data, answe…
  12. Q12Consider the following algorithm for learning action-value estimates in an episodic MDP: Based on the above data, answe…
  13. Q14Recall from the lectures, the forward-view\ -return is defined as: where\ denotes the\ -step return starting at time\ .…
  14. Q15Using every-visit Monte Carlo, estimate\ and\ . What is the value of\ ? (use\ )
  15. Q16Using every-visit Monte Carlo, estimate\ and\ . What is the value of\ ? (use )
  16. Q17Batch Monte Carlo (MC) methods estimate the value function by finding the least-squares fit to the sampled returns gene…