Quiz Space

Reinforcement Learning · Quiz 2 · 23 Nov 2025 · September 2025 term

Question 12: Consider the following algorithm for learning action-val…

Question 12

+4 marksOne or more correct options

Consider the following algorithm for learning action-value estimates in an episodic MDP:

Based on the above data, answer the given subquestions.

In the given pseudocode, the algorithm maintains two value functions,
and
, and updates them alternately.
Which of the following best describes the purpose and effect of this design choice?

Select all that apply.

  1. A

    It ensures that the same target network is used for both selection and evaluation, improving stability.

  2. B

    It ensures the overall estimate equals the true expected return by averaging

    and
    .

  3. C

    It decorrelates the selection and evaluation of the greedy action, reducing overestimation bias.

  4. D

    It doubles the effective learning rate by updating two estimators in parallel.

Show answer

Correct answer

  • C

    It decorrelates the selection and evaluation of the greedy action, reducing overestimation bias.

See the whole paperTake it as a mock test

Question 12 of 17 in the IIT Madras BS Reinforcement Learning (Reinforcement Learning) Quiz 2 paper sat on 23 Nov 2025, in the September 2025 term (IIT M DEGREE AN EXAM QDB2 23 Nov 2025 NEW). It carries 4 marks.

More questions from this paper

  1. Q1Consider the following assertion reason pair:\ Assertion: In Monte Carlo (MC), the value function converges to the cert…
  2. Q2Which of the following is the correct way to represent policy for policy search methods? Assume represents a real-value…
  3. Q3Recall the incremental update rule for REINFORCE: Consider the following binary-bandit problem: Which of the following …
  4. Q4In Q-Learning, the update rule for the action-value function is based on bootstrapping from the current estimate. Which…
  5. Q5Consider an episodic task where the\ -step returns follow a decaying exponential pattern: Assume the episode is suffici…
  6. Q6What is the primary reason for using a separate target network with periodically updated weights in DQN?
  7. Q7What are the effects of periodically updating the weights of the target network in DQN?
  8. Q8Find the Q values for all three actions\ for the state\ . Express your answer as a vector q, where
  9. Q9Compute the TD target for this transition using SARSA with\ . Use a greedy policy derived from the Q-values at\ to sele…
  10. Q10Perform one Step of semi-gradient TD using this transition. What would be\ if\ ?
  11. Q11Consider the following algorithm for learning action-value estimates in an episodic MDP: Based on the above data, answe…
  12. Q13Recall from the lectures, the forward-view\ -return is defined as: where\ denotes the\ -step return starting at time\ .…
  13. Q14Recall from the lectures, the forward-view\ -return is defined as: where\ denotes the\ -step return starting at time\ .…
  14. Q15Using every-visit Monte Carlo, estimate\ and\ . What is the value of\ ? (use\ )
  15. Q16Using every-visit Monte Carlo, estimate\ and\ . What is the value of\ ? (use )
  16. Q17Batch Monte Carlo (MC) methods estimate the value function by finding the least-squares fit to the sampled returns gene…