Quiz Space

Reinforcement Learning · Quiz 2 · 23 Nov 2025 · September 2025 term

Question 7: What are the effects of periodically updating the weights…

Question 7

+2 marksOne or more correct options

In Deep Q-Networks (DQN), a separate target network is maintained, whose weights are periodically copied from the main (online) network instead of being updated at every step. Based on the above data, answer the given subquestions.

What are the effects of periodically updating the weights of the target network in DQN?

Select all that apply.

  1. A

    Provides stable targets to the main network during training.

  2. B

    Causes the learning targets to be non-stationary.

  3. C

    Mitigates instabilities in Q-learning updates.

  4. D

    Avoids the need for a replay buffer.

Show answer

Correct answers

  • A

    Provides stable targets to the main network during training.

  • C

    Mitigates instabilities in Q-learning updates.

See the whole paperTake it as a mock test

Question 7 of 17 in the IIT Madras BS Reinforcement Learning (Reinforcement Learning) Quiz 2 paper sat on 23 Nov 2025, in the September 2025 term (IIT M DEGREE AN EXAM QDB2 23 Nov 2025 NEW). It carries 2 marks.

More questions from this paper

  1. Q1Consider the following assertion reason pair:\ Assertion: In Monte Carlo (MC), the value function converges to the cert…
  2. Q2Which of the following is the correct way to represent policy for policy search methods? Assume represents a real-value…
  3. Q3Recall the incremental update rule for REINFORCE: Consider the following binary-bandit problem: Which of the following …
  4. Q4In Q-Learning, the update rule for the action-value function is based on bootstrapping from the current estimate. Which…
  5. Q5Consider an episodic task where the\ -step returns follow a decaying exponential pattern: Assume the episode is suffici…
  6. Q6What is the primary reason for using a separate target network with periodically updated weights in DQN?
  7. Q8Find the Q values for all three actions\ for the state\ . Express your answer as a vector q, where
  8. Q9Compute the TD target for this transition using SARSA with\ . Use a greedy policy derived from the Q-values at\ to sele…
  9. Q10Perform one Step of semi-gradient TD using this transition. What would be\ if\ ?
  10. Q11Consider the following algorithm for learning action-value estimates in an episodic MDP: Based on the above data, answe…
  11. Q12Consider the following algorithm for learning action-value estimates in an episodic MDP: Based on the above data, answe…
  12. Q13Recall from the lectures, the forward-view\ -return is defined as: where\ denotes the\ -step return starting at time\ .…
  13. Q14Recall from the lectures, the forward-view\ -return is defined as: where\ denotes the\ -step return starting at time\ .…
  14. Q15Using every-visit Monte Carlo, estimate\ and\ . What is the value of\ ? (use\ )
  15. Q16Using every-visit Monte Carlo, estimate\ and\ . What is the value of\ ? (use )
  16. Q17Batch Monte Carlo (MC) methods estimate the value function by finding the least-squares fit to the sampled returns gene…