Quiz Space

Reinforcement Learning · Quiz 2 · 23 Nov 2025 · September 2025 term

Question 15: Using every-visit Monte Carlo, estimate\ and\ . What is …

Question 15

+3 marksNumerical answer

Consider an episodic task with four non-terminal states: A, B, C and D. The following are some episodes experienced by an agent following a fixed policy. The terminal state is not explicitly mentioned for any of the episodes.
A,1, C,1, D,0, C,0, A,1, B,1
A,0, C,0, D,1, C,1, B,1
D,1, C,0, A,0, C,1, B,0
C,0, A,1, C,1, B,0
C,1, B,0
D,0, B,1
B,1
B,0
(Enter your answer correct to three decimal places.)
Based on the above data, answer the given subquestions.

Using every-visit Monte Carlo, estimate
and
. What is the value of
?

(use
)

Show answer

Correct answer: 1.975 (accepted within ±0.005)

See the whole paperTake it as a mock test

Question 15 of 17 in the IIT Madras BS Reinforcement Learning (Reinforcement Learning) Quiz 2 paper sat on 23 Nov 2025, in the September 2025 term (IIT M DEGREE AN EXAM QDB2 23 Nov 2025 NEW). It carries 3 marks.

More questions from this paper

  1. Q1Consider the following assertion reason pair:\ Assertion: In Monte Carlo (MC), the value function converges to the cert…
  2. Q2Which of the following is the correct way to represent policy for policy search methods? Assume represents a real-value…
  3. Q3Recall the incremental update rule for REINFORCE: Consider the following binary-bandit problem: Which of the following …
  4. Q4In Q-Learning, the update rule for the action-value function is based on bootstrapping from the current estimate. Which…
  5. Q5Consider an episodic task where the\ -step returns follow a decaying exponential pattern: Assume the episode is suffici…
  6. Q6What is the primary reason for using a separate target network with periodically updated weights in DQN?
  7. Q7What are the effects of periodically updating the weights of the target network in DQN?
  8. Q8Find the Q values for all three actions\ for the state\ . Express your answer as a vector q, where
  9. Q9Compute the TD target for this transition using SARSA with\ . Use a greedy policy derived from the Q-values at\ to sele…
  10. Q10Perform one Step of semi-gradient TD using this transition. What would be\ if\ ?
  11. Q11Consider the following algorithm for learning action-value estimates in an episodic MDP: Based on the above data, answe…
  12. Q12Consider the following algorithm for learning action-value estimates in an episodic MDP: Based on the above data, answe…
  13. Q13Recall from the lectures, the forward-view\ -return is defined as: where\ denotes the\ -step return starting at time\ .…
  14. Q14Recall from the lectures, the forward-view\ -return is defined as: where\ denotes the\ -step return starting at time\ .…
  15. Q16Using every-visit Monte Carlo, estimate\ and\ . What is the value of\ ? (use )
  16. Q17Batch Monte Carlo (MC) methods estimate the value function by finding the least-squares fit to the sampled returns gene…