Quiz Space

Reinforcement Learning · Quiz 1 · 15 Mar 2026 · January 2026 term

Question 20: Consider a 2-state MDP with states and , discount factor…

Question 20

+1 markWritten answer

Consider a 2-state MDP with states and , discount factor . There are two actions in : and . Action gives immediate reward and moves deterministically to ; action gives immediate reward and keeps the agent in . In , there is a single action that yields an immediate reward and transitions back to . Assume initial value estimates are , Based on the above data, answer the given subquestions.

Show answer

A written answer, not marked automatically.

See the whole paperTake it as a mock test

Question 20 of 22 in the IIT Madras BS Reinforcement Learning (Reinforcement Learning) Quiz 1 paper sat on 15 Mar 2026, in the January 2026 term (Reinforcement Learning 15 Mar 26). It carries 1 mark.

More questions from this paper

  1. Q1Which of the following is/are correct and valid reasons to consider sampling actions from a softmax distribution instea…
  2. Q2Consider a discounted return: in an infinite-horizon MDP with bounded rewards and discount factor 1.For fixed rewards, …
  3. Q3Consider the following assertion and reason pair and select the correct option: Assertion:In the UCB algorithm for mult…
  4. Q4In the context of MDPs, what does it mean for a policy to be greedy with respect to an action- value function ?
  5. Q5In policy iteration for finite MDPs, how are the Bellman equations used during the policy evaluation step?
  6. Q6In a -armed bandit setting, we maintain a running estimate of the action-value for each arm , denoted . We now consider…
  7. Q7Which of the following equations best represents the Bellman optimality equation for the optimal state-value function
  8. Q8The sequence of rewards for a continuing task with is given below: Find the return . Your answer should have exactly tw…
  9. Q9You have a 3-armed bandit where each arm, when active, yields a Bernoulli reward with success probabilities , , and . H…
  10. Q10Assume the initial value estimates are , , and . After performing one synchronous policy evaluation update (under the f…
  11. Q11Starting from , , and . perform two successive synchronous policy evaluation sweeps (under the fixed policy that follow…
  12. Q12If you perform a single value iteration Bellman optimality update for , what is the new value ?
  13. Q13Suppose after some value iterations your current estimates are , . If you perform one value iteration update for , what…
  14. Q14Consider a faulty -armed bandit with -noise activation and base arm success probabilities . If you always choose the ar…
  15. Q15Consider a faulty -armed bandit with -noise activation and distinct base success probabilities . Suppose the agent foll…
  16. Q16In a faulty -armed bandit with -noise activation, suppose you mistakenly model the environment as a standard bandit wit…
  17. Q17Which of the following best explains why a contextual bandit approach is preferred over a standard multi-armed bandit i…
  18. Q18After many rounds of user interaction, what is the main optimality objective of the contextual bandit algorithm in this…
  19. Q19Consider a Markov Decision Process (MDP) with three states , , and , where is a terminal state and . Each non-terminal …
  20. Q21Consider an N-armed bandit problem in which each arm , for , produces a reward of 1 with probability and otherwise, whe…
  21. Q22Consider a large-scale news recommendation platform that uses a contextual bandit approach to decide which headline to …