uiz Space

January 2025 term · Reinforcement Learning · BSDA5007

Reinforcement Learning End Term: 13 April 2025, Set 1-6 (January 2025 term)

The IIT Madras BS Reinforcement Learning (Reinforcement Learning) End Term paper sat on 13 Apr 2025, in the January 2025 term, set 1-6: 16 questions for 43 marks in 180 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.

Questions
16
Marks
43
Duration
180 min
MCQ
6
MSQ
3
Numerical
7
Take as mock testPractise with answers

Updated

Official paper: IIT M IMPROVEMENT FN EXAM QIM2 13 Apr · No negative marking.

Question 1

+2 marksOne correct option

Consider the following assertion reason pair and select the correct option:
Assertion: Reinforcement learning is a type of unsupervised learning algorithm as both don’t have correct labels.
Reason: In unsupervised learning, a reward like quantity is not maximized.

  1. A

    Assertion and Reason are both true and Reason is a correct explanation ofAssertion.

  2. B

    Assertion and Reason are both true and Reason is not a correct explanation ofAssertion.

  3. C

    Assertion is true but Reason is false.

  4. D

    Assertion is false but Reason is true.

Show answer

Correct answer

  • D

    Assertion is false but Reason is true.

Question 1 on its own pageAlso asked in End Term 13 Apr 2025

Question 2

+2 marksOne correct option

Consider a reinforcement learning agent navigating a grid world environment. The agent receives rewards of +1 for reaching the goal state and 0 otherwise. Which of the following statements accurately describes the differences between Monte Carlo and TD learning in this scenario?

  1. A

    Monte Carlo updates are unbiased estimators of the true value function, whileTD updates may introduce bias.

  2. B

    TD updates are guaranteed to converge to the optimal value function, whileMonte Carlo updates may not converge.

  3. C

    Monte Carlo updates require less memory and computational resourcescompared to TD updates.

  4. D

    TD updates are more robust to noise and stochasticity in the environmentcompared to Monte Carlo updates.

Show answer

Correct answer

  • A

    Monte Carlo updates are unbiased estimators of the true value function, whileTD updates may introduce bias.

Question 2 on its own pageAlso asked in End Term 13 Apr 2025

Question 3

+2 marksOne correct option

In Q-learning, how does maximization bias affect the performance of the algorithm in complex environments?

  1. A

    Maximization bias can lead to overestimation of action values, resulting insuboptimal policies and slower convergence to the optimal policy.

  2. B

    Maximization bias helps to accelerate learning by prioritizing actions withhigher estimated values, leading to faster convergence to the optimal policy.

  3. C

    Maximization bias reduces the exploration-exploitation trade-off, resulting inmore exploratory behavior and improved generalization to unseen states.

  4. D

    Maximization bias has minimal impact on the performance of Q-learning incomplex environments, as it tends to balance out over time through exploration.

Show answer

Correct answer

  • A

    Maximization bias can lead to overestimation of action values, resulting insuboptimal policies and slower convergence to the optimal policy.

Question 3 on its own pageAlso asked in End Term 13 Apr 2025

Question 4

+3 marksOne correct option

In which of the following scenarios is Expected SARSA a good fit?

  1. A

    When the environment has a high degree of stochasticity, makingbootstrapping unstable in value-based methods

  2. B

    When function approximation is necessary due to large state spaces, requiringdeep networks for learning representations

  3. C

    When learning needs to prioritize separating state value and advantagefunctions for better decision-making

  4. D

    When experience replay is essential for stable learning and sample efficiency

Show answer

Correct answer

  • A

    When the environment has a high degree of stochasticity, makingbootstrapping unstable in value-based methods

Question 4 on its own pageAlso asked in End Term 13 Apr 2025, End Term 13 Apr 2025, End Term 13 Apr 2025, End Term 13 Apr 2025 and 1 more

Question 5

+3 marksOne or more correct options

Which of the following methods are a form of Generalized Policy Iteration?

Select all that apply.

  1. A

    Policy Improvement

  2. B

    Value Iteration

  3. C

    Q-Learning and SARSA

  4. D

    Monte Carlo control methods

Show answer

Correct answers

  • B

    Value Iteration

  • C

    Q-Learning and SARSA

  • D

    Monte Carlo control methods

Question 5 on its own pageAlso asked in End Term 13 Apr 2025, End Term 13 Apr 2025, End Term 13 Apr 2025, End Term 13 Apr 2025 and 1 more

Question 6

+3 marksOne or more correct options

In the context of actor-critic methods, what is the effect of replacing the return Gt with the TD target?

Select all that apply.

  1. A

    It increases the variance in the estimate of the gradient of the performance.

  2. B

    It decreases the variance in the estimate of the gradient of the performance.

  3. C

    It introduces a bias in the estimate of the gradient of the performance.

  4. D

    It doesn’t introduce any bias in the estimate of the gradient of theperformance.

  5. E

    None of these.

Show answer

Correct answers

  • B

    It decreases the variance in the estimate of the gradient of the performance.

  • C

    It introduces a bias in the estimate of the gradient of the performance.

Question 6 on its own pageAlso asked in End Term 13 Apr 2025

Question 7

+2 marksOne or more correct options

Select all that apply.

  1. A

    b can be a function of s

  2. B

    b can be a function of a

  3. C

    b can be a random variable independent of s, a

  4. D

    None of these

Show answer

Correct answers

  • A

    b can be a function of s

  • C

    b can be a random variable independent of s, a

Question 7 on its own page

Question 8

+3 marksNumerical answer
Show answer

Correct answer: -7

Question 8 on its own page

Question 9

+3 marksNumerical answer

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 32

Question 9 on its own page

Question 10

+3 marksNumerical answer

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 2

Question 10 on its own page

Question 11

+3 marksNumerical answer

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 116

Question 11 on its own page

Question 12

+3 marksNumerical answer

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 0.97 (accepted within ±0.01)

Question 12 on its own page

Question 13

+3 marksNumerical answer

Based on the above data, answer the given subquestions.

Show answer

Correct answer: -3.5

Question 13 on its own page

Question 14

+2 marksOne correct option

Based on the above data, answer the given subquestions.

Which of the routes illustrated on the grid is taken when the Recursively Optimal policy is executed?

  1. A

    Route 1

  2. B

    Route 2

  3. C

    Route 3

Show answer

Correct answer

  • A

    Route 1

Question 14 on its own page

Question 15

+2 marksOne correct option

Based on the above data, answer the given subquestions.

Which of the routes illustrated on the grid is taken when the Flat Optimal policy is executed?

  1. A

    Route 1

  2. B

    Route 2

  3. C

    Route 3

Show answer

Correct answer

  • C

    Route 3

Question 15 on its own page

Question 16

+3 marksNumerical answer

Based on the above data, answer the given subquestions.

Calculate the number of time-steps taken to finish the episode when the hierarchically optimal policy is executed.

Show answer

Correct answer: 10

Question 16 on its own page