Quiz Space

May 2023 term · Reinforcement Learning · BSDA5007

Reinforcement Learning End Term: 3 September 2023, Set QPE1-S1 (May 2023 term)

The IIT Madras BS Reinforcement Learning (Reinforcement Learning) End Term paper sat on 3 Sept 2023, in the May 2023 term, set QPE1-S1: 17 questions for 50 marks in 180 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.

Questions
17
Marks
50
Duration
180 min
MCQ
8
MSQ
2
Numerical
7
Take as mock testPractise with answers

Updated

Official paper: IIT M DEGREE ET1 EXAM QPE1 S2 03 Sep · No negative marking.

Question 1

+3 marksOne correct option
  1. A
  2. B
  3. C
  4. D
  5. E
Show answer

Correct answer

  • C
Question 1 on its own page

Question 2

+3 marksOne correct option
  1. A
  2. B
  3. C
  4. D
Show answer

Correct answer

  • A
Question 2 on its own page

Question 3

+3 marksOne correct option

Select the most appropriate statement concerning the behaviour policy in Q-learning.

  1. A
  2. B
  3. C
  4. D
Show answer

Correct answer

  • A
Question 3 on its own page

Question 4

+3 marksOne correct option
  1. A
  2. B
  3. C
  4. D
Show answer

Correct answer

  • B
Question 4 on its own page

Question 5

+3 marksOne correct option

Given a problem with a well defined hierarchy, what ordering would you expect on the total expected reward for a hierarchically optimal policy (H), a recursively optimal policy (R) and a flat optimal policy (F)?

  1. A
  2. B
  3. C
  4. D
Show answer

Correct answer

  • C
Question 5 on its own page

Question 6

+3 marksOne correct option

Which of the following corresponds to an update for the actor in the case of one-step actor-critic method?

  1. A
  2. B
  3. C
  4. D
Show answer

Correct answer

  • A
Question 6 on its own page

Question 7

+4 marksOne or more correct options

Select all that apply.

  1. A
  2. B
  3. C
  4. D
Show answer

Correct answers

  • B
  • C
Question 7 on its own page

Question 8

+3 marksOne or more correct options

Select all true statements.

Select all that apply.

  1. A

    Policy gradient methods use a parameterized policy that can select actions without consulting a value function.

  2. B

    Policy gradient methods must use a value function to learn the policy parameters.

  3. C

    According to the policy gradient theorem, computing the gradient of the performance requires the computation of the gradient of the state distribution μ(s).

  4. D

    In REINFORCE with baseline, the baseline cannot be a function of the actions.

Show answer

Correct answers

  • A

    Policy gradient methods use a parameterized policy that can select actions without consulting a value function.

  • D

    In REINFORCE with baseline, the baseline cannot be a function of the actions.

Question 8 on its own page

Question 9

+3 marksNumerical answer
Show answer

Correct answer: 0.315 (accepted within ±0.015)

Question 9 on its own page

Question 10

+3 marksNumerical answer
Show answer

Correct answer: 3

Question 10 on its own page

Question 11

+3 marksNumerical answer
Show answer

Correct answer: 0.51

Question 11 on its own page

Question 12

+3 marksNumerical answer
Show answer

Correct answer: 0.9

Question 12 on its own page

Question 13

+4 marksOne correct option

Which of the following is the TD error used in the TD(0) algorithm?

  1. A
  2. B
  3. C
  4. D
Show answer

Correct answer

  • B
Question 13 on its own page

Question 14

+2.5 marksNumerical answer

Based on the above data, answer the given subquestions.

Find the TD error for this transition.

Show answer

Correct answer: 10

Question 14 on its own page

Question 15

+1.5 marksNumerical answer

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 1

Question 15 on its own page

Question 16

+3.5 marksNumerical answer

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 0.725 (accepted within ±0.015)

Question 16 on its own page

Question 17

+1.5 marksOne correct option

Based on the above data, answer the given subquestions.

If the eligibility traces are replacing in nature, which state would have highest eligibility trace at the end of a trajectory, and what will be the value of the eligibility trace of the corresponding state?

  1. A
  2. B
  3. C
  4. D
Show answer

Correct answer

  • B
Question 17 on its own page