The IIT Madras BS Reinforcement Learning (Reinforcement Learning) Quiz 1 paper sat on 29 Oct 2023, in the September 2023 term: 17 questions for 40 marks in 120 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.
Select correct statements about UCB1 algorithm:
Each arm is selected atleast twice before any arm is selected third time.
Each arm is selected atleast once before any arm is selected second time.
Constant c controls degree of exploration, higher its value, higher the exploration.
Constant c controls degree of exploration, higher its value, lower the exploration.
Correct answers
Each arm is selected atleast once before any arm is selected second time.
Constant c controls degree of exploration, higher its value, higher the exploration.
For the value iteration algorithm, which of the following statements are correct?
Correct answers
Choose the correct statement(s) regarding explore-exploit dilemma:
Always exploring is not optimal.
Always exploiting is optimal.
None of these.
Correct answer
Always exploring is not optimal.
Which of the following is the correct Bellman equation for deterministic transitions? The symbols have the usual meaning.
Correct answer
Consider the following statements and select the correct option.
Assertion: Monte Carlo value function approximation methods do not need knowledge of the model to be implemented.
Reason: Monte Carlo value function approximation methods require only a way to sample trajectories from the environment and aggregate the results.
Assertion and Reason are both true and Reason is a correct explanation of Assertion.
Assertion and Reason are both true and Reason is not a correct explanation of Assertion.
Assertion is true but Reason is false.
Assertion is false but Reason is true.
Correct answer
Assertion and Reason are both true and Reason is a correct explanation of Assertion.
Based on the above data, answer the given subquestions.
What is the probability of choosing arm A2 at timestamp t = 6?
Correct answer: 0.09 (accepted within ±0.01)
Based on the above data, answer the given subquestions.
At time stamp t = 6, suppose the arm with least estimate so far is pulled and the reward is ln(4). Which of the following is correct after timestamp t = 6?
Arm A1 is optimal arm.
Arm A2 is optimal arm.
Arm A3 is optimal arm.
An optimal arm can not be determined.
There is a tie for optimal arm.
Correct answer
Arm A3 is optimal arm.
Correct answer: 7
Correct answer: 1
Based on the above data, answer the given subquestions.
Consider the following deterministic policies in table (1) :
Which of the following are optimal policies?
π1
π2
π3
π4
Can not be determined.
Correct answers
π1
π2
π3
Based on the above data, answer the given subquestions.
Refer to table (1) and choose the correct statements from the following:
π1 = π2
π2 < π3
π1 ≤ π4
π3 ≥ π4
None of these
Correct answers
π1 = π2
π3 ≥ π4
Based on the above data, answer the given subquestions.
Correct answer: -3
Based on the above data, answer the given subquestions.
Correct answer: -3
Based on the above data, answer the given subquestions.
1 and 2.
1 and 3.
2 and 3.
2 and 5.
2 and 4.
4 and 5.
Correct answers
1 and 3.
2 and 5.
Based on the above data, answer the given subquestions.
What will be the value of v(hot) after one round of value iteration? Assuming v(hot) and v(cold) are initialized with 0. Note the value function is updated synchronously.
Correct answer: 6
Based on the above data, answer the given subquestions.
What will be the value of v(cold) after one round of value iteration? Assuming v(hot) and v(cold) are initialized with 0. Note the value function is updated synchronously.
Correct answer: -3
Based on the above data, answer the given subquestions.
What will be the value of v(hot) + v(cold) after two rounds of value iteration? Assuming v(hot) and v(cold) are initialized with 0. Note the value function is updated synchronously.
Correct answer: 10.8 (accepted within ±0.05)
