Reinforcement Learning End Term: 13 April 2025, Set 1-3 (January 2025 term)

1. A

   When the environment has a high degree of stochasticity, makingbootstrapping unstable in value-based methods

2. B

   When function approximation is necessary due to large state spaces, requiringdeep networks for learning representations

3. C

   When learning needs to prioritize separating state value and advantagefunctions for better decision-making

4. D

   When experience replay is essential for stable learning and sample efficiency

1. A

   SARSA updates its action-value function based on the action actually taken inthe next state, while Q-learning updates its action-value function based on the maximum action-value in the next state.

2. B

   SARSA is guaranteed to converge to the optimal policy under certainconditions, while Q-learning may diverge or oscillate without additional modifications.

3. C

   SARSA is more computationally efficient than Q-learning, requiring fewerupdates to converge to the optimal policy.

4. D

   SARSA and Q-learning exhibit similar performance in terms of convergencespeed and solution quality in this scenario.

Select all that apply.

1. A

   If there are n arms, the lower bound of finding the optimal arm is ω(n).

2. B

   If an agent doesn’t sufficiently pull each arm, then it can incorrectly pick asuboptimal arm as the optimal arm.

3. C

   Exploration is very important step and an agent should keep exploringregularly, to minimize the regret.

4. D

   The optimal arm can be determined by pulling each arm once and then itshould be pulled every time afterwards.