Question 1

Which of the following is/are correct and valid reasons to consider sampling actions from a softmax distribution instead of using an greedy approach? 1.Under softmax exploration, the probability of selecting an action increases with its estimated action value, which reduces unnecessary exploration of clearly inferior actions. 2.Unlike the -greedy method, softmax exploration does not require careful, gradual decay of the exploration parameter and still yields asymptotically correct behaviour even if the temperature is reduced sharply. 3.It enables more fine-grained discrimination among actions whose estimated Q-values are close to the maximum, allowing more nuanced preference for slightly better actions. Which of the above statements is/are correct? Consider a discounted return: in an infinite-horizon MDP with bounded rewards and discount factor 1.For fixed rewards, the contribution of to decays geometrically as . 2.If and rewards are uniformly bounded, the infinite sum is always finite. 3.For , the discounted return is bounded above in magnitude by . Here, Rmax be the maximum reward for any transition. Which of the above statements is/are correct? Consider a discounted return: Consider a discounted return: Consider the following assertion and reason pair and select the correct option: Assertion:In the UCB algorithm for multi-armed bandits, replacing the upper confidence bound with a lower confidence bound (and greedily selecting actions based on that lower bound) would still promote effective exploration or lead to optimal reward maximisation. Reason:Even when lower confidence bounds are used instead of upper bounds, if the algorithm greedily selects arms based on these lower bounds, it would still encourage exploration and ultimately achieve optimal reward maximisation.