Question 14: Consider the following optimization algorithms used for …

Select all that apply.

1. A

   SGD with Momentum accumulates an exponentially decaying moving average of past gradients to accelerate convergence along consistent directions.

2. B

   Momentum methods adapt individual learning rates for each parameter based on past squared gradients.

3. C

   RMSProp maintains a running average of squared gradients to scale learning rate.

4. D

   Adam combines ideas from Momentum and RMSProp by using both first and second moment estimates of gradients.

5. E

   Adam requires computing second-order derivatives of the loss function.

6. F

   Adaptive methods such as RMSProp and Adam can use different effective learning rates for different parameters.