Question 6: Which of the following statements are TRUE about training…

Select all that apply.

1. A

   Stochastic gradient descent (batch size ) computes the true gradient of the total loss function at each step.

2. B

   In mini-batch gradient descent with batch size on a dataset of points, one epoch consists of parameter updates.

3. C

   Momentum-based gradient descent can overshoot a narrow valley because accumulated velocity carries the parameters past the minimum.

4. D

   In vanilla (batch) gradient descent, each step is guaranteed to decrease the loss for a sufficiently small learning rate on a smooth loss function.