Opening the paper…
Deep Learning, Quiz 2
What are the possible outcomes if we don’t do bias correction in ADAM?
What are the possible outcomes if we don’t do bias correction in ADAM? Consider the assertion and the corresponding reason,\ Assertion (A): The RMSProp Algorithm starts oscillating around the minima.\ Reason (R): RMSProp is sensitive to initial learning rate value. Suppose you have a deep neural network trained on a dataset with a small amount of labeled examples. After applying L2 regularization during training with a fixed value of regularization parameter, the following observations are made:\ • The empirical error decreases significantly.\ • The true error remains high.\ • The variance of the error on different subsets of the training data is relatively low.\ • When tested on a diverse set of unseen data, the model’s performance is inconsistent. Which of the following statements provides a plausible explanation for this scenario?