Question 3
Suppose you have a deep neural network trained on a dataset with a small amount of labeled examples. After applying L2 regularization during training with a fixed value of regularization parameter, the following observations are made:
• The empirical error decreases significantly.
• The true error remains high.
• The variance of the error on different subsets of the training data is relatively low.
• When tested on a diverse set of unseen data, the model’s performance is inconsistent. Which of the following statements provides a plausible explanation for this scenario?
The model is underfitting due to excessive regularization.
L2 regularization effectively reduces both overfitting and underfitting.
The inconsistency in performance on unseen data is expected due to the limited labeled examples in the training set.
The model is still overfitting to the training data, and the regularization parameter needs adjustment.