Question 3
The Huber Loss function is introduced as a robust alternative to Mean Squared Error (MSE) and Mean Absolute Error (MAE). What is the primary characteristic of Huber Loss that makes it robust to outliers?
It always produces a gradient of zero for all errors, preventing large updates.
It exclusively uses a quadratic penalty, but only for errors exactly at the threshold.
It transitions from a quadratic loss for small errors to a linear loss for large errors, thereby down-weighting the impact of outliers.
It entirely ignores errors beyond a certain delta, considering them irrelevant.
It applies a logarithmic transformation to all error values before computing the loss.