Question 3: A dataset is given by X = \begin{bmatrix} 1 & 2 & 0 & 5 &…

A dataset is given by

$X = \begin{bmatrix} 1 & 2 & 0 & 5 & 0 \\ 1 & 0 & 3 & 0 & 0 \\ 1 & 1 & 0 & 1 & 1 \\ 1 & -1 & 0 & 0 & 0 \end{bmatrix}, y = \begin{bmatrix} 10 \\ 6 \\ 5 \\ 2 \end{bmatrix}$

The rows of $X$ represent samples and the columns represent features, with the first column corresponds the bias term. We use a linear regression neuron where the prediction $\hat{y}_i$ for a sample $x_i$ is given by the linear combination $\hat{y}_i = z_i = \sum_{j=0}^{4} w_j x_{ij}$.

The weights are updated using Stochastic Gradient Descent (SGD) for one epoch (i.e., once for each of the 4 samples). The loss function is the Mean Squared Error, $L = (\hat{y} - y)^2$. If all weights are initialized to $w_j = 0.5$, which of the following weights is updated the fewest number of times?