MLT End Term: 13 April 2025, Set QDD1 (January 2025 term)

For a given dataset, a 1-Nearest Neighbor (1-NN) and a 3-Nearest Neighbor (3-NN) classifier are applied. Which classifier is likely to exhibit a higher Leave-One-Out Cross Validation (LOOCV) error? In case of tie-breaker, assign a positive class (+) to the data point.

Consider the following three weight vectors obtained by minimizing the ridge regression objective with penalty parameters $\lambda = 0.1, 10, 50$.

$\theta_1 = \begin{bmatrix} 0.5 & 0.56 & 2.5 \end{bmatrix}^T$
$\theta_2 = \begin{bmatrix} 0.05 & 0.1 & 1.23 \end{bmatrix}^T$
$\theta_3 = \begin{bmatrix} 1.2 & 0.84 & 3.15 \end{bmatrix}^T$

Select the most appropriate match for each weight vector corresponding to penalty parameter $\lambda$ from the following options:

Consider the following dataset with two features and the corresponding labels:

Fit the linear regression model $y = w_1 x_1 + w_2 x_2$ using the normal equation obtained from the squared error loss.

Hint: The normal equation for linear regression is:

$w = (XX^T)^{-1} X y$