Question 1

Below is the constrained version of the ridge regression optimization problem: $$\min_{w \in \mathbb{R}} \sum_{i=1}^{n} (w^T x_i - y_i)^2$$ $$\text{subject to } ||w||^2 \leq \theta.$$ Following are the weight vectors to be considered, along with the mean squared error (MSE) produced by each: $w_1 = \begin{bmatrix} 1 & 1 & 1 & 1 \end{bmatrix}^T$, $MSE = 2$\ $w_2 = \begin{bmatrix} 0 & 2 & 1 & 3 \end{bmatrix}^T$, $MSE = 7$\ $w_3 = \begin{bmatrix} 1 & 2 & 0 & 1 \end{bmatrix}^T$, $MSE = 1$\ $w_4 = \begin{bmatrix} 2 & 1 & 1 & 2 \end{bmatrix}^T$, $MSE = 8$ If $\theta = 10$, which of the following weight vectors will be selected as the final weight vector by ridge regression? Figure from the original question paper *p* is the proportion of points with label 1 in some node in a decision tree. Which of the following statements are true?