Machine Learning Techniques Quiz 2: 16 March 2025 (January 2025 term)

Below is the constrained version of the ridge regression optimization problem:

$\min_{w \in \mathbb{R}} \sum_{i=1}^{n} (w^T x_i - y_i)^2$

$\text{subject to } ||w||^2 \leq \theta.$

Following are the weight vectors to be considered, along with the mean squared error (MSE) produced by each:

$w_1 = \begin{bmatrix} 1 & 1 & 1 & 1 \end{bmatrix}^T$, $MSE = 2$
$w_2 = \begin{bmatrix} 0 & 2 & 1 & 3 \end{bmatrix}^T$, $MSE = 7$
$w_3 = \begin{bmatrix} 1 & 2 & 0 & 1 \end{bmatrix}^T$, $MSE = 1$
$w_4 = \begin{bmatrix} 2 & 1 & 1 & 2 \end{bmatrix}^T$, $MSE = 8$

If $\theta = 10$, which of the following weight vectors will be selected as the final weight vector by ridge regression?

p is the proportion of points with label 1 in some node in a decision tree. Which of the following statements are true?