Machine Learning Techniques End Term: 13 April 2025, Set QDD1 (January 2025 term)

For a given dataset, a 1-Nearest Neighbor (1-NN) and a 3-Nearest Neighbor (3-NN) classifier are applied. Which classifier is likely to exhibit a higher Leave-One-Out Cross Validation (LOOCV) error? In case of tie-breaker, assign a positive class (+) to the data point.

Consider the following three weight vectors obtained by minimizing the ridge regression objective with penalty parameters $\lambda = 0.1, 10, 50$.

$\theta_1 = \begin{bmatrix} 0.5 & 0.56 & 2.5 \end{bmatrix}^T$
$\theta_2 = \begin{bmatrix} 0.05 & 0.1 & 1.23 \end{bmatrix}^T$
$\theta_3 = \begin{bmatrix} 1.2 & 0.84 & 3.15 \end{bmatrix}^T$

Select the most appropriate match for each weight vector corresponding to penalty parameter $\lambda$ from the following options:

Consider the following dataset with two features and the corresponding labels:

Fit the linear regression model $y = w_1 x_1 + w_2 x_2$ using the normal equation obtained from the squared error loss.

Hint: The normal equation for linear regression is:

$w = (XX^T)^{-1} X y$

Consider a dataset with 4 datapoints:

$\{(\mathbf{x}_1, y_1), (\mathbf{x}_2, y_2), (\mathbf{x}_3, y_3), (\mathbf{x}_4, y_4)\},$

where $y_i \in \{+1, -1\}$ and $\mathbf{x}_i \in \mathbf{R}^2$. In the first iteration of the AdaBoost algorithm, suppose a decision stump $f_1$ is chosen, which correctly classifies the first three data points and incorrectly classifies the last data point. Assume the initial distribution of the dataset assigns equal weights to all data points, i.e., $D_0(i) = \frac{1}{4}$, for $i = 1, 2, 3, 4$. What will be the updated distribution of the weights of the data points after the first iteration?

Which of the following statements correctly differentiates PCA and Kernel PCA?

Consider the following formualtion of the soft margin SVM:

$\min_{w, \epsilon} \frac{1}{2} ||w||^2 + C \sum_{i=1}^{n} \epsilon_i, \qquad C \geq 0$

$\text{subject to } (w^T x_i) y_i + \epsilon_i \geq 1, \ \forall i$

$\epsilon_i \geq 0, \ \forall i.$

Which of the following statements is/are correct?

Which of the following statements are true for bagging?

For a decision tree, each node has exactly two child nodes (balanced tree). If the tree has a depth of 3, how many leaf nodes are there?

Let $X_1, X_2, \ldots, X_n$ be i.i.d. samples from a Uniform distribution on the interval $[0, \theta]$, where $\theta$ is an unknown parameter. The probability density function (PDF) of continuous uniform distribution is given by:

$f(x; \theta) = \begin{cases} \frac{1}{\theta}, & 0 \leq x \leq \theta \\ 0, & \text{otherwise} \end{cases}.$

Find the Maximum Likelihood Estimate (MLE) of $\theta$ based on a given sample $10, 15, 12, 20, 17$.

Based on the above data, answer the given subquestions.

Find the squared length of the updated weight vector after one iteration (one pass through all the data points) of the perceptron algorithm, assuming $w_0 = \begin{bmatrix} 0 & 0 \end{bmatrix}^T$. While looking for mistakes, cycle through the data points form left to right.

Based on the above data, answer the given subquestions.

Will the algorithm converge after this update?

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

Which of the following training data points are certainly not support vectors?

Consider a single-layer neural network with two neurons in the hidden layer. The weight parameters of the network are given as follows:

$w_1^{(1)} = \frac{1}{2},\ w_2^{(1)} = \frac{1}{2}$

$w_1^{(2)} = \frac{1}{2},\ w_2^{(2)} = -\frac{1}{2}$

$w^{out} = \begin{bmatrix} 1 & -1 \end{bmatrix}^T,$

where $w_i^{(j)}$ represents the weight associated with the $j$-th neuron for the $i$-th input feature. Assume that we are solving a binary classification problem.

Based on the above data, answer the given subquestions.

The output layer of the neural network will return a probability $p$ for the input $x_{test} = \begin{bmatrix} 2 & 4 \end{bmatrix}^T$. The sigmoid function is used as the activation function in both the hidden and the output layer of the neural network. Find the value of $p$. Enter the answer correct to two decimal places.

Consider a single-layer neural network with two neurons in the hidden layer. The weight parameters of the network are given as follows:

$w_1^{(1)} = \frac{1}{2},\ w_2^{(1)} = \frac{1}{2}$

$w_1^{(2)} = \frac{1}{2},\ w_2^{(2)} = -\frac{1}{2}$

$w^{out} = \begin{bmatrix} 1 & -1 \end{bmatrix}^T,$

where $w_i^{(j)}$ represents the weight associated with the $j$-th neuron for the $i$-th input feature. Assume that we are solving a binary classification problem.

Based on the above data, answer the given subquestions.