MLT End Term: 22 December 2024, Set QDF3 (September 2024 term)

NOTE: Enter the answer correct to one decimal place

Consider a dataset of 200 points where the $i$-th data point is given by:

$\mathbf{x}_i = a_i \cdot \begin{bmatrix} 3 \\ 1 \\ 0 \\ 2 \end{bmatrix} + b_i \cdot \begin{bmatrix} 1 \\ 0 \\ 3 \\ 0 \end{bmatrix},$

where $a_i$ and $b_i$ are real numbers such that

$\sum_{i=1}^{200} a_i = \sum_{i=1}^{200} b_i = 0.$

Standard PCA is performed on this dataset. If the top two principal components are retained and used to reconstruct the dataset, what is the reconstruction error?

A train running between two stations A and B will be late on any day by a random amount $X$, where $X \sim \text{Uniform}[0, \theta]$. Suppose the train is late by random amounts (in minutes)

$10, 45, 17, 20, 49, 52, 15, 4, 32, 35$

independently on 10 days. Consider a $\text{Uniform}[0, 60]$ prior for the parameter $\theta$. Find the maximum aposteriori (MAP) estimate of $\theta$.

Recall that $\hat{\theta}_{MAP} = \underset{\theta}{\arg\max}\ f(\theta \mid \{X_1, \ldots, X_n\})$, where $f$ is the posterior distribution.

Consider a logistic regression model trained to detect spam emails. Emails containing harmful phishing links are labeled as spam (1), while regular emails are labeled as not spam (0). A good spam detector should correctly identify almost all emails with phishing links. Even a single phishing email incorrectly classified as not spam could expose users to significant risks. However, the detector may classify some regular emails as spam. This trade-off is acceptable to ensure safety.
To ensure user safety, if the threshold is set to a low value to prioritize the detection of phishing emails, type 1. If the threshold is set to a high value to minimize false positives (misclassifying regular emails as spam), type 0.

Select the correct statements from the following for k-means algorithm:

Perform one round of the gradient descent algorithm for the linear regression on the below data

$X = \begin{bmatrix} 1 & -2 & 0 \\ 2 & 5 & 1 \end{bmatrix}, y = \begin{bmatrix} 1 \\ 6 \\ 1 \end{bmatrix}$

with the initial weight vector $w_0 = \begin{bmatrix} 1 \\ 0.5 \end{bmatrix}$ and learning rate $\eta = 0.01$. Find the updated weight vector $w_1$ after one iteration of gradient descent.
Note: The objective function is given by $f(w) = ||X^T w - y||^2$.

While training a perceptron model, the weight vector at some iteration $t$ is $\mathbf{w}^t$. The next two data points picked up by the perceptron algorithm in the course of its execution are $(\mathbf{x}_1, y_1)$ and $(\mathbf{x}_2, y_2)$, where $y$ is the true label:

$\mathbf{w}^t = \begin{bmatrix} 1 \\ 2 \\ -1 \end{bmatrix}, \quad \mathbf{x}_1 = \begin{bmatrix} 0 \\ -1 \\ 0 \end{bmatrix}, y_1 = -1, \quad \mathbf{x}_2 = \begin{bmatrix} -1 \\ 0 \\ 2 \end{bmatrix}, y_2 = 1.$

What are the values of $\mathbf{w}^{t+1}$ and $\mathbf{w}^{t+2}$?

Consider the following dataset for a binary classification problem in which the features are from $\{0,1\}^3$ and the labels are from $\{0,1\}$.

$\mathbf{X} = \begin{bmatrix} 1 & 0 & 1 & 1 & 0 & 1 & 0 & 0 \\ 0 & 1 & 0 & 1 & 1 & 0 & 1 & 1 \\ 1 & 1 & 0 & 1 & 1 & 0 & 0 & 1 \end{bmatrix}$

$\mathbf{y} = \begin{bmatrix} 1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 \end{bmatrix}^T$

A Naive Bayes classifier is trained on this dataset. The parameters to be estimated are represented as $\hat{p}_i^y$, which are presented in the form of the table given below:

Assume that there is no smoothing. Answer the given sub-questions.

Consider the following dataset for a binary classification problem in which the features are from $\{0,1\}^3$ and the labels are from $\{0,1\}$.

$\mathbf{X} = \begin{bmatrix} 1 & 0 & 1 & 1 & 0 & 1 & 0 & 0 \\ 0 & 1 & 0 & 1 & 1 & 0 & 1 & 1 \\ 1 & 1 & 0 & 1 & 1 & 0 & 0 & 1 \end{bmatrix}$

$\mathbf{y} = \begin{bmatrix} 1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 \end{bmatrix}^T$

A Naive Bayes classifier is trained on this dataset. The parameters to be estimated are represented as $\hat{p}_i^y$, which are presented in the form of the table given below:

Assume that there is no smoothing. Answer the given sub-questions.

Consider the following dataset for a binary classification problem in which the features are from $\{0,1\}^3$ and the labels are from $\{0,1\}$.

$\mathbf{X} = \begin{bmatrix} 1 & 0 & 1 & 1 & 0 & 1 & 0 & 0 \\ 0 & 1 & 0 & 1 & 1 & 0 & 1 & 1 \\ 1 & 1 & 0 & 1 & 1 & 0 & 0 & 1 \end{bmatrix}$

$\mathbf{y} = \begin{bmatrix} 1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 \end{bmatrix}^T$

A Naive Bayes classifier is trained on this dataset. The parameters to be estimated are represented as $\hat{p}_i^y$, which are presented in the form of the table given below:

Assume that there is no smoothing. Answer the given sub-questions.

Given a training set $A_n = \{(x_1, y_1), \ldots, (x_n, y_n)\}$, let $A_n^{-i}$ denote the set obtained by removing the label of the $i$-th training example. Leave one out cross validation (LOOCV) is performed as follows: for each $(x_i, y_i)$ in the training set, we train the model on the remaining $n - 1$ points of $A_n^{-i}$ and test the prediction on $(x_i, y_i)$. The error is defined as

$error_L(A_n) = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{f}^{-i}(x_i))^2,$

where $\hat{f}^{-i}(x_i)$ denote the predicted value of $y_i$ using the model trained on $A_n^{-i}$. Now, for the dataset:

$D = \{(1, 2), (2, 3), (3, 4)\},$

fit the model $\hat{f} = \beta x_i$ and find $error_L(A_3)$. Enter the answer correct to two decimal places.

Consider the following optimization problem involving the variable $\mathbf{x} \in \mathbb{R}^2$:

$\min_{\mathbf{x}}\ x_1^2 + 2x_2 \quad \text{subject to} \quad x_1 - x_2 \leq 3.$

What is the output of the following expression for $\mathbf{x}' = \begin{bmatrix} 1 \\ 4 \end{bmatrix}$? Here, $L(\mathbf{x}', \lambda)$ is the Lagrangian function for this optimization problem evaluated at $\mathbf{x}'$.

$\max_{\lambda \geq 0} L(\mathbf{x}', \lambda)$

Which of the following options are not correct? Select all that apply.

If we remove all the non-support vectors from the dataset, what will be the impact on the model using SVM algorithm?

In each round of AdaBoost, the weight for a particular training observation is decreased from round t to round t + 1 if the observation was:

Is the following statement true or false?
For a fixed size of the training and the test set, increasing the complexity of the model always leads to an increment in the test error.

Which of the following is correct ?