MLT End Term: 22 December 2024, Set QDF1 (September 2024 term)

NOTE: Enter the answer correct to one decimal place

Consider a dataset of 200 points where the $i$-th data point is given by:

$\mathbf{x}_i = a_i \cdot \begin{bmatrix} 3 \\ 1 \\ 0 \\ 2 \end{bmatrix} + b_i \cdot \begin{bmatrix} 1 \\ 0 \\ 3 \\ 0 \end{bmatrix},$

where $a_i$ and $b_i$ are real numbers such that

$\sum_{i=1}^{200} a_i = \sum_{i=1}^{200} b_i = 0.$

Standard PCA is performed on this dataset. If the top two principal components are retained and used to reconstruct the dataset, what is the reconstruction error?

A train running between two stations A and B will be late on any day by a random amount $X$, where $X \sim \text{Uniform}[0, \theta]$. Suppose the train is late by random amounts (in minutes)

$10, 55, 17, 20, 49, 51, 15, 4, 32, 35$

independently on 10 days. Consider a $\text{Uniform}[0, 60]$ prior for the parameter $\theta$. Find the maximum aposteriori (MAP) estimate of $\theta$.

Recall that $\hat{\theta}_{MAP} = \underset{\theta}{\arg\max}\ f(\theta \mid \{X_1, \ldots, X_n\})$, where $f$ is the posterior distribution.

Consider a logistic regression model trained to detect spam emails. Emails containing harmful phishing links are labeled as spam (1), while regular emails are labeled as not spam (0). A good spam detector should correctly identify almost all emails with phishing links. Even a single phishing email incorrectly classified as not spam could expose users to significant risks. However, the detector may classify some regular emails as spam. This trade-off is acceptable to ensure safety.
To ensure user safety, if the threshold is set to a low value to prioritize the detection of phishing emails, type 1. If the threshold is set to a high value to minimize false positives (misclassifying regular emails as spam), type 0.

Given a training set $A_n = \{(x_1, y_1), \ldots, (x_n, y_n)\}$, let $A_n^{-i}$ denote the set obtained by removing the label of the $i$-th training example. Leave one out cross validation (LOOCV) is performed as follows: for each $(x_i, y_i)$ in the training set, we train the model on the remaining $n - 1$ points of $A_n^{-i}$ and test the prediction on $(x_i, y_i)$. The error is defined as

$error_L(A_n) = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{f}^{-i}(x_i))^2,$

where $\hat{f}^{-i}(x_i)$ denote the predicted value of $y_i$ using the model trained on $A_n^{-i}$. Now, for the dataset:

$D = \{(1, 2), (2, 2), (0, 1)\},$

fit the model $\hat{f} = \beta x_i$ and find $error_L(A_3)$.

Consider the following optimization problem involving the variable $\mathbf{x} \in \mathbb{R}^2$:

$\min_{\mathbf{x}}\ x_1^2 + 2x_2 \quad \text{subject to} \quad x_1 - x_2 \geq 3.$

What is the output of the following expression for $\mathbf{x}' = \begin{bmatrix} 1 \\ 4 \end{bmatrix}$? Here, $L(\mathbf{x}', \lambda)$ is the Lagrangian function for this optimization problem evaluated at $\mathbf{x}'$.

$\max_{\lambda \geq 0} L(\mathbf{x}', \lambda)$

Perform one round of the gradient descent algorithm for the linear regression on the below data

$X = \begin{bmatrix} 1 & -2 & 0 \\ 2 & 5 & 1 \end{bmatrix}, y = \begin{bmatrix} 1 \\ 6 \\ 1 \end{bmatrix}$

with the initial weight vector $w_0 = \begin{bmatrix} 1 \\ 0.5 \end{bmatrix}$ and learning rate $\eta = 0.05$. Find the updated weight vector $w_1$ after one iteration of gradient descent.
Note: The objective function is given by $f(w) = ||X^T w - y||^2$.

While training a perceptron model, the weight vector at some iteration $t$ is $\mathbf{w}^t$. The next two data points picked up by the perceptron algorithm in the course of its execution are $(\mathbf{x}_1, y_1)$ and $(\mathbf{x}_2, y_2)$, where $y$ is the true label:

$\mathbf{w}^t = \begin{bmatrix} 0 \\ 1 \\ 2 \end{bmatrix}, \quad \mathbf{x}_1 = \begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}, y_1 = 1, \quad \mathbf{x}_2 = \begin{bmatrix} -1 \\ 1 \\ 1 \end{bmatrix}, y_2 = -1.$

What are the values of $\mathbf{w}^{t+1}$ and $\mathbf{w}^{t+2}$?

If we remove all the non-support vectors from the dataset, what will be the impact on the model using SVM algorithm?

In each round of AdaBoost, the weight for a particular training observation is decreased from round t to round t + 1 if the observation was:

Is the following statement true or false?
For a fixed size of the training and the test set, increasing the complexity of the model always leads to an increment in the test error.

Consider a supervised ML problem where a model is trained using a training dataset, and its performance is evaluated on both the training and validation datasets. During training, the model’s loss is plotted over time for both the datasets. Assume that the loss L(t) is decreasing as the model learns.
The following graph shows two curves, Curve (A) and Curve (B), representing the loss for the training and validation datasets, respectively:

Which of the following options are correct? Select all that apply.

Consider the following dataset for a binary classification problem in which the features are from $\{0,1\}^3$ and the labels are from $\{0,1\}$.

$\mathbf{X} = \begin{bmatrix} 1 & 0 & 1 & 1 & 0 & 1 & 0 & 0 \\ 0 & 1 & 0 & 1 & 1 & 0 & 1 & 1 \\ 1 & 1 & 0 & 1 & 1 & 0 & 0 & 1 \end{bmatrix}$

$\mathbf{y} = \begin{bmatrix} 1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 \end{bmatrix}^T$

A Naive Bayes classifier is trained on this dataset. The parameters to be estimated are represented as $\hat{p}_i^y$, which are presented in the form of the table given below:

Assume that there is no smoothing. Answer the given sub-questions.

Consider the following dataset for a binary classification problem in which the features are from $\{0,1\}^3$ and the labels are from $\{0,1\}$.

$\mathbf{X} = \begin{bmatrix} 1 & 0 & 1 & 1 & 0 & 1 & 0 & 0 \\ 0 & 1 & 0 & 1 & 1 & 0 & 1 & 1 \\ 1 & 1 & 0 & 1 & 1 & 0 & 0 & 1 \end{bmatrix}$

$\mathbf{y} = \begin{bmatrix} 1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 \end{bmatrix}^T$

A Naive Bayes classifier is trained on this dataset. The parameters to be estimated are represented as $\hat{p}_i^y$, which are presented in the form of the table given below:

Assume that there is no smoothing. Answer the given sub-questions.

Consider the following dataset for a binary classification problem in which the features are from $\{0,1\}^3$ and the labels are from $\{0,1\}$.

$\mathbf{X} = \begin{bmatrix} 1 & 0 & 1 & 1 & 0 & 1 & 0 & 0 \\ 0 & 1 & 0 & 1 & 1 & 0 & 1 & 1 \\ 1 & 1 & 0 & 1 & 1 & 0 & 0 & 1 \end{bmatrix}$

$\mathbf{y} = \begin{bmatrix} 1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 \end{bmatrix}^T$

A Naive Bayes classifier is trained on this dataset. The parameters to be estimated are represented as $\hat{p}_i^y$, which are presented in the form of the table given below:

Assume that there is no smoothing. Answer the given sub-questions.

Which of the following is correct ?