MLT End Term: 24 December 2023, Set FDB1 (September 2023 term)

Common Data:

A team was given a dataset $X \in \mathbb{R}^{d \times n}$ where $d$ denotes the number of features and $n$ denotes the number of samples. They found that there are 10 samples in the dataset and each sample contains 100 features. Assume that the datapoints $x_4$ to $x_{10}$ are all linear combination of linearly independent data points $(x_1, x_2, x_3)$.

Based on the above data, answer the given subquestions.

Suppose the team applies linear PCA on the dataset and reconstructs the data points with zero error using k principal components (directions). For which value of k the reconstruction error would become zero?

Common Data:

A team was given a dataset $X \in \mathbb{R}^{d \times n}$ where $d$ denotes the number of features and $n$ denotes the number of samples. They found that there are 10 samples in the dataset and each sample contains 100 features. Assume that the datapoints $x_4$ to $x_{10}$ are all linear combination of linearly independent data points $(x_1, x_2, x_3)$.

Based on the above data, answer the given subquestions.

(Enter your answer correctly to two decimal places)

We wish to fit a GMM with $K = 2$ for a dataset having 4 points. At the beginning of the $t^{th}$ time step of the EM algorithm, we have $\theta^{(t)}$ as follows:

$\begin{gathered} \pi_1 = 0.3, \pi_2 = 0.7 \\ \mu_1 = 2, \sigma_1^2 = 1 \\ \mu_2 = 3, \sigma_2^2 = 1 \end{gathered}$

The density of the points given a particular mixture is given to you for all four points. $f$ is the density of a Gaussian.

What is the value of $\lambda_k^i$ for $i = 1$ and $k = 2$ after the E-step? Enter your answer correct to two decimal places

Consider a training set $\{X, y\}$, where $X \in \mathbb{R}^{d \times n}$ and the target $y \in \mathbb{R}^n$. Suppose a team decided to use linear regression model, $h = w^T X$ where $w \in \mathbb{R}^{d \times 1}$ that minimizes the objective function $L(w)$ given below

$L(w) = \sum_{i=1}^{n} (w^T x_i - y_i)^2$

Suppose we obtain $w = w^*$ such that $L(w^*) = \sum_{i=1}^{n} ({w^*}^T x_i - y_i)^2 = 0$. Choose the following correct statements

Consider a Ridge Regression scenario where the feature vector $\mathbf{x}$ is given by $\mathbf{x} = [x_1, x_2, x_3, x_4]$, and the weight vector is determined using the formula:

$\mathbf{w}_\lambda = \left(\mathbf{X}^T\mathbf{X} + \lambda\mathbf{I}\right)^{-1}\mathbf{X}^T\mathbf{y}$

Here, $\mathbf{X}$ is the design matrix, $\mathbf{y}$ is the target vector, and $\lambda$ is the regularization parameter.

Suppose we obtain the weight vector $\mathbf{w}_{\lambda=4} = [2, 5, 7, 9]$ when $\lambda$ is set to $4$. If we increase $\lambda$ to $8$, which of the following is most likely to be the updated weight vector?

Consider the following 2D dataset with points in $\mathbb{R}^2$:

Suppose we run the perceptron learning algorithm by initializing the weight vector to zero. Does the algorithm converge (with zero error) in a finite number of iterations?

Consider a logistic regression model that has been trained for a binary classification problem on a dataset in $\mathbb{R}^2$. The weight vector is $\begin{bmatrix} 1/2 \\ 2/3 \end{bmatrix}$. Given a test data-point as input to the model, it returns $1$ as the predicted label if the probability output by the model is greater than $0.60$ and $0$ otherwise. What is the predicted label for the test data-point $\begin{bmatrix} 1 \\ 0 \end{bmatrix}$?

Note that the probability output by a logistic regression model is $P(y = 1 \mid \mathbf{x})$

Based on the above data, answer the given subquestions.

What is the equation of the decision boundary? Select all options that are correct.

Based on the above data, answer the given subquestions.

Consider a soft-margin Support Vector Machine (SVM) for a binary classification problem with a dataset in a two-dimensional space. The optimization problem for the soft-margin SVM is formulated as:

$\text{Minimize } \frac{1}{2}\|\mathbf{w}\|^2 + C\sum_{i=1}^{N} \xi_i$

subject to the constraints:

$y_i(\mathbf{w} \cdot \mathbf{x}_i + b) \ge 1 - \xi_i \text{ and } \xi_i \ge 0 \text{ for all } i$

Where $C$ is a positive constant.
Which of the following statements about the soft-margin SVM is correct?

Which of the following statements are correct?

We have trained four models in the same dataset with different hyperparameters. In the following table, we have recorded the training and testing errors for each of the models.

Based on the above information, which of the following statement(s) is/are correct?