MLT End Term: 24 December 2023, Set ADD3 (September 2023 term)

Based on the above data, answer the given subquestions.

Suppose the team applies linear PCA on the dataset and reconstructs the data points with zero error using k principal components (directions). For which value of k the reconstruction error would become zero?

Based on the above data, answer the given subquestions.

Consider Lloyd’s algorithm used for k-means clustering and choose the correct statements:

We wish to fit a GMM with $K = 2$ for a dataset having 4 points. At the beginning of the $t^{th}$ time step of the EM algorithm, we have $\theta^{(t)}$ as follows:

$\begin{gathered} \pi_1 = 0.4, \pi_2 = 0.6 \\ \mu_1 = 2, \sigma_1^2 = 1 \\ \mu_2 = 3, \sigma_2^2 = 1 \end{gathered}$

The density of the points given a particular mixture is given to you for all four points. $f$ is the density of a Gaussian.

What is the value of $\lambda_k^i$ for $i = 1$ and $k = 2$ after the E-step? Enter your answer correct to two decimal places

Consider a linearly independent set of data points

$X = \begin{bmatrix} 1 & 1 & -1 \\ 1 & 0 & 1 \\ -1 & 1 & 1 \end{bmatrix}$

and the corresponding label $y = \begin{bmatrix} 0.5 \\ 0 \\ -0.5 \end{bmatrix}$. Suppose we fit the data points using a simple linear regression model that minimizes squared error loss $L(w) = \sum (w^T x_i - y_i)^2$. Compute the value of loss at $w = w^*$, where $w^* = (XX^T)^{-1}Xy$

Consider the following dataset with 3 features and 5 data points:

$(d_1, d_2, d_3 \in \mathbb{R})$

You decide to train a linear regression model on this dataset. After training, you obtain the following weight vector

$\mathbf{w} = \begin{bmatrix} 1.2 \\ 0.5 \\ 2.5 \end{bmatrix}$

Now, you decide to introduce L2 regularization to your model. You train the model again with a regularization parameter $(\lambda)$ set to 0.5. The new regularized weight vector is:

$\mathbf{w}_{\text{regularized}} = \begin{bmatrix} 0.9 \\ 0.4 \\ 1.5 \end{bmatrix}$

Based on the above data, answer the given subquestions.

Consider the following dataset with 3 features and 5 data points:

$(d_1, d_2, d_3 \in \mathbb{R})$

You decide to train a linear regression model on this dataset. After training, you obtain the following weight vector

$\mathbf{w} = \begin{bmatrix} 1.2 \\ 0.5 \\ 2.5 \end{bmatrix}$

Now, you decide to introduce L2 regularization to your model. You train the model again with a regularization parameter $(\lambda)$ set to 0.5. The new regularized weight vector is:

$\mathbf{w}_{\text{regularized}} = \begin{bmatrix} 0.9 \\ 0.4 \\ 1.5 \end{bmatrix}$

Based on the above data, answer the given subquestions.

If you further reduce the regularization parameter (λ) to zero. What would be the new prediction for the target variable?

Consider a logistic regression model that has been trained for a binary classification problem on a dataset in $\mathbb{R}^2$. The final weight vector learned by the model is $\mathbf{w} = \begin{bmatrix} 3/2 \\ 5/12 \end{bmatrix}$. Given a test data point as input to the model, it returns $1$ as the predicted label if the probability output by the model is greater than $0.75$ and $0$ otherwise.

What is the predicted label for the test data point $\mathbf{x} = \begin{bmatrix} 0 \\ 2 \end{bmatrix}$? Note that the probability output by a logistic regression model is $P(y = 1 \mid \mathbf{x})$.

Based on the above data, answer the given subquestions.

What is the equation of the decision boundary? Select all options that are correct.

Based on the above data, answer the given subquestions.

Consider a soft-margin Support Vector Machine (SVM) for a binary classification problem with a dataset in a two-dimensional space. The optimization problem for the soft-margin SVM is formulated as:

$\text{Minimize } \frac{1}{2}\|\mathbf{w}\|^2 + C\sum_{i=1}^{N} \xi_i$

subject to the constraints:

$y_i(\mathbf{w} \cdot \mathbf{x}_i + b) \ge 1 - \xi_i \text{ and } \xi_i \ge 0 \text{ for all } i$

Where $C$ is a positive constant.
Which of the following statements about the soft-margin SVM is correct?

Which of the following statements are correct?

Suppose we have trained four different models using the same training set from the dataset $D$ and recorded the training error. The testing error for each model was also recorded using a separate test set. The recorded values are summarized in the table below

Based on the above information, which of the following statement(s) is/are correct?

Choose all the correct statements about neural networks