MLT End Term: 11 December 2022, Set ETD1-2 (September 2022 term)

Which of the following estimators are more likely to be preferred for bagging? Select all that apply.

What will be the result of applying ReLU to the following values?:
-3.7, 4.9, -2.0, 5.2, 7.4, -8.3

Suppose you run gradient descent for linear regression for 100 iterations with a learning rate 0.01. You observe that the training loss (sum of squared loss) is increasing after every iteration. What may be the reason? What changes would you make to the set-up for the gradient descent to converge to a solution?

Using a fixed number of training examples, Model 1 will have _____________ bias than Model 2, and Model 2 is more likely to _______________

Consider a neural network with 10 inputs and 2 outputs. If there are 5 hidden layers each with 5 neurons, how many parameters need to be learnt if there is a bias associated with each neuron in the hidden and output layers?

What is the maximum number of mistakes that the Perceptron algorithm can make in this data set?

Consider the following linearly separable training dataset for a binary classification problem in $\mathbb{R}^2$ :

$\mathbf{x_1} = \begin{bmatrix} 2 \\ 0 \end{bmatrix}, y_1 = 1 \qquad \mathbf{x_2} = \begin{bmatrix} 6 \\ 2 \end{bmatrix}, y_2 = 1$

$\mathbf{x_3} = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, y_3 = -1 \qquad \mathbf{x_4} = \begin{bmatrix} -4 \\ -1 \end{bmatrix}, y_4 = -1$

A hard-margin, linear-SVM is trained on this dataset. Among the four options given below, one of them is the optimal weight vector $\mathbf{w}^*$. Identify this vector. Recall that the optimal weight vector is the solution to the primal problem.

Consider the following data set:

Which of the following will have larger leave-one-out cross-validation error?

Based on the above data, answer the given subquestions.

Which of the two models is more likely to fit the training data better?

Based on the above data, answer the given subquestions.

Which model is more likely to give less test error?

Upon performing standard PCA on a centered dataset in $\mathbb{R}^3$, we get the principal components to be:

$\mathbf{w}_1 = \begin{bmatrix} 1 & 0 & 0 \end{bmatrix}^T, \quad \mathbf{w}_2 = \begin{bmatrix} 0 & 1 & 0 \end{bmatrix}^T, \quad \mathbf{w}_3 = \begin{bmatrix} 0 & 0 & 1 \end{bmatrix}^T$

$\mathbf{C}$ is the covariance matrix of the centered dataset. The off-diagonal entries are hidden from your view:

$\mathbf{C} = \begin{bmatrix} 15 & a & b \\ a & 9 & c \\ b & c & 3 \end{bmatrix}$

$\begin{bmatrix} x_1 & x_2 & x_3 \end{bmatrix}^T$ denotes a data-point. Here, $x_1, x_2, x_3$ are the three features.

Note: The word standard indicates that no kernel has been used.

Based on the above data, answer the given subquestions.

What is the variance along the first principal component?

Upon performing standard PCA on a centered dataset in $\mathbb{R}^3$, we get the principal components to be:

$\mathbf{w}_1 = \begin{bmatrix} 1 & 0 & 0 \end{bmatrix}^T, \quad \mathbf{w}_2 = \begin{bmatrix} 0 & 1 & 0 \end{bmatrix}^T, \quad \mathbf{w}_3 = \begin{bmatrix} 0 & 0 & 1 \end{bmatrix}^T$

$\mathbf{C}$ is the covariance matrix of the centered dataset. The off-diagonal entries are hidden from your view:

$\mathbf{C} = \begin{bmatrix} 15 & a & b \\ a & 9 & c \\ b & c & 3 \end{bmatrix}$

$\begin{bmatrix} x_1 & x_2 & x_3 \end{bmatrix}^T$ denotes a data-point. Here, $x_1, x_2, x_3$ are the three features.

Note: The word standard indicates that no kernel has been used.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

How many mistakes does your algorithm make?

Based on the above data, answer the given subquestions.

What is the squared length of the weight vector corresponding to the final linear separator your algorithm outputs?

Consider that we introduce negative marking in this exam. After getting the results, you observe that eight of your friends $\{f_1, \ldots, f_8\}$ have scored the following marks respectively:

{5, 6, -2, -3, 1, 7, -4, -1}

You want to cluster your friends into two groups based on their marks by using the Lloyd's algorithm.

You initialize the algorithm by keeping the first four friends, i.e., $\{f_1, f_2, f_3, f_4\}$ in cluster 1 ($C_1$) and the last four friends, i.e., $\{f_5, f_6, f_7, f_8\}$ in cluster 2 ($C_2$).

How would the clusters look like after executing one step of Lloyd's algorithm?

Based on the above data, answer the given subquestions.

What is the value of a?

Based on the above data, answer the given subquestions.

What is the value of b?

Based on the above data, answer the given subquestions.

What is the value of c?

Based on the above data, answer the given subquestions.

What is the value of the following expression?
(p+r) (1+q+s)

Consider a single iteration of the AdaBoost algorithm that was run on three sample points, starting with uniform weights on the sample points. The labels are either +1 or −1. In the table below, some values have been omitted.

Based on the above data, answer the given subquestions.

Consider a single iteration of the AdaBoost algorithm that was run on three sample points, starting with uniform weights on the sample points. The labels are either +1 or −1. In the table below, some values have been omitted.

Based on the above data, answer the given subquestions.

Consider a single iteration of the AdaBoost algorithm that was run on three sample points, starting with uniform weights on the sample points. The labels are either +1 or −1. In the table below, some values have been omitted.

Based on the above data, answer the given subquestions.

How much training error will be incurred by the first estimator? The training examples consist of given three points. Enter your answer correct to two decimal places.

There are $8$ points in a training dataset in $\mathbb{R}^2$ for a binary classification problem that is linearly separable. Use the following notation: $\mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$ for a data-point and $\mathbf{w}^* = \begin{bmatrix} w_1 \\ w_2 \end{bmatrix}$ for the optimal weight vector of a hard-margin, linear-SVM.

Based on the above data, answer the given subquestions.

There are $8$ points in a training dataset in $\mathbb{R}^2$ for a binary classification problem that is linearly separable. Use the following notation: $\mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$ for a data-point and $\mathbf{w}^* = \begin{bmatrix} w_1 \\ w_2 \end{bmatrix}$ for the optimal weight vector of a hard-margin, linear-SVM.

Based on the above data, answer the given subquestions.