MLT End Term: 11 December 2022, Set ETD1 (September 2022 term)

What will be the result of applying ReLU to the following values?:
-2.7, 3.9, -1.0, 4.2, 6.4, -7.3

Suppose you run gradient descent for linear regression for 500 iterations with a learning rate 0.02. You observe that the training loss (sum of squared loss) is increasing after every iteration. What may be the reason? What changes would you make to the set-up for the gradient descent to converge to a solution?

A set of data points is generated by the following process:

$y_i = w_0 + w_1x_i + w_2x_i^2 + w_3x_i^3 + w_4x_i^4 + w_5x_i^5 + \epsilon$ where $\epsilon$ is a Gaussian noise.

You use two models to fit the data:

Model 1: $\hat{y} = a_0 + a_1x + a_2x^2$

Model 2: $\hat{y} = a_0 + a_1x + a_2x^2 + a_3x^3 + \ldots + a_{10}x^{10}$

Using a fixed number of training examples, Model 2 will have __________ bias than Model 1, and Model 1 is more likely to __________

Consider the following linearly separable training dataset for a binary classification problem in $\mathbb{R}^2$ :

$\mathbf{x_1} = \begin{bmatrix} 0 \\ -1 \end{bmatrix}, y_1 = 1 \qquad \mathbf{x_2} = \begin{bmatrix} -2 \\ 3 \end{bmatrix}, y_2 = 1$

$\mathbf{x_3} = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, y_3 = -1 \qquad \mathbf{x_4} = \begin{bmatrix} 1 \\ -1 \end{bmatrix}, y_4 = -1$

A hard-margin, linear-SVM is trained on this dataset. Among the four options given below, one of them is the optimal weight vector $\mathbf{w}^*$. Identify this vector. Recall that the optimal weight vector is the solution to the primal problem.

Consider the following data set:

Which of the following will have lower leave-one-out cross-validation error?

Consider that we introduce negative marking in this exam. After getting the results, you observe that eight of your friends $\{f_1, \ldots, f_8\}$ have scored the following marks respectively:

{5, 6, -2, -3, 1, 7, -4, -1}

You want to cluster your friends into two groups based on their marks by using the Lloyd's algorithm.

You initialize the algorithm by keeping the first four friends, i.e., $\{f_1, f_2, f_3, f_4\}$ in cluster 1 ($C_1$) and the last four friends, i.e., $\{f_5, f_6, f_7, f_8\}$ in cluster 2 ($C_2$).

How would the clusters look like after executing one step of Lloyd's algorithm?

Which of the following estimators are more likely to be preferred for bagging? Select all that apply.

Consider a neural network with 8 inputs and 2 outputs. If there are 4 hidden layers each with 4 neurons, how many parameters need to be learnt if there is a bias associated with each neuron in the hidden and output layers?

Based on the above data, answer the given subquestions.

Which of the two models is more likely to fit the training data better?

Based on the above data, answer the given subquestions.

Which model is more likely to give less test error?

Upon performing standard PCA on a centered dataset in $\mathbb{R}^3$, we get the principal components to be:

$\mathbf{w}_1 = \begin{bmatrix} 1 & 0 & 0 \end{bmatrix}^T, \quad \mathbf{w}_2 = \begin{bmatrix} 0 & 1 & 0 \end{bmatrix}^T, \quad \mathbf{w}_3 = \begin{bmatrix} 0 & 0 & 1 \end{bmatrix}^T$

$\mathbf{C}$ is the covariance matrix of the centered dataset. The off-diagonal entries are hidden from your view:

$\mathbf{C} = \begin{bmatrix} 12 & a & b \\ a & 6 & c \\ b & c & 3 \end{bmatrix}$

$\begin{bmatrix} x_1 & x_2 & x_3 \end{bmatrix}^T$ denotes a data-point. Here, $x_1, x_2, x_3$ are the three features.

Note: The word standard indicates that no kernel has been used.

Based on the above data, answer the given subquestions.

What is the variance along the first principal component?

Upon performing standard PCA on a centered dataset in $\mathbb{R}^3$, we get the principal components to be:

$\mathbf{w}_1 = \begin{bmatrix} 1 & 0 & 0 \end{bmatrix}^T, \quad \mathbf{w}_2 = \begin{bmatrix} 0 & 1 & 0 \end{bmatrix}^T, \quad \mathbf{w}_3 = \begin{bmatrix} 0 & 0 & 1 \end{bmatrix}^T$

$\mathbf{C}$ is the covariance matrix of the centered dataset. The off-diagonal entries are hidden from your view:

$\mathbf{C} = \begin{bmatrix} 12 & a & b \\ a & 6 & c \\ b & c & 3 \end{bmatrix}$

$\begin{bmatrix} x_1 & x_2 & x_3 \end{bmatrix}^T$ denotes a data-point. Here, $x_1, x_2, x_3$ are the three features.

Note: The word standard indicates that no kernel has been used.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

How many mistakes does your algorithm make?

Based on the above data, answer the given subquestions.

What is the squared length of the weight vector corresponding to the final linear separator your algorithm outputs?

Based on the above data, answer the given subquestions.

What is the value of a?

Based on the above data, answer the given subquestions.

What is the value of b?

Based on the above data, answer the given subquestions.

What is the value of c?

Based on the above data, answer the given subquestions.

Consider a single iteration of the AdaBoost algorithm that was run on three sample points, starting with uniform weights on the sample points. The labels are either +1 or −1. In the table below, some values have been omitted.

Based on the above data, answer the given subquestions.

What will be the true label for point x1 ? Enter 1 or -1 .

Consider a single iteration of the AdaBoost algorithm that was run on three sample points, starting with uniform weights on the sample points. The labels are either +1 or −1. In the table below, some values have been omitted.

Based on the above data, answer the given subquestions.

What will be the updated weight for point x2 ? Enter your answer correct to two decimal places.

Consider a single iteration of the AdaBoost algorithm that was run on three sample points, starting with uniform weights on the sample points. The labels are either +1 or −1. In the table below, some values have been omitted.

Based on the above data, answer the given subquestions.

How much training error will be incurred by the first estimator? The training examples consist of given three points. Enter your answer correct to two decimal places.

There are $8$ points in a training dataset in $\mathbb{R}^2$ for a binary classification problem that is linearly separable. Use the following notation: $\mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$ for a data-point and $\mathbf{w}^* = \begin{bmatrix} w_1 \\ w_2 \end{bmatrix}$ for the optimal weight vector of a hard-margin, linear-SVM.

Based on the above data, answer the given subquestions.

There are $8$ points in a training dataset in $\mathbb{R}^2$ for a binary classification problem that is linearly separable. Use the following notation: $\mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$ for a data-point and $\mathbf{w}^* = \begin{bmatrix} w_1 \\ w_2 \end{bmatrix}$ for the optimal weight vector of a hard-margin, linear-SVM.

Based on the above data, answer the given subquestions.