MLT End Term: 7 August 2022, Set QPD1 (May 2022 term)

During classification of linearly separable data-set using perceptron algorithm, as the value of learning rate α is decreased,

A knn algorithm with k = 30 gives high training error and high validation error. What value of the k we should choose to get the better performance of the
algorithm?

Consider a binary classification problem. Let p1 denote the proportion of class 0 examples in a particular node. Which of the following graphs shows correct curves for the Gini-index, Entropy and misclassification error of that node?

The following is the activation vector output by some hidden layer in a neural network when some input vector is given to it:

Which of the following could be the activation function used in this layer?

Which of the following are multi label problems? For each option, assume suitable features are available.

In a random forest model let p < m be the number of randomly selected features that are used to identify the best split at any node of a tree. Which of the
following are true? (m is the original number of features)

Consider feature matrix $\mathbf{X} = \begin{bmatrix} 1 & 2 \\ 2 & 1 \\ 2 & 3 \end{bmatrix}$, label vector $\mathbf{y} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}$ and the weight vector is $\mathbf{w} = \begin{bmatrix} 1 \\ -2 \\ 3 \end{bmatrix}$. Add a dummy feature to $\mathbf{X}$.

What will be the value of the loss function if a regression model without regularization is fitted?

Consider a test-dataset of 100 points for a binary classification problem, where 60 belong to the positive class (true label) and the rest belong to the negative class (true label). The following is a table for some classifier that has been prepared by an ML engineer:

If this is a valid confusion matrix (just by looking at the numbers), enter the classifier’s recall as the answer. If this is not a valid confusion matrix, enter 0 as the answer. Your answer should be in the interval [0, 1], endpoints inclusive. Enter your answer correct to two decimal places.

Consider a modified loss function for linear regression that is of the following form for a training dataset that has $n$ points:

$L(\mathbf{w}) = \frac{1}{2}\sum_{i=1}^{n} r_i(\mathbf{w}^T\mathbf{x}_i - y_i)^2$

Here, $r_i$ is some constant in $[0, 1]$ associated with each data-point in the training dataset. The dummy feature and the corresponding weight are already included in the vectors $\mathbf{x}$ and $\mathbf{w}$ respectively. What is the expression of the gradient of $L(\mathbf{w})$ with respect to $\mathbf{w}$?

Consider unlabeled data with two features X1 and X2 as shown in the figure.

C1 and C2 are the coordinates of centroids obtained after certain iterations of K-means algorithm with K = 2. Which of the following options is correct about the silhouette score S? Note: Euclidean distance is used to calculate the distances.

Consider a neural network for a multi-class, image classification problem. When the network is trained on the images as they are, it does a good job on the test data. Call the dataset (train + test) for this setup D1 and the network N1. Assume that we now turn all images upside down, in both the training and test dataset. Now, the network with the same architecture is trained from scratch on this modified dataset. Call the dataset (train + test) for this setup D2 and network N2. Select the most appropriate option.

Two classifiers are trained on a dataset for a binary classification problem. They are then tested on the same dataset. The $F_1$ scores of both classifiers are the same. $(p_1, r_1)$ and $(p_2, r_2)$ are the precision-recall scores for the two classifiers. Consider the following statements:

Statement-1

If $p_1 > p_2$, then $r_1 < r_2$

Statement-2

If $p_1 = p_2$, then $r_1 = r_2$

Select the most appropriate option.

Which of the following code blocks will produce the same value for the variable ‘Ans’? Assume the ‘numpy’ library is imported as ‘np’.

Rajesh trained a model on house price prediction problem. After training he came up with following model:

$y = 4000 - 20 \cdot x_1 + 15 \cdot x_2 + 500 \cdot x_3 - 30 \cdot x_4$

where,
$y$ is predicted house price,
$x_1$ is number of years the house was built before,
$x_2$ is the carpet area in square feet,
$x_3$ is the number of floors the apartment/house has,
$x_4$ represents distance of the property from railway station.

According to Rajesh's model, which of the following statements are correct?

What is the output of the following code?

Consider the feature matrix $\mathbf{X} = \begin{bmatrix} 1 & 2 & 3 \\ 3 & 2 & 4 \end{bmatrix}$ and corresponding label vector $\mathbf{y} = \begin{bmatrix} 2 \\ 4 \end{bmatrix}$. Let regularization rate, $\lambda = 0.1$. Compute lasso regression loss using weight $\mathbf{w} = \begin{bmatrix} 1 \\ 0.01 \\ -0.5 \\ 0 \end{bmatrix}$.

You are given a linearly separable dataset with feature matrix X, for which a perceptron has been trained until it converges (perfectly separates the data). The weight vector corresponding to it is w. This dataset has 60 points from the positive class and 40 from the negative class. What is the output of the following snippet of code?

Notes

• The labels for a perceptron lie in $\{-1, 1\}$.
• The NumPy arrays $\mathbf{X}$ and $\mathbf{w}$ are compatible for matrix multiplication, the dummy feature and the corresponding weight are already built into the arrays.
• None of the 100 points lie on the decision boundary.

A Gaussian Naive Bayes model is trained for a multi-class classification problem that has 12 features and 5 classes. Find the total number of parameters that have to be estimated for this model. Consider each parameter to be a scalar value. In other words, if we decide to store all the parameters in a Python list, with each element of the list being a float value corresponding to a single parameter, what is the size of this list? Ignore the priors in the calculation. Only focus on the parameters of the class conditional densities.

Consider the following network architecture:

How many parameters (weights + biases) does this network have?

A multi-class classification problem has 5 classes. The training dataset has n data-points, with an equal number of points from each of the 5 classes. Consider a dummy classifier that does prediction as follows: for each input data-point, it picks one of the 5 classes at random (uniformly) and outputs that as its prediction. What is the accuracy of the model on the training dataset as n becomes very large? Your answer should be between 0 and 1. Enter your answer correct to two decimal places.
Hint: Think about it in the probabilistic sense.

Consider a logistic regression model for a binary classification problem with two features $x_1$ and $x_2$. The feature vector is $\begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$ and labels lie in $\{0, 1\}$. The threshold for inference is 0.5. The dummy feature and the weight corresponding to it can be ignored for this problem. Let $x_1$ be the horizontal axis and $x_2$ be the vertical axis. You are given two feature vectors:

$\mathbf{x_1} = \begin{bmatrix} \sqrt{3} \\ 1 \end{bmatrix}, \mathbf{x_2} = \begin{bmatrix} -\sqrt{3} \\ 1 \end{bmatrix}$

The weight vector makes an angle of $\theta$ with the positive $x_1$ axis (horizontal). Each $\theta$ corresponds to a different classifier. For what range of values of $\theta$ are both $\mathbf{x_1}$ and $\mathbf{x_2}$ predicted to belong to class-1?

Hints:

• To draw the weight vector $\mathbf{w} = \begin{bmatrix} w_1 \\ w_2 \end{bmatrix}$, plot the point $(w_1, w_2)$ and draw an arrow starting at the origin to this point.
• $\tan(30^\circ) = \frac{1}{\sqrt{3}}$

Consider the following data points

We perform k-means clustering on the above data, with $k = 2$, using Manhattan distance as the distance measure. At $t$th iteration, we have (2,2) as the centroid for cluster-1 and (6,5) as the centroid for cluster-2. After performing $(t + 1)$th iteration the point (3.4,4) will belong to

If Wij is the weight of the edge from neuron i in layer l − 1 to neuron j in layer l, which of the following statements about the matrix W are true? Neurons in a layer are processed (indexed) from top to bottom. So, the first neuron in a layer is the top-most neuron in that layer.

A perceptron model is trained on the following binary classification data-set.

At one of the iterations, weights are: $w_0 = -0.7, \;\; w_1 = 0.8$ and $w_2 = 0.1$. Assume that $w_0', w_1'$ and $w_2'$ are the updated weights after one epoch (one epoch is completed on going through all the data points). Find the value of $w_0' + w_1' + w_2'$. Consider the learning rate to be one. Don't change the order of samples while updating the weight vector. Take the original order of samples.

Inference using logistic regression happens as follows. $T$ is called the threshold and is some real number in the interval $(0, 1)$. $\hat{y}$ stands for the predicted label.

$\hat{y} = \begin{cases} 1, & P(y = 1 \mid \mathbf{x}) \geq T \\ 0, & \text{otherwise} \end{cases}$

Given this setup, the equation of the decision boundary is given below:

$\mathbf{w}^T\mathbf{x} - u = 0$

$\mathbf{w}$ has the same dimensions as $\mathbf{x}$. The dummy feature is included in $\mathbf{x}$ and the corresponding weight is included in $\mathbf{w}$. If $T = \frac{e^2}{1 + e^2}$, what is the value of the unknown quantity $u$? Here, $e$ is the Euler's number that is associated with the natural logarithm and is the same one found on your calculator. Enter the closest integer as your answer.

An SVM has been trained for a 2D problem. The feature vector is $\mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$.

It has the following weight vector and bias:

$\mathbf{w} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}, b = -2$

Recall that the labels in the SVM setup are $+1$ and $-1$. Consider a unit square whose vertices are at:

$(0, 0), (1, 0), (0, 1), (1, 1)$

The horizontal axis corresponds to $x_1$ and the vertical axis corresponds to $x_2$. A point is picked at random (uniformly) from the region bounded by the square. What is the probability that this point is predicted as belonging to class $+1$ by the SVM? Enter your answer correct to two decimal places.

Find the hinge loss for this soft-margin SVM classifier on the dataset that is given in the table. The weight vector and bias are as follows:

$\mathbf{w} = \begin{bmatrix} 1 \\ -1 \end{bmatrix}, \quad b = 1$

The coefficient $C$ can be assumed to be 1.

Note that you just need to report the hinge loss. Do not compute the margin loss which involves only the term $\mathbf{w}$. Also note that the hinge loss does not have a factor of 0.5 before it. Enter the closest integer as your answer.

Consider two networks $N_1$ and $N_2$ for a binary classification task. $N_1$ has two neurons at the output layer and uses the softmax activation function. $N_2$ has one neuron at the output layer and uses sigmoid activation function. We don't need the information about the hidden layers for this problem.

For some test data-point $\mathbf{x}$, the pre-activations at the output layer for $N_1$ is given below. The first neuron corresponds to class-0 and the second corresponds to class-1:

$\mathbf{z} = \begin{bmatrix} 1 \\ 3 \end{bmatrix}$

It turns out that for this data-point, both networks predict the same probability of this point belonging to class-1. That is $P(y = 1 \mid \mathbf{x})$ is the same for both networks. If this is the case, what should be the pre-activation value at the output layer of $N_2$, call it $z$, corresponding to this data-point? Enter the closest integer as your answer.

Notes

• The activation of $N_2$ at the output-layer is interpreted as $P(y = 1 \mid \mathbf{x})$.
• The $\mathbf{z}$ for $N_1$ is a vector and the $z$ for $N_2$ is a scalar.
• Be careful about the distinction between pre-activation and activation.

Based on the above data, answer the given subquestions.

How many samples of the training data-set belong to Leaf 2?

Based on the above data, answer the given subquestions.

If the present price of a car which is driven for 5 years is 9.5 lac. If the car has an LPG engine, what will be the prediction for the selling price of the car according to the given regression tree? Enter your answer in lac. If your answer is 5.6 lac, enter the answer 5.6.

Consider that adaboost model is trained on the following binary classification data-set.

The data-set is split according to feature $X_2$ to create the first stump. Equal sample weights are assigned to each examples to create the first stump and Gini-index measure is used to split the data.

Based on the above data, answer the given subquestions.

What will be the performance of the first stump? Enter your answer correct to two decimal places.

Consider that adaboost model is trained on the following binary classification data-set.

The data-set is split according to feature $X_2$ to create the first stump. Equal sample weights are assigned to each examples to create the first stump and Gini-index measure is used to split the data.

Based on the above data, answer the given subquestions.

What sample weight will be assigned to the last example to create the next stump? Don’t normalize the sample weights. Enter your answer correct to two decimal places.