Machine Learning Techniques Quiz 2: 10 July 2022 (May 2022 term)

A logistic regression model is being trained on a dataset of size $2n$. The first $n$ data-points belong to class-1 and the rest in class-0. Note that we are talking about the true label here.

$\begin{aligned} &\text{Class-1} = \{x_1, \cdots, x_n\} \\ &\text{Class-0} = \{x_{n+1}, \cdots, x_{2n}\} \end{aligned}$

The probability output by the model at any step in the training process is given by:

$P(y = 1 \mid x_i) = p_i$

Which of the following expressions is the loss of the model?

NOTE: We use the binary cross entropy loss for logistic regression. Labels are 1 and 0 for the two classes.

Which of the following could be the decision boundary learned by a hard-margin SVM? Choose the most appropriate option.
NOTE: The dataset is the same for all four options.

Which of the following could be the vector of probabilities output by a softmax regression model for 5 classes? Note that the options are independent of each other.

Consider a logistic regression model that is trained on videos to detect objectionable content. Videos with objectionable content belong to the positive class (label 1). Harmless videos belong to the negative class (label 0).
A good detector should be able to correctly identify almost all videos that are objectionable. If it incorrectly classifies even a single video that has inappropriate content in it, that could have serious consequences, as millions of people might end up watching it. In this process the detector may classify some harmless videos as belonging to the positive class. But that is a price we are willing to pay.
How should we choose the threshold (for inference) of this logistic regression model?

Consider a binary classification task that has 2 features. Assume that we train a soft-margin, linear SVM (decision boundary is a line in 2D space). We know nothing about the distribution of points in the training set. The points need not be linearly separable.
This model is now tested on the following dataset that has four points.

What are the possible values of the accuracy of the model? All options are independent of each other. Assume that the decision boundary of the model does not pass through any one of the four points.

An SVM model that has been trained for a binary classification task has the following weight vector and bias:

$w = \begin{bmatrix} 5 \\ 7 \end{bmatrix}, \quad b = -35$

This model is tested on a dataset with 10 samples as given below. Here, $x = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$ is a feature vector and $y$ is the true label.

Compute the accuracy of the model on this dataset. Enter your answer between 0 and 1.

Hint: The Cartesian coordinate system was named after Rene Descartes.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

What is the predicted label of the test point if k = 1? The label is 1 for the positive class and –1 for the negative class.

How many points are at a distance of 2 from the test point?
Note:
(1) We use the Manhattan distance.
(2) We want those points that are exactly 2 units away from the test point.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

Consider a binary classification problem. The training-data has several features out of which we have access to only two binary features $(x_1, x_2)$. The labels are $1$ and $2$ for the two classes. The training dataset has the following distribution of points:

The table is to be parsed as follows. The first row of the table states that there are $30$ points from class $1$ that have $x_1 = 0, x_2 = 0$. A Bernoulli Naive Bayes model is fit for this data with the following matrix of probabilities:

$\begin{bmatrix} w_{11} & w_{12} \\ w_{21} & w_{22} \end{bmatrix}$

Each entry in this matrix can be understood as follows. For $i, j \in \{1, 2\}$:

$w_{ij} = P(x_i = 1 \mid y = j)$

You can ignore smoothing. For all questions, report the answer up to two decimal places. Do not round-up or round-down the answer.
For example, if you get a value of $0.37914$, just report $0.37$.

NOTE: If the last row of the table is not clear: for feature vector $(1, 1)$ in class-2, there are fifty points.

Based on the above data, answer the given subquestions.

Consider a binary classification problem. The training-data has several features out of which we have access to only two binary features $(x_1, x_2)$. The labels are $1$ and $2$ for the two classes. The training dataset has the following distribution of points:

The table is to be parsed as follows. The first row of the table states that there are $30$ points from class $1$ that have $x_1 = 0, x_2 = 0$. A Bernoulli Naive Bayes model is fit for this data with the following matrix of probabilities:

$\begin{bmatrix} w_{11} & w_{12} \\ w_{21} & w_{22} \end{bmatrix}$

Each entry in this matrix can be understood as follows. For $i, j \in \{1, 2\}$:

$w_{ij} = P(x_i = 1 \mid y = j)$

You can ignore smoothing. For all questions, report the answer up to two decimal places. Do not round-up or round-down the answer.
For example, if you get a value of $0.37914$, just report $0.37$.

NOTE: If the last row of the table is not clear: for feature vector $(1, 1)$ in class-2, there are fifty points.

Based on the above data, answer the given subquestions.

Consider a binary classification problem. The training-data has several features out of which we have access to only two binary features $(x_1, x_2)$. The labels are $1$ and $2$ for the two classes. The training dataset has the following distribution of points:

The table is to be parsed as follows. The first row of the table states that there are $30$ points from class $1$ that have $x_1 = 0, x_2 = 0$. A Bernoulli Naive Bayes model is fit for this data with the following matrix of probabilities:

$\begin{bmatrix} w_{11} & w_{12} \\ w_{21} & w_{22} \end{bmatrix}$

Each entry in this matrix can be understood as follows. For $i, j \in \{1, 2\}$:

$w_{ij} = P(x_i = 1 \mid y = j)$

You can ignore smoothing. For all questions, report the answer up to two decimal places. Do not round-up or round-down the answer.
For example, if you get a value of $0.37914$, just report $0.37$.

NOTE: If the last row of the table is not clear: for feature vector $(1, 1)$ in class-2, there are fifty points.

Based on the above data, answer the given subquestions.

Consider a binary classification problem. The training-data has several features out of which we have access to only two binary features $(x_1, x_2)$. The labels are $1$ and $2$ for the two classes. The training dataset has the following distribution of points:

The table is to be parsed as follows. The first row of the table states that there are $30$ points from class $1$ that have $x_1 = 0, x_2 = 0$. A Bernoulli Naive Bayes model is fit for this data with the following matrix of probabilities:

$\begin{bmatrix} w_{11} & w_{12} \\ w_{21} & w_{22} \end{bmatrix}$

Each entry in this matrix can be understood as follows. For $i, j \in \{1, 2\}$:

$w_{ij} = P(x_i = 1 \mid y = j)$

You can ignore smoothing. For all questions, report the answer up to two decimal places. Do not round-up or round-down the answer.
For example, if you get a value of $0.37914$, just report $0.37$.

NOTE: If the last row of the table is not clear: for feature vector $(1, 1)$ in class-2, there are fifty points.

Based on the above data, answer the given subquestions.