Machine Learning Techniques Quiz 2: 1 December 2024 (September 2024 term)

Kernel regression with a polynomial kernel is applied on the following dataset with two features:

$X = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix}, \qquad y = [2, 1, 2]^T$

Weight vector can be written as $w = \phi(X)\alpha$, where $\phi$ is the transformation mapping corresponding to the kernel $k(x_i, x_j) = (1 + x_i^T x_j)^2$. The vector $\alpha$ is given by $(K)^{-1}y$, where $K$ is the kernel matrix.

Based on the above data, answer the given subquestions.

Kernel regression with a polynomial kernel is applied on the following dataset with two features:

$X = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix}, \qquad y = [2, 1, 2]^T$

Weight vector can be written as $w = \phi(X)\alpha$, where $\phi$ is the transformation mapping corresponding to the kernel $k(x_i, x_j) = (1 + x_i^T x_j)^2$. The vector $\alpha$ is given by $(K)^{-1}y$, where $K$ is the kernel matrix.

Based on the above data, answer the given subquestions.

A binary classification dataset has 2000 data points belonging to $\{0,1\}^2$. A Naive Bayes algorithm was run on the same dataset, resulting in the following estimates:

$\begin{aligned} \hat{p}, &\text{ estimate for } P(y = 1) = 0.4 \\ \hat{p}_1^0, &\text{ estimate for } P(f_1 = 1 \mid y = 0) = 0.25 \\ \hat{p}_2^0, &\text{ estimate for } P(f_2 = 1 \mid y = 0) = 0.35 \\ \hat{p}_1^1, &\text{ estimate for } P(f_1 = 1 \mid y = 1) = 0.15 \\ \hat{p}_2^1, &\text{ estimate for } P(f_2 = 1 \mid y = 1) = 0.05 \end{aligned}$

Based on the above data, answer the given subquestions.

A binary classification dataset has 2000 data points belonging to $\{0,1\}^2$. A Naive Bayes algorithm was run on the same dataset, resulting in the following estimates:

$\begin{aligned} \hat{p}, &\text{ estimate for } P(y = 1) = 0.4 \\ \hat{p}_1^0, &\text{ estimate for } P(f_1 = 1 \mid y = 0) = 0.25 \\ \hat{p}_2^0, &\text{ estimate for } P(f_2 = 1 \mid y = 0) = 0.35 \\ \hat{p}_1^1, &\text{ estimate for } P(f_1 = 1 \mid y = 1) = 0.15 \\ \hat{p}_2^1, &\text{ estimate for } P(f_2 = 1 \mid y = 1) = 0.05 \end{aligned}$

Based on the above data, answer the given subquestions.

Consider the following feature vectors in $\mathbb{R}^3$:

$x_1 = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}, \quad x_2 = \begin{bmatrix} -1 \\ -2 \\ 0 \end{bmatrix}, \quad x_3 = \begin{bmatrix} 3 \\ 0 \\ 1 \end{bmatrix}, \quad x_4 = \begin{bmatrix} 0 \\ 1 \\ 4 \end{bmatrix}, \quad x_5 = \begin{bmatrix} 2 \\ -1 \\ -3 \end{bmatrix}$

The labels of these points are:

$y_1 = 0, \quad y_2 = 1, \quad y_3 = 1, \quad y_4 = 0, \quad y_5 = 1$

If we use a $k$-NN algorithm with $k = 3$, what would be the predicted label for the following test point:

$x_{test} = \begin{bmatrix} 1 \\ 1 \\ 2 \end{bmatrix}$

Suppose you want to use a Naive Bayes classifier to predict whether a student will pass or fail an exam based on two features: the number of hours they studied and whether they attended review sessions. Assume that the features are conditionally independent given the exam outcome and that the variances of the study hours distributions are equal for both pass and fail categories. How many parameters are required to classify a new student using this Naive Bayes classifier?

Select all true statements.

Consider a binary classification problem with a training dataset of 100 points, evenly distributed between two classes (50 points in each class). You decide to train a k-NN algorithm with k = 3. Each point is considered its own neighbor during classification.
Based on the above data, answer the given subquestions.

What is the minimum number of misclassifications that can occur in the training dataset when using this k-NN algorithm?

Consider a binary classification problem with a training dataset of 100 points, evenly distributed between two classes (50 points in each class). You decide to train a k-NN algorithm with k = 3. Each point is considered its own neighbor during classification.
Based on the above data, answer the given subquestions.

Assuming there are outliers, the decision boundary becomes smoother with decreasing value of k in a k-NN algorithm.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

Consider the Bayesian formulation of the linear regression problem, where the prior for $w$ is assumed to be

$w \sim \text{Laplace}(0, 2/\lambda).$

Then, which among the following is true?

Hint: If $X \sim \text{Laplace}(\mu, b)$, then

$f_X(x) = \frac{1}{2b} e^{-|x-\mu|/b}.$

Consider a binary classification problem in which a decision tree is classifying data points into two classes, A and B. In a particular node of the tree, 60% of the data points belong to class A, while the remaining 40% belong to class B.
Based on the above data, answer the given subquestions.

Do you have enough information to find the entropy of this node?

Consider a binary classification problem in which a decision tree is classifying data points into two classes, A and B. In a particular node of the tree, 60% of the data points belong to class A, while the remaining 40% belong to class B.
Based on the above data, answer the given subquestions.

If the answer to the previous question is ”Yes,” calculate the entropy of this node to three decimal places. If the answer to the previous question is ”No,” enter -1 as your answer.