Machine Learning Techniques Quiz 2: 6 August 2023 (May 2023 term)

Consider a training dataset of $n$ points for a regression problem. Assume that the model is linear. Let $\mathbf{w}_1$ and $\mathbf{w}_2$ be the optimal weight vectors obtained from solving the following optimization problems.

$\mathbf{w}_1 = \underset{\mathbf{w}}{\arg\min} \ \sum_{i=1}^{n} (\mathbf{w}^T\mathbf{x}_i - y_i)^2$

$\mathbf{w}_2 = \underset{\mathbf{w}}{\arg\min} \ \sum_{i=1}^{n} (\mathbf{w}^T\mathbf{x}_i - y_i)^3$

Choose the most appropriate answer.

Which of the following statements are true about the decision tree algorithm?

Which of the following statements is/are true regarding solution of Ridge regression problem?

Consider kernel regression with the kernel function $(\mathbf{x}_1^T\mathbf{x}_2 + 2)^2$ applied on the following dataset.

$\mathbf{X} = \begin{bmatrix} 1 & 0 & 2 & 0 & 3 & 0 \\ 0 & 1 & 0 & 2 & 0 & 3 \\ 0 & 0 & 0 & 0 & 0 & 0 \end{bmatrix}$

The optimal weight vector $\mathbf{w}^*$ is given by:

$\mathbf{w}^* = \phi(X)[0.1, 2, 3.9, 5, 6, 8]^T$

where $\phi$ is transformation mapping corresponding to the given kernel. What will be the prediction for the data point $[0, 0, 1]^T$?

Consider the following data set:

$X = [8, 6, 10]$

Assuming a ridge penalty $\lambda = 100$, what will be the value of $\frac{\hat{w}_{ridge}}{\hat{w}_{MLE}}$?

Here $\hat{w}_{ridge}$ and $\hat{w}_{MLE}$ are the Ridge and MLE estimates of the weight vectors, respectively. Assume that the label vector $y$ of shape $(3, 1)$ is known. Enter your answer correct to two decimal places.

Consider a binary classification problem and a decision tree that is being trained to classify the points. In one of the internal nodes in this tree, 75% of the data-points belong to one of the two classes and the rest belong to the other class. You are not given the information about which class is more numerous in this node.
Based on the above data, answer the given subquestions.

Do you have enough information to find the entropy of this node?

Consider a binary classification problem and a decision tree that is being trained to classify the points. In one of the internal nodes in this tree, 75% of the data-points belong to one of the two classes and the rest belong to the other class. You are not given the information about which class is more numerous in this node.
Based on the above data, answer the given subquestions.

If the answer to the previous questions is "Yes", find the entropy of the node. Use log2 and enter your answer correct to three decimal places.
If the answer to the previous question is "No", enter -1 as your answer.

Based on the above data, answer the given subquestions.

If p = P ( y = 1 ) is estimated to be 0.4, what will be the prediction for the point x = 3 using the Bayes classifier? Enter 1 or -1.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

If p = P (y = 1) is estimated to be 0.5 using MLE on a given training dataset, what will be the training error of the Bayes classifier for this problem?

Consider a naive Bayes model is trained on the following data matrix $X$ of shape $(d, n)$ and corresponding label vector $y$:

$X = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 1 & 1 & 0 \end{bmatrix} \quad y = \begin{bmatrix} 1 & 0 & 1 & 0 \end{bmatrix}^T$

Assume that $\hat{p}$ and $\hat{p}_j^{y_i}$ are estimates for $P(y = 1)$ and $P(f_j = 1 | y = y_i)$, respectively. Here, $f_i;\ i = 1, 2, 3$ is the $i^{th}$ feature. These parameters are estimated using MLE. Do not apply any smoothing on the dataset.

Based on the above data, answer the given subquestions.

Consider a naive Bayes model is trained on the following data matrix $X$ of shape $(d, n)$ and corresponding label vector $y$:

$X = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 1 & 1 & 0 \end{bmatrix} \quad y = \begin{bmatrix} 1 & 0 & 1 & 0 \end{bmatrix}^T$

Assume that $\hat{p}$ and $\hat{p}_j^{y_i}$ are estimates for $P(y = 1)$ and $P(f_j = 1 | y = y_i)$, respectively. Here, $f_i;\ i = 1, 2, 3$ is the $i^{th}$ feature. These parameters are estimated using MLE. Do not apply any smoothing on the dataset.

Based on the above data, answer the given subquestions.