MLT Quiz 2: 3 December 2023 (September 2023 term)

Suppose we transform the data points $X$ using a mapping $\Phi(\cdot)$. Assume there exists a kernel matrix $K$ for the mapping $\Phi(\cdot)$. Moreover, we categorize the model as parametric and non-parametric according to the following definitions.

• Parametric: The samples in the training set are not necessary for making predictions on a test sample
• non-Parametric: All the samples in the training set are necessary for making predictions on a test sample

Check all that is true about the kernel regression.

The team modifies the relation as $y = X^T w + \epsilon$ where $\epsilon \sim \mathcal{N}(0, \sigma^2)$. Suppose that $\sigma^2 = 1$, $d = 3$ and $n = 100$. Assume that $\sum_{i=1}^{n} (w^T x_i - y_i)^2 = 0$ for $w = w^*$. What is the negative log-likelihood of the dataset $(X, y)$ at $w^*$? Use logarithm to base $10$.

In a Ridge regression scenario with the following dataset:

$X = \begin{bmatrix} -2 & 3 & 4 \end{bmatrix}$

And the corresponding target vector:

$Y = \begin{bmatrix} -15 \\ 18 \\ 24 \end{bmatrix}$

The regularization parameter is set at $\lambda = 3$. Calculate the ratio of the Maximum Likelihood Estimate (MLE) weight vector $(w_{\text{MLE}})$ to the Ridge weight vector $(w_{\text{Ridge}})$ and select the correct range.

Which of the following is/are the primary advantages of using L1 regularization

Select all the statements that are true about decision trees and k-Nearest Neighbors (k-NN) in machine learning:

Consider that the three weight vectors $\mathbf{w}_1, \mathbf{w}_2$, and $\mathbf{w}_3$ are learned for an eight-dimensional dataset using different regression models (Not in any particular order).

$\begin{aligned} \mathbf{w}_1 &= [0.32, -0.12, 0, 0.42, -0.18, -0.05, 0.2, -0.09] \\ \mathbf{w}_2 &= [0.25, -0.08, 0.38, -0.22, 0.14, -0.31, 0.19, -0.12] \\ \mathbf{w}_3 &= [0.22, -0.11, 0.04, 0.16, 0.08, -0.03, 0.1, -0.14] \end{aligned}$

Select the most appropriate match for these weight vectors.

Given a design matrix $X \in \mathbb{R}^{d \times n}$ and a target vector $Y \in \mathbb{R}^{n \times 1}$, where $d$ represents the number of features, $n$ represents the number of data points, and the data is defined as:

$X = \begin{bmatrix} 2 & 3 \\ 4 & 1 \end{bmatrix}$

$Y = \begin{bmatrix} 2 \\ 4 \end{bmatrix}$

Calculate the coefficients $\beta$ for Ridge regression with $\lambda = 2$.

You are working on a decision tree algorithm to classify whether a bank loan applicant will default on their loan based on several financial factors. The dataset includes Credit Score, Annual Income, Loan Amount, Loan Term as features.
The target variable is binary: 1 for ”Default” and 0 for ”No Default.”
You have a dataset of 500 loan applicants, and you want to construct a decision tree to predict loan default. To determine the first split (root node), you'll use the information gain as the criterion. Here’s the distribution of loan default in the dataset:
Default: 150 applicants No Default: 350 applicants
Calculate the Entropy for the initial dataset.

Consider the problem of classifying an input text as positive sentiment or negative sentiment. For example, the text “I am happy” is positive and the text “ I am a bit worried” is negative. The dictionary that is used to encode the text into a vector contains 12 words. Suppose we prefer to use a generative learning algorithm that estimates the joint probability $P(x, y)$, where $x \in \{0, 1\}^{12}$ and $y \in \{0, 1\}$. Assume that the features $x_i$ in a sample are not independent given the label. How many parameters do we need to estimate from the given data? (Enter -1 if you think the given information is insufficient to find the answer)

Given a training dataset with 100 data points, how many distances would we have to compute in the process of predicting the label of test-point in the k-NN algorithm with k = 5

If the proportion of points belonging to class 1 in a node is p, for what value of p is the node’s entropy maximum?

Consider the following dataset with 6 samples along with the corresponding labels. Each sample has three binary features $f_1, f_2$ and $f_3$.

Assume that the features are conditionally independent given the label $y$. Suppose the test sample is $x_{test} = [0, 1, 1]^T$.

Based on the above data answer the given subquestions.

Consider the following dataset with 6 samples along with the corresponding labels. Each sample has three binary features $f_1, f_2$ and $f_3$.

Assume that the features are conditionally independent given the label $y$. Suppose the test sample is $x_{test} = [0, 1, 1]^T$.

Based on the above data answer the given subquestions.

What will be the predicted label according to the Naive Bayes decision rule?