Machine Learning Techniques Quiz 2: 3 August 2025 (May 2025 term)

In the Bayesian modeling for linear regression, we assume the following:

• $y \mid x \sim \text{Normal}(w^Tx, \sigma^2 I)$
• $w \sim \text{Normal}(0, \gamma^2 I)$

Under this setup, the maximum aposteriori (MAP) estimator of the weights $\hat{w}_{\text{MAP}}$ is given by:

$\hat{w}_{\text{MAP}} = \arg\min_{w} \left\{\|y - X^Tw\|^2 + \lambda\|w\|^2\right\}$

Using the given information, find the expression for the regularization parameter $\lambda$.

You are evaluating the performance of a linear regression model using different hyperparameter settings (learning rate and number of iterations). Below are the results:

Which hyperparameter setting would you choose based on the training and validation errors? Why?

Consider the linear regression model:

$y = w^Tx + \epsilon$

where $\epsilon \sim \mathcal{N}(0, \sigma^2)$.

Note:

• $\hat{w}_{\text{MLE}}$: Maximum Likelihood Estimator (MLE) of $w$
• $\hat{w}_{\text{MAP}}$: Maximum A Posteriori (MAP) estimator of $w$
• $\mathbb{E}\left(\|\hat{w} - w\|^2\right)$: Mean Squared Error (MSE) between an estimator $\hat{w}$ and the true parameter $w$

Which of the following statements is/are true?

Which among the following are true?

Perform a 3-NN regressor for the following dataset:

where, $x = \begin{bmatrix} x_1 & x_2 \end{bmatrix}^T$ is a feature vector and $y$ is the label. What will be the predicted value for the test data point $\begin{bmatrix} 3 & 2 \end{bmatrix}^T$? Enter the answer correct to one decimal place.

Based on the above data, answer the given subquestions.

Which among the following represents the correct decision boundary for the decision tree? Note: Here + represents the label 1 and − represents the label 0.

Based on the above data, answer the given subquestions.

What will be the prediction for the new datapoint (2, 2)?

Consider a 2-class Naive Bayes classifier with first feature $f_1$ as binary and the second feature $f_2$ as Gaussian. Following are the information that are provided to us:

Estimate of $P(Y = 1), \hat{p} = 0.4$
Estimate of $P(f_1 = 1 \mid y = 0)$, $\hat{p}_1^0 = 0.4$
Estimate of $P(f_1 = 1 \mid y = 1)$, $\hat{p}_1^1 = 0.5$
$(f_2 \mid y = 0) \sim \text{Normal}(0, 1)$
$(f_2 \mid y = 1) \sim \text{Normal}(1, 1)$

Based on the above data, answer the given subquestions.

Consider a 2-class Naive Bayes classifier with first feature $f_1$ as binary and the second feature $f_2$ as Gaussian. Following are the information that are provided to us:

Estimate of $P(Y = 1), \hat{p} = 0.4$
Estimate of $P(f_1 = 1 \mid y = 0)$, $\hat{p}_1^0 = 0.4$
Estimate of $P(f_1 = 1 \mid y = 1)$, $\hat{p}_1^1 = 0.5$
$(f_2 \mid y = 0) \sim \text{Normal}(0, 1)$
$(f_2 \mid y = 1) \sim \text{Normal}(1, 1)$

Based on the above data, answer the given subquestions.

Suppose you want to build a decision tree to help diagnose a certain disease based on patient data. You have a dataset of 1000 patients labeled as either “positive” or “negative”. Here, positive represents patients with the disease and negative represents no disease. One of the feature used to diagnose is headache.
Below is the breakdown of the patients reporting headache:
Headache present: 300 positive, 50 negative
Headache not present: 100 positive, 550 negative

Use the following values if required:

• $\log_2(0.4) = -1.32$
• $\log_2(0.6) = -0.74$
• $\log_2(0.86) = -0.22$
• $\log_2(0.14) = -2.84$
• $\log_2(0.15) = -2.74$
• $\log_2(0.85) = -0.23$

Note: Use $\log_2$ for the entropy calculations.

Based on the above data, answer the given subquestions.

What is the entropy of the root node? Enter the answer correct to two decimal places.

Suppose you want to build a decision tree to help diagnose a certain disease based on patient data. You have a dataset of 1000 patients labeled as either “positive” or “negative”. Here, positive represents patients with the disease and negative represents no disease. One of the feature used to diagnose is headache.
Below is the breakdown of the patients reporting headache:
Headache present: 300 positive, 50 negative
Headache not present: 100 positive, 550 negative

Use the following values if required:

• $\log_2(0.4) = -1.32$
• $\log_2(0.6) = -0.74$
• $\log_2(0.86) = -0.22$
• $\log_2(0.14) = -2.84$
• $\log_2(0.15) = -2.74$
• $\log_2(0.85) = -0.23$

Note: Use $\log_2$ for the entropy calculations.

Based on the above data, answer the given subquestions.

What is the information gain for choosing the feature “headache” for splitting at the root node? Enter the answer correct to two decimal places.