Machine Learning Techniques End Term: 31 August 2025, Set QDB3 (May 2025 term)

Consider the following dataset:

$D = \left\{\left(\begin{bmatrix} -2 \\ 1 \end{bmatrix}, +1\right), \left(\begin{bmatrix} -1 \\ 1 \end{bmatrix}, +1\right), \left(\begin{bmatrix} -1 \\ -1 \end{bmatrix}, -1\right), \left(\begin{bmatrix} 2 \\ 1 \end{bmatrix}, -1\right)\right\}.$

The perceptron algorithm is applied on this data set with the weight vector initialized to $\begin{bmatrix} 0 & 0 \end{bmatrix}^T$. Will the algorithm converge after the second round of update of the weight vector? While looking for mistakes, cycle through the data points form left to right.

Consider $K$-means clustering algorithm with $K = 2$ on a one-dimensional dataset $D = \{x_1, x_2, \ldots, x_n\}$ aiming to minimize:

$F(z_1, z_2, \ldots, z_n) = \sum_{i=1}^{n} ||x_i - \mu_{z_i}||^2,$

where $z_i \in \{1, 2\}$ is the cluster assignment for point $x_i$ and $\mu_{z_i}$ is the mean of the assigned cluster.

For greedy initialization, choose the first center uniformly at random from $D$, and the second as the point farthest from it (by Euclidean distance). Let $F_{\text{optimal}}$ be the minimum possible value of $F$. Which of the following statements about greedy initialization is true?

Suppose we are solving a binary classification task using a neural network with the following architecture:

• Input: $x = \begin{bmatrix} 1 \\ 1 \end{bmatrix}$
• Hidden layer: 2 neurons (ReLU activation)
• Output layer: 1 neuron (sigmoid activation)

The weight parameters of the network are given as:

$W^{(1)} = \begin{bmatrix} w_1^{(1)} & w_2^{(1)} \\ w_1^{(2)} & w_2^{(2)} \end{bmatrix} = \begin{bmatrix} 2 & -1 \\ 1 & 1 \end{bmatrix}, \quad W^{(out)} = \begin{bmatrix} w_1^{(out)} & w_2^{(out)} \end{bmatrix}^T = \begin{bmatrix} 1 & 2 \end{bmatrix}^T,$

where $w_i^{(j)}$ represents the weight associated with the $j$-th neuron for the $i$-th input feature. What are the predicted probabilities?

Which of the following statements are true about the decision tree algorithm?

Consider a 1D dataset which consists of 2 positive data points (points with label 1) $\{1, 2\}$ and 3 negative data points (points with label $-1$) $\{-3, -2, -1\}$. Suppose that we want to learn a soft-margin linear SVM for this data set. Consider the formulation of the soft-margin SVM:

$\min_{w, \epsilon} \frac{1}{2}||w||^2 + C\sum_{i=1}^{n} \epsilon_i, \qquad C \geq 0$

$\text{subject to } \; (w^Tx_i)y_i + \epsilon_i \geq 1, \; \forall i$

$\epsilon_i \geq 0, \; \forall i.$

Select the correct options from the following:

Given a two-dimensional data set where points from class 1 are: {(−2, 3), (−1, 1)} and points from class 0 are: {(1, 3), (1, 4)}. Which of the following statements are true?

Consider a dataset $\{x_1, x_2, x_3, x_4\}$, where each $x_i \in \mathbb{R}^3$. The covariance matrix $C$ has three non-zero eigenvalues $\lambda_1 > \lambda_2 > \lambda_3$ satisfying:

$2\lambda_1 - \lambda_2 + \lambda_3 = 8$

$\lambda_1 + 2\lambda_2 - \lambda_3 = 5$

The difference between the largest and smallest eigenvalue is 3. Determine the total variance of the dataset.

Consider a logistic regression model trained to predict whether a customer will buy a product ($Y = 1$) or not ($Y = 0$) based on two features: $X_1$: number of website visits per week and $X_2$: time spent on product pages per visit (minutes). The fitted model is:

$\hat{p} = P(y = 1 \mid x_1, x_2) = \frac{1}{1 + e^{-(0.4x_1 + 0.1x_2)}}.$

For a customer who spends 4 minutes per visit, find how many weekly visits are required to make the predicted probability of purchase equal to 0.7. Enter the answer correct to 2 decimal places.

A random forest model consists of three decision trees. For a given test point, each tree outputs the following class probabilities for class $+1$ and class $-1$:

$\begin{aligned} &\text{Tree 1:} \quad P(+1) = 0.7, \; P(-1) = 0.3 \\ &\text{Tree 2:} \quad P(+1) = 0.4, \; P(-1) = 0.6. \\ &\text{Tree 3:} \quad P(+1) = 0.8, \; P(-1) = 0.2 \end{aligned}$

The random forest predicts the final label by averaging the class probabilities across the trees and selecting the class with the higher average probability. What is the final predicted label for the test point? (Give the answer as 1 for class $+1$ or $-1$ for class $-1$.)

A machine's lifetime (in hours) is modeled as a random variable $X \sim \text{Exponential}(\lambda)$. Suppose the observed lifetimes (in hours) of the machine over 8 trials are:

$5.2,\ 0.8,\ 3.4,\ 2.5,\ 1.1,\ 4.0,\ 6.3,\ 2.9$

Assume that the parameter $\lambda$ has a prior distribution given by $\lambda \sim \text{Uniform}(0, 2)$. Find the maximum a posteriori (MAP) estimate of $\lambda$. Enter the answer correct to two decimal plaes.

Recall that:

$\hat{\lambda}_{\text{MAP}} = \arg\max_{\lambda} \; f(\lambda \mid \{X_1, \ldots, X_n\}),$

where $f$ is the posterior distribution.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

Consider a naive Bayes classification problem with three binary features $f_1, f_2, f_3$ for a binary classification problem. We have a total of 6 training samples given in the table below:

Based on the above data, answer the given subquestions.

Consider a naive Bayes classification problem with three binary features $f_1, f_2, f_3$ for a binary classification problem. We have a total of 6 training samples given in the table below:

Based on the above data, answer the given subquestions.

If a test point has label 1, what is the probability of $x_{test} = \begin{bmatrix} 1 & 1 & 1 \end{bmatrix}^T$? Assume that the Laplacian smoothing is done, (i.e., add two pseudo training exmaples $x_{pseudo}^0 = \begin{bmatrix} 1 & 1 & 1 \end{bmatrix}^T$ for class 0 and $x_{pseudo}^1 = \begin{bmatrix} 1 & 1 & 1 \end{bmatrix}^T$ for class 1).

Enter the answer correct to four decimal places.