MLT Quiz 1: 29 October 2023 (September 2023 term)

In the K-means algorithm, cluster centroids $(\mu_k)$ are updated at each iteration to minimize the sum of squared distances between data points and their assigned centroids. The formula used for this update is:

$\mu_k = \frac{\sum x_i \mathbb{1}(z_i = k)}{\sum \mathbb{1}(z_i = k)}$

Here:

• $x_i$ represents $i^{\text{th}}$ data point in the dataset.
• $z_i$ represents the cluster assignment for data point $x_i$, where $z_i$ is an integer indicating the cluster index $(k)$.

Suppose you have a dataset of 9 two-dimensional data points and are performing K-means clustering with $K = 3$ clusters. After the first iteration, clusters $C_1, C_2$, and $C_3$ have the following data points:

$C_1$: $\{(2, 2), (3, 3), (4,4) \}$
$C_2$: $\{ (6, 6), (7, 7),(8,8 )\}$
$C_3$: $\{(10, 10), (11, 11), (12, 12)\}$

After the second iteration, which of the following options represents the updated cluster centroid $\mu_k$ for $C_1$ using the provided formula?

Consider 9 data points $x_1, x_2, \ldots, x_9$: $(2,3),(1,2),(3,1),(5,6),(7,8),(9,9),(6,7),(10,10),(11,11)$. Suppose we aim to form 3 clusters from these points using the K-Means algorithm. After the first iteration, clusters $C_1, C_2, C_3$ have the following data points:

$C_1$: $(1, 2), (3, 1)$
$C_2$: $(2, 3), (5, 6), (6,7)$
$C_3$: $(9, 9), (7, 8), (10, 10), (11, 11)$

After the second iteration, to which cluster will the data point $(2, 3)$ be reassigned?

Which of the following statements are true regarding applying standard PCA on a centered dataset?

Consider a dataset that has 200 data-points, each of which is either a zero or one. The Bernoulli distribution is used to model this data. If the MLE estimate for the parameter p of the Bernoulli distribution is 0.3, how many zeros does the dataset have?

Suppose you have been given a task of estimating the conversion rate of google click of new online advertisement campaign. You have collected data from a limited sample of $50$ users, where $15$ of them have converted. You want to use Bayesian estimation with a prior distribution to provide a more robust estimate of the conversion rate.

Assume you have prior information and you decide to use a beta distribution as your prior. You choose a beta distribution with parameters $\alpha = 3$ and $\beta = 4$ to capture your prior beliefs.

Calculate the posterior mean?

Enter your answer correctly up to two decimals

A k-means++ algorithm with k= 3 was applied to the following 2D points:
(2, 2), (3, 3), (4, 4), (10, 10), (12, 12), (13, 13)
(2, 2) is chosen as the first cluster mean.
Based on the above data, answer the given subquestions.

Which point has the highest probability of being chosen as the 2*^(nd)* cluster mean? Use the Manhattan distance to compute the distances.

A k-means++ algorithm with k= 3 was applied to the following 2D points:
(2, 2), (3, 3), (4, 4), (10, 10), (12, 12), (13, 13)
(2, 2) is chosen as the first cluster mean.
Based on the above data, answer the given subquestions.

Let’s assume that (3, 3), (10, 10), and (12, 12) are the cluster means. You introduce a new data point, (−5, 10). Which cluster mean is the data point (−5, 10) closest to when using the Euclidean distance method?

Suppose that we have a dataset (data matrix) $X \in \mathbb{R}^{dxn}$, where $d$ denotes the number of features and $n$ denotes the number of samples. Assume that $d >> n$ and the data is centered. The representative line is given by the unit vector, pointing in the direction of the line, $\mathbf{w}$.

In all the cases, we maximize the objective function given below

$f(\mathbf{w}) = \mathbf{w}^T C \mathbf{w}$

where $C = \frac{1}{n} X X^T$ is a covariance matrix. The kernel matrix is defined to be $K = X^T X$.

Based on the above data, answer the given subquestions.

Which of the following statements are true?

Suppose that we have a dataset (data matrix) $X \in \mathbb{R}^{dxn}$, where $d$ denotes the number of features and $n$ denotes the number of samples. Assume that $d >> n$ and the data is centered. The representative line is given by the unit vector, pointing in the direction of the line, $\mathbf{w}$.

In all the cases, we maximize the objective function given below

$f(\mathbf{w}) = \mathbf{w}^T C \mathbf{w}$

where $C = \frac{1}{n} X X^T$ is a covariance matrix. The kernel matrix is defined to be $K = X^T X$.

Based on the above data, answer the given subquestions.

Which of the following matrices helps in transforming the eigenvectors of K to the eigenvectors of C?

Suppose that we have a dataset (data matrix) $X \in \mathbb{R}^{dxn}$, where $d$ denotes the number of features and $n$ denotes the number of samples. Assume that $d >> n$ and the data is centered. The representative line is given by the unit vector, pointing in the direction of the line, $\mathbf{w}$.

In all the cases, we maximize the objective function given below

$f(\mathbf{w}) = \mathbf{w}^T C \mathbf{w}$

where $C = \frac{1}{n} X X^T$ is a covariance matrix. The kernel matrix is defined to be $K = X^T X$.

Based on the above data, answer the given subquestions.

Suppose that the 95% of the variance of the data is captured using d − 1 eigenvectors. This implies that