Linear Statistical Models Quiz 1: 23 February 2025 (January 2025 term)

1. A linear regression model $\hat{y} = a_0 + a_1 x$ is fit for the cars dataset. The dataset has 50 observations with 2 variables dist and speed. Study the code and answer the following subquestions:

(a) Find the best fit line $\hat{y}$. [2 Marks]

(b) Find the residual corresponding to the data point $(x, y) = (4, 2)$. [3 Marks]

1. Let $X_1, X_2, \ldots, X_n$ be i.i.d. samples from a normal distribution with mean $\mu$ and variance $\sigma^2$. Consider the sample mean and sample variance as

$\overline{X} = \frac{1}{n}\sum_{i=1}^{n} X_i, \qquad\qquad S^2 = \frac{1}{(n-1)}\sum_{i=1}^{n}(X_i - \overline{X})^2,$

then which of the following are true? Select all that apply [3 Marks]

(a) $\overline{X} \sim \text{Normal}(\mu, \sigma^2/n)$

(b) $\overline{X} \sim \text{Normal}(\mu, n\sigma^2)$

(c) $\dfrac{(n-1)S^2}{\sigma^2} \sim \chi^2_n$

(d) $\dfrac{\sqrt{n}(\overline{X} - \mu)}{S} \sim t_n$

(e) $\dfrac{\sqrt{n}(\overline{X} - \mu)}{S} \sim t_{(n-1)}$

1. Consider the first 10 rows of the iris dataset as below:

(a) What will be the output of iris[7, 5]? [1 Mark]

(b) What will be the output of the following code snippet? [2 Marks]

(c) The iris dataset contains a total of 150 observations. Consider the following output:

Then, find the population variance of the variable Sepal.Length. [2 Marks]

1. Consider the linear model :

$y_{ij} = \alpha + \beta x_{ij} + \gamma_i + \epsilon_{ij} \quad ; \quad 1 \leq j \leq n_i \, , \, 1 \leq i \leq 3,$

where :

• $\gamma_i \sim N(0, \sigma^2_\gamma)$ is a random effect for group $i$, independent of $\epsilon_{ij}$.
• $\epsilon_{ij} \sim N(0, \sigma^2_\epsilon)$ are independent errors.
• $x_{ij}$ is a known covariate.

Find the distribution of $y_{ij}$. [2 Marks]

1. Suppose you fit a linear regression model to the following data:

and the estimated regression equation is : $\hat{\vec{y}} = 2.2 + 0.6x$. If the prediction errors be defined by $\vec{e}_i = \vec{y}_i - \hat{\vec{y}}_i$, then calculate the value of $\sum_{i=1}^{n}(\vec{e}_i)^2$. [4 Marks]

1. Let $(\vec{Y}, X\vec{\beta}, \sigma^2 I_n)$ be a linear model and $p \in \mathbb{R}^m$. Prove that $p^T\vec{\beta}$ is estimable $\Leftrightarrow p \in C(X^T)$. [4 Marks]

2. Consider a model:

$\begin{aligned} y_1 &= 2\beta_1 + 2\beta_2 + 2\beta_3 + 2\beta_4 + \epsilon_1 \,; \\ y_2 &= 2\beta_1 + 2\beta_3 - 2\beta_2 - 2\beta_4 + \epsilon_2 \,; \\ y_3 &= 2\beta_1 + 2\beta_2 - 2\beta_3 - 2\beta_4 + \epsilon_3 \,; \\ y_4 &= 2\beta_1 + 2\beta_4 - 2\beta_2 - 2\beta_3 + \epsilon_4 \,, \end{aligned}$

where $\beta_i \in R$ and $\epsilon_i$'s are uncorrelated random variables with variance $\sigma^2$; $i = 1, 2, 3, 4$.

(a) If we want to rewrite the model as $(\vec{Y}, X\vec{\beta}, I_{4\times4})$, then find $\vec{Y}, X, \vec{\beta}$ and $\vec{\epsilon}$. [2Marks]

(b) Define Normal equations and using normal equations find the least square estimates of $\beta_1, \beta_2, \beta_3$ and $\beta_4$. [5 Marks]