Linear Statistical Models Quiz 2: 23 November 2025 (September 2025 term)

Cheat Sheet:

1. For a linear model:

$\underset{\sim}{y} = X\underset{\sim}{\beta} + \underset{\sim}{\epsilon}$

where, $\epsilon_i$'s are uncorrelated with mean 0 and variance $\sigma^2$.

• Residual sum of square; RSS $= ||\underset{\sim}{y} - X\underset{\sim}{\hat{\beta}}||^2$.
• $\hat{\sigma^2} = \dfrac{RSS}{n-2}$.
• Total sum of square; TSS $= ||\underset{\sim}{y} - \underset{\sim}{\overline{y}}||^2$.
• Coefficient of Determination; $R^2 = 1 - \dfrac{RSS}{TSS}$.

1. In hypothesis testing, if p-value is less than the significance level $\alpha$, i.e. (p-value $< \alpha$), then there is evidence to reject the null hypothesis.

1. Consider a simple linear model $(\underset{\sim}{Y}, X\underset{\sim}{\beta}, \sigma^2 I)$ given by

$y_i = \beta_0 + \beta_1 x_i + \epsilon_i; \;\; 1 \leq i \leq n.$

where, $\epsilon_i \overset{\text{i.i.d.}}{\sim} N(0, \sigma^2)$.

The least square estimates of $\beta_0$ and $\beta_1$ are given as

$\hat{\beta_1} = \frac{\sum_{i=1}^{n}(x_i - \overline{x})y_i}{\sum_{i=1}^{n}(x_i - \overline{x})^2} \quad \text{and} \quad \hat{\beta_0} = \overline{y} - \hat{\beta_1}\overline{x}, \quad \text{respectively.}$

Prove that $\hat{\beta_1}$ and $\hat{\beta_0}$ are unbiased estimates of $\beta_1$ and $\beta_0$ respectively. [5 Marks]

1. Consider a linear model as

$y_i = \beta_0 + \epsilon_i \quad ; \quad 1 \leq i \leq n$

where, $\epsilon_i \overset{\text{i.i.d.}}{\sim} N(0, \sigma^2)$.

Let $\hat{\beta}_{01} = k(2y_1) + \dfrac{y_2}{4} + ky_3$ and $\hat{\beta}_{02} = \dfrac{y_1 + 2y_2 + 2y_3}{5}$ be two linear estimators of $\beta_0$, where $k$ is a constant.

(i) Find the value of $k$ such that $\hat{\beta}_{01}$ is an unbiased estimator of $\beta_0$. [3 Marks]

(ii) Which among the two estimators, $\hat{\beta}_{01}$ and $\hat{\beta}_{02}$, would you choose to use? Justify. [5 Marks]

1. A university professor wants to understand how students' performance in a Data Science course (measured by a quiz marks ‘$y$’) depend on their study habits. The Professor collects the data which contains the three variables : “Hours of self study per week”, “Hours spent watching recorded lectures” and “Quiz marks”. The target variable is “Quiz marks (out of 10)”, while the remaining two variables are predictors. Each variable has 5 observations.

Consider the following R code and output.

1x1 = c(1,2,3,4,5)
2x2 = c(1,3,2,4,5)
3y = c(2,4,5,4,5)
4model_1 = lm(y~x1)
5summary(model_1)
6model_2 = lm(y~x2)
7summary(model_2)
8model_3 = lm(y~x1+x2)
9summary(model_3)
10
11
12## Output-1
13Call:

1lm(formula = y ~ x1)
2
3Residuals:
4   1    2    3    4    5
5-0.8  0.6  1.0 -0.6 -0.2
6
7Coefficients:
8            Estimate Std. Error t value Pr(>|t|)
9(Intercept)   2.2000     0.9381   2.345    0.101
10x1            0.6000     0.2828   2.121    0.124
11
12Residual standard error: 0.8944 on 3 degrees of freedom
13Multiple R-squared:    0.6, Adjusted R-squared:  0.4667
14F-statistic:   4.5 on 1 and 3 DF,  p-value: 0.124
15
16## Output-2
17Call:
18lm(formula = y ~ x2)
19
20Residuals:
21         1          2          3          4          5
22-1.000e+00 -4.996e-16  1.500e+00 -5.000e-01  0.000e+00
23
24Coefficients:
25            Estimate Std. Error t value Pr(>|t|)
26(Intercept)   2.5000     1.1328   2.207    0.114
27x2            0.5000     0.3416   1.464    0.239
28
29Residual standard error: 1.08 on 3 degrees of freedom
30Multiple R-squared:  0.4167, Adjusted R-squared:  0.2222
31F-statistic: 2.143 on 1 and 3 DF,  p-value: 0.2394
32
33 ## Output 3
34
35 Call:
36lm(formula = y ~ x1 + x2)
37
38Residuals:
39      1       2       3       4       5
40-0.8421  0.7895  0.7895 -0.5789 -0.1579
41
42Coefficients:
43            Estimate Std. Error t value Pr(>|t|)
44(Intercept)   2.2632     1.1526   1.963    0.189
45x1            0.7895     0.7807   1.011    0.418
46x2           -0.2105     0.7807  -0.270    0.813
47
48Residual standard error: 1.076 on 2 degrees of freedom
49Multiple R-squared:  0.614, Adjusted R-squared:  0.2281
50F-statistic: 1.591 on 2 and 2 DF,  p-value: 0.386

Based on the given information, answer the following subquestions.

(i) Write the fitted linear regression model for all the different models. [3 Marks]

(ii) Write null and alternative hypotheses for the first model and check if we can reject the null hypothesis at 5% significance level, i.e., $\alpha = 0.05$. Also, what can you conclude about the relationship between the variables Hours of self study per week and Quiz marks. [3 Marks]

(iii) Interpret $R^2$ for the first two models and explain which model will perform better. [2 Marks]

1. Consider a linear model as

$y_i = \beta_1 x_i + \epsilon_i \quad ; \quad 1 \leq i \leq n$

where, $\epsilon_i \overset{\text{i.i.d.}}{\sim} N(0, \sigma^2)$.

(i). Find the value of $\beta_1$ for which RSS (Residual Sum of Squares) will be minimum. Provide all the steps. [3 Marks]

(ii). Find the value of $Var(\hat{\beta_1})$. [2 Marks ]

(iii). Suppose $\epsilon_i \overset{\text{i.i.d.}}{\sim} N(\mu, \sigma^2)$ for the model $y_i = \beta_1 x_i + \epsilon_i \;\; ; \;\; 1 \leq i \leq n$. Find bias$(\hat{\beta_1})$ [2 Marks]

1. Define Linear Zero Estimators. [2 Marks]