Linear Statistical Models Quiz 2: 3 December 2023 (September 2023 term)

1. State Gauss-Markov Theorem. [4 Marks]

2. Define BLUE. [3 Marks]

3. Literacy rate is a reflection of the educational facilities and quality of education available in a country, and mass communication plays a large part in the educational process. In an effort to relate the literacy rate of a country to various mass communication outlets, a demographer has proposed to relate literacy rate to the following variables: number of daily newspaper copies (per 10 population) and number of radios (per 10 population). Here are the data for a sample of 3 countries:

(a) Identify the dependent and independent variables. [1 Mark]

(b) Suppose we assume the linear model:

$\textit{Literacy rate} = \beta_0 + \beta_1 \times \textit{Radios} + \beta_2 \times Newspapers + \epsilon$

where, $\epsilon \sim N(0, \sigma^2)$

Find the least square estimate for $\vec{\beta}$. [5 Marks]

(c) Define and compute the RSS (Residual sum of squares) and TSS (Total sum of squares) for the above defined model. [4 Marks]

(d) Find the estimate value of $\sigma^2$ [2 Marks]

(e) Compute $R^2$ for the above defined model and interpret it. [4 Marks]

(f) Suppose we modify our linear model by deleting a variable as follows:

$\textit{Literacy rate} = \beta_0 + \beta_1 \times \textit{Radios} + \epsilon$

where, $\epsilon \sim N(0, \sigma^2)$

Compute $R^2$ for the modified model. [10 Marks]

(g) Compare the value of $R^2$ for the two models. Elaborate your answer. [2 Marks]

1. Consider a linear model as

$y_i = \beta_0 + \beta_1 x_i + \epsilon_i \quad ; \quad 1 \leq i \leq n$

where, $\epsilon_i \sim N(0, \sigma^2)$.

For the given linear model, we want to perform hypothesis testing at 5% significance level, i.e. $\alpha = 0.05$, to check if there's a relationship between the variables $x$ and $y$. Based on the given information, answer the following questions:

(a) Write down the null and alternative hypothesis. [3 Marks]

(b) Using R, we fit the linear model:

$y_i = 1.5 + \epsilon_i \quad ; \quad 1 \leq i \leq n$

The output obtained is as follows:

1Call:
2lm(formula = y ~ x)
3
4Residuals:
5    Min      1Q  Median      3Q     Max
6-3.9053 -0.7122  0.0545  0.7265  2.3814
7
8Coefficients:
9            Estimate Std. Error t value Pr(>|t|)
10(Intercept)  1.58118    0.11591  13.642   <2e-16 ***
11x           -0.02647    0.01893  -1.399    0.165
12---
13Signif. codes:
140 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
15
16Residual standard error: 1.128 on 98 degrees of freedom
17Multiple R-squared:  0.01957,    Adjusted R-squared:  0.009564
18F-statistic: 1.956 on 1 and 98 DF,  p-value: 0.1651

Based on the output obtained, justify if we can reject the null hypothesis? Also, what can you conclude about the relationship between $x$ and $y$? [4 Marks]

(c) Using R, on the same dataset, now we fit the linear model:

$y_i = 1.5 + 2x_i + \epsilon_i \quad ; \quad 1 \leq i \leq n$

The output obtained is as follows:

1Call:
2lm(formula = y ~ x)
3
4Residuals:
5    Min      1Q  Median      3Q     Max
6-3.9053 -0.7122  0.0545  0.7265  2.3814
7
8Coefficients:
9            Estimate Std. Error t value Pr(>|t|)
10(Intercept)  1.58118    0.11591   13.64   <2e-16 ***
11x            1.97353    0.01893  104.26   <2e-16 ***
12---
13Signif. codes:
140 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
15
16Residual standard error: 1.128 on 98 degrees of freedom
17Multiple R-squared:  0.9911,    Adjusted R-squared:  0.991
18F-statistic: 1.087e+04 on 1 and 98 DF,  p-value: < 2.2e-16

Based on the output obtained, justify if we can reject the null hypothesis? Also, what can you conclude about the relationship between $x$ and $y$? [4 Marks]

(d) In which among the above two models, obtained in part $(b)$ and $(c)$, the independent variables collectively explain more variability in the dependent variable? Elaborate on your answer. [4 Marks]