Linear Statistical Models Quiz 2: 16 March 2025 (January 2025 term)

1. Suppose $(\underset{\sim}{Y}, X\underset{\sim}{\beta}, \sigma^2 I_{n \times n})$ is a linear model and $p^T\underset{\sim}{\beta}$ is a linear estimable function, where $p \in \mathbb{R}^m$. If there exists a $l \in \mathbb{R}^n$, such that $l^T\underset{\sim}{Y}$ is a linear zero estimator, then which of the following holds? Select all that apply. [2 Marks]

(a) $l \in C(X)^\perp$

(b) $p \in \text{columnspace}(X)$

(c) $l \in \text{Null}(X^T)$

(d) $l \in \text{Null}(X)$

(e) $p \in \text{columnspace}(X^T X)$

1. Show that $\underset{\sim}{p}^T\underset{\sim}{\beta}$ is estimable, (where, $\underset{\sim}{p}^T\underset{\sim}{\beta} = p_1\beta_1 + p_2\beta_2 + \ldots + p_m\beta_m \;\forall\; p \in \mathbb{R}^m$) under the linear model $(\underset{\sim}{Y}, X\underset{\sim}{\beta}, \sigma^2 I)$ if and only if $p$ is orthogonal to the Null space of $X$. [4 Marks]

2. Consider a linear model as

$y_i = \beta_1 x_i + \epsilon_i \quad ; \quad 1 \leq i \leq n$

where, $\epsilon_i \sim N(0, \sigma^2)$.The $i$-th fitted values that result from performing this model is

$\hat{y_i} = x_i\hat{\beta},$

where

$\hat{\beta} = \frac{\sum_i^n x_i y_i}{\sum_i^n x_i^2}$

If $\hat{y_i} = \displaystyle\sum_{j=1}^{n} a_j y_j$, then find $a_j$? [3 marks]

1. Consider the following data points:

We fit a linear regression model

$y_i = \beta_0 + \beta_1 x_i + \epsilon_i, \qquad i : 1 \to 5,$

where $\epsilon_i \sim \text{Normal}(0, \sigma^2)$.

(a) Find the least square estimates of $\beta_0$ and $\beta_1$. [4 marks]

(b) Calculate the value of RSS (Residual Sum of Squares) [3 Marks]

1. Study the code and answer the following sub-questions:

1
2X = runif(10000,min = 0, max = 20)
3u = rnorm(10000, mean = 0, sd = 20)
4y = -2 + 6*X + u
5df1 = data.frame(X=X, Y=y)
6l1 = lm(Y~X, data = df1)
7l1
8df2 = df[sample(nrow(df1),100),]
9l2 = lm(Y~X, data = df2)
10l2
11
12## Output-1
13Call:
14lm(formula = Y ~ X, data = df1)
15
16Coefficients:
17(Intercept)            X
18     -1.771        5.981
19
20## Output-2
21Call:
22lm(formula = Y ~ X, data = df2)
23
24Coefficients:
25(Intercept)            X
26      3.278        5.515

(a) Explain the functionality of each line of the code. [4 Marks]

(b) Interpret the “Output-1” and “Output-2”. [2 Marks]

(c) Write the fitted linear regression model for both datasets, “df1” and “df2.” Do you observe any differences in the outputs? Provide your comments on the possible reason. [3 Marks]

1. Consider a linear model as

$y_i = \beta_1 x_i + \epsilon_i \quad ; \quad 1 \leq i \leq n$

where, $\epsilon_i \sim N(0, \sigma^2)$.

Find the value of $\beta_1$ for which RSS (Residual Sum of Squares) will be minimum. Write all the steps properly. [4 Marks]

1. A real estate analyst is studying the relationship between the size of a house ($x$, in square feet) and its selling price ( $y$, in thousands of dollars). The analyst collects data from $n$ houses in a city and wants to determine if there is a statistically significant relationship between the size of the house and its selling price.

The relationship between the variables is modeled as:

$y_i = \beta_0 + \beta_1 x_i + \epsilon_i \quad ; \quad 1 \leq i \leq n$

where:

• $y_i$ : Selling price of the $i^{th}$ house (in thousands of dollars).
• $x_i$ : Size of the $i^{th}$ house (in square feet).
• $\beta_0$ : The intercept, representing the average selling price of a house with size $x = 0$.
• $\beta_1$ : The slope, representing the change in selling price for every additional square foot of house size.
• $\epsilon_i$: Random error terms, assumed to follow a normal distribution $\epsilon_i \sim N(0, \sigma^2)$.

The following output has been obtained for a particular dataset :

1Call:
2lm(formula = y ~ x, data = house_data)
3
4Residuals:
5   Min     1Q Median     3Q    Max
6-6.458 -4.247 -2.036  2.631 12.253
7
8Coefficients:
9             Estimate Std. Error t value Pr(>|t|)
10(Intercept) 26.857367   7.262252   3.698  0.00606 **
11x            0.183111   0.002945  62.172 4.98e-12 ***
12---
13Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
14
15Residual standard error: 6.654 on 8 degrees of freedom
16Multiple R-squared:  0.9979,    Adjusted R-squared:  0.9977
17F-statistic:  3865 on 1 and 8 DF,  p-value: 4.98e-12

(a) Define the null and alternative hypotheses. [1 Mark]

(b) What are the estimates of $\beta_0$ and $\beta_1$. Interpret the meaning with respect to the given model. [2 Marks]

(c) Write the fitted linear regression model. [1 Mark]

(d) Based on the output obtained, justify if we can reject the null hypothesis. Also, what can you conclude about the relationship between $x$ (Size of house) and $y$ (Selling price)? [2 Marks]