Question 3
You are evaluating the performance of a linear regression model using different hyperparameter settings (learning rate and number of iterations). Below are the results:
| Setting | Learning Rate | Iterations | Training Error | Validation Error |
|---|---|---|---|---|
| 1 | 0.05 | 1000 | 150 | 160 |
| 2 | 0.10 | 800 | 145 | 150 |
| 3 | 0.15 | 500 | 140 | 190 |
| 4 | 0.25 | 350 | 170 | 180 |
Which hyperparameter setting would you choose based on the training and validation errors? Why?
Setting 3, because it has the lowest training error, indicating the best fit to the training data.
Setting 2, because it has the lowest validation error and a relatively low training error, indicating a good generalization.
Setting 1, because it has the highest number of iterations and therefore likely to be more stable.
Setting 4, because it uses the highest learning rate and converges the fastest.