Figure from the original question paper

(B) Option figure from the original question paper

(A) Option figure from the original question paper

(C) Option figure from the original question paper

(D) Option figure from the original question paper

Figure from the original question paper

(E) Option figure from the original question paper

(A) Option figure from the original question paper

(B) Option figure from the original question paper

(C) Option figure from the original question paper

(D) Option figure from the original question paper

You are experimenting with L1 and L2 regularization techniques on a deep neural network trained on a moderately sized dataset. You decide to compare their effects and make the following observations: - When using L1 regularization, the number of non-zero weights in the network is significantly reduced. - When using L2 regularization, the magnitudes of the weights are smaller, but most weights remain non-zero. - L1 regularization leads to a slightly higher empirical error on the training set compared to L2 regularization. - L2 regularization leads to better performance on the test set compared to L1 regularization. Which of the following is the most plausible explanation for these observations?

(A) L1 regularization induces sparsity, which can lead to feature selection, but might cause underfitting if too many features are removed.

(B) L2 regularization encourages smaller weights, which can improve generalization but might result in a more complex model.

(C) L1 regularization is generally inferior to L2 regularization in terms of generalization performance.

(D) The differences in performance are primarily due to the characteristics of L1 and L2 regularization, not the size of the dataset.

You are working on a project where you aim to predict housing prices based on various features. You have split your data into a training set and a test set. You decide to use neural network for this task and are experimenting with different regularization techniques (L1, L2).\ You have the following observations from your experiments: - Model A: Neural Network with L1 Regularization (Lasso) - Empirical Test Error: 0.25 - Train Error: 0.30 - Model B: Neural network with L2 Regularization (Ridge) - Empirical Test Error: 0.20 - Train Error: 0.27 - Model C: Neural Network without Regularization - Empirical Test Error: 0.67 - Train Error: 0.11 Given the observations from the experiments above, which of the following statements are correct? (Select all that apply)

(B) Model C shows signs of overfitting.

(C) Model B strikes a balance between bias and variance, making it a potentially good choice for regularization.

(A) Model C has the lowest train error, suggesting it generalizes the best.

(D) Model A has the highest train error, indicating it is underfitting the data.

May 2024 term · Deep Learning · BSCS3004

Deep Learning Quiz 2: 4 August 2024 (May 2024 term)

The IIT Madras BS Deep Learning (Deep Learning) Quiz 2 paper sat on 4 Aug 2024, in the May 2024 term: 16 questions for 50 marks in 120 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.

Questions: 16
Marks: 50
Duration: 120 min
MCQ: 11
MSQ: 1
Numerical: 4

Take as mock test Practise with answers

Updated 25 Sept 2026

Official paper: IIT M DEGREE AN EXAM QDB2 4 Aug 2024 · No negative marking.

Question 1

+3 marksOne correct option

Show answer

Correct answer

Question 1 on its own page

Question 2

+3 marksOne correct option

Which of the following statements accurately describes the behavior of the learning rate in the AdaGrad algorithm?

A
In each iteration, the learning rate monotonically increases.
B
In each iteration, the learning rate can increase, decrease, or remain constant.
C
In each iteration, the learning rate monotonically decreases, although it may remain constant for some periods.
D
The learning rate remains constant throughout the iterations.

Show answer

Correct answer

C
In each iteration, the learning rate monotonically decreases, although it may remain constant for some periods.

Question 2 on its own page

Question 3

+3 marksOne correct option

Show answer

Correct answer

Question 3 on its own page

Question 4

+3 marksOne correct option

What is the primary purpose of using cosine annealing with warm restarts in optimization algorithms?

A
Cosine annealing with warm restarts aims to maintain a constant learning rate throughout the optimization process.
B
Cosine annealing with warm restarts is a technique to dynamically adjust the learning rate based on the cosine function, allowing for smoother optimization trajectories.
C
Cosine annealing with warm restarts helps prevent the optimization process from getting stuck in sharp minima by periodically restarting with a higher learning rate.
D
Cosine annealing with warm restarts is a method to gradually decrease the learning rate over iterations, allowing for more precise convergence to the optimal solution.

Show answer

Correct answer

C
Cosine annealing with warm restarts helps prevent the optimization process from getting stuck in sharp minima by periodically restarting with a higher learning rate.

Question 4 on its own page

Question 5

+3 marksOne correct option

How does dropout affect the behavior of the neural network during the inference phase?

A
All neurons are dropped randomly as in the training phase.
B
Only a subset of neurons is kept active, and their outputs are scaled by the keep probability.
C
All neurons are active, and their outputs are typically scaled by the keep probability to maintain consistency with the training phase.
D
Dropout is not applied during the inference phase, and no scaling is necessary.

Show answer

Correct answer

C
All neurons are active, and their outputs are typically scaled by the keep probability to maintain consistency with the training phase.

Question 5 on its own page

Question 6

+3 marksOne correct option

Which of the following options correctly describes the effect of early stopping as a regularization technique in training ANNs?

A
It increases the number of epochs until the validation error starts to increase.
B
It stops training when the validation error stops decreasing and begins to increase.
C
It reduces the learning rate gradually as training progresses.
D
It adds a penalty to the loss function based on the weight values.

Show answer

Correct answer

B
It stops training when the validation error stops decreasing and begins to increase.

Question 6 on its own page

Question 7

+3 marksOne correct option

You are experimenting with L1 and L2 regularization techniques on a deep neural network trained on a moderately sized dataset. You decide to compare their effects and make the following observations:

When using L1 regularization, the number of non-zero weights in the network is significantly reduced.
When using L2 regularization, the magnitudes of the weights are smaller, but most weights remain non-zero.
L1 regularization leads to a slightly higher empirical error on the training set compared to L2 regularization.
L2 regularization leads to better performance on the test set compared to L1 regularization.

Which of the following is the most plausible explanation for these observations?

A
L1 regularization induces sparsity, which can lead to feature selection, but might cause underfitting if too many features are removed.
B
L2 regularization encourages smaller weights, which can improve generalization but might result in a more complex model.
C
L1 regularization is generally inferior to L2 regularization in terms of generalization performance.
D
The differences in performance are primarily due to the characteristics of L1 and L2 regularization, not the size of the dataset.

Show answer

Correct answer

A
L1 regularization induces sparsity, which can lead to feature selection, but might cause underfitting if too many features are removed.

Question 7 on its own page

Question 8

+3 marksOne correct option

What is the impact of using a sigmoid activation function on the gradients when the input to the neuron is a very large negative number?

A
The output approaches 0 and the gradient approaches 0.
B
The output approaches 0.5 and the gradient approaches 0.25.
C
The output approaches 1 and the gradient approaches 0.
D
The output approaches 0 and the gradient approaches 1.

Show answer

Correct answer

A
The output approaches 0 and the gradient approaches 0.

Question 8 on its own pageAlso asked in Quiz 2 16 Mar 2025

Question 9

+3 marksOne correct option

What is the derivative of the Leaky ReLU activation function with respect to its input?

A
0 for negative inputs, 1 for positive inputs.
B
1 for negative inputs, 0 for positive inputs.
C
0.01 for negative inputs, 1 for positive inputs.
D
0.05 for negative inputs, 1 for positive inputs.
E
1 for all inputs.

Show answer

Correct answer

C
0.01 for negative inputs, 1 for positive inputs.

Question 9 on its own pageAlso asked in Quiz 2 1 Dec 2024

Question 10

+3 marksOne correct option

Which of the following do not suffer from the vanishing gradient problem as well as the dead neurons problem?
1. Sigmoid
2. Tanh
3. ReLU
4. Leaky ReLU

A
Only 1
B
Only 2
C
Only 3
D
Only 4
E
Both 3 and 4
F
Both 1 and 2

Show answer

Correct answer

D
Only 4

Question 10 on its own page

Question 11

+3 marksOne correct option

Both sigmoid and tanh activation functions saturate:

A
near 0
B
near 1
C
at large positive values
D
at large negative values
E
Both at large positive values and at large negative values

Show answer

Correct answer

E
Both at large positive values and at large negative values

Question 11 on its own page

Question 12

+3 marksOne or more correct options

You are working on a project where you aim to predict housing prices based on various features. You have split your data into a training set and a test set. You decide to use neural network for this task and are experimenting with different regularization techniques (L1, L2).
You have the following observations from your experiments:

Model A: Neural Network with L1 Regularization (Lasso)
- Empirical Test Error: 0.25
- Train Error: 0.30
Model B: Neural network with L2 Regularization (Ridge)
- Empirical Test Error: 0.20
- Train Error: 0.27
Model C: Neural Network without Regularization
- Empirical Test Error: 0.67
- Train Error: 0.11

Given the observations from the experiments above, which of the following statements are correct? (Select all that apply)

Select all that apply.

A
Model C has the lowest train error, suggesting it generalizes the best.
B
Model C shows signs of overfitting.
C
Model B strikes a balance between bias and variance, making it a potentially good choice for regularization.
D
Model A has the highest train error, indicating it is underfitting the data.

Show answer

Correct answers

B
Model C shows signs of overfitting.
C
Model B strikes a balance between bias and variance, making it a potentially good choice for regularization.

Question 12 on its own page

Question 13

+3 marksNumerical answer

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 1

Question 13 on its own page

Question 14

+3 marksNumerical answer

Based on the above data, answer the given subquestions.

Pass $a$ through the ReLU activation function and assign the output to $h$ .

$ReLU(x) = \begin{cases} x, & if\ x \ge 0 \\ 0, & \text{otherwise} \end{cases}$

Compute the output $\hat{y}$ by averaging $h$ and calculate the squared error loss

$L = 0.5(\hat{y} - y)^2$

Enter the loss value

Show answer

Correct answer: 0.09 (accepted within ±0.01)

Question 14 on its own page

Question 15

+4 marksNumerical answer

Consider a Convolutional Neural Network (CNN) architecture for image classification with the following layers:
1. Convolutional layer with 6 filters of size 3 × 3, with a stride of 1 and no padding.
2. Max pooling layer with a pool size of 2 × 2 and a stride of 2.
3. Convolutional layer with 4 filters of size 4 × 4, with a stride of 1 and no padding.
4. Max pooling layer with a pool size of 2 × 2 and a stride of 2.
5. Fully connected layer with 20 neurons.
6. Output layer with 10 neurons (for 10 classes) using softmax activation.
If the input image size is 64 × 64 × 3, and the network has no bias term, how many parameters are there in the CNN?

Show answer

Correct answer: 16426

Question 15 on its own page

Question 16

+4 marksNumerical answer

Consider an intermediate feature map $H$ obtained after applying convolution operation on the input $X$ using kernel $F$ .

$H = \begin{bmatrix} 3 & 2 & 2 & 0 \\ 1 & 2 & 1 & 1 \\ 3 & 1 & -1 & 2 \\ 1 & 0 & 1 & 2 \end{bmatrix}$

Apply the max-pooling operation using a filter of size $2 \times 2$ with stride $s = 2$ and no padding ( $p = 0$ ) and store the resultant output in matrix $H_m$ . The prediction $\hat{y}$ is simply the sum of elements in $H_m$ . Suppose $\frac{\partial L}{\partial \hat{y}} = 1$ (that is, the gradient of loss with respect to the prediction). What is the gradient $\frac{\partial L}{\partial H_{00}}$ where $H_{00}$ is the element at the 0-th row and 0-th column? If you think the given info is insufficient, enter -1 as the answer.

Show answer

Correct answer: 1

Question 16 on its own page