Figure from the original question paper

(A) The weight update is tiny at a gentle loss surface.

(C) The weight update is large at a steep loss surface.

(B) The weight update is tiny at a steep loss surface.

(D) The weight update is large at a gentle loss surface.

Deep Learning Quiz 1 PYQ Jan 2025: 23 Feb 2025 · IITM BS

Question 1

+3 marksOne correct option

Which of the following statements correctly distinguishes between Momentum Optimizer and Nesterov Accelerated Momentum (NAG) Optimizer?

A
Momentum uses the current gradient to update the velocity, while NAG uses a lookahead gradient based on a tentative future position.
B
Momentum is computationally faster than NAG because it skips calculating the gradient at a lookahead position.
C
NAG is a variant of momentum that adjusts the step size dynamically during training.
D
Momentum uses an adaptive learning rate, whereas NAG uses a fixed learning rate.

Show answer

Correct answer

A
Momentum uses the current gradient to update the velocity, while NAG uses a lookahead gradient based on a tentative future position.

Question 1 on its own page

Question 2

+3 marksOne correct option

Which of the functions given below has the steepest slope at the point x = 1?

A
B
C
D

Show answer

Correct answer

D

Question 2 on its own page

Question 3

+3 marksOne or more correct options

Select all that apply.

A
The weight update is tiny at a gentle loss surface.
B
The weight update is tiny at a steep loss surface.
C
The weight update is large at a steep loss surface.
D
The weight update is large at a gentle loss surface.

Show answer

Correct answers

A
The weight update is tiny at a gentle loss surface.
C
The weight update is large at a steep loss surface.

Question 3 on its own page

Question 4

+3 marksNumerical answer

Consider a network trained on a dataset of 5000 data-points using mini-batch gradient descent with a batch size of 50. If the network is trained for 100 epochs, how many times does each parameter in the network get updated?

Show answer

Correct answer: 10000

Question 4 on its own page

Question 5

+2 marksNumerical answer

Consider a perceptron for a binary classification task. There are four features each of which is binary. The dataset has 16 data-points. The feature vector for no two data-points is the same. The weight vector is w = (1, 1,−1,−1). Ignore the bias.

If the number of misclassifications is zero on this dataset, answer the given subquestions.

Find the number of data-points in the dataset that have true label 1.

Show answer

Correct answer: 11

Question 5 on its own page

Question 6

+2 marksNumerical answer

Consider a perceptron for a binary classification task. There are four features each of which is binary. The dataset has 16 data-points. The feature vector for no two data-points is the same. The weight vector is w = (1, 1,−1,−1). Ignore the bias.

If the number of misclassifications is zero on this dataset, answer the given subquestions.

Find the number of data-points in the dataset that have true label 0.

Show answer

Correct answer: 5

Question 6 on its own page

Question 7

+2 marksOne correct option

Based on the above data, answer the given subquestions.

A
(−0.8, 0.8)
B
(0.08,−0.08)
C
(−0.08, 0.08)
D
(1,−1)

Show answer

Correct answer

A
(−0.8, 0.8)

Question 7 on its own page

Question 8

+2 marksNumerical answer

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 180

Question 8 on its own page

Question 9

+2 marksOne or more correct options

Consider a dataset for a binary classification problem in which the labels are in $\{0,1\}$ :

$x_1$	$x_2$	$y$
1	1	1
$-1$	1	1
1	$-1$	0

A perceptron is trained on this dataset using the perceptron learning algorithm starting with the initial weight vector as $(0,0)$ . Ignore the bias. The data-points are processed in the order in which they appear in the table, from top to bottom. If $\mathbf{w}$ is the weight vector, use the following rule for prediction:

$\hat{y} = \begin{cases} 1, & \mathbf{w}^T\mathbf{x} \geqslant 0 \\ 0, & \mathbf{w}^T\mathbf{x} < 0 \end{cases}$

The perceptron converges to $\mathbf{w}^* = (w_1^*, w_2^*)$ after $k$ updates to the weight vector. Note that the number of iterations of the training loop is not the same as the number of updates to the weight vector.

Based on the above data, answer the given subquestions.

Select all that apply.

A
(0, 0)
B
(0, 1)
C
(−1, 1)
D
(1, 1)

Show answer

Correct answer

C
(−1, 1)

Question 9 on its own page

Question 10

+2 marksNumerical answer

Consider a dataset for a binary classification problem in which the labels are in $\{0,1\}$ :

$x_1$	$x_2$	$y$
1	1	1
$-1$	1	1
1	$-1$	0

A perceptron is trained on this dataset using the perceptron learning algorithm starting with the initial weight vector as $(0,0)$ . Ignore the bias. The data-points are processed in the order in which they appear in the table, from top to bottom. If $\mathbf{w}$ is the weight vector, use the following rule for prediction:

$\hat{y} = \begin{cases} 1, & \mathbf{w}^T\mathbf{x} \geqslant 0 \\ 0, & \mathbf{w}^T\mathbf{x} < 0 \end{cases}$

The perceptron converges to $\mathbf{w}^* = (w_1^*, w_2^*)$ after $k$ updates to the weight vector. Note that the number of iterations of the training loop is not the same as the number of updates to the weight vector.

Based on the above data, answer the given subquestions.

What is the value of k?

Show answer

Correct answer: 1

Question 10 on its own page

Question 11

+2 marksOne or more correct options

Consider a dataset for a binary classification problem in which the labels are in $\{0,1\}$ :

$x_1$	$x_2$	$y$
1	1	1
$-1$	1	1
1	$-1$	0

A perceptron is trained on this dataset using the perceptron learning algorithm starting with the initial weight vector as $(0,0)$ . Ignore the bias. The data-points are processed in the order in which they appear in the table, from top to bottom. If $\mathbf{w}$ is the weight vector, use the following rule for prediction:

$\hat{y} = \begin{cases} 1, & \mathbf{w}^T\mathbf{x} \geqslant 0 \\ 0, & \mathbf{w}^T\mathbf{x} < 0 \end{cases}$

The perceptron converges to $\mathbf{w}^* = (w_1^*, w_2^*)$ after $k$ updates to the weight vector. Note that the number of iterations of the training loop is not the same as the number of updates to the weight vector.

Based on the above data, answer the given subquestions.

Select all that apply.

A
B
C
D

Show answer

Correct answers

A
B

Question 11 on its own page

Question 12

+1 markNumerical answer

Consider a sigmoid neuron that takes in an input vector $\mathbf{x} = \begin{bmatrix} 1 \\ 0.5 \\ 1 \end{bmatrix}$ . The weight vector $\mathbf{w}$ is initialized to $\begin{bmatrix} 0.5 \\ 1 \\ 0.5 \end{bmatrix}$ and $b = 1$ . The output of the sigmoid neuron is

$z = \mathbf{w}^T\mathbf{x} + b$

$\hat{y} = \frac{1}{1 + e^{-z}}$

Suppose that we use the following loss function:

$\mathcal{L} = \frac{1}{2}(y - \hat{y})^2$

Based on the above data, answer the given subquestions.

Calculate the value of z (correct up to two decimal places).

Show answer

Correct answer: 2.5 (accepted within ±0.03)

Question 12 on its own page

Question 13

+1 markNumerical answer

Consider a sigmoid neuron that takes in an input vector $\mathbf{x} = \begin{bmatrix} 1 \\ 0.5 \\ 1 \end{bmatrix}$ . The weight vector $\mathbf{w}$ is initialized to $\begin{bmatrix} 0.5 \\ 1 \\ 0.5 \end{bmatrix}$ and $b = 1$ . The output of the sigmoid neuron is

$z = \mathbf{w}^T\mathbf{x} + b$

$\hat{y} = \frac{1}{1 + e^{-z}}$

Suppose that we use the following loss function:

$\mathcal{L} = \frac{1}{2}(y - \hat{y})^2$

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 0.92 (accepted within ±0.03)

Question 13 on its own page

Question 14

+1 markNumerical answer

Consider a sigmoid neuron that takes in an input vector $\mathbf{x} = \begin{bmatrix} 1 \\ 0.5 \\ 1 \end{bmatrix}$ . The weight vector $\mathbf{w}$ is initialized to $\begin{bmatrix} 0.5 \\ 1 \\ 0.5 \end{bmatrix}$ and $b = 1$ . The output of the sigmoid neuron is

$z = \mathbf{w}^T\mathbf{x} + b$

$\hat{y} = \frac{1}{1 + e^{-z}}$

Suppose that we use the following loss function:

$\mathcal{L} = \frac{1}{2}(y - \hat{y})^2$

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 0.42 (accepted within ±0.03)

Question 14 on its own page

Question 15

+3 marksNumerical answer

Consider a sigmoid neuron that takes in an input vector $\mathbf{x} = \begin{bmatrix} 1 \\ 0.5 \\ 1 \end{bmatrix}$ . The weight vector $\mathbf{w}$ is initialized to $\begin{bmatrix} 0.5 \\ 1 \\ 0.5 \end{bmatrix}$ and $b = 1$ . The output of the sigmoid neuron is

$z = \mathbf{w}^T\mathbf{x} + b$

$\hat{y} = \frac{1}{1 + e^{-z}}$

Suppose that we use the following loss function:

$\mathcal{L} = \frac{1}{2}(y - \hat{y})^2$

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 0.39 (accepted within ±0.03)

Question 15 on its own page

Question 16

+1 markNumerical answer

Based on the above data, answer the given subquestions.

What is the value of the output of the first sigmoid neuron in the first hidden layer (i.e. h11). (Answer correct upto two digits after the decimal)

Show answer

Correct answer: 0.95 (accepted within ±0.04)

Question 16 on its own page

Question 17

+1 markNumerical answer

Based on the above data, answer the given subquestions.

What is the value of the output of the first sigmoid neuron in the second hidden layer (i.e. h21) (Answer correct upto two digits after the decimal)

Show answer

Correct answer: 0.67 (accepted within ±0.04)

Question 17 on its own page

Question 18

+1 markNumerical answer

Based on the above data, answer the given subquestions.

What is the value of the output of the first neuron in the third or output layer with the softmax activation function (i.e. h31) (Answer correct upto two digits after the decimal)

Show answer

Correct answer: 0.26 (accepted within ±0.04)

Question 18 on its own page

Question 19

+1 markNumerical answer

Based on the above data, answer the given subquestions.

What is the value of the cross entropy loss? (use natural log). (Answer correct upto two digits after the decimal)

Show answer

Correct answer: 1.33 (accepted within ±0.04)

Question 19 on its own page

Question 20

+2 marksNumerical answer

Based on the above data, answer the given subquestions.

Show answer

Correct answer: -0.5 (accepted within ±0.04)

Question 20 on its own page

Question 21

+2 marksNumerical answer

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 0.15 (accepted within ±0.04)

Question 21 on its own page