The IIT Madras BS Deep Learning (Deep Learning) Quiz 2 paper sat on 24 Mar 2024, in the January 2024 term: 16 questions for 50 marks in 120 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.
What are the possible outcomes if we don’t do bias correction in ADAM?
The Algorithm will not converge to a local or global minima.
It would lead to initial updates that are much smaller.
It would lead to initial updates which are much larger.
Correct answer
It would lead to initial updates which are much larger.
Consider the assertion and the corresponding reason,
Assertion (A): The RMSProp Algorithm starts oscillating around the minima.
Reason (R): RMSProp is sensitive to initial learning rate value.
A is true but R is not the correct reason
A is True and R is the correct reason
A is False and therefore R
Correct answer
A is true but R is not the correct reason
Suppose you have a deep neural network trained on a dataset with a small amount of labeled examples. After applying L2 regularization during training with a fixed value of regularization parameter, the following observations are made:
• The empirical error decreases significantly.
• The true error remains high.
• The variance of the error on different subsets of the training data is relatively low.
• When tested on a diverse set of unseen data, the model’s performance is inconsistent. Which of the following statements provides a plausible explanation for this scenario?
The model is underfitting due to excessive regularization.
L2 regularization effectively reduces both overfitting and underfitting.
The inconsistency in performance on unseen data is expected due to the limited labeled examples in the training set.
The model is still overfitting to the training data, and the regularization parameter needs adjustment.
Correct answer
The model is still overfitting to the training data, and the regularization parameter needs adjustment.
What are the maximum and minimum possible values of the derivative of sigmoid function?
1, 0
0.5, 0
0.5, 0.25
0.25, 0
Correct answer
0.25, 0
Consider a feed-forward neural network containing three inputs and one output. It contains 50 hidden layers, each containing two neurons. The activation function used in all the hidden layers is ReLU, the output layer uses the Sigmoid activation function for binary classification. The network is trained with a binary cross-entropy loss function.
Assume all weights are initialized to 1 and all biases to 0. For an input of [-10, 5, -20], what will be the output of the neural network?
0
1
0.5
0.25
-25
Insufficient information
Correct answer
0.5
Suppose we have a neural network with two hidden layers, each containing two neurons. The network contains two inputs and one output.
Weights: wij = 1 for all weights
Biases: bi = 0 for all biases
Input: x = [10, -1].
We consider the following two scenarios:
Scenario 1: We use Maxout activation for the hidden layers with k = 2 linear functions. Scenario 2: We use ReLU activation with dropout with a dropout probability of 0.5
Which of the following is true concerning the outputs in each scenario?
The outputs of both scenarios will be the same.
The output of Scenario 1 is expected to be greater than that of Scenario 2.
The output of Scenario 1 is expected to be lesser than that of Scenario 2.
Correct answer
The output of Scenario 1 is expected to be greater than that of Scenario 2.
Consider a neural network with one hidden layer containing two neurons. The input layer contains 2 neurons and the output layer contains one neuron. ReLU activation is used for the hidden layers and sigmoid is used for the output layer.
Considering weights wij = 0.5 for all weights and biases bi = 0 for all biases. Assuming a dropout probability of 0.5, if an input vector x = [1, -2] is fed to this network, which of the following seems to be the expected output?
0
0.5
0.25
1
Correct answer
0.5
Which of the following statements are true w.r.t. AdaDelta Algorithm?
In a surface with steep curvature, both vt and ut decrease which stops learning rate from reducing aggressively.
At low curvature regions vt starts decreasing and learning rate starts increasing.
AdaDelta allows numerator to increase or decrease based on current gradients only.
AdaDelta converges quicker than RMSProp Algorithm.
Correct answers
At low curvature regions vt starts decreasing and learning rate starts increasing.
AdaDelta converges quicker than RMSProp Algorithm.
After training a neural network using vanilla gradient descent, you notice that it performs exceptionally well on the training set (99% accuracy) but poorly on the test set (50% accuracy). Which of the following techniques could be employed to address this overfitting issue and improve generalization performance?
RMSprop optimizer
Dropout
Data augmentation
Decaying learning rate
Correct answers
Dropout
Data augmentation
Which of the following optimization algorithms encounter problem of oscillating around the minima?
AdaGrad
RMSProp
Nesterov Accelerated Gradient Descent
ADAM
Momentum Based Gradient Descent
Correct answers
RMSProp
Nesterov Accelerated Gradient Descent
ADAM
Momentum Based Gradient Descent
Consider a network where sigmoid activation is used at all hidden layers. Which of the following will be true?
Initializing the weights to very low values causes the gradients to vanish.
Initializing the weights to very high values causes the gradients to vanish.
The weight updates are possible only in the direction of the first or third quadrant.
The weight updates are possible only in the direction of the second or fourth quadrant.
Correct answers
Initializing the weights to very high values causes the gradients to vanish.
The weight updates are possible only in the direction of the first or third quadrant.
Consider an Input matrix X of shape 4×4 and the kernel (filter) of shape 3×3 with a bias b = 1. Assume, stride equals 1 and no padding.
Note: In all your calculations, truncate the result to 2 decimal points, that is, if the intermediate result is 2.34434, then take it as 2.34 for the next step.
Based on the above data answer the given subquestions.
Compute the convolution sum by sliding the kernel over the input block.
where, denotes the convolution operation. Enter the sum of elements in A. Enter your answer correct to two decimal points.
Correct answer: 1
Consider an Input matrix X of shape 4×4 and the kernel (filter) of shape 3×3 with a bias b = 1. Assume, stride equals 1 and no padding.
Note: In all your calculations, truncate the result to 2 decimal points, that is, if the intermediate result is 2.34434, then take it as 2.34 for the next step.
Based on the above data answer the given subquestions.
The feature map A is then passed through a max pooling layer with the kernel size of 2 × 2. The result is then passed as an input to the sigmoid (logistic) function.
What is the output value y? Enter your answer to two decimal places.
Correct answer: 0.88 (accepted within ±0.02)
Consider an Input matrix X of shape 4×4 and the kernel (filter) of shape 3×3 with a bias b = 1. Assume, stride equals 1 and no padding.
Note: In all your calculations, truncate the result to 2 decimal points, that is, if the intermediate result is 2.34434, then take it as 2.34 for the next step.
Based on the above data answer the given subquestions.
Suppose the gradient of the loss with respect to is -1 (that is, ). What will be the value of , where is the element of kernel at the position.
Correct answer: 0
An MRI image of size 500 × 500 × 10 is feed as an input to a convolutional layer. Suppose that we want to produce an output of size 241 × 241 × 5 using a set of kernels. Suppose the kernel size is 20 × 20 × 10.
Answer the given subquestions
What is the value of stride s?. Assume zero padding (p = 0)
Correct answer: 2
An MRI image of size 500 × 500 × 10 is feed as an input to a convolutional layer. Suppose that we want to produce an output of size 241 × 241 × 5 using a set of kernels. Suppose the kernel size is 20 × 20 × 10.
Answer the given subquestions
How many parameters (excluding bias) are there in the convolutional layer?
Correct answer: 20000
