The IIT Madras BS Deep Learning (Deep Learning) End Term paper sat on 21 Dec 2025, in the September 2025 term: 32 questions for 79 marks in 180 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.
What is a co-occurrence matrix in the context of Natural Language Processing?
A matrix representing similarity scores between documents
A matrix where rows and columns represent words, and each cell indicates how frequently two words appear together in a context
A matrix that stores only the most frequent word in each document
A neural network used to compute word embeddings directly
Correct answer
A matrix where rows and columns represent words, and each cell indicates how frequently two words appear together in a context
What is the primary advantage of word embeddings compared to one-hot encoding?
Simpler implementation
Captures semantic relationships
They automatically correct spelling errors
Word embeddings are sparser than one-hot encoding
Correct answer
Captures semantic relationships
Which of the following is not true about multi-head cross attention?
Keys and values are taken from the encoder outputs
Queries come from the decoder outputs
All heads attend to identical features due to shared projection matrices
Each head learns to attend to different representation subspaces
Correct answer
All heads attend to identical features due to shared projection matrices
What is the primary advantage of Transformer encoders over RNN encoders for long sequences?
Lower memory usage
Parallelization during training
Smaller model size
Simpler architecture
Correct answer
Parallelization during training
0
Indeterminate
Correct answer
0
Correct answers
Cannot be determined
Correct answer
Which of the following optimizers use momentum (or momentum-like mechanisms) in updating gradients?
NAG
AdaGrad
RMSProp
AdaDelta
ADAM
Correct answers
NAG
ADAM
Regarding the specific gating mechanisms used to control information flow and gradients in LSTMs and GRUs, which statement is correct?
GRUs use a three-gate architecture (Input, Forget, Output) similar to LSTMs but remove the cell state
LSTMs couple the input and forget gates into a single 'update' gate, whereas GRUs keep them separate.
GRUs simplify the architecture by combining the forget and input gates into a single 'update' gate and merging the cell state and hidden state.
Both LSTMs and GRUs require a separate 'reset' gate to handle the exploding gradient problem explicitly.
Correct answer
GRUs simplify the architecture by combining the forget and input gates into a single 'update' gate and merging the cell state and hidden state.
Which of the following statements are true for the saturated neurons problem? Select all that apply.
ReLU activation function is preferred over tanh, as it is less likely to saturate.
Tanh is preferred over sigmoid as it solves the problem of neuron saturation.
Leaky ReLU mitigates the problem of neuron saturation compared to ReLU.
If a neuron saturates in a network, the weights do not get updated further.
Correct answers
ReLU activation function is preferred over tanh, as it is less likely to saturate.
Leaky ReLU mitigates the problem of neuron saturation compared to ReLU.
If a neuron saturates in a network, the weights do not get updated further.
0.1
0.05
0.025
0.0125
Correct answer
0.05
You are training a deep feedforward neural network (100 layers) for binary classification with a sigmoid output with tanh activation in some hidden layers and ReLU activations in some hidden layers. You notice that gradients in some layers vanish early, preventing those weights from updating, even though the network hasn’t converged. Which of the following fixes could help?
Increase the size of your training set
Replace ReLU activations with leaky ReLUs
Use appropriate weight initialization
Correct answers
Replace ReLU activations with leaky ReLUs
Use appropriate weight initialization
Why is the sigmoid activation function often preferred over the step activation function in neural networks?
The sigmoid is differentiable, allowing gradient-based optimization methods like backpropagation.
The step function is non-differentiable and does not support learning through gradient descent.
The sigmoid function completely avoids vanishing gradient problems.
The step function outputs values in the range (0,1), whereas the sigmoid outputs only discrete values {0,1}.
Correct answers
The sigmoid is differentiable, allowing gradient-based optimization methods like backpropagation.
The step function is non-differentiable and does not support learning through gradient descent.
Correct answer: 1152
Consider the sigmoid activation function defined as:
Correct answers
Correct answer: 143
Correct answer: 41
Correct answer: -1.5 (accepted within ±0.03)
Based on the above data, answer the given subquestions.
Correct answer: -7.5
Correct answer: 30
Based on the above data, answer the given subquestions.
Identify the word in the vocabulary with the highest predicted probability for the missing word (???), and provide its corresponding integer index (key) from the vocabulary list as your final answer.
Correct answer: 6
Based on the above data, answer the given subquestions.
Correct answer: 24
Correct answer: -4.00
Based on the above data, answer the given subquestions.
Correct answer
Identify the word in the vocabulary that has the highest probability of being predicted as a context word given the center word 'networks'. Provide the corresponding integer index (key) from the vocabulary list as your answer.
Correct answer: 4
Based on the above data, answer the given subquestions.
Correct answer
How many total weights from Win and Wout combined will be updated?
Correct answer: 18
Based on the above data, answer the given subquestions.
Correct answer: 1.705 (accepted within ±0.055)
The previous decoder hidden state is:
and
Based on the above data, answer the given subquestions.
Correct answer: 1.88 (accepted within ±0.04)
Based on the above data, answer the given subquestions.
Correct answer: 0.18 (accepted within ±0.05)
The previous decoder hidden state is:
and
Based on the above data, answer the given subquestions.
Correct answer: 0.43 (accepted within ±0.04)
The previous decoder hidden state is:
and
Based on the above data, answer the given subquestions.
Correct answer: 1.72 (accepted within ±0.04)
