The IIT Madras BS Introduction to Deep Learning and Generative AI (Deep Learning and GenAI) End Term paper sat on 10 May 2026, in the January 2026 term, set S2: 24 questions for 100 marks in 180 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.
Which of the following orders of operations is/are CORRECT for a single training iteration in PyTorch?
Forward → Backward → Zero gradients → Optimizer step
Forward → Zero gradients → Backward → Optimizer step
Zero gradients → Optimizer step → Forward → Backward
Zero gradients → Forward → Backward → Optimizer step
Backward → Forward → Optimizer step → Zero gradients
Correct answers
Forward → Zero gradients → Backward → Optimizer step
Zero gradients → Forward → Backward → Optimizer step
Which of the following is/are NOT present in a standard GPT (decoder-only) model?
Positional Embeddings
Word Embeddings
MultiHead Cross Attention Layer
MultiHead Self Attention Layer for the Encoder
MultiHead Self Attention Layer for the Decoder
Correct answers
MultiHead Cross Attention Layer
MultiHead Self Attention Layer for the Encoder
Consider the following RAG pipeline:
Which of the following is/are true?
The pipeline is incomplete as a RAG generation pipeline because no LLM is present to generate the final response.
The retriever and formatter will still be executed as part of the pipeline.
The pipeline correctly performs retrieval-augmented generation.
The pipeline behaves exactly the same as a complete RAG pipeline with an LLM.
Correct answers
The pipeline is incomplete as a RAG generation pipeline because no LLM is present to generate the final response.
The retriever and formatter will still be executed as part of the pipeline.
Consider the following PyTorch network:
Correct answer
Correct answer
Consider the following code snippet:
What will be the output?
Correct answer
Consider the following single-layer RNN:
10
20
30
40
Correct answer
20
3840, 5120
1920, 2560
6400, 8000
1280, 2560
Correct answer
3840, 5120
AD and AE
BD and CD
BD and BF
CD and CE
Correct answer
BD and BF
You are given:
• Image size: 64 × 64 • Patch size: 16 × 16 • Number of input channels: 3 (RGB) • Embedding dimension: 128
You want to implements patch embedding using a single nn.Conv2d layer in PyTorch such that:
• Each patch becomes one token. • The convolution extracts non-overlapping patches. • The output shape becomes:
(B, 128, 4, 4)
where B is batch size.
You write:
Which of the following configurations correctly implement non-overlapping 16×16 patches?
kernel_size = 3, stride = 1, padding = 1
kernel_size = 16, stride = 8, padding = 0
kernel_size = 16, stride = 16, padding = 0
kernel_size = 4, stride = 4, padding = 0
Correct answer
kernel_size = 16, stride = 16, padding = 0
We build a decoder RNN from scratch with the following hyperparameters:
• Batch_size=32 • Vocab_size=28 • Context_length=10 • Hidden_size=64
The following code is used to create the hidden layer:
What will be the final dimension of the hidden layer?
32x28
32x64
64x32
28x64
64x28
32x10
Correct answer
32x64
The following code computes a metric -
Which of the following situations does the above metric becomes useful in?
Generating next tokens in a text summarization task.
Identifying which token is most semantically similar to a given token.
Generating fixed positional embeddings for tokens.
Creating word embedding vectors to be fed into a transformer model.
Correct answer
Identifying which token is most semantically similar to a given token.
You are preparing a dataset to train an encoder-decoder model with the first batch of training data as follows:
Training data:
[Good evening sir,
Amazing data today and a great meeting too,
The sky is blue]
What will be the output dimension when you process the data with the following line of code? Assume word-level tokenization.
3x8
8x3
8x8
8x1
Correct answer
8x3
In a Transformer layer, the residual connection is typically implemented as:
output = SubLayer(input)
output = input × SubLayer(input)
output = concat(input, SubLayer(input))
output = input + SubLayer(input)
Correct answer
output = input + SubLayer(input)
Given a tensor with shape (batch_size = 32, seq_len = 50, d_model = 768), how is Layer Normalization applied in a Transformer?
It is applied once per batch (32 times), normalizing across all tokens and features.
It is applied once per sequence position (50 times), normalizing across batch and features.
It is applied independently to each token (32 × 50 = 1600 times), normalizing across the 768 feature dimensions.
It is applied once per feature dimension (768 times), normalizing across batch and sequence length.
Correct answer
It is applied independently to each token (32 × 50 = 1600 times), normalizing across the 768 feature dimensions.
In Layer Normalization, which parameters are learned during training?
Mean and variance
Gain (γ) and bias (β)
Only the normalization threshold
No parameters are learned; it's a fixed operation
Correct answer
Gain (γ) and bias (β)
The following tokenizer is applied on the input data given in the code:
0
-1
1
It will randomly take a value from −1, 0, or 1.
Correct answer
0
Consider a RAG application (uses chunking and cosine similarity to compare chunks) .
The input query to the RAG is given who's embedding vector is as follows: [1,0.5,1]
The vector store contains 3 chunk embeddings as follows:
C1 = [0,1,0.9] C2 = [1,1,1] C3 = [0.2,0,0.5]
Which chunk will be retrieved when the RAG application executes the input query?
C1
C2
C3
C1 and C2 are equally likely.
Correct answer
C2
A tokenizer is configured as follows:
You preprocess the following text:
text = "Deep learning is amazing"
Which of the following is the most likely format of tokens["input_ids"]?
Correct answer
Which of the following statements about the Universal Approximation Theorem (UAT) is/are correct?
A neural network must have at least two hidden layers to approximate any continuous function.
A single hidden layer neural network with a non-linear activation function can approximate any continuous function on a compact domain, given sufficient neurons.
The Universal Approximation Theorem guarantees that the network can learn the function efficiently during training.
The Universal Approximation Theorem holds only for ReLU activation functions.
The Universal Approximation Theorem does not apply if the activation function is linear.
Correct answers
A single hidden layer neural network with a non-linear activation function can approximate any continuous function on a compact domain, given sufficient neurons.
The Universal Approximation Theorem does not apply if the activation function is linear.
The following table lists optimizers and hyperparameter sets in shuffled order.
Which of the following matchings is correct?
A-4, B-3, C-2, D-1
A-2, B-1, C-3, D-4
A-2, B-4, C-3, D-1
A-3, B-4, C-2, D-1
Correct answer
A-2, B-4, C-3, D-1
Consider the following code snippets:
Which of the above snippets is more likely to reduce overfitting?
Snippet 1
Snippet 2 is better because it avoids noise
Both are equally resistant
None of them is resistant to overfitting
Correct answer
Snippet 1
Correct answer: 34.85 (accepted within ±0.35)
Consider a word embedding model with a vocabulary size of 3200 and a hidden dimension of 64.
The model uses fixed sinusoidal positional encodings based on token positions.
How many trainable parameters are present in the positional encoding layer? (Ignore bias terms.)
Correct answer: 0
