Quiz Space

Introduction to Deep Learning and Generative AI · End Term · 10 May 2026 · January 2026 term · Set S2

Question 14: In a Transformer layer, the residual connection is typic…

Question 14

+4 marksOne correct option

In a Transformer layer, the residual connection is typically implemented as:

  1. A

    output = SubLayer(input)

  2. B

    output = input × SubLayer(input)

  3. C

    output = concat(input, SubLayer(input))

  4. D

    output = input + SubLayer(input)

Show answer

Correct answer

  • D

    output = input + SubLayer(input)

Video solution

The video solution to this question is being recorded and will play here. Until then, the answer key is above.

See the whole paperTake it as a mock test

Question 14 of 24 in the IIT Madras BS Introduction to Deep Learning and Generative AI (Deep Learning and GenAI) End Term paper sat on 10 May 2026, in the January 2026 term (Introduction To Deep Learning And Generative Ai 06 May 26 (Session 2)). It carries 4 marks.

More questions from this paper

  1. Q1Which of the following orders of operations is/are CORRECT for a single training iteration in PyTorch?
  2. Q2Which of the following is/are NOT present in a standard GPT (decoder-only) model?
  3. Q3Consider the following RAG pipeline: Which of the following is/are true?
  4. Q4Consider the following PyTorch network:
  5. Q5Figure question
  6. Q6Consider the following code snippet: What will be the output?
  7. Q7Consider the following single-layer RNN:
  8. Q8Figure question
  9. Q9Figure question
  10. Q10You are given:\ Image size: 64 × 64 • Patch size: 16 × 16 • Number of input channels: 3 (RGB) • Embedding dimension: 12…
  11. Q11We build a decoder RNN from scratch with the following hyperparameters:\ Batch_size=32 • Vocab_size=28 • Context_length…
  12. Q12The following code computes a metric - Which of the following situations does the above metric becomes useful in?
  13. Q13You are preparing a dataset to train an encoder-decoder model with the first batch of training data as follows:\ Traini…
  14. Q15Given a tensor with shape (batch_size = 32, seq_len = 50, d_model = 768), how is Layer Normalization applied in a Trans…
  15. Q16In Layer Normalization, which parameters are learned during training?
  16. Q17The following tokenizer is applied on the input data given in the code:
  17. Q18Consider a RAG application (uses chunking and cosine similarity to compare chunks) .\ The input query to the RAG is giv…
  18. Q19A tokenizer is configured as follows: You preprocess the following text:\ text = "Deep learning is amazing" Which of th…
  19. Q20Which of the following statements about the Universal Approximation Theorem (UAT) is/are correct?
  20. Q21The following table lists optimizers and hyperparameter sets in shuffled order. Which of the following matchings is cor…
  21. Q22Consider the following code snippets: Which of the above snippets is more likely to reduce overfitting?
  22. Q23Figure question
  23. Q24Consider a word embedding model with a vocabulary size of 3200 and a hidden dimension of 64.\ The model uses fixed sinu…