Quiz Space

Large Language Models · Quiz 1 · 13 Jul 2025 · May 2025 term

Question 5: Why are residual connections important in transformer arc…

Question 5

+2 marksOne correct option

Why are residual connections important in transformer architectures?

  1. A

    They reduce memory consumption.

  2. B

    They add extra cost of compute by adding batch normalization.

  3. C

    They help in training deep networks by enabling gradient flow.

  4. D

    They remove the need for layer normalization.

Show answer

Correct answer

  • C

    They help in training deep networks by enabling gradient flow.

See the whole paperTake it as a mock test

Question 5 of 21 in the IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 13 Jul 2025, in the May 2025 term (IIT M DEGREE AN EXAM QDB2 13 July 2025). It carries 2 marks.

More questions from this paper

  1. Q1Transformers process input tokens:
  2. Q2What is the purpose of the softmax function in the attention mechanism?
  3. Q3What is the main difference between GPT and BERT pre-training objectives?
  4. Q4In Top-K sampling for language generation, increasing the value of K typically has which of the following effects?
  5. Q6Which of the following statements are true regarding causal language modeling (CLM)?
  6. Q7Which of the following are valid reasons why transformer-based large language models are widely used in natural languag…
  7. Q8Which elements are included in BERT’s input representation for Next Sentence Prediction?
  8. Q9How does Top-p (nucleus) sampling differ from Top-K sampling in language generation?
  9. Q10Consider the embedding vector for a word, x = [0.5, -0.4, 0.3, -0.2]^T. Suppose the word is at position 3 in the given …
  10. Q11Suppose a language model outputs the following logits for a vocabulary of size 4: | Token | Logit | |---|---| | the | 2…
  11. Q12Figure question
  12. Q13Question Description Embedding dimension : d_{model} = 4 Input sequence (sequence length = 2, d_{model} = 4) : X = \beg…
  13. Q14Question Description Embedding dimension : d_{model} = 4 Input sequence (sequence length = 2, d_{model} = 4) : X = \beg…
  14. Q15Question Description Embedding dimension : d_{model} = 4 Input sequence (sequence length = 2, d_{model} = 4) : X = \beg…
  15. Q16For the given input matrix X and multihead attention output MHA(X), apply a residual connection and store the result in…
  16. Q17Based on the above data, answer the given subquestions.
  17. Q18Based on the above data, answer the given subquestions.
  18. Q19The table presents the conditional probability distribution over vocabulary tokens at each timestep during sequence gen…
  19. Q20The table presents the conditional probability distribution over vocabulary tokens at each timestep during sequence gen…
  20. Q21The table presents the conditional probability distribution over vocabulary tokens at each timestep during sequence gen…