Quiz Space

Introduction to Deep Learning and Generative AI · End Term · 10 May 2026 · January 2026 term · Set S2

Question 15: Given a tensor with shape (batch_size = 32, seq_len = 50…

Question 15

+4 marksOne correct option

Given a tensor with shape (batch_size = 32, seq_len = 50, d_model = 768), how is Layer Normalization applied in a Transformer?

  1. A

    It is applied once per batch (32 times), normalizing across all tokens and features.

  2. B

    It is applied once per sequence position (50 times), normalizing across batch and features.

  3. C

    It is applied independently to each token (32 × 50 = 1600 times), normalizing across the 768 feature dimensions.

  4. D

    It is applied once per feature dimension (768 times), normalizing across batch and sequence length.

Show answer

Correct answer

  • C

    It is applied independently to each token (32 × 50 = 1600 times), normalizing across the 768 feature dimensions.

See the whole paperTake it as a mock test

Question 15 of 24 in the IIT Madras BS Introduction to Deep Learning and Generative AI (Deep Learning and GenAI) End Term paper sat on 10 May 2026, in the January 2026 term (Introduction To Deep Learning And Generative Ai 06 May 26 (Session 2)). It carries 4 marks.

More questions from this paper

  1. Q1Which of the following orders of operations is/are CORRECT for a single training iteration in PyTorch?
  2. Q2Which of the following is/are NOT present in a standard GPT (decoder-only) model?
  3. Q3Consider the following RAG pipeline: Which of the following is/are true?
  4. Q4Consider the following PyTorch network:
  5. Q5Figure question
  6. Q6Consider the following code snippet: What will be the output?
  7. Q7Consider the following single-layer RNN:
  8. Q8Figure question
  9. Q9Figure question
  10. Q10You are given:\ Image size: 64 × 64 • Patch size: 16 × 16 • Number of input channels: 3 (RGB) • Embedding dimension: 12…
  11. Q11We build a decoder RNN from scratch with the following hyperparameters:\ Batch_size=32 • Vocab_size=28 • Context_length…
  12. Q12The following code computes a metric - Which of the following situations does the above metric becomes useful in?
  13. Q13You are preparing a dataset to train an encoder-decoder model with the first batch of training data as follows:\ Traini…
  14. Q14In a Transformer layer, the residual connection is typically implemented as:
  15. Q16In Layer Normalization, which parameters are learned during training?
  16. Q17The following tokenizer is applied on the input data given in the code:
  17. Q18Consider a RAG application (uses chunking and cosine similarity to compare chunks) .\ The input query to the RAG is giv…
  18. Q19A tokenizer is configured as follows: You preprocess the following text:\ text = "Deep learning is amazing" Which of th…
  19. Q20Which of the following statements about the Universal Approximation Theorem (UAT) is/are correct?
  20. Q21The following table lists optimizers and hyperparameter sets in shuffled order. Which of the following matchings is cor…
  21. Q22Consider the following code snippets: Which of the above snippets is more likely to reduce overfitting?
  22. Q23Figure question
  23. Q24Consider a word embedding model with a vocabulary size of 3200 and a hidden dimension of 64.\ The model uses fixed sinu…