Quiz Space

Large Language Models · Quiz 2 · 12 Apr 2026 · January 2026 term

Question 7: Which of the following are components found within a stan…

Question 7

+3 marksOne or more correct options

Which of the following are components found within a standard Transformer Encoder layer?

Select all that apply.

  1. A

    Multi-Head Self-Attention mechanism

  2. B

    Position-wise Feed-Forward Networks

  3. C

    Cross-Attention mechanism (Encoder-Decoder attention)

  4. D

    Masked Multi-Head Self-Attention

Show answer

Correct answers

  • A

    Multi-Head Self-Attention mechanism

  • B

    Position-wise Feed-Forward Networks

See the whole paperTake it as a mock test

Question 7 of 21 in the IIT Madras BS Large Language Models (LLM) Quiz 2 paper sat on 12 Apr 2026, in the January 2026 term (Large Language Models 07 Apr 26). It carries 3 marks.

More questions from this paper

  1. Q1Figure question
  2. Q2Figure question
  3. Q3Why is the standard GPT architecture (Decoder-only with causal masking) generally unsuitable for the Masked Language Mo…
  4. Q4In the T5 (Text-to-Text Transfer Transformer) framework, every NLP task is cast as a text generation problem. If you us…
  5. Q5Figure question
  6. Q6When constructing training datasets for large language models (LLMs), which of the following best describes the key fac…
  7. Q8Select all correct statements regarding the comparison between RNNs and Transformers.
  8. Q9Select all correct findings from the T5 paper regarding Unsupervised Pre-training Objectives.
  9. Q10Consider the concept of Zero-Shot Transfer as popularized by GPT-2. Why might this be preferred over Supervised Fine-Tu…
  10. Q11Figure question
  11. Q12Is the given attention matrix valid for a causal language modelling task?
  12. Q13At time step t = 5 (word: data), what is the attention weight assigned to the word science? (Provide exact answer)
  13. Q14Which of the following character pairs occurs with the highest frequency across the entire sentence before any BPE merg…
  14. Q15Calculate the total vocabulary size immediately after the first merge operation is completed.\ Note: The vocabulary siz…
  15. Q16Figure question
  16. Q17Figure question
  17. Q18What is a major disadvantage of character-level tokenization?
  18. Q19When using the SentencePiece, how are subword units selected?
  19. Q20To obtain high-quality training text from raw web data for large language models, which of the following mechanisms are…
  20. Q21Which of the following statements correctly explain the importance of deduplication during preprocessing of large-scale…