Quiz Space

Large Language Models · Quiz 1 · 26 Oct 2025 · September 2025 term

Question 11: Which of the following statements about multiple attenti…

Question 11

+2 marksOne or more correct options

Which of the following statements about multiple attention heads in a Transformer are true?

Select all that apply.

  1. A

    They allow learning diverse local/global relations.

  2. B

    They reduce memory usage compared to one head.

  3. C

    Increasing the number of heads while keeping dmodel fixed, decreases the dimensionality handled by each head.

  4. D

    All heads necessarily learn completely independent information about the input.

Show answer

Correct answers

  • A

    They allow learning diverse local/global relations.

  • C

    Increasing the number of heads while keeping dmodel fixed, decreases the dimensionality handled by each head.

See the whole paperTake it as a mock test

Question 11 of 20 in the IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 26 Oct 2025, in the September 2025 term (IIT M DEGREE AN EXAM QDB2 26 Oct 2025). It carries 2 marks.

More questions from this paper

  1. Q1Figure question
  2. Q2Which of the following is true regarding sinusoidal encoding?
  3. Q3We have a transformer model translating English to French. Suppose the target sequence is: Target (French): [‘je’, ‘sui…
  4. Q4Figure question
  5. Q5In the context of language model training, teacher forcing is a technique where:
  6. Q6Figure question
  7. Q7You have a mini-batch of 2 samples, each with 2 features: X = \begin{bmatrix} 1 & 2 \ 3 & 4 \end{bmatrix} Rows represen…
  8. Q8Figure question
  9. Q9For a vocabulary size of 10, how many beams will there be in greedy search, beam search with beam size 3, and exhaustiv…
  10. Q10Which of the following decoding strategies is/are inherently non-deterministic?
  11. Q12Which of the following statements about the Transformer architecture are true?
  12. Q13A transformer decoder uses a vocabulary of size 5. At step 1, the predicted token probabilities are: | Token | Probabil…
  13. Q14Figure question
  14. Q15A Transformer model processes a sequence of T = 1000 tokens. Within the self-attention layer, the attention scores are …
  15. Q16Figure question
  16. Q17How many next-token prediction targets are generated during training a full batch?
  17. Q18For one sequence, how many non-zero attention scores remain in the masked attention score matrix (per head)?
  18. Q19You are analyzing a GPT-style decoder-only Transformer trained as a causal language model with the following setup: Seq…
  19. Q20During autoregressive inference with GPT: