Quiz Space

Large Language Models · Quiz 1 · 15 Mar 2026 · January 2026 term

Question 8: Select all statements that correctly describe the motivat…

Question 8

+3 marksOne or more correct options

Select all statements that correctly describe the motivation and behavior of Multi-Head Attention as compared to single-head attention.

Select all that apply.

  1. A

    It allows the model to jointly attend to information from different representation subspaces at different positions.

  2. B

    It is mathematically similar to having multiple filters/kernels in a CNN to capture different features.

  3. C

    It reduces the total number of parameters required compared to a single head with the same total dimension.

  4. D

    Each head can theoretically learn to capture different linguistic relationships (e.g., one head links "it" to "animal", another links "it" to "tired").

Show answer

Correct answers

  • A

    It allows the model to jointly attend to information from different representation subspaces at different positions.

  • B

    It is mathematically similar to having multiple filters/kernels in a CNN to capture different features.

  • D

    Each head can theoretically learn to capture different linguistic relationships (e.g., one head links "it" to "animal", another links "it" to "tired").

See the whole paperTake it as a mock test

Question 8 of 19 in the IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 15 Mar 2026, in the January 2026 term (Large Language Models 15 Mar 26). It carries 3 marks.

More questions from this paper

  1. Q1Figure question
  2. Q2In the scaled dot-product attention equation
  3. Q3Figure question
  4. Q4In a standard Transformer encoder-decoder architecture, is masking typically applied in the cross-attention layer?
  5. Q5Figure question
  6. Q6Figure question
  7. Q7Figure question
  8. Q9Given the vectorized self-attention calculation
  9. Q10Figure question
  10. Q11Figure question
  11. Q12The previous decoder hidden state is: and Based on the above data, answer the given subquestions.
  12. Q13The previous decoder hidden state is: and Based on the above data, answer the given subquestions.
  13. Q14The previous decoder hidden state is: and Based on the above data, answer the given subquestions.
  14. Q15What is the number of parameters in the token embedding matrix?
  15. Q16What is the total number of parameters in the positional embedding matrix?
  16. Q17For ONE complete attention head, what is the total number of parameters (Q + K + V projections combined)?
  17. Q18After the multi-head attention computation, all head outputs are concatenated and projected back to the model dimension…
  18. Q19What is the total number of parameters in the complete FFN (including both weight matrices and bias vectors)?