Quiz Space

Large Language Models · End Term · 31 Aug 2025 · May 2025 term · Set QDB3

Question 12: What percentage of entries in the attention matrix are z…

Question 12

+2 marksNumerical answer

Based on the above data, answer the given subquestions.

What percentage of entries in the attention matrix are zero (i.e., the sparsity)?

Show answer

Correct answer: 80

See the whole paperTake it as a mock test

Question 12 of 17 in the IIT Madras BS Large Language Models (LLM) End Term paper sat on 31 Aug 2025, in the May 2025 term (IIT M DEGREE AN EXAM QDB3 31 Aug 2025). It carries 2 marks.

More questions from this paper

  1. Q1Given the input string: sunshine And the following vocabulary of subword tokens with their corresponding log-probabilit…
  2. Q2A GPT model is trained using causal language modeling. During training, for a sequence of T = 4 tokens, which of the fo…
  3. Q3Figure question
  4. Q4Which of the following preprocessing steps are commonly used when preparing data for transformer models like BERT?
  5. Q5Which modifications are used in transformers to handle longer sequences efficiently?
  6. Q6In Transformer models that use relative position embeddings (such as in Transformer-XL or T5), clipping is often applie…
  7. Q7Figure question
  8. Q8For the input “you enjoy tea often”, compute the final representation of the word “enjoy” after the attention layer (i.…
  9. Q9Suppose the input sentence is “tea you enjoy often”. Using the same matrices and processing method, what is the attenti…
  10. Q10Choose the correct representation of π for the given mask image.
  11. Q11How many permutations of π are possible for n = 5?
  12. Q13Consider the short sentence: “small models sometimes beat big ones” A model processes this sequence using the naive rel…
  13. Q14Consider the short sentence: “small models sometimes beat big ones” A model processes this sequence using the naive rel…
  14. Q15Consider the short sentence: “small models sometimes beat big ones” A model processes this sequence using the naive rel…
  15. Q16Consider the short sentence: “small models sometimes beat big ones” A model processes this sequence using the naive rel…
  16. Q17Consider the short sentence: “small models sometimes beat big ones” A model processes this sequence using the naive rel…