Quiz Space

Large Language Models · End Term · 13 Sept 2026 · May 2026 term · Set S2

Question 2: For T5-Base, the model dimension is 768 and FFN dimension…

Question 2

+2 marksNumerical answer

For T5-Base, the model dimension is 768 and FFN dimension is 3072. Ignore bias. Find the number of parameters in the FFN of one encoder layer.

Show answer

Correct answer: 4718592

See the whole paperTake it as a mock test

Question 2 of 20 in the IIT Madras BS Large Language Models (LLM) End Term paper sat on 13 Sept 2026, in the May 2026 term (Large Language Models 13 Sep 26 (Session 2)). It carries 2 marks.

More questions from this paper

  1. Q1Figure question
  2. Q3Figure question
  3. Q4Calculate the KV Cache memory size in Megabytes for a single request (batch size = 1) with the following parameters:\ N…
  4. Q5In BERT's Next Sentence Prediction task, how are the negative (NotNext) examples created?
  5. Q6What happens when a very large model is trained for many epochs on a relatively small dataset in the T5 scaling-law set…
  6. Q7PagedAttention is designed to address which specific inefficiency in Large Language Model (LLM) serving?
  7. Q8Figure question
  8. Q9In a standard Transformer layer utilizing Rotary Positional Embeddings (RoPE), at which specific stage of the forward p…
  9. Q10Figure question
  10. Q11Comparing the architectures of BERT and GPT, select all the structural differences that are true.
  11. Q12Based on the architectural comparison experiments (Encoder-Decoder vs. Decoder-only vs. Encoder-only) discussed in the …
  12. Q13Select all true statements regarding KV Caching during the autoregressive inference of LLMs.
  13. Q14Select all correct statements regarding Decoding Strategies in language generation.
  14. Q15In a feature-based approach using a pretrained transformer (like BERT) for a Multiple-Choice QA task, which of the foll…
  15. Q16A Transformer processes a sequence of length 5 using 6 identical self-attention layers. Count the total number of allow…
  16. Q17Consider the architecture of BART. Which of the following descriptions best match its structural design?
  17. Q18Why is Flash Attention considered an "IO-aware" algorithm?
  18. Q19Figure question
  19. Q20An engineer is designing a Transformer model capable of length extrapolation (handling sequences longer than those seen…