Quiz Space

Large Language Models · End Term · 13 Sept 2026 · May 2026 term · Set 1

Question 14: What is the primary trade-off made by utilizing KV Cache…

Question 14

+3 marksOne correct option

What is the primary trade-off made by utilizing KV Cache during autoregressive decoding?

  1. A

    It reduces inference latency but utilizes smaller batch size

  2. B

    It increases generation speed but reduces model accuracy

  3. C

    It reduces computational overhead but increases memory usage

  4. D

    It reduces training time by storing intermediate token representations

Show answer

Correct answer

  • C

    It reduces computational overhead but increases memory usage

See the whole paperTake it as a mock test

Question 14 of 18 in the IIT Madras BS Large Language Models (LLM) End Term paper sat on 13 Sept 2026, in the May 2026 term (Large Language Models 13 Sep 26 (Session 2)). It carries 3 marks.

More questions from this paper

  1. Q1Consider the following Assertion (A) and Reason (R) about the Transformer encoder. Assertion (A): In a Transformer enco…
  2. Q2Which of the following statements regarding one-hot positional encoding and sinusoidal positional encoding is incorrect?
  3. Q3You are designing an LLM to translate a sentence from English to Spanish with low computational complexity. The system …
  4. Q4In the Next Sentence Prediction task, how are the two input sentences distinguished by the BERT architecture?
  5. Q5Consider a mini-batch of 3 samples, each with 2 features. The input matrix X is given below, where rows represent indiv…
  6. Q6Consider a tokenizer processing the word "lion".\ The vocabulary contains the following tokens along with the assigned …
  7. Q7Figure question
  8. Q8Consider an LLM operating on a vocabulary consisting of 5 tokens. For a given context, the model produces the following…
  9. Q9Consider an LLM operating on a vocabulary consisting of 5 tokens. For a given context, the model produces the following…
  10. Q10How did the GPT-2 model initially demonstrate "zero-shot domain transfer" capabilities on complex tasks like summarizat…
  11. Q11Match the following unsupervised objectives (I,II,III) to their corresponding Target output structures (A,B,C) based on…
  12. Q12Consider the original text "The quick brown fox jumps over the lazy dog". If the input for a prefix language modelling …
  13. Q13Consider a transformer model that uses strided local attention with a fixed window size c. If the sequence length T is …
  14. Q15A Large Language model uses Grouped Query Attention with a KV cache during inference. Consider the following model conf…
  15. Q16Which of the following statements about length generalization in Large Language Models (LLMs) are correct?
  16. Q17The query and key projection matrices are and RoPE Frequency For RoPE, use the frequency formula Using the property the…
  17. Q18The query and key projection matrices are and RoPE Frequency For RoPE, use the frequency formula Using the property the…