Select all true statements regarding KV Caching during the autoregressive inference of LLMs.
It trades off increased memory usage for reduced computational latency.
It avoids recomputing the Key and Value vectors for tokens that have already been processed in previous steps.
It is essential during the training phase to speed up backpropagation.
The memory required for the KV cache grows linearly with the sequence length and batch size.
Correct answers
It trades off increased memory usage for reduced computational latency.
It avoids recomputing the Key and Value vectors for tokens that have already been processed in previous steps.
The memory required for the KV cache grows linearly with the sequence length and batch size.
Question 13 of 20 in the IIT Madras BS Large Language Models (LLM) End Term paper sat on 13 Sept 2026, in the May 2026 term (Large Language Models 13 Sep 26 (Session 2)). It carries 3 marks.
