Question 14
1 Key/Value head (same as MQA).
LLM End Term 10 May 2026 — Question 14
Show answer
Correct answer
Question 14 of 20 in the IIT Madras BS Large Language Models (LLM) End Term paper sat on 10 May 2026, in the January 2026 term (Large Language Models 10 May 26). It carries 3 marks.
More questions from this paper
- In the standard Transformer Decoder, the Multi-Head Attention layer is "Masked". What is the specific purpose of this m…
- Figure question
- Figure question
- A transformer layer with absolute positional encoding and another with ALiBi are compared on sequences extended from le…
- Based on the BERT pre-training phase, select all correct constraints and properties regarding the input data.
- Select the statements that correctly distinguish fine-tuning from zero-shot prompting for GPT-style models.
- Figure question
- Figure question
- Figure question
- Suppose a language model outputs the following logits for a vocabulary of size 4:
- Figure question
- Adapter Layers are a parameter-efficient fine-tuning strategy. Where are these layers typically inserted within a Trans…
- Why do standard LLM architectures include explicit Positional Encodings (like RoPE or Sinusoidal) despite the causal ma…
- Consider Multi-Query Attention (MQA). How does its architecture differ from standard Multi-Head Attention (MHA) to impr…
- Figure question
- Figure question
- Which of the following statements about fine-tuning strategies such as gradual unfreezing and multi-task learning in th…
- Select all correct statements regarding Local Attention Variants (like Sparse, Dilated, or Windowed Attention).
- Which of the following techniques are used in Flash Attention to speed up attention computation and save memory?