The IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 19 Jul 2026, in the May 2026 term: 16 questions for 50 marks in 120 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.
Which statements are true about traditional attention used in sequence-to-sequence models?
The decoder decides which encoder states are important.
Attention weights are computed over all decoder hidden states.
The context vector is a weighted sum of encoder states.
Encoder hidden states are ignored after encoding.
Correct answers
The decoder decides which encoder states are important.
The context vector is a weighted sum of encoder states.
The query, key, and value projection matrices are given by:
Based on the above data, answer the given subquestions.
Correct answer: 4
The query, key, and value projection matrices are given by:
Based on the above data, answer the given subquestions.
Choose the token pair with the least attention score.
{"learn", "easy"}
{"learn", "math"}
{"easy", "math"}
{"learn", "learn"}
{"easy" , "easy"}
Correct answer
{"learn", "easy"}
A Transformer model processes a sequence containing 6 tokens using Multi-Head Attention. The model initially uses 4 attention heads. If the number of attention heads is doubled to 8, while the sequence length remains unchanged, how many additional attention scores are computed across all heads?
Correct answer: 144
Correct answer: 32768
Correct answer: 8128
Correct answer: 256
Consider the following statements regarding the use of Teacher Forcing while training autoregressive sequence-to-sequence models and select the appropriate option:
Statement 1: Teacher Forcing involves passing the input token to a larger Teacher model whose output is then input as the next token Statement 2: Teacher Forcing helps to prevent compounding of early incorrect predictions that can destabilize training
Statement 1 is True but Statement 2 is False
Statement 1 is False but Statement 2 is True
Both the statements are True
Both the statements are False
Correct answer
Statement 1 is False but Statement 2 is True
Based on the above data, answer the given subquestions.
Correct answer: 0.18 (accepted within ±0.01)
Based on the above data, answer the given subquestions.
Choose the option corresponding to the sentence with the highest joint probability under the given language model.
"i eat apples"
"apples like You"
"i eat bananas"
"you like bananas
"you like apples
Correct answer
"i eat bananas"
Based on the above data, answer the given subquestions.
Using the given probability tree calculate the conditional probability of predicting "apples" given "like" [i.e P(apples | like)]. (Submit -1 if the provided information is insufficient)
Correct answer: -1
Which of the following are components found within a GPT decoder layer?
Causal (Masked) Multi-Head Self-Attention
Position-wise Feed-Forward Network
Multi Head (Cross) Attention
Recurrent Neural Network (RNN) Layer
Add & Norm Layer
Correct answers
Causal (Masked) Multi-Head Self-Attention
Position-wise Feed-Forward Network
Add & Norm Layer
The table below represents the conditional probability distribution over the vocabulary. Each column corresponds to a decoding timestep and the values represent the conditional probability of selecting a token at the timestep given the previously generated tokens
For all calculations, consider only the tokens mentioned in the first column as the Vocabulary
Based on the above data, answer the given subquestions.
Use greedy decoding to determine the most likely sequence of 6 tokens and compute the probability of generating that sequence. Enter the value rounded off to 3 decimal places
Correct answer: 0.00475 (accepted within ±0.00075)
The table below represents the conditional probability distribution over the vocabulary. Each column corresponds to a decoding timestep and the values represent the conditional probability of selecting a token at the timestep given the previously generated tokens
For all calculations, consider only the tokens mentioned in the first column as the Vocabulary
Based on the above data, answer the given subquestions.
Use exhaustive search strategy to identify the most probable sequence consisting of 4 tokens. How many decoder runs are required in total to identify the most probable sequence containing 4 tokens?
Correct answer: 400
The table below represents the conditional probability distribution over the vocabulary. Each column corresponds to a decoding timestep and the values represent the conditional probability of selecting a token at the timestep given the previously generated tokens
For all calculations, consider only the tokens mentioned in the first column as the Vocabulary
Based on the above data, answer the given subquestions.
Use Top-k sampling at timestep t = 4 and k = 3. Compute the renormalized probability of the most probable token from the candidate tokens and enter the value rounded off to three decimal places
Correct answer: 0.412 (accepted within ±0.002)
The table below represents the conditional probability distribution over the vocabulary. Each column corresponds to a decoding timestep and the values represent the conditional probability of selecting a token at the timestep given the previously generated tokens
For all calculations, consider only the tokens mentioned in the first column as the Vocabulary
Based on the above data, answer the given subquestions.
Consider the sequence "the castle was abandoned". What is the minimum beam width required to guarantee that this sequence is retained during beam search at timestep t = 4 ?
Correct answer: 5
