Question 1
What happens when you use Batch Normalization with a batch size of 1?
It works perfectly fine
The statistics become meaningless (mean=0, std=0)
It automatically switches to Layer Normalization
It uses the running statistics from training
Large Language Models End Term: 21 December 2025, Set 1 (September 2025 term)
The IIT Madras BS Large Language Models (LLM) End Term paper sat on 21 Dec 2025, in the September 2025 term, set 1: 24 questions for 50 marks in 180 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.
- 24
- 50
- 180 min
- 16
- 5
- 3
Show answer
Correct answer
The statistics become meaningless (mean=0, std=0)
Question 2
A language model outputs the following logits for the next token: {cat: 3.2, dog: 2.9, bird: 1.1, fish: 0.5, snake: -0.4} Before sampling, the decoding pipeline performs: • Temperature scaling with T = 2.0 • Top-K filtering with K = 3 After applying both steps, which tokens remain eligible for sampling?
Only ''cat''
'cat'' or ''dog''
'cat'', ''dog'' or ''bird''
'dog'', ''bird'' or ''fish''
Show answer
Correct answer
'cat'', ''dog'' or ''bird''
Question 3
Tokens A, B, C, D, E
Tokens A, B
Tokens A, B, C
Tokens A, B, C, D
Tokens A, B, D
Show answer
Correct answer
Tokens A, B, C
Question 4
Insufficient information to determine
Show answer
Correct answer
Insufficient information to determine
Question 5
A research team wants to classify scientific abstracts into multiple topics simultaneously (e.g., ''ML'', ''biology'', ''statistics''), where each abstract may belong to more than one topic. They fine-tune BERT for this task. Which modification is MOST appropriate?
Replace the [CLS] vector with an average of the top-4 attention heads
Feed the [CLS] embedding into a linear layer with a sigmoid activation per label
Use token embeddings individually and classify each token
Use the [SEP] token embedding for multi-label prediction
Show answer
Correct answer
Feed the [CLS] embedding into a linear layer with a sigmoid activation per label
Question 6
A company wants to automatically correct noisy OCR text extracted from scanned documents. The text contains spelling mistakes, missing words, and scrambled phrases. Which model should they fine-tune?
BERT, because it masks tokens and predicts them independently
BART, because it is trained with text corruption and autoregressive reconstruction
GPT, because it is optimal for bidirectional correction
BERT, because [CLS] captures global structure
Show answer
Correct answer
BART, because it is trained with text corruption and autoregressive reconstruction
Question 7
Computing the softmax for each row
Performing layer normalization
Show answer
Correct answer
Question 8
During autoregressive inference, key-value (KV) caching is used to avoid recomputing keys and values for previously generated tokens. If the KV cache is allowed to grow without any limit, which of the following failure modes can occur?
GPU memory exhaustion
Latency becoming quadratic in the output length
Inability to perform parallel token generation
Beam search collapsing to a single token
Show answer
Correct answer
GPU memory exhaustion
Question 9
(A figure from the original paper is missing from the source site.)
Show answer
Correct answer
Question 10
Show answer
Correct answer
Question 11
If a model uses Absolute Positional Embeddings (APE), increasing the sequence length from 100 to 1000 requires:
No new parameters
900 additional positional embedding vectors
A new rotation matrix
A distance-based bias matrix
Show answer
Correct answer
900 additional positional embedding vectors
Question 12
3
5
6
7
Show answer
Correct answer
6
Question 13
APE = 10, RPE = 19, ALiBi = 0, NoPE = 0
APE = 10, RPE = 10, ALiBi = 19, NoPE = 0
APE = 20, RPE = 19, ALiBi = 1, NoPE = 1
APE = 10, RPE = 2T = 20, ALiBi = 10, NoPE = 0
Show answer
Correct answer
APE = 10, RPE = 19, ALiBi = 0, NoPE = 0
Question 14
Which of the following methods uses/use the concept: "The farther apart two tokens are, the less they should attend to each other''?
RoPE
APE
ALiBi
RPE
Show answer
Correct answer
ALiBi
Question 15
You trained a model with max sequence length 512. At inference, you need to process sequence length 4096 without retraining. Which positional encoding will perform best out-of-the-box?
APE
RPE
RoPE
ALiBi
Show answer
Correct answer
ALiBi
Question 16
7000
8000
Depends on the number of distances
0
Show answer
Correct answer
0
Question 17
Show answer
Correct answer: 100000
Question 18
Show answer
Correct answer: 121
Question 19
Show answer
Correct answer: 30
Question 20
Show answer
Correct answer: 110592
Question 21
Show answer
Correct answer: 64
Question 22
A GPT-style causal language model is trained using the next-token prediction objective:
During inference, the model must generate tokens autoregressively from left to right using only past context. Consider the following statements about GPT-style causal models.
Removing positional encodings would force GPT to treat all permutations of the same set of tokens as equivalent contexts
Causal masking ensures that the computational cost of training scales linearly with sequence length
Show answer
Correct answers
Removing positional encodings would force GPT to treat all permutations of the same set of tokens as equivalent contexts
Question 23
Under which of the following decoding settings can the model produce different outputs across multiple runs on the same prompt?
Show answer
Correct answers
Question 24
Show answer
Correct answers