The IIT Madras BS Large Language Models (LLM) Quiz 2 paper sat on 23 Nov 2025, in the September 2025 term: 22 questions for 50 marks in 120 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.
A corpus containing the following five words is tokenized using WordPiece algorithm.
| Word | Frequency |
|---|---|
play | 3 |
played | 2 |
pray | 1 |
prey | 5 |
reply | 4 |
| Total | 15 |
Table 1: Word frequencies in the corpus before any merges
The initial character-level vocabulary is:
Based on the above data, answer the given subquestions.
Correct answer: 0
A corpus containing the following five words is tokenized using WordPiece algorithm.
| Word | Frequency |
|---|---|
play | 3 |
played | 2 |
pray | 1 |
prey | 5 |
reply | 4 |
| Total | 15 |
Table 1: Word frequencies in the corpus before any merges
The initial character-level vocabulary is:
Based on the above data, answer the given subquestions.
Correct answer: 0.065 (accepted within ±0.005)
A corpus containing the following five words is tokenized using WordPiece algorithm.
| Word | Frequency |
|---|---|
play | 3 |
played | 2 |
pray | 1 |
prey | 5 |
reply | 4 |
| Total | 15 |
Table 1: Word frequencies in the corpus before any merges
The initial character-level vocabulary is:
Based on the above data, answer the given subquestions.
Which token will be generated in the first merge?
Correct answer
A study is comparing three architectures, all trained with the same compute budget:
•
Model A (Encoder-Decoder): A 12-layer encoder and a 12-layer decoder with separate parameters. Its total size is 220 million parameters.
•
Model B (Decoder-Only): A 24-layer decoder-only model. Its total size is also approximately 220 million parameters.
•
Model C (Shared Enc-Dec): A 12-layer encoder and a 12-layer decoder where the parameters
are shared (
).
Assuming that the 12-layer encoder and 12-layer decoder in Model A have an equal number of parameters, what is the approximate total parameter count (in millions) of Model C?
Correct answer: 110
Consider a text corpus being processed using the Byte Pair Encoding (BPE) algorithm. The initial
vocabulary contains
tokens (representing characters), and the corpus contains a total of tokens.
During BPE training, the following merge operations are performed. Each merge combines all occurrences of a particular token pair into a single new token, reducing the total token count accordingly:
1.
Merge 1: Pair ('t', 'h') appears 24 times → merged into new token 'th'
2.
Merge 2: Pair ('i', 'n') appears 18 times → merged into new token 'in'
3.
Merge 3: Pair ('th', 'e') appears 10 times → merged into new token 'the'
4.
Merge 4: Pair ('i', 's') appears 8 times → merged into new token 'is'
What is the final total number of tokens in the corpus after all 4 merges are completed?
Correct answer: 940
If Byte-Pair Encoding (BPE) starts with
unique characters and performs
merge operations, what will be the final vocabulary size?
Correct answer: 10100
In a 4-token sequence, a Transformer is calculating the attention for the third Token. Its Query
vector (
) must be compared against the Key vectors (
) of all tokens. The Causal LM mask is applied to the raw scores before the softmax function. The raw dot-product scores are:
•
•
•
•
After the Causal LM mask and the subsequent softmax function are applied, what will be the final attention probability assigned to Token 4?
Correct answer: 0
Which of the following is/are primarily multilingual datasets? Select all that apply.
Sangraha
DOLMA
BookCorpus
CommonCrawl
Correct answers
Sangraha
CommonCrawl
Which decoding method ensures the same response every time for a given prompt (assuming no randomness in model weights)? Select all that apply.
Top-k with k = 3, temperature > 1
Top-k with k = 3, temperature = 1
Top-p with p = 0.2, Temperature > 1
Top-p with p = 0.2, Temperature = 1
Beam Search with beam-size = 5
Beam Search with beam-size = 1
Correct answers
Beam Search with beam-size = 5
Beam Search with beam-size = 1
A data preprocessing pipeline removed data from the Common Crawl snapshot in three stages: 50% after language identification, 24% of the remaining after quality filtering, and 12% of what remained after deduplication. What percentage of the original Common Crawl snapshot remains in the final dataset? (Choose the closest number)
21%
66%
33%
50%
Correct answer
33%
A data preprocessing pipeline needs to apply the following steps on raw Common Crawl data: 1.
Fuzzy deduplication (using MinHash)
2.
Toxicity detection (using ML-based classifiers)
3.
PII removal (using regex patterns)
4.
Exact deduplication (using hash-based matching)
5.
Quality filtering (using several rule-based heuristics)
6.
URL Filtering
What is the most efficient sequence to apply these steps?
6
4
1
5
2
3
1
4
2
5
3
6
5
6
4
1
2
3
4
5
6
1
3
2
Correct answer
6
4
1
5
2
3
Consider the following four pre-training scenarios for four different language models:
(Here, the symbols <X>, <Y>, and <extra_id_0>, <extra_id_1> represent different kinds of placeholder tokens used by some language models during pre-training.)
Match each scenario to the correct model:
A = BART, B = T5, C = BERT, D = GPT
A = T5, B = BERT, C = GPT, D = BART
A = BERT, B = GPT, C = BART, D = T5
A = BERT, B = BART, C = GPT, D = T5
Correct answer
A = BERT, B = GPT, C = BART, D = T5
Consider the scaling law:
where
= model parameters,
= training tokens,
= scale constants,
= scaling exponents.
A team trains an LLM and observes the following losses for their model:
1.
Training Loss
2.
Test Loss (same distribution as training)
3.
Test Loss (different distribution from training)
Which loss does
represent?
Both 2 and 3
Both 1 and 2
1 only
2 only
Correct answer
2 only
A company is building a sentiment analysis system for customer reviews. They have two options: Option A: Fine-tune BERT for sentiment classification
Option B: Fine-tune BART for sentiment classification
Their requirements are:
•
Process 10,000 reviews per minute
•
Deploy on cost-effective cloud instances (memory-constrained)
•
Simple positive/negative/neutral classification
•
No need for explanations or text generation
Which statement best describes the trade-offs they should consider?
BART is expected to provide significantly better accuracy and is worth the extra computational cost since it's pre-trained on more diverse corruption tasks.
BART is better because its decoder can verify the encoder's sentiment understanding, providing a built-in validation mechanism that improves robustness.
They should use BART's encoder only and discard the decoder, which will give them the same efficiency as BERT with better pre-training.
BERT is more suitable because sentiment classification only requires understanding (encoding), not generation (decoding), making it faster and more memory-efficient without sacrificing accuracy.
Correct answer
BERT is more suitable because sentiment classification only requires understanding (encoding), not generation (decoding), making it faster and more memory-efficient without sacrificing accuracy.
Which of the following statements best describes the fundamental difference between the unsupervised pre-training objective used in T5 (span corruption) and its supervised fine-tuning objectives such as translation or summarization?
The pre-training objective is a denoising task, while the fine-tuning objective is a Causal Language Model (CLM) task.
There is no fundamental difference; both are treated as "text-to-text" tasks, where the model is trained to generate a target sequence given an input sequence.
The pre-training objective trains the model to fill in [MASK] tokens, while the fine-tuning objective trains the model to generate text from a [CLS] token.
The pre-training objective trains the entire encoder-decoder, while the fine- tuning objective freezes the encoder and only trains the decoder.
Correct answer
There is no fundamental difference; both are treated as "text-to-text" tasks, where the model is trained to generate a target sequence given an input sequence.
An original document is:
“The quick brown fox jumps over the lazy dog.”
A corrupted input for pre-training is: “The quick <X> fox <Y> over the lazy dog.”
Assuming a T5-style Span Corruption, what is the exact target sequence the decoder is trained to generate?
Correct answer
A Transformer model is designed to handle both understanding and generation tasks. When given
the input sequence
, its attention mask allows the following:
•
When processing
or
, the model can see both
and
.
•
When processing
, the model can see
and
but not
.
This attention scheme is the defining characteristic of which training objective?
Causal Language Modeling (CLM)
Masked Language Modeling (MLM)
Denoising Objective (like BART)
Prefix Language Modeling (Prefix-LM)
Correct answer
Prefix Language Modeling (Prefix-LM)
BART is an encoder-decoder model. During its denoising pre-training, what is fed as input to the decoder?
The corrupted text from the encoder.
The original, uncorrupted text (shifted right by one token).
A sequence of [MASK] tokens, one for each original word.
Nothing; the decoder starts with only the < s > token.
Correct answer
The original, uncorrupted text (shifted right by one token).
Correct answer
How does SentencePiece handle whitespace characters (like spaces) during tokenization?
It only keeps whitespace that appears next to punctuation marks.
It discards all whitespace as a pre-processing step.
It replaces all whitespace with a special [SPACE] token.
It treats whitespace as part of the text stream and encodes it, often using a
symbol like
(underscore).
Correct answer
It treats whitespace as part of the text stream and encodes it, often using a
symbol like
(underscore).
For a layer with
features, how many learnable parameters do BatchNorm and LayerNorm have?
BatchNorm: 512, LayerNorm: 512
BatchNorm: 1024, LayerNorm: 512
BatchNorm: 512, LayerNorm: 1024
BatchNorm: 1024, LayerNorm: 1024
Correct answer
BatchNorm: 1024, LayerNorm: 1024
In a standard Encoder-Decoder Transformer, where do the Query
, Key
, and Value inputs for the Decoder's Cross-Attention sub-layer come from?
comes from the decoder's previous sub-layer;
and
come from the final hidden states of the encoder.
comes from the final hidden states of the encoder;
and
come from the decoder's previous sub-layer.
,
, and
all come from the final hidden states of the encoder.
,
, and
all come from the decoder's previous sub-layer.
Correct answer
comes from the decoder's previous sub-layer;
and
come from the final hidden states of the encoder.
