Large Language Models Quiz 2: 12 April 2026 (January 2026 term)
The IIT Madras BS Large Language Models (LLM) Quiz 2 paper sat on 12 Apr 2026, in the January 2026 term: 21 questions for 50 marks in 120 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.
- 21
- 50
- 120 min
- 10
- 6
- 5
Show answer
Correct answer
Question 2
To reduce the computational cost of the softmax layer during training.
To act as a regularizer similar to Dropout.
Show answer
Correct answer
Question 3
Why is the standard GPT architecture (Decoder-only with causal masking) generally unsuitable for the Masked Language Modeling (MLM) objective as implemented in BERT?
GPT models are too small to learn bidirectional contexts.
The causal mask in GPT prevents the model from attending to future tokens, making it impossible to use right-side context to predict a masked token.
GPT does not have positional embeddings, which are required for MLM.
GPT uses ReLU activation, while BERT uses GELU, which is required for MLM.
Show answer
Correct answer
The causal mask in GPT prevents the model from attending to future tokens, making it impossible to use right-side context to predict a masked token.
Question 4
In the T5 (Text-to-Text Transfer Transformer) framework, every NLP task is cast as a text generation problem. If you use T5 for a Semantic Textual Similarity (STS-B) task, where the goal is to predict a similarity score (e.g., 3.8) between two sentences, how does the model output this score?
It outputs a single scalar value from a regression head on top of the encoder.
It generates the string "3.8" token-by-token using the decoder.
It outputs a class label corresponding to a bucketed score range (e.g., "High Similarity").
T5 cannot be used for regression tasks like STS-B.
Show answer
Correct answer
It generates the string "3.8" token-by-token using the decoder.
Question 5
The model trains almost exclusively on the largest dataset (highest resource task).
The sampling distribution approaches a uniform distribution, where all tasks (large and small) are sampled with nearly equal probability.
The model trains almost exclusively on the smallest dataset (lowest resource task).
The sampling distribution remains proportional to the original dataset sizes.
Show answer
Correct answer
The sampling distribution approaches a uniform distribution, where all tasks (large and small) are sampled with nearly equal probability.
Question 6
When constructing training datasets for large language models (LLMs), which of the following best describes the key factors that must be balanced to achieve strong and reliable performance?
Model depth, number of parameters, and learning rate
Scale, diversity, and quality of the training data
Vocabulary size, tokenization method, and batch size
Compute budget, optimizer choice, and hardware efficiency
Show answer
Correct answer
Scale, diversity, and quality of the training data
Question 7
Which of the following are components found within a standard Transformer Encoder layer?
Multi-Head Self-Attention mechanism
Position-wise Feed-Forward Networks
Cross-Attention mechanism (Encoder-Decoder attention)
Masked Multi-Head Self-Attention
Show answer
Correct answers
Multi-Head Self-Attention mechanism
Position-wise Feed-Forward Networks
Question 8
Select all correct statements regarding the comparison between RNNs and Transformers.
Transformers process tokens strictly sequentially during training in the same way as RNNs.
Transformers allow for significantly more parallelization during training compared to RNNs.
Attention mechanisms in Transformers utilize Query, Key, and Value vectors derived from input embeddings.
Show answer
Correct answers
Transformers allow for significantly more parallelization during training compared to RNNs.
Attention mechanisms in Transformers utilize Query, Key, and Value vectors derived from input embeddings.
Question 9
Select all correct findings from the T5 paper regarding Unsupervised Pre-training Objectives.
The specific corruption rate (e.g., 10%, 15%, 25%) had a minimal effect on downstream performance.
Using a span length of around 3 tokens performed slightly better than masking single tokens.
The "Deshuffling" objective (reordering shuffled sentences) significantly outperformed the Span Corruption objective.
Replacing a corrupted span with a unique sentinel token worked better than simply dropping the tokens from the input.
Show answer
Correct answers
The specific corruption rate (e.g., 10%, 15%, 25%) had a minimal effect on downstream performance.
Using a span length of around 3 tokens performed slightly better than masking single tokens.
Replacing a corrupted span with a unique sentinel token worked better than simply dropping the tokens from the input.
Question 10
Consider the concept of Zero-Shot Transfer as popularized by GPT-2. Why might this be preferred over Supervised Fine-Tuning?
It allows the model to handle tasks for which no labeled training data is available.
It always achieves higher accuracy than a fine-tuned SOTA model.
It avoids the need to store a separate specialized model (checkpoint) for every downstream task.
It mimics the human ability to perform tasks based on instructions without needing thousands of examples.
Show answer
Correct answers
It allows the model to handle tasks for which no labeled training data is available.
It avoids the need to store a separate specialized model (checkpoint) for every downstream task.
It mimics the human ability to perform tasks based on instructions without needing thousands of examples.
Question 11
Show answer
Correct answer: 1.335 (accepted within ±0.035)
Question 12
Based on the above data, answer the given subquestions.
Is the given attention matrix valid for a causal language modelling task?
True
False
Insufficient information
Show answer
Correct answer
False
Question 13
Based on the above data, answer the given subquestions.
At time step t = 5 (word: data), what is the attention weight assigned to the word science? (Provide exact answer)
Show answer
Correct answer: 0.2
Question 14
Which of the following character pairs occurs with the highest frequency across the entire sentence before any BPE merges are performed?
Show answer
Correct answer
Question 15
Calculate the total vocabulary size immediately after the first merge operation is completed.
Note: The vocabulary size includes all individual base characters/symbols plus the newly created merge token.
Show answer
Correct answer: 20
Question 16
Show answer
Correct answer: 0.62 (accepted within ±0.03)
Question 17
Show answer
Correct answer: 648448
Question 18
What is a major disadvantage of character-level tokenization?
Character-level tokenization cannot represent punctuation marks or special symbols
It has very large vocabulary size because each word is broken into multiple characters
It produces much longer input sequences which increases computational cost.
It fails to capture morphological patterns such as prefixes and suffixes, reducing the model's ability to understand word structure
Show answer
Correct answer
It produces much longer input sequences which increases computational cost.
Question 19
When using the SentencePiece, how are subword units selected?
Based on a probabilistic model that maximizes the likelihood of the training data.
By randomly selecting character n-grams until the vocabulary limit is reached.
By selecting only the top 50,000 most frequent words in the corpus.
By iteratively merging the most frequent pair of adjacent characters.
Show answer
Correct answer
Based on a probabilistic model that maximizes the likelihood of the training data.
Question 20
To obtain high-quality training text from raw web data for large language models, which of the following mechanisms are typically included in the pre-processing pipeline?
Tokenizing text into subword units using Byte Pair Encoding (BPE)
Deduplicating content at line, paragraph, and document levels
Detecting and filtering toxic content such as hate speech and profanity
Identifying the language of web pages
Assessing the quality of content to remove low-value or spam text
Detecting and removing Personally Identifiable Information (PII)
Fine-tuning the model using reinforcement learning from human feedback (RLHF)
Show answer
Correct answers
Deduplicating content at line, paragraph, and document levels
Detecting and filtering toxic content such as hate speech and profanity
Identifying the language of web pages
Assessing the quality of content to remove low-value or spam text
Detecting and removing Personally Identifiable Information (PII)
Question 21
Which of the following statements correctly explain the importance of deduplication during preprocessing of large-scale datasets used for training Deep Learning or Large Language Models?
Deduplication reduces the risk of overfitting by preventing repeated samples from dominating the gradient updates.
Deduplication guarantees that the trained model will achieve higher accuracy on all downstream tasks.
Deduplication helps avoid data leakage between training and evaluation sets, leading to more reliable performance metrics.
Deduplication eliminates the need for regularization techniques such as dropout and weight decay.
Show answer
Correct answers
Deduplication reduces the risk of overfitting by preventing repeated samples from dominating the gradient updates.
Deduplication helps avoid data leakage between training and evaluation sets, leading to more reliable performance metrics.