Figure from the original question paper

(C) k = 0 will perform better compared to k = 600

(A) k = 0 will perform same as compared to k = 600

(B) k = 0 will perform worse compared to k = 600

(D) Value of k is irrelevant as the difference between position embeddings of two consicutive tokens remain the same.

Figure from the original question paper

(A) Option figure from the original question paper

(B) Option figure from the original question paper

(C) Option figure from the original question paper

(D) Option figure from the original question paper

(E) Option figure from the original question paper

September 2024 term · Large Language Models · BSDA5004

LLM End Term: 22 December 2024, Set QDB4 (September 2024 term)

The IIT Madras BS Large Language Models (LLM) End Term paper sat on 22 Dec 2024, in the September 2024 term, set QDB4: 21 questions for 50 marks in 180 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.

Questions: 21
Marks: 50
Duration: 180 min
Numerical: 6
MCQ: 9
MSQ: 5
Written: 1

Take as mock test Practise with answers

Updated 25 Sept 2026

Official paper: IIT M DEGREE AN EXAM QDB4 22 Dec 2024 · No negative marking.

Question 1

+3 marksNumerical answer

The input embeddings for the words “learning”, “brings” and “joy” are $h_1 = [1.0, 0.5, 1]$ , $h_2 = [1, 0.25, 0]$ , and $h_3 = [0.1, 0.1, 0.9]$ , respectively. Note that the embeddings are row vectors. The projection matrices are as follows

$W_Q = \begin{bmatrix} 1 & 1 \\ -1 & 1 \\ 0 & 1 \end{bmatrix} \quad W_K = \begin{bmatrix} 1 & 1 \\ 1 & 0 \\ -1 & 1 \end{bmatrix} \quad W_V = \begin{bmatrix} 0 & 0 \\ -1 & -1 \\ 1 & 1 \end{bmatrix}$

The following quantities are computed as

$Q = HW_Q \quad K = HW_K \quad V = HW_V$

Let $e_j$ denote the unnormalized attention score, $a_j$ denote the normalized attention score (ignore the scaling by $\sqrt{d_k}$ ) and $z_j$ denote the linear combination of the value vectors for the $j - th$ word.

Enter the value of first element i.e. with index (0,0) of $\frac{\partial a_3}{\partial e_3}$

Show answer

Correct answer: 0.25 (accepted within ±0.05)

Question 1 on its own page

Question 2

+3 marksOne correct option

Assume that we have a large corpus of text. The vocabulary constructed from the text contains 10000 words. Of these, 100 words occurred only once in the entire corpus of text. The parameters of the embedding layer and the output layer of the model are shared. Suppose we create a batch of 256 samples (each sample is a sentence from the corpus). None of these samples contains any of the 100 rare words. Suppose we pre-train the model for one iteration using the batch of samples, then:

A
it is certain that the embeddings of none of these 100 rare words will get updated.
B
there is a chance that the embeddings of all or some of these 100 rare words will get updated
C
the embeddings of all these 100 rare words will defintely get updated
D
None of these

Show answer

Correct answer

B
there is a chance that the embeddings of all or some of these 100 rare words will get updated

Question 2 on its own page

Question 3

+3 marksOne correct option

Suppose we use a pre-trained model for text generation with the given prompt “I am going to”. Which of the following decoding strategies can be used such that the pre-trained model generates same text completion each time it is executed

A
Beam search with beam size 4
B
Greedy approach
C
Top-K with k = 2
D
None of these

Show answer

Correct answer

B
Greedy approach

Question 3 on its own page

Question 4

+3 marksOne or more correct options

Assume that we have 12 sufficiently large supervised NLP datasets for different tasks like sentiment classification, textual entailment, language understanding and so on. Suppose we take all the samples (dropping labels) from these datasets to train the GPT model using the CLM objective.Then which of the following transfer learning approaches is (are) appropriate to transfer the knowledge to any of these 12 downstream tasks?

Select all that apply.

A
Zero shot learning
B
Few-shot learning
C
Full Fine-tuning
D
None of the given approaches are appropriate as the model has already been trained using samples from all the 12 datasets

Show answer

Correct answers

A
Zero shot learning
B
Few-shot learning
C
Full Fine-tuning

Question 4 on its own page

Question 5

+2 marksOne correct option

What is the order of the language modeling pipeline?

A
First, the model, which handles text and returns raw predictions.The tokenizer then makes sense of these predictions and converts them back to text when needed.
B
First, the tokenizer, which handles text and returns IDs. The model handles these IDs and outputs a prediction, which can be some text.
C
The tokenizer handles text and returns IDs. The model handles these IDs and outputs a prediction. The tokenizer can then be used once again to convert these predictions back to some text.
D
None of these

Show answer

Correct answer

C
The tokenizer handles text and returns IDs. The model handles these IDs and outputs a prediction. The tokenizer can then be used once again to convert these predictions back to some text.

Question 5 on its own pageAlso asked in End Term 22 Dec 2024

Question 6

+2 marksOne correct option

Consider following statment and mark if it is true or false:
Repeating examples in the pre-training datasets are completely harmless for downstream performance.

A
TRUE
B
FALSE

Show answer

Correct answer

B
FALSE

Question 6 on its own page

Question 7

+2 marksOne correct option

What is the time complexity of the computing attention matrix A with strided local attention (parameterized by c) and batch size of one? The symbols have the usual meaning.

Show answer

Correct answer

Question 7 on its own page

Question 8

+2 marksOne correct option

A
k = 0 will perform same as compared to k = 600
B
k = 0 will perform worse compared to k = 600
C
k = 0 will perform better compared to k = 600
D
Value of k is irrelevant as the difference between position embeddings of two consicutive tokens remain the same.

Show answer

Correct answer

C
k = 0 will perform better compared to k = 600

Question 8 on its own page

Question 9

+2 marksOne correct option

Show answer

Correct answer

Question 9 on its own page

Question 10

+2 marksNumerical answer

Consider the following dictionary with the number of word occurrences in a corpus:

python

1wo =  { "blast" : 2,
2        "bitterest" : 1,
3        "we" : 1,}

Note: Append </w> to each word at the end.

You will be working with word piece algorithm, answer the given sub questions in that context:

How many tokens are there in the initial vocabulary?

Show answer

Correct answer: 10

Question 10 on its own page

Question 11

+3 marksOne correct option

Consider the following dictionary with the number of word occurrences in a corpus:

python

1wo =  { "blast" : 2,
2        "bitterest" : 1,
3        "we" : 1,}

Note: Append </w> to each word at the end.

You will be working with word piece algorithm, answer the given sub questions in that context:

Which of the following pairs has the least score before any merge?

Show answer

Correct answer

Question 11 on its own page

Question 12

+3 marksWritten answer

Consider the following dictionary with the number of word occurrences in a corpus:

python

1wo =  { "blast" : 2,
2        "bitterest" : 1,
3        "we" : 1,}

Note: Append </w> to each word at the end.

You will be working with word piece algorithm, answer the given sub questions in that context:

Which pair will be merged in the very first merge? Say the pair is (‘a’,‘b’), then enter “ab” (without quotes and white spaces). If there is a tie between two or more candidates, pick the one that occurs first in the original vocabulary.
NOTE: Enter the exact answer without any space in the beginning or at the end.

Show answer

Correct answer: la

Question 12 on its own page

Question 13

+3 marksNumerical answer

Consider the following dictionary with the number of word occurrences in a corpus:

python

1wo =  { "blast" : 2,
2        "bitterest" : 1,
3        "we" : 1,}

Note: Append </w> to each word at the end.

You will be working with word piece algorithm, answer the given sub questions in that context:

What is the score of the pair merged in the very first merge?

Show answer

Correct answer: 1.0

Question 13 on its own page

Question 14

+3 marksOne correct option

Consider the following dictionary with the number of word occurrences in a corpus:

python

1wo =  { "blast" : 2,
2        "bitterest" : 1,
3        "we" : 1,}

Note: Append </w> to each word at the end.

You will be working with word piece algorithm, answer the given sub questions in that context:

Show answer

Correct answer

Question 14 on its own page

Question 15

+2 marksOne or more correct options

Consider the following statements about decreasing the size of the vocabulary

Select all that apply.

A
It increases number of tokens required to represent a sentence
B
It Increases the computational complexity computing attention matrix
C
It increases the context length T
D
None of these

Show answer

Correct answers

A
It increases number of tokens required to represent a sentence
B
It Increases the computational complexity computing attention matrix
C
It increases the context length T

Question 15 on its own page

Question 16

+2 marksOne or more correct options

Choose the correct statements regarding an encoder-decoder transformer model:

Select all that apply.

A
Multi head masked attention block in decoder makes use of a mask.
B
Transformer model is an improvement over its RNN based predecessor for machine translation.
C
For machine tranlation task the input and output embedding layers have to be identical.
D
None of these.

Show answer

Correct answers

A
Multi head masked attention block in decoder makes use of a mask.
B
Transformer model is an improvement over its RNN based predecessor for machine translation.

Question 16 on its own page

Question 17

+2 marksOne or more correct options

Consider C4 pipeline. Which of the following will NOT pass through it as it is?

Select all that apply.

Show answer

Correct answers

Question 17 on its own page

Question 18

+2 marksOne or more correct options

Which of the following are correct about GPT-1 model:

Select all that apply.

A
It is a decoder only model.
B
It uses absolute positional encoding.
C
It uses a composite dataset for pre-training.
D
None of these.

Show answer

Correct answers

A
It is a decoder only model.
B
It uses absolute positional encoding.

Question 18 on its own page

Question 19

+2 marksNumerical answer

How many entries of unnormalized attention matrix QK^(T) will be calculated with strided attention mechanism with c = 5. The sequence length is T = 32

Show answer

Correct answer: 164

Question 19 on its own page

Question 20

+2 marksNumerical answer

How many non zero entries will be there in the mask matrix M?

Show answer

Correct answer: 256

Question 20 on its own page

Question 21

+2 marksNumerical answer

Show answer

Correct answer: 3

Question 21 on its own page