uiz Space

January 2026 term · Large Language Models · BSDA5004

Large Language Models Quiz 1: 15 March 2026 (January 2026 term)

The IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 15 Mar 2026, in the January 2026 term: 19 questions for 40 marks in 120 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.

Questions
19
Marks
40
Duration
120 min
MCQ
12
MSQ
3
Numerical
4
Take as mock testPractise with answers

Updated

Official paper: Large Language Models 15 Mar 26 · No negative marking.

Question 1

+2 marksOne correct option
  1. A
  2. B
  3. C
  4. D
Show answer

Correct answer

  • C
Question 1 on its own page

Question 2

+2 marksOne correct option

In the scaled dot-product attention equation

  1. A
  2. B

    To reduce the number of trainable parameters in the model

  3. C

    To prevent the dot product values from growing too large, which would push the softmax function into regions with extremely small gradients

  4. D

    To normalize the embedding vectors to have a unit length

Show answer

Correct answer

  • C

    To prevent the dot product values from growing too large, which would push the softmax function into regions with extremely small gradients

Question 2 on its own page

Question 3

+2 marksOne correct option
  1. A
  2. B
  3. C
  4. D
Show answer

Correct answer

  • C
Question 3 on its own page

Question 4

+2 marksOne correct option

In a standard Transformer encoder-decoder architecture, is masking typically applied in the cross-attention layer?

  1. A

    Yes, both causal masking and padding masking are applied to prevent the decoder from attending to future encoder positions

  2. B

    Yes, only causal masking is applied to maintain the autoregressive property

  3. C

    No causal masking is needed, but padding masking may be applied to ignore padded positions in the source sequence

  4. D

    No masking is ever applied in cross-attention since the encoder processes the complete input sequence

Show answer

Correct answer

  • C

    No causal masking is needed, but padding masking may be applied to ignore padded positions in the source sequence

Question 4 on its own page

Question 5

+2 marksOne correct option
  1. A

    Segment embeddings are not used; the embedding layer is disabled for single-sentence tasks.

  2. B
  3. C
  4. D

    Segment embeddings are randomly initialized for every new single-sentence input.

Show answer

Correct answer

  • B
Question 5 on its own page

Question 6

+2 marksOne correct option
  1. A

    It approaches a uniform distribution where all tokens have equal probability.

  2. B

    It approaches a "one-hot" distribution where the token with the highest logit has probability 1.0 (Greedy Search).

  3. C

    It remains unchanged from the original softmax distribution.

  4. D

    It causes numerical instability and results in NaNs.

Show answer

Correct answer

  • B

    It approaches a "one-hot" distribution where the token with the highest logit has probability 1.0 (Greedy Search).

Question 6 on its own page

Question 7

+2 marksOne correct option
  1. A

    The average pooling of all token vectors.

  2. B
  3. C
  4. D

    The concatenation of all hidden state vectors.

Show answer

Correct answer

  • C
Question 7 on its own page

Question 8

+3 marksOne or more correct options

Select all statements that correctly describe the motivation and behavior of Multi-Head Attention as compared to single-head attention.

Select all that apply.

  1. A

    It allows the model to jointly attend to information from different representation subspaces at different positions.

  2. B

    It is mathematically similar to having multiple filters/kernels in a CNN to capture different features.

  3. C

    It reduces the total number of parameters required compared to a single head with the same total dimension.

  4. D

    Each head can theoretically learn to capture different linguistic relationships (e.g., one head links "it" to "animal", another links "it" to "tired").

Show answer

Correct answers

  • A

    It allows the model to jointly attend to information from different representation subspaces at different positions.

  • B

    It is mathematically similar to having multiple filters/kernels in a CNN to capture different features.

  • D

    Each head can theoretically learn to capture different linguistic relationships (e.g., one head links "it" to "animal", another links "it" to "tired").

Question 8 on its own page

Question 9

+3 marksOne or more correct options

Given the vectorized self-attention calculation

Select all that apply.

  1. A
  2. B

    The softmax operation is applied to the entire matrix at once (globally), not row-wise.

  3. C
  4. D
Show answer

Correct answers

  • A
  • C
  • D
Question 9 on its own page

Question 10

+2 marksOne or more correct options

Select all that apply.

  1. A
  2. B
  3. C
  4. D

    This mechanism allows the Decoder to focus on relevant parts of the input sequence processed by the Encoder.

Show answer

Correct answers

  • B
  • D

    This mechanism allows the Decoder to focus on relevant parts of the input sequence processed by the Encoder.

Question 10 on its own page

Question 11

+2 marksNumerical answer
Show answer

Correct answer: -0.62 (accepted within ±0.03)

Question 11 on its own page

Question 12

+1 markNumerical answer

The previous decoder hidden state is:

and

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 1.925 (accepted within ±0.035)

Question 12 on its own page

Question 13

+3 marksNumerical answer

The previous decoder hidden state is:

and

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 0.35 (accepted within ±0.04)

Question 13 on its own page

Question 14

+2 marksNumerical answer

The previous decoder hidden state is:

and

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 1.36 (accepted within ±0.04)

Question 14 on its own page

Question 15

+2 marksOne correct option

Consider the following Configuration for a GPT model:
• Vocabulary size: 40,000 tokens • Embedding dimension (d_model): 768 • Maximum sequence length: 512 • Number of transformer blocks: 12 • Number of attention heads per block: 12 • Feed-forward network hidden dimension: 3,072 • Activation function: GELU
Note: In multi-head attention, the model dimension is split equally among all heads.
Based on the above data, answer the given subquestions.

What is the number of parameters in the token embedding matrix?

  1. A

    30,720,000

  2. B

    40,768

  3. C

    393,216

  4. D

    15,728,640,000

Show answer

Correct answer

  • A

    30,720,000

Question 15 on its own page

Question 16

+2 marksOne correct option

Consider the following Configuration for a GPT model:
• Vocabulary size: 40,000 tokens • Embedding dimension (d_model): 768 • Maximum sequence length: 512 • Number of transformer blocks: 12 • Number of attention heads per block: 12 • Feed-forward network hidden dimension: 3,072 • Activation function: GELU
Note: In multi-head attention, the model dimension is split equally among all heads.
Based on the above data, answer the given subquestions.

What is the total number of parameters in the positional embedding matrix?

  1. A

    512,768

  2. B

    393,216

  3. C

    3,932,160

  4. D

    39,321

Show answer

Correct answer

  • B

    393,216

Question 16 on its own page

Question 17

+2 marksOne correct option

Consider the following Configuration for a GPT model:
• Vocabulary size: 40,000 tokens • Embedding dimension (d_model): 768 • Maximum sequence length: 512 • Number of transformer blocks: 12 • Number of attention heads per block: 12 • Feed-forward network hidden dimension: 3,072 • Activation function: GELU
Note: In multi-head attention, the model dimension is split equally among all heads.
Based on the above data, answer the given subquestions.

For ONE complete attention head, what is the total number of parameters (Q + K + V projections combined)?

  1. A

    49,152

  2. B

    196,608

  3. C

    589,824

  4. D

    147,456

Show answer

Correct answer

  • D

    147,456

Question 17 on its own page

Question 18

+2 marksOne correct option

Consider the following Configuration for a GPT model:
• Vocabulary size: 40,000 tokens • Embedding dimension (d_model): 768 • Maximum sequence length: 512 • Number of transformer blocks: 12 • Number of attention heads per block: 12 • Feed-forward network hidden dimension: 3,072 • Activation function: GELU
Note: In multi-head attention, the model dimension is split equally among all heads.
Based on the above data, answer the given subquestions.

After the multi-head attention computation, all head outputs are concatenated and projected back to the model dimension.
How many parameters are in the output projection matrix (weights only, excluding bias)?

  1. A

    768

  2. B

    49,152

  3. C

    589,824

  4. D

    294,912

Show answer

Correct answer

  • C

    589,824

Question 18 on its own page

Question 19

+2 marksOne correct option

Consider the following Configuration for a GPT model:
• Vocabulary size: 40,000 tokens • Embedding dimension (d_model): 768 • Maximum sequence length: 512 • Number of transformer blocks: 12 • Number of attention heads per block: 12 • Feed-forward network hidden dimension: 3,072 • Activation function: GELU
Note: In multi-head attention, the model dimension is split equally among all heads.
Based on the above data, answer the given subquestions.

What is the total number of parameters in the complete FFN (including both weight matrices and bias vectors)?

  1. A

    2,359,296

  2. B

    4,718,592

  3. C

    4,722,432

  4. D

    9,437,184

Show answer

Correct answer

  • C

    4,722,432

Question 19 on its own page