Quiz Space

Large Language Models · Quiz 1 · 26 Oct 2025 · September 2025 term

LLM Quiz 1 26 Oct 2025 — Question 1

Question 1

+2 marksOne correct option
  1. A

    To increase the numerical precision of the attention scores.

  2. B

    To reduce the overall computational cost of the attention mechanism.

  3. C

    To ensure that the sum of the attention scores for each query equals 1.

  4. D

    To avoid numerical issues and loss of gradients during training.

Show answer

Correct answer

  • D

    To avoid numerical issues and loss of gradients during training.

See the whole paperTake it as a mock test

Question 1 of 20 in the IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 26 Oct 2025, in the September 2025 term (IIT M DEGREE AN EXAM QDB2 26 Oct 2025). It carries 2 marks.

More questions from this paper

  1. Q2Which of the following is true regarding sinusoidal encoding?
  2. Q3We have a transformer model translating English to French. Suppose the target sequence is: Target (French): [‘je’, ‘sui…
  3. Q4Figure question
  4. Q5In the context of language model training, teacher forcing is a technique where:
  5. Q6Figure question
  6. Q7You have a mini-batch of 2 samples, each with 2 features: X = \begin{bmatrix} 1 & 2 \ 3 & 4 \end{bmatrix} Rows represen…
  7. Q8Figure question
  8. Q9For a vocabulary size of 10, how many beams will there be in greedy search, beam search with beam size 3, and exhaustiv…
  9. Q10Which of the following decoding strategies is/are inherently non-deterministic?
  10. Q11Which of the following statements about multiple attention heads in a Transformer are true?
  11. Q12Which of the following statements about the Transformer architecture are true?
  12. Q13A transformer decoder uses a vocabulary of size 5. At step 1, the predicted token probabilities are: | Token | Probabil…
  13. Q14Figure question
  14. Q15A Transformer model processes a sequence of T = 1000 tokens. Within the self-attention layer, the attention scores are …
  15. Q16Figure question
  16. Q17How many next-token prediction targets are generated during training a full batch?
  17. Q18For one sequence, how many non-zero attention scores remain in the masked attention score matrix (per head)?
  18. Q19You are analyzing a GPT-style decoder-only Transformer trained as a causal language model with the following setup: Seq…
  19. Q20During autoregressive inference with GPT: