Quiz Space

Large Language Models · Quiz 1 · 13 Jul 2025 · May 2025 term

Question 11: Suppose a language model outputs the following logits fo…

Question 11

+3 marksNumerical answer

Suppose a language model outputs the following logits for a vocabulary of size 4:

TokenLogit
the2.0
sky1.0
is0.5
blue−1.0-1.0

You apply temperature scaling with temperature T=0.5T = 0.5. After applying the softmax on the temperature-scaled logits, what is the probability assigned to the token “the”? (Give your answer up to 2 decimal places)

Show answer

Correct answer: 0.84 (accepted within ±0.03)

See the whole paperTake it as a mock test

Question 11 of 21 in the IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 13 Jul 2025, in the May 2025 term (IIT M DEGREE AN EXAM QDB2 13 July 2025). It carries 3 marks.

More questions from this paper

  1. Q1Transformers process input tokens:
  2. Q2What is the purpose of the softmax function in the attention mechanism?
  3. Q3What is the main difference between GPT and BERT pre-training objectives?
  4. Q4In Top-K sampling for language generation, increasing the value of K typically has which of the following effects?
  5. Q5Why are residual connections important in transformer architectures?
  6. Q6Which of the following statements are true regarding causal language modeling (CLM)?
  7. Q7Which of the following are valid reasons why transformer-based large language models are widely used in natural languag…
  8. Q8Which elements are included in BERT’s input representation for Next Sentence Prediction?
  9. Q9How does Top-p (nucleus) sampling differ from Top-K sampling in language generation?
  10. Q10Consider the embedding vector for a word, x = [0.5, -0.4, 0.3, -0.2]^T. Suppose the word is at position 3 in the given …
  11. Q12Figure question
  12. Q13Question Description Embedding dimension : d_{model} = 4 Input sequence (sequence length = 2, d_{model} = 4) : X = \beg…
  13. Q14Question Description Embedding dimension : d_{model} = 4 Input sequence (sequence length = 2, d_{model} = 4) : X = \beg…
  14. Q15Question Description Embedding dimension : d_{model} = 4 Input sequence (sequence length = 2, d_{model} = 4) : X = \beg…
  15. Q16For the given input matrix X and multihead attention output MHA(X), apply a residual connection and store the result in…
  16. Q17Based on the above data, answer the given subquestions.
  17. Q18Based on the above data, answer the given subquestions.
  18. Q19The table presents the conditional probability distribution over vocabulary tokens at each timestep during sequence gen…
  19. Q20The table presents the conditional probability distribution over vocabulary tokens at each timestep during sequence gen…
  20. Q21The table presents the conditional probability distribution over vocabulary tokens at each timestep during sequence gen…