Quiz Space

Large Language Models · Quiz 1 · 13 Jul 2025 · May 2025 term

Question 6: Which of the following statements are true regarding caus…

Question 6

+3 marksOne or more correct options

Which of the following statements are true regarding causal language modeling (CLM)?

Select all that apply.

  1. A

    The model only attends to past and current tokens during training.

  2. B

    The model is trained by predicting the next token in a sequence.

  3. C

    The model uses bidirectional context.

  4. D

    The CLM objective is commonly used for encoder-only models.

Show answer

Correct answers

  • A

    The model only attends to past and current tokens during training.

  • B

    The model is trained by predicting the next token in a sequence.

See the whole paperTake it as a mock test

Question 6 of 21 in the IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 13 Jul 2025, in the May 2025 term (IIT M DEGREE AN EXAM QDB2 13 July 2025). It carries 3 marks.

More questions from this paper

  1. Q1Transformers process input tokens:
  2. Q2What is the purpose of the softmax function in the attention mechanism?
  3. Q3What is the main difference between GPT and BERT pre-training objectives?
  4. Q4In Top-K sampling for language generation, increasing the value of K typically has which of the following effects?
  5. Q5Why are residual connections important in transformer architectures?
  6. Q7Which of the following are valid reasons why transformer-based large language models are widely used in natural languag…
  7. Q8Which elements are included in BERT’s input representation for Next Sentence Prediction?
  8. Q9How does Top-p (nucleus) sampling differ from Top-K sampling in language generation?
  9. Q10Consider the embedding vector for a word, x = [0.5, -0.4, 0.3, -0.2]^T. Suppose the word is at position 3 in the given …
  10. Q11Suppose a language model outputs the following logits for a vocabulary of size 4: | Token | Logit | |---|---| | the | 2…
  11. Q12Figure question
  12. Q13Question Description Embedding dimension : d_{model} = 4 Input sequence (sequence length = 2, d_{model} = 4) : X = \beg…
  13. Q14Question Description Embedding dimension : d_{model} = 4 Input sequence (sequence length = 2, d_{model} = 4) : X = \beg…
  14. Q15Question Description Embedding dimension : d_{model} = 4 Input sequence (sequence length = 2, d_{model} = 4) : X = \beg…
  15. Q16For the given input matrix X and multihead attention output MHA(X), apply a residual connection and store the result in…
  16. Q17Based on the above data, answer the given subquestions.
  17. Q18Based on the above data, answer the given subquestions.
  18. Q19The table presents the conditional probability distribution over vocabulary tokens at each timestep during sequence gen…
  19. Q20The table presents the conditional probability distribution over vocabulary tokens at each timestep during sequence gen…
  20. Q21The table presents the conditional probability distribution over vocabulary tokens at each timestep during sequence gen…