Quiz Space

Large Language Models · Quiz 2 · 23 Nov 2025 · September 2025 term

Question 15: Which of the following statements best describes the fun…

Question 15

+2 marksOne correct option

Which of the following statements best describes the fundamental difference between the unsupervised pre-training objective used in T5 (span corruption) and its supervised fine-tuning objectives such as translation or summarization?

  1. A

    The pre-training objective is a denoising task, while the fine-tuning objective is a Causal Language Model (CLM) task.

  2. B

    There is no fundamental difference; both are treated as "text-to-text" tasks, where the model is trained to generate a target sequence given an input sequence.

  3. C

    The pre-training objective trains the model to fill in [MASK] tokens, while the fine-tuning objective trains the model to generate text from a [CLS] token.

  4. D

    The pre-training objective trains the entire encoder-decoder, while the fine- tuning objective freezes the encoder and only trains the decoder.

Show answer

Correct answer

  • B

    There is no fundamental difference; both are treated as "text-to-text" tasks, where the model is trained to generate a target sequence given an input sequence.

See the whole paperTake it as a mock test

Question 15 of 22 in the IIT Madras BS Large Language Models (LLM) Quiz 2 paper sat on 23 Nov 2025, in the September 2025 term (IIT M DEGREE AN EXAM QDB2 23 Nov 2025 NEW). It carries 2 marks.

More questions from this paper

  1. Q1A corpus containing the following five words is tokenized using WordPiece algorithm. | Word | Frequency | |---|---| | p…
  2. Q2A corpus containing the following five words is tokenized using WordPiece algorithm. | Word | Frequency | |---|---| | p…
  3. Q3A corpus containing the following five words is tokenized using WordPiece algorithm. | Word | Frequency | |---|---| | p…
  4. Q4A study is comparing three architectures, all trained with the same compute budget:\ •\ Model A (Encoder-Decoder): A 12…
  5. Q5Consider a text corpus being processed using the Byte Pair Encoding (BPE) algorithm. The initial vocabulary contains\ t…
  6. Q6If Byte-Pair Encoding (BPE) starts with\ unique characters and performs\ merge operations, what will be the final vocab…
  7. Q7In a 4-token sequence, a Transformer is calculating the attention for the third Token. Its Query vector (\ ) must be co…
  8. Q8Which of the following is/are primarily multilingual datasets? Select all that apply.
  9. Q9Which decoding method ensures the same response every time for a given prompt (assuming no randomness in model weights)…
  10. Q10A data preprocessing pipeline removed data from the Common Crawl snapshot in three stages: 50% after language identific…
  11. Q11A data preprocessing pipeline needs to apply the following steps on raw Common Crawl data: 1.\ Fuzzy deduplication (usi…
  12. Q12Consider the following four pre-training scenarios for four different language models: Model A: Given “The cat sat on t…
  13. Q13Consider the scaling law: where\ \= model parameters,\ \= training tokens,\ \= scale constants,\ \= scaling exponents.\…
  14. Q14A company is building a sentiment analysis system for customer reviews. They have two options: Option A: Fine-tune BERT…
  15. Q16An original document is: “The quick brown fox jumps over the lazy dog.” A corrupted input for pre-training is: “The qui…
  16. Q17A Transformer model is designed to handle both understanding and generation tasks. When given the input sequence\ , its…
  17. Q18BART is an encoder-decoder model. During its denoising pre-training, what is fed as input to the decoder?
  18. Q19Figure question
  19. Q20How does SentencePiece handle whitespace characters (like spaces) during tokenization?
  20. Q21For a layer with\ features, how many learnable parameters do BatchNorm and LayerNorm have?
  21. Q22In a standard Encoder-Decoder Transformer, where do the Query\ , Key\ , and Value inputs for the Decoder's Cross-Attent…