Quiz Space

Large Language Models · Quiz 2 · 16 Aug 2026 · May 2026 term

Question 6: A baseline T5 model with denoising objective on a span of…

Question 6

+3 marksOne or more correct options

A baseline T5 model with denoising objective on a span of corrupted tokens is used for unsupervised pre-training. Consider the phrase "life is like a box of chocolates". If the word "life" and the span "box of chocolates" are selected for corruption, which of the following statements correctly describe the training process?

Select all that apply.

  1. A

    The input sequence replaces the corrupted spans with unique sentinal tokens as: "<X> is like a <Y>".

  2. B

    The output sequence generated takes the form "<X> life <Y> box of chocolates <Z>", with the final sentinal token to indicate sequence completion.

  3. C

    The loss is calculated exclusively over the generated sentinal tokens and the missing text.

  4. D

    The decoder reconstructs the entire original sequence autoregressively to compute the total loss over all positions.

Show answer

Correct answers

  • A

    The input sequence replaces the corrupted spans with unique sentinal tokens as: "<X> is like a <Y>".

  • B

    The output sequence generated takes the form "<X> life <Y> box of chocolates <Z>", with the final sentinal token to indicate sequence completion.

  • C

    The loss is calculated exclusively over the generated sentinal tokens and the missing text.

See the whole paperTake it as a mock test

Question 6 of 18 in the IIT Madras BS Large Language Models (LLM) Quiz 2 paper sat on 16 Aug 2026, in the May 2026 term (Large Language Models 16 Aug 26). It carries 3 marks.

More questions from this paper

  1. Q1Answer the subquestions for the given transformer architecture for one (N = 1) Encoder Decoder Block In the given Trans…
  2. Q2Answer the subquestions for the given transformer architecture for one (N = 1) Encoder Decoder Block Identify the funct…
  3. Q3Answer the subquestions for the given transformer architecture for one (N = 1) Encoder Decoder Block Based on the archi…
  4. Q4Figure question
  5. Q5Which of the following options correctly reflect the internal state updates performed during a Byte Pair Encoding (BPE)…
  6. Q7During the fine-tuning of GPT-1 for Textual Entailment tasks, what is the purpose of the delimiter token (\$)?
  7. Q8During the fine-tuning of a decoder-only Transformer for sequence classification, why is the hidden representation of t…
  8. Q9Figure question
  9. Q10In the WordPiece tokenization algorithm, how does the scoring formula prioritize token merges?
  10. Q11Consider the T5 "text-to-text" paradigm. How does the model process fundamentally different tasks such as Semantic Simi…
  11. Q12In the context of the BART (Bidirectional AutoRegressive Transformer) architecture, how does the model handle corrupted…
  12. Q13When comparing the C4 dataset to an 'Unfiltered-C4' dataset that is 8 times larger, what was the observed effect on dow…
  13. Q14Which of the following describes the 'Adapter Layers' fine-tuning strategy?
  14. Q15Which of the following is a reported negative effect of including significantly duplicated content in LLM training data?
  15. Q16Figure question
  16. Q17Consider the following corpus consisting of 4 words (ignore spaces and punctuation). there is no spoon The vocabulary i…
  17. Q18Consider the following corpus consisting of 4 words (ignore spaces and punctuation). there is no spoon The vocabulary i…