Quiz Space

Deep Learning Practice · Quiz 1 · 15 Mar 2026 · January 2026 term

Deep Learning Practice Quiz 1 15 Mar 2026 — Question 14

Question 14

+3 marksOne or more correct options
Figure from the original question paper

Select all that apply.

  1. A

    Task Conditioning: The model learns to follow natural language formatting (Instruction/Input/Answer) rather than just mapping a sequence to a single integer ID.

  2. B

    Open Vocabulary: The model's prediction head remains the size of its full vocabulary (e.g., 50k+ tokens), allowing it to generate any string as an answer instead of being restricted to fixed logits for classes.

  3. C

    Guaranteed Zero-Shot: This training format ensures the model will generalize perfectly to any unseen task prompt without further data.

  4. D

    Objective Shift: The training objective moves from minimizing cross-entropy loss over a discrete class index to minimizing next-token prediction loss over the sequence tokens.

Show answer

Correct answers

  • A

    Task Conditioning: The model learns to follow natural language formatting (Instruction/Input/Answer) rather than just mapping a sequence to a single integer ID.

  • B

    Open Vocabulary: The model's prediction head remains the size of its full vocabulary (e.g., 50k+ tokens), allowing it to generate any string as an answer instead of being restricted to fixed logits for classes.

  • D

    Objective Shift: The training objective moves from minimizing cross-entropy loss over a discrete class index to minimizing next-token prediction loss over the sequence tokens.

See the whole paperTake it as a mock test

Question 14 of 19 in the IIT Madras BS Deep Learning Practice (Deep Learning Practice) Quiz 1 paper sat on 15 Mar 2026, in the January 2026 term (Deep Learning Practice 15 Mar 26). It carries 3 marks.

More questions from this paper

  1. Q1Consider subword tokenization methods such as Byte Pair Encoding (BPE) and WordPiece as employed in transformer-based l…
  2. Q2Consider two subword tokenizers trained on the same text corpus. Tokenizer A uses a vocabulary of 8,000 tokens, whereas…
  3. Q3A subword tokenizer decomposes rare chemical entity names into a large number of fragments. During downstream fine-tuni…
  4. Q4Figure question
  5. Q5Figure question
  6. Q6In the standard Transformer architecture (e.g., GPT or BERT), which component exhibits a computational complexity that …
  7. Q7A model with 4B parameters (fp32) is being trained using the AdamW optimizer on a single 32GB GPU. Why is full fine-tun…
  8. Q8During the full fine-tuning of a Large Language Model, you aim to regularize the training process by penalizing the gro…
  9. Q9Why does full fine-tuning of large language models often require more memory than inference using the same model?
  10. Q10Which technique is most effective at reducing GPU memory usage during training without modifying the model architecture?
  11. Q11Consider the output of a Hugging Face tokenizer when applied to a batch of sequences with padding= "max_length" and ret…
  12. Q12Figure question
  13. Q13Which of the following statements correctly distinguish different Transformer architectures and their typical training …
  14. Q15A model is trained using the Causal Language Modeling (CLM) objective with a standard cross-entropy loss. Which of the …
  15. Q16When configuring TrainingArguments for a Transformer model, we include a learning rate warmup phase (e.g., warmup_steps…
  16. Q17A transformer-based language model has 1.2 billion parameters and is fine-tuned using AdamW. Assume parameters, gradien…
  17. Q18A GPT-style transformer block has an embedding dimension of 768 and uses 12 attention heads. Consider a specific proces…
  18. Q19You are training a decoder-only language model with a vocabulary size of 32,000, maximum context length of 2048, and em…