Quiz Space

Large Language Models · Quiz 2 · 23 Nov 2025 · September 2025 term

Question 13: Consider the scaling law: where\ \= model parameters,\ \…

Question 13

+2 marksOne correct option

Consider the scaling law:

where
= model parameters,
= training tokens,
= scale constants,
= scaling exponents.
A team trains an LLM and observes the following losses for their model:
1.
Training Loss
2.
Test Loss (same distribution as training)
3.
Test Loss (different distribution from training)

Which loss does
represent?

  1. A

    Both 2 and 3

  2. B

    Both 1 and 2

  3. C

    1 only

  4. D

    2 only

Show answer

Correct answer

  • D

    2 only

See the whole paperTake it as a mock test

Question 13 of 22 in the IIT Madras BS Large Language Models (LLM) Quiz 2 paper sat on 23 Nov 2025, in the September 2025 term (IIT M DEGREE AN EXAM QDB2 23 Nov 2025 NEW). It carries 2 marks.

More questions from this paper

  1. Q1A corpus containing the following five words is tokenized using WordPiece algorithm. | Word | Frequency | |---|---| | p…
  2. Q2A corpus containing the following five words is tokenized using WordPiece algorithm. | Word | Frequency | |---|---| | p…
  3. Q3A corpus containing the following five words is tokenized using WordPiece algorithm. | Word | Frequency | |---|---| | p…
  4. Q4A study is comparing three architectures, all trained with the same compute budget:\ •\ Model A (Encoder-Decoder): A 12…
  5. Q5Consider a text corpus being processed using the Byte Pair Encoding (BPE) algorithm. The initial vocabulary contains\ t…
  6. Q6If Byte-Pair Encoding (BPE) starts with\ unique characters and performs\ merge operations, what will be the final vocab…
  7. Q7In a 4-token sequence, a Transformer is calculating the attention for the third Token. Its Query vector (\ ) must be co…
  8. Q8Which of the following is/are primarily multilingual datasets? Select all that apply.
  9. Q9Which decoding method ensures the same response every time for a given prompt (assuming no randomness in model weights)…
  10. Q10A data preprocessing pipeline removed data from the Common Crawl snapshot in three stages: 50% after language identific…
  11. Q11A data preprocessing pipeline needs to apply the following steps on raw Common Crawl data: 1.\ Fuzzy deduplication (usi…
  12. Q12Consider the following four pre-training scenarios for four different language models: Model A: Given “The cat sat on t…
  13. Q14A company is building a sentiment analysis system for customer reviews. They have two options: Option A: Fine-tune BERT…
  14. Q15Which of the following statements best describes the fundamental difference between the unsupervised pre-training objec…
  15. Q16An original document is: “The quick brown fox jumps over the lazy dog.” A corrupted input for pre-training is: “The qui…
  16. Q17A Transformer model is designed to handle both understanding and generation tasks. When given the input sequence\ , its…
  17. Q18BART is an encoder-decoder model. During its denoising pre-training, what is fed as input to the decoder?
  18. Q19Figure question
  19. Q20How does SentencePiece handle whitespace characters (like spaces) during tokenization?
  20. Q21For a layer with\ features, how many learnable parameters do BatchNorm and LayerNorm have?
  21. Q22In a standard Encoder-Decoder Transformer, where do the Query\ , Key\ , and Value inputs for the Decoder's Cross-Attent…