Quiz Space

Large Language Models · Quiz 2 · 3 Aug 2025 · May 2025 term

Question 7: Suppose you are given 4x more compute budget to scale a T…

Question 7

+3 marksOne correct option

Suppose you are given 4x more compute budget to scale a Transformer-based language model. According to scaling laws, what is the best way to utilize this increased compute for optimal model performance?

  1. A

    Increase the model size (number of parameters) by 4x and keep the dataset size and training steps the same.

  2. B

    Train the same model on 4x more data for 4x more steps without changing the model size.

  3. C

    Increase the model size by approximately 2x and increase the amount of training data by approximately 2x.

  4. D

    Double the model size and halve the training data size to compensate.

Show answer

Correct answer

  • C

    Increase the model size by approximately 2x and increase the amount of training data by approximately 2x.

See the whole paperTake it as a mock test

Question 7 of 17 in the IIT Madras BS Large Language Models (LLM) Quiz 2 paper sat on 3 Aug 2025, in the May 2025 term (IIT M IMPROVEMENT AN EXAM QIA2 03 Aug 2025). It carries 3 marks.

More questions from this paper

  1. Q1What is a primary motivation for using subword tokenization (like BPE or WordPiece) instead of word-level tokenization …
  2. Q2What is a key implication of scaling laws in large language models?
  3. Q3Consider the Following Assertion and Reason pair :\ Assertion : In Byte Pair Encoding (BPE), each merge operation adds …
  4. Q4What is the primary advantage of T5’s text-to-text approach for transfer learning in NLP?
  5. Q5What is the format of output the T5 model is expected to generate during its span-corruption pretraining?
  6. Q6Given the input string: peanutbutter And the following vocabulary of subword tokens with their corresponding log-probab…
  7. Q8In a data preprocessing stage designed to prepare clean, diverse, and safe data, which of the following examples is lik…
  8. Q9What are the key challenges encountered when building a vocabulary for NLP models?
  9. Q10Select all the correct input-output representations used for different pretraining objectives based on the base sentenc…
  10. Q11In multi-task pretraining, tasks are sampled according to the Proportional Mixing Strategy: r_m = \frac{\min(e_m, K)}{\…
  11. Q12Consider the scaling law for pretraining test loss: L(N, D) = \left(\frac{N}{N_c}\right)^{-\alpha_N} + \left(\frac{D}{D…
  12. Q13Consider the following dictionary for the subword tokenization: Append </w> to each word and consider it for all …
  13. Q14Consider the following dictionary for the subword tokenization: Append </w> to each word and consider it for all …
  14. Q15Consider the following dictionary for the subword tokenization: Append </w> to each word and consider it for all …
  15. Q16Consider the following dictionary for the subword tokenization: Append </w> to each word and consider it for all …
  16. Q17Consider the following dictionary for the subword tokenization: Append </w> to each word and consider it for all …