Quiz Space

Large Language Models · Quiz 2 · 3 Aug 2025 · May 2025 term

Question 9: What are the key challenges encountered when building a v…

Question 9

+3 marksOne or more correct options

What are the key challenges encountered when building a vocabulary for NLP models?

Select all that apply.

  1. A

    A larger vocabulary leads to a bigger embedding matrix and increased softmax computation

  2. B

    Difficulty in handling out-of-vocabulary (OOV) words

  3. C

    The size of each embedding vector increases with the number of out-of- vocabulary (OOV) words

  4. D

    More vocabulary entries always guarantee better model generalization

Show answer

Correct answers

  • A

    A larger vocabulary leads to a bigger embedding matrix and increased softmax computation

  • B

    Difficulty in handling out-of-vocabulary (OOV) words

See the whole paperTake it as a mock test

Question 9 of 17 in the IIT Madras BS Large Language Models (LLM) Quiz 2 paper sat on 3 Aug 2025, in the May 2025 term (IIT M IMPROVEMENT AN EXAM QIA2 03 Aug 2025). It carries 3 marks.

More questions from this paper

  1. Q1What is a primary motivation for using subword tokenization (like BPE or WordPiece) instead of word-level tokenization …
  2. Q2What is a key implication of scaling laws in large language models?
  3. Q3Consider the Following Assertion and Reason pair :\ Assertion : In Byte Pair Encoding (BPE), each merge operation adds …
  4. Q4What is the primary advantage of T5’s text-to-text approach for transfer learning in NLP?
  5. Q5What is the format of output the T5 model is expected to generate during its span-corruption pretraining?
  6. Q6Given the input string: peanutbutter And the following vocabulary of subword tokens with their corresponding log-probab…
  7. Q7Suppose you are given 4x more compute budget to scale a Transformer-based language model. According to scaling laws, wh…
  8. Q8In a data preprocessing stage designed to prepare clean, diverse, and safe data, which of the following examples is lik…
  9. Q10Select all the correct input-output representations used for different pretraining objectives based on the base sentenc…
  10. Q11In multi-task pretraining, tasks are sampled according to the Proportional Mixing Strategy: r_m = \frac{\min(e_m, K)}{\…
  11. Q12Consider the scaling law for pretraining test loss: L(N, D) = \left(\frac{N}{N_c}\right)^{-\alpha_N} + \left(\frac{D}{D…
  12. Q13Consider the following dictionary for the subword tokenization: Append </w> to each word and consider it for all …
  13. Q14Consider the following dictionary for the subword tokenization: Append </w> to each word and consider it for all …
  14. Q15Consider the following dictionary for the subword tokenization: Append </w> to each word and consider it for all …
  15. Q16Consider the following dictionary for the subword tokenization: Append </w> to each word and consider it for all …
  16. Q17Consider the following dictionary for the subword tokenization: Append </w> to each word and consider it for all …