Quiz Space

Deep Learning Practice · Quiz 1 · 27 Oct 2024 · September 2024 term

Question 6: Suppose that we pre-train a Causal Language Model. Choose…

Question 6

+3 marksOne correct option

Suppose that we pre-train a Causal Language Model. Choose the data collator function from the Hugging Face library that is suitable for this task

  1. A

    DataCollator(tokenizer)

  2. B

    DefaultDataCollator(tokenizer)

  3. C

    DataCollatorForLanguageModelling(tokenizer,mlm=False)

  4. D

    DataCollatorForCausalLanguageModelling(tokenizer)

  5. E

    DataLoader(tokenizer)

Show answer

Correct answer

  • C

    DataCollatorForLanguageModelling(tokenizer,mlm=False)

See the whole paperTake it as a mock test

Question 6 of 15 in the IIT Madras BS Deep Learning Practice (Deep Learning Practice) Quiz 1 paper sat on 27 Oct 2024, in the September 2024 term (IIT M DEGREE AN EXAM QDB2 27 Oct 2024). It carries 3 marks.

More questions from this paper

  1. Q1Figure question
  2. Q2Choose the Hugging Face module that helps us train a tokenizer from scratch on a specific dataset.
  3. Q3A dataset contains 10 billion words ( separated by a single white space). Suppose we use a pre- trained tokenizer that …
  4. Q4Which of the following tokenization algorithms can be applied to languages that do not have any word delimiters?
  5. Q5Consider the Wikipedia dataset scraped from the web that contains 2 billion words. A team decided to use the BPE tokeni…
  6. Q7The IMDB dataset has 25000 samples in the training split. It contains two columns, named, text and label. Consider the …
  7. Q8Consider two datasets namely "ds1" and "ds2" . The structure of the dataset with the number of samples in each split is…
  8. Q9Figure question
  9. Q10Figure question
  10. Q11Here is a configuration of the GPTNeo model from the Hugging Face hub. Figure 1: GPTNeoConfig Based on the above data, …
  11. Q12Here is a configuration of the GPTNeo model from the Hugging Face hub. Figure 1: GPTNeoConfig Based on the above data, …
  12. Q13Here is a set of training arguments used by the GPT-2 model that was pre-trained on a dataset that contains 10 billion …
  13. Q14Here is a set of training arguments used by the GPT-2 model that was pre-trained on a dataset that contains 10 billion …
  14. Q15Figure question