The IMDB dataset has 25000 samples in the training split. It contains two columns, named, text and label. Consider the code snippet given below and choose all the correct statements from datasets import load_dataset imdb_dataset = load_dataset("stanfordnlp/imdb",split='train') def get_num_words(example): num_words = len(example["text"].split()) return {'num_words':num_words} ds = imdb_dataset.map(get_num_words)

(A) Option figure from the original question paper

(F) Option figure from the original question paper

(B) Option figure from the original question paper

(C) Option figure from the original question paper

(D) Option figure from the original question paper

(E) Option figure from the original question paper

Consider two datasets namely "ds1" and "ds2" . The structure of the dataset with the number of samples in each split is given below. Suppose we create a new dataset in the following ways. Assume necessary DatasetDict({ train: Dataset({ features: ['text', 'label'], num_rows: 25000 }) test: Dataset({ features: ['text', 'label'], num_rows: 25000 }) unsupervised: Dataset({ features: ['text', 'label'], num_rows: 50000 }) }) ds1 DatasetDict({ train: Dataset({ features: ['text', 'label'], num_rows: 8530 }) test: Dataset({ features: ['text', 'label'], num_rows: 1066 }) unsupervised: Dataset({ features: ['text', 'label'], num_rows: 1066 }) }) ds2 imports and the statements are executed independently (i.e., an error in executing a statement does not affect the execution of other statements). Select all the correct statements. ds3 = datasets.concatenate_datasets([ds1,ds2]) ds4 = datasets.concatenate_datasets([ds1['train'],ds2['train']]) ds5 = datasets.concatenate_datasets([ds1['train'],ds2['test']]) ds6 = datasets.concatenate_datasets( [ds1['train'],ds1['test'], ds2['train'],ds2['validation']])

(B) Option figure from the original question paper

(C) Option figure from the original question paper

(D) Option figure from the original question paper

(A) Option figure from the original question paper

Figure from the original question paper

(A) Option figure from the original question paper

(C) Option figure from the original question paper

(B) Option figure from the original question paper

September 2024 term · Deep Learning Practice · BSDA5013

Deep Learning Practice Quiz 1: 27 October 2024 (September 2024 term)

The IIT Madras BS Deep Learning Practice (Deep Learning Practice) Quiz 1 paper sat on 27 Oct 2024, in the September 2024 term: 15 questions for 50 marks in 120 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.

Questions: 15
Marks: 50
Duration: 120 min
MCQ: 6
MSQ: 4
Numerical: 5

Take as mock test Practise with answers

Updated 25 Sept 2026

Official paper: IIT M DEGREE AN EXAM QDB2 27 Oct 2024 · No negative marking.

Question 1

+4 marksOne correct option

A
Pre-Tokenization
B
Normalization
C
Post Processor
D
Tokenization Algorithm
E
Decoder

Show answer

Correct answer

C
Post Processor

Question 1 on its own page

Question 2

+3 marksOne correct option

Choose the Hugging Face module that helps us train a tokenizer from scratch on a specific dataset.

A
tokenizers
B
transformers
C
evaluate
D
Autotrain
E
Accelerate

Show answer

Correct answer

A
tokenizers

Question 2 on its own page

Question 3

+3 marksOne correct option

A dataset contains 10 billion words ( separated by a single white space). Suppose we use a pre- trained tokenizer that has a vocabulary of size 10,000 to tokenize the dataset, then the number of tokens in the dataset will always be greater than or equal to the number of words in the dataset. The statement is

A
True
B
False

Show answer

Correct answer

A
True

Question 3 on its own page

Question 4

+3 marksOne correct option

Which of the following tokenization algorithms can be applied to languages that do not have any word delimiters?

A
BPE (Byte Pair Encoding)
B
Wordpiece
C
Sentencepiece

Show answer

Correct answer

C
Sentencepiece

Question 4 on its own page

Question 5

+3 marksOne correct option

Consider the Wikipedia dataset scraped from the web that contains 2 billion words. A team decided to use the BPE tokenization algorithm with the varying vocabulary size from 2K to 52K, then the statement that increasing the number of merges will increase the size of the vocabulary is

A
True
B
False
C
Insufficient information

Show answer

Correct answer

A
True

Question 5 on its own page

Question 6

+3 marksOne correct option

Suppose that we pre-train a Causal Language Model. Choose the data collator function from the Hugging Face library that is suitable for this task

A
DataCollator(tokenizer)
B
DefaultDataCollator(tokenizer)
C
DataCollatorForLanguageModelling(tokenizer,mlm=False)
D
DataCollatorForCausalLanguageModelling(tokenizer)
E
DataLoader(tokenizer)

Show answer

Correct answer

C
DataCollatorForLanguageModelling(tokenizer,mlm=False)

Question 6 on its own page

Question 7

+4 marksOne or more correct options

The IMDB dataset has 25000 samples in the training split. It contains two columns, named, text and label. Consider the code snippet given below and choose all the correct statements

python

1from datasets import load_dataset
2imdb_dataset = load_dataset("stanfordnlp/imdb",split='train')
3
4def get_num_words(example):
5    num_words = len(example["text"].split())
6    return {'num_words':num_words}
7
8ds = imdb_dataset.map(get_num_words)

Select all that apply.

Show answer

Correct answers

Question 7 on its own page

Question 8

+4 marksOne or more correct options

Consider two datasets namely "ds1" and "ds2" . The structure of the dataset with the number of samples in each split is given below. Suppose we create a new dataset in the following ways. Assume necessary

python

1DatasetDict({
2    train: Dataset({
3        features: ['text', 'label'],
4        num_rows: 25000
5    })
6    test: Dataset({
7        features: ['text', 'label'],
8        num_rows: 25000
9    })
10    unsupervised: Dataset({
11        features: ['text', 'label'],
12        num_rows: 50000
13    })
14})

ds1

python

1DatasetDict({
2    train: Dataset({
3        features: ['text', 'label'],
4        num_rows: 8530
5    })
6    test: Dataset({
7        features: ['text', 'label'],
8        num_rows: 1066
9    })
10    unsupervised: Dataset({
11        features: ['text', 'label'],
12        num_rows: 1066
13    })
14})

ds2

imports and the statements are executed independently (i.e., an error in executing a statement does not affect the execution of other statements). Select all the correct statements.

python

1ds3 = datasets.concatenate_datasets([ds1,ds2])
2ds4 = datasets.concatenate_datasets([ds1['train'],ds2['train']])
3ds5 = datasets.concatenate_datasets([ds1['train'],ds2['test']])
4ds6 = datasets.concatenate_datasets(
5            [ds1['train'],ds1['test'],
6            ds2['train'],ds2['validation']])

Select all that apply.

Show answer

Correct answers

Question 8 on its own page

Question 9

+3 marksOne or more correct options

Select all that apply.

Show answer

Correct answers

Question 9 on its own page

Question 10

+3 marksOne or more correct options

Select all that apply.

A
ids
B
tokens
C
offsets
D
attention_mask
E
special_token_mask
F
type_ids
G
vocab_size

Show answer

Correct answers

A
ids
B
tokens
C
offsets
D
attention_mask
E
special_token_mask
F
type_ids

Question 10 on its own page

Question 11

+3 marksNumerical answer

Here is a configuration of the GPTNeo model from the Hugging Face hub.

text

1"bos_token_id": 50256,
2"classifier_dropout": 0.1,
3"embed_dropout": 0.0,
4"eos_token_id": 50256,
5"hidden_size": 2048,
6"initializer_range": 0.02,
7"intermediate_size": null,
8"layer_norm_epsilon": 1e-05,
9"max_position_embeddings": 2048,
10"model_type": "gpt_neo",
11"num_heads": 16,
12"num_layers": 24,
13"resid_dropout": 0.0,
14"transformers_version": "4.44.2",
15"use_cache": true,
16"vocab_size": 50257,
17"window_size": 256

Figure 1: GPTNeoConfig

Based on the above data, answer the given subquestions.

Enter the number of parameters in the embedding layer of the model in millions. For example, if the answer is 1234567. Then enter it as 1.23

Show answer

Correct answer: 102.5 (accepted within ±0.5)

Question 11 on its own page

Question 12

+3 marksNumerical answer

Here is a configuration of the GPTNeo model from the Hugging Face hub.

text

1"bos_token_id": 50256,
2"classifier_dropout": 0.1,
3"embed_dropout": 0.0,
4"eos_token_id": 50256,
5"hidden_size": 2048,
6"initializer_range": 0.02,
7"intermediate_size": null,
8"layer_norm_epsilon": 1e-05,
9"max_position_embeddings": 2048,
10"model_type": "gpt_neo",
11"num_heads": 16,
12"num_layers": 24,
13"resid_dropout": 0.0,
14"transformers_version": "4.44.2",
15"use_cache": true,
16"vocab_size": 50257,
17"window_size": 256

Figure 1: GPTNeoConfig

Based on the above data, answer the given subquestions.

Enter the context length.

Show answer

Correct answer: 2048

Question 12 on its own page

Question 13

+5 marksNumerical answer

Here is a set of training arguments used by the GPT-2 model that was pre-trained on a dataset that contains 10 billion tokens. The context length of the model is modified to 2048, the vocabulary size is 50,257 and the embedding dimension is 768. The length of all the samples in a batch is equal to the context length of the model.

python

1training_args = TrainingArguments( output_dir='out',
2                              evaluation_strategy="steps",
3                              eval_steps=500,
4                              num_train_epochs=1,
5                              per_device_train_batch_size=16,
6                              per_device_eval_batch_size=16,
7                              tf32=True,
8                              gradient_accumulation_steps=2,
9                              adam_beta1=0.9,
10                              adam_beta2=0.999,
11                              learning_rate=2e-5,
12                              weight_decay=0.01,
13                              logging_dir='logs',
14                              logging_strategy="steps",
15                              logging_steps = 500,
16                              save_steps=5000,
17                              save_total_limit=20,
18                              report_to='wandb',
19                            )

Based on the above data, answer the given subquestions.

Enter the number of tokens (in millions) processed by the model after 1000 steps. Enter the answer to 2 decimal places. For example, if your answer is 123456789, then enter it as 123.45.

Show answer

Correct answer: 65.5 (accepted within ±0.2)

Question 13 on its own page

Question 14

+3 marksNumerical answer

python

1training_args = TrainingArguments( output_dir='out',
2                              evaluation_strategy="steps",
3                              eval_steps=500,
4                              num_train_epochs=1,
5                              per_device_train_batch_size=16,
6                              per_device_eval_batch_size=16,
7                              tf32=True,
8                              gradient_accumulation_steps=2,
9                              adam_beta1=0.9,
10                              adam_beta2=0.999,
11                              learning_rate=2e-5,
12                              weight_decay=0.01,
13                              logging_dir='logs',
14                              logging_strategy="steps",
15                              logging_steps = 500,
16                              save_steps=5000,
17                              save_total_limit=20,
18                              report_to='wandb',
19                            )

Based on the above data, answer the given subquestions.

How many steps does it take to complete one epoch of training? Enter the answer in thousands (round down to an integer). For example, if your answer is 1234567.89, then enter it as 1234567.

Show answer

Correct answer: 152.5 (accepted within ±0.5)

Question 14 on its own page

Question 15

+3 marksNumerical answer

Show answer

Correct answer: 500

Question 15 on its own page