Figure from the original question paper

(A) ["hugging", "face", "is", "awesome"]

(C) ["hugging", "face", "is", "awesome", ""]

Consider the following code where a tokenizer is created using the BPE model: from tokenizers import Tokenizer from tokenizers.models import BPE from tokenizers.trainers import BpeTrainer from tokenizers.normalizers import Lowercase from tokenizers.pre_tokenizers import Whitespace data = ["existent", "non", "exist", "word"] # Create a tokenizer with BPE model without specifying the unk_token model = BPE() tokenizer = Tokenizer(model) # Normalizer and Pre-tokenizer tokenizer.normalizer = Lowercase() tokenizer.pre_tokenizer = Whitespace() # Train the tokenizer trainer = BpeTrainer(vocab_size=5000, special_tokens=[" ", " ", " "]) tokenizer.train_from_iterator(data, trainer) # Access the tokenizer's vocabulary vocab = tokenizer.get_vocab() # Test tokenization with a word not present in the vocabulary tokens = tokenizer.encode("nonexistentword").tokens print("Tokens:", tokens) Which of the following statements is correct about the tokenization output when the word "nonexistentword" is encountered?

(C) The tokenizer will output "non", "existent", "word" as separate tokens since it has split the word into known subwords.

(A) The tokenizer will output the token [UNK] because "nonexistentword" is not in the vocabulary.

(B) The tokenizer will output the word "nonexistentword" as a single token.

(D) The tokenizer will output "non", "exist", "ent", "word" as separate tokens since it has split the word into known subwords.

(E) The tokenizer will output an error because "nonexistentword" is not present in the vocabulary and the unk_token was not defined.

Given the following code snippet: from transformers import AutoModelForSequenceClassification from peft import LoraConfig base_model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=2) lora_config = LoraConfig( r=8, lora_alpha=16, lora_dropout=0.1 ) Which of the following options correctly adds LoRA to the base_model? (Whichever option you choose, assume the corresponding imports already done for them)

(D) Option figure from the original question paper

(A) Option figure from the original question paper

(B) Option figure from the original question paper

(C) Option figure from the original question paper

Given the following code for setting up the trainer: from transformers import Trainer, TrainingArguments training_args = TrainingArguments( output_dir="./results", evaluation_strategy="epoch", save_strategy="epoch", logging_dir="./logs" ) trainer = Trainer( model=peft_model, args=training_args, train_dataset=dummy_train_dataset, eval_dataset=dummy_eval_dataset ) Which of the following lines correctly starts the fine-tuning process?

(B) Option figure from the original question paper

(A) Option figure from the original question paper

(C) Option figure from the original question paper

(D) Option figure from the original question paper

What would happen if you set the target_modules parameter in the LoraConfig to an empty list []? lora_config = LoraConfig( task_type="SEQ_CLASSIFICATION", r=8, lora_alpha=32, lora_dropout=0.1, target_modules=[] )

(C) LoRA will not be applied to any layer of the model.

(A) LoRA will be applied to the entire model, including the embedding layers.

(B) LoRA will only be applied to the final classification layer of the model.

(D) The model will throw an error because an empty list is invalid.

Consider the following code where the tokenizer is trained using two different vocabulary sizes: 5K and 50K. The function train_and_tokenize trains a tokenizer with the specified vocabulary size and tokenizes the word "unhappiness". The number of tokens produced for each vocabulary size is stored in n_t_5k and n_t_50k. (Assume get_examples() returns an iterator on some dataset) from tokenizers import Tokenizer from tokenizers.models import BPE from tokenizers.trainers import BpeTrainer from tokenizers.normalizers import Lowercase from tokenizers.pre_tokenizers import Whitespace # Function to train and tokenize using a given vocabulary size def train_and_tokenize(vocab_size): tokenizer = Tokenizer(BPE()) tokenizer.normalizer = Lowercase() tokenizer.pre_tokenizer = Whitespace() # Train the tokenizer with the specified vocabulary size trainer = BpeTrainer(vocab_size=vocab_size, special_tokens=[" ", " ", " ", " "]) # tokenizer.train_from_iterator(["unhappiness", "happiness", "unhappy", "joy"], trainer) tokenizer.train_from_iterator(get_examples(batch_size = 1000), trainer) # Tokenize the dataset (let's assume we are using a simple list of words) tokens = tokenizer.encode("unhappiness").tokens return tokens print(train_and_tokenize(5000)) print(train_and_tokenize(50000)) # Tokenize using vocab size 5K and 50K n_t_5k = len(train_and_tokenize(5000)) n_t_50k = len(train_and_tokenize(50000)) # Print results for both vocab sizes print("Tokens with vocab size 5K:", n_t_5k)…

(A) n_t_5k > n_t_50k because with a smaller vocabulary size, the tokenizer will split more words into smaller subwords, leading to a higher number of tokens.

(B) n_t_5k \< n_t_50k because a smaller vocabulary size means the tokenizer can only split the words into fewer subwords, resulting in fewer tokens.

(C) n_t_5k = n_t_50k because the number of tokens is independent of the vocabulary size and depends on the input words.

(D) n_t_5k = n_t_50k because both vocabularies will represent the word "unhappiness" with the same number of tokens due to the use of subword tokenization.

Figure from the original question paper

(A) Option figure from the original question paper

(C) Option figure from the original question paper

(B) Option figure from the original question paper

(D) Option figure from the original question paper

Figure from the original question paper Select all the correct statements.

(A) Option figure from the original question paper

(B) Option figure from the original question paper

(D) Option figure from the original question paper

(C) Option figure from the original question paper

You are working with two datasets and trying to combine them using the following code: from datasets import concatenate_datasets, load_dataset ds1 = load_dataset("imdb", split="train[:4000]") ds2 = load_dataset("csv", data_files="reviews.csv") combined_dataset = concatenate_datasets([ds1, ds2]) Assume the following: The ds1 dataset has columns: text (string), label (int).\ The ds2 dataset has columns: review (string), sentiment (int). Select all the correct statements.

(A) Option figure from the original question paper

(C) Option figure from the original question paper

(B) Option figure from the original question paper

(D) Option figure from the original question paper

(E) Option figure from the original question paper

Below is a snippet for loading the GPT-2 model configuration: from transformers import GPT2Config config = GPT2Config.from_pretrained("gpt2-medium") print(config) the output of the above is GPT2Config { "activation_function": "gelu_new", "architectures": [ "GPT2LMHeadModel" ], "attn_pdrop": 0.1, "bos_token_id": 50256, "embd_pdrop": 0.1, "eos_token_id": 50256, "initializer_range": 0.02, "layer_norm_epsilon": 1e-05, "model_type": "gpt2", "n_ctx": 1024, "n_embd": 1024, "n_head": 16, "n_inner": null, "n_layer": 24, "n_positions": 1024, "n_special": 0, "predict_special_tokens": true, "reorder_and_upcast_attn": false, "resid_pdrop": 0.1, "scale_attn_by_inverse_layer_idx": false, "scale_attn_weights": true, "summary_activation": null, "summary_first_dropout": 0.1, "summary_proj_to_labels": true, "summary_type": "cls_index", "summary_use_proj": true, "task_specific_params": { "text-generation": { "do_sample": true, "max_length": 50 } }, "transformers_version": "4.47.1", "use_cache": true, "vocab_size": 50257 } Based on the above data, answer the given subquestions. What does the n_ctx parameter in the GPT-2 configuration represent?

(B) The maximum length of the input sequence in tokens.

(A) The number of attention heads in the model.

(C) The embedding size of each token.

(D) The total number of parameters in the model.

January 2025 term · Deep Learning Practice · BSDA5013

Deep Learning Practice Quiz 1: 23 February 2025 (January 2025 term)

The IIT Madras BS Deep Learning Practice (Deep Learning Practice) Quiz 1 paper sat on 23 Feb 2025, in the January 2025 term: 16 questions for 50 marks in 120 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.

Questions: 16
Marks: 50
Duration: 120 min
MCQ: 9
MSQ: 4
Numerical: 3

Take as mock test Practise with answers

Updated 25 Sept 2026

Official paper: IIT M DEGREE AN EXAM QDB2 23 Feb 2025 · No negative marking.

Question 1

+3 marksOne correct option

A
["hugging", "face", "is", "awesome"]
B
[102, 463, 509, 101, 2020]
C
["hugging", "face", "is", "awesome", ""]
D
"Hugging Face is awesome!"

Show answer

Correct answer

B
[102, 463, 509, 101, 2020]

Question 1 on its own page

Question 2

+3 marksOne correct option

Consider the following code where a tokenizer is created using the BPE model:

python

1from tokenizers import Tokenizer
2from tokenizers.models import BPE
3from tokenizers.trainers import BpeTrainer
4from tokenizers.normalizers import Lowercase
5from tokenizers.pre_tokenizers import Whitespace
6
7data = ["existent", "non", "exist", "word"]
8
9# Create a tokenizer with BPE model without specifying the unk_token
10model = BPE()
11tokenizer = Tokenizer(model)
12
13# Normalizer and Pre-tokenizer
14tokenizer.normalizer = Lowercase()
15tokenizer.pre_tokenizer = Whitespace()
16
17# Train the tokenizer
18trainer = BpeTrainer(vocab_size=5000, special_tokens=["<s>", "</s>", "<pad>"])
19tokenizer.train_from_iterator(data, trainer)
20
21# Access the tokenizer's vocabulary
22vocab = tokenizer.get_vocab()
23
24# Test tokenization with a word not present in the vocabulary
25
26tokens = tokenizer.encode("nonexistentword").tokens
27print("Tokens:", tokens)

Which of the following statements is correct about the tokenization output when the word "nonexistentword" is encountered?

A
The tokenizer will output the token [UNK] because "nonexistentword" is not in the vocabulary.
B
The tokenizer will output the word "nonexistentword" as a single token.
C
The tokenizer will output "non", "existent", "word" as separate tokens since it has split the word into known subwords.
D
The tokenizer will output "non", "exist", "ent", "word" as separate tokens since it has split the word into known subwords.
E
The tokenizer will output an error because "nonexistentword" is not present in the vocabulary and the unk_token was not defined.

Show answer

Correct answer

C
The tokenizer will output "non", "existent", "word" as separate tokens since it has split the word into known subwords.

Question 2 on its own page

Question 3

+3 marksOne correct option

Consider the following code snippet:

python

1from tokenizers import Tokenizer
2from tokenizers.models import BPE
3from tokenizers.trainers import BpeTrainer
4from tokenizers.normalizers import Lowercase
5from tokenizers.pre_tokenizers import Whitespace
6from tokenizers.processors import TemplateProcessing
7
8data = ["hello", "world"]
9
10# Create a tokenizer with BPE model
11model = BPE()
12tokenizer = Tokenizer(model)
13
14# Normalizer and Pre-tokenizer
15tokenizer.normalizer = Lowercase()
16tokenizer.pre_tokenizer = Whitespace()
17
18# Trainer for the tokenizer
19trainer = BpeTrainer(vocab_size=5000, special_tokens=["<s>", "</s>", "<pad>", "<unk>"])
20tokenizer.train_from_iterator(data, trainer)
21
22# Post-processor
23tokenizer.post_processor = TemplateProcessing(single="[CLS] $0 [SEP]",
24                                              special_tokens=[("[CLS]", 2), ("[SEP]", 3)])
25
26# Encode input and get the token IDs
27encoded = tokenizer.encode("Hello world")
28print("Token IDs:", encoded.ids)

What will likely be printed as the output?

A
[2, 3]
B
[3, 0, 2]
C
[2, 0, 1, 3]
D
[3, 1, 0, 2]

Show answer

Correct answer

C
[2, 0, 1, 3]

Question 3 on its own page

Question 4

+3 marksOne correct option

Given the following code snippet:

python

1from transformers import AutoModelForSequenceClassification
2from peft import LoraConfig
3
4base_model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=2)
5lora_config = LoraConfig(
6    r=8,
7    lora_alpha=16,
8    lora_dropout=0.1
9)

Which of the following options correctly adds LoRA to the base_model? (Whichever option you choose, assume the corresponding imports already done for them)

Show answer

Correct answer

Question 4 on its own page

Question 5

+3 marksOne correct option

Given the following code for setting up the trainer:

python

1from transformers import Trainer, TrainingArguments
2
3training_args = TrainingArguments(
4    output_dir="./results",
5    evaluation_strategy="epoch",
6    save_strategy="epoch",
7    logging_dir="./logs"
8)
9
10trainer = Trainer(
11    model=peft_model,
12    args=training_args,
13    train_dataset=dummy_train_dataset,
14    eval_dataset=dummy_eval_dataset
15)

Which of the following lines correctly starts the fine-tuning process?

Show answer

Correct answer

Question 5 on its own page

Question 6

+3 marksOne correct option

What would happen if you set the target_modules parameter in the LoraConfig to an empty list []?

python

1lora_config = LoraConfig(
2    task_type="SEQ_CLASSIFICATION",
3    r=8,
4    lora_alpha=32,
5    lora_dropout=0.1,
6    target_modules=[]
7)

A
LoRA will be applied to the entire model, including the embedding layers.
B
LoRA will only be applied to the final classification layer of the model.
C
LoRA will not be applied to any layer of the model.
D
The model will throw an error because an empty list is invalid.

Show answer

Correct answer

C
LoRA will not be applied to any layer of the model.

Question 6 on its own page

Question 7

+4 marksOne correct option

Consider the following code where the tokenizer is trained using two different vocabulary sizes: 5K and 50K. The function train_and_tokenize trains a tokenizer with the specified vocabulary size and tokenizes the word "unhappiness". The number of tokens produced for each vocabulary size is stored in n_t_5k and n_t_50k. (Assume get_examples() returns an iterator on some dataset)

python

1from tokenizers import Tokenizer
2from tokenizers.models import BPE
3from tokenizers.trainers import BpeTrainer
4from tokenizers.normalizers import Lowercase
5from tokenizers.pre_tokenizers import Whitespace
6
7# Function to train and tokenize using a given vocabulary size
8def train_and_tokenize(vocab_size):
9    tokenizer = Tokenizer(BPE())
10    tokenizer.normalizer = Lowercase()
11    tokenizer.pre_tokenizer = Whitespace()
12
13    # Train the tokenizer with the specified vocabulary size
14    trainer = BpeTrainer(vocab_size=vocab_size, special_tokens=["<unk>", "<s>", "</s>", "<pad>"])
15    # tokenizer.train_from_iterator(["unhappiness", "happiness", "unhappy", "joy"], trainer)
16    tokenizer.train_from_iterator(get_examples(batch_size = 1000), trainer)
17
18    # Tokenize the dataset (let's assume we are using a simple list of words)
19    tokens = tokenizer.encode("unhappiness").tokens
20    return tokens
21print(train_and_tokenize(5000))
22print(train_and_tokenize(50000))
23
24# Tokenize using vocab size 5K and 50K
25n_t_5k = len(train_and_tokenize(5000))
26n_t_50k = len(train_and_tokenize(50000))
27
28# Print results for both vocab sizes
29print("Tokens with vocab size 5K:", n_t_5k)
30print("Tokens with vocab size 50K:", n_t_50k)

Which of the following is the likely behavior regarding the number of tokens (n_t_5k and n_t_50k) produced when using a vocabulary size of 5K vs. 50K?

A
n_t_5k > n_t_50k because with a smaller vocabulary size, the tokenizer will split more words into smaller subwords, leading to a higher number of tokens.
B
n_t_5k < n_t_50k because a smaller vocabulary size means the tokenizer can only split the words into fewer subwords, resulting in fewer tokens.
C
n_t_5k = n_t_50k because the number of tokens is independent of the vocabulary size and depends on the input words.
D
n_t_5k = n_t_50k because both vocabularies will represent the word "unhappiness" with the same number of tokens due to the use of subword tokenization.

Show answer

Correct answer

A
n_t_5k > n_t_50k because with a smaller vocabulary size, the tokenizer will split more words into smaller subwords, leading to a higher number of tokens.

Question 7 on its own page

Question 8

+2 marksOne correct option

What type of language modeling objective is used during the pretraining of GPT-2?

A
Masked Language Modeling (MLM)
B
Sequence-to-Sequence Modeling
C
Causal Language Modeling (CLM)
D
Text Classification Objective

Show answer

Correct answer

C
Causal Language Modeling (CLM)

Question 8 on its own page

Question 9

+3 marksOne or more correct options

Which of the following is/are gradient based (fine tuning) methods?

Select all that apply.

A
Zero-shot Tuning
B
Few-shots Tuning
C
Instruction Fine tuning
D
Parameter Efficient Fine Tuning
E
Preference Tuning Via RLHF

Show answer

Correct answers

C
Instruction Fine tuning
D
Parameter Efficient Fine Tuning
E
Preference Tuning Via RLHF

Question 9 on its own page

Question 10

+3 marksOne or more correct options

Select all that apply.

Show answer

Correct answers

Question 10 on its own page

Question 11

+3 marksOne or more correct options

Select all the correct statements.

Select all that apply.

Show answer

Correct answers

Question 11 on its own page

Question 12

+3 marksOne or more correct options

You are working with two datasets and trying to combine them using the following code:

python

1from datasets import concatenate_datasets, load_dataset
2
3ds1 = load_dataset("imdb", split="train[:4000]")
4ds2 = load_dataset("csv", data_files="reviews.csv")
5
6combined_dataset = concatenate_datasets([ds1, ds2])

Assume the following:

The ds1 dataset has columns: text (string), label (int).
The ds2 dataset has columns: review (string), sentiment (int).

Select all the correct statements.

Select all that apply.

Show answer

Correct answers

Question 12 on its own page

Question 13

+3 marksNumerical answer

The original train split contains 25,000 samples, and 60% of the samples have a text length greater than 200. If the dataset's text length distribution is uniform across all splits, how many samples will the subset dataset contain?

Show answer

Correct answer: 3750

Question 13 on its own page

Question 14

+4 marksNumerical answer

Below is a snippet for loading the GPT-2 model configuration:

python

1from transformers import GPT2Config
2
3config = GPT2Config.from_pretrained("gpt2-medium")
4print(config)

the output of the above is

text

1GPT2Config {
2  "activation_function": "gelu_new",
3  "architectures": [
4    "GPT2LMHeadModel"
5  ],
6  "attn_pdrop": 0.1,
7  "bos_token_id": 50256,
8  "embd_pdrop": 0.1,
9  "eos_token_id": 50256,
10  "initializer_range": 0.02,
11  "layer_norm_epsilon": 1e-05,
12  "model_type": "gpt2",
13  "n_ctx": 1024,
14  "n_embd": 1024,
15  "n_head": 16,
16  "n_inner": null,
17  "n_layer": 24,
18  "n_positions": 1024,
19  "n_special": 0,
20  "predict_special_tokens": true,
21  "reorder_and_upcast_attn": false,
22  "resid_pdrop": 0.1,
23  "scale_attn_by_inverse_layer_idx": false,
24  "scale_attn_weights": true,
25  "summary_activation": null,
26  "summary_first_dropout": 0.1,
27  "summary_proj_to_labels": true,
28  "summary_type": "cls_index",
29  "summary_use_proj": true,
30  "task_specific_params": {
31    "text-generation": {
32      "do_sample": true,
33      "max_length": 50
34    }
35   },
36   "transformers_version": "4.47.1",
37   "use_cache": true,
38   "vocab_size": 50257
39 }

Based on the above data, answer the given subquestions.

Calculate the number of parameters in the query, key, and value matrices of a single transformer block.

Show answer

Correct answer: 3145728

Question 14 on its own page

Question 15

+4 marksNumerical answer

Below is a snippet for loading the GPT-2 model configuration:

python

1from transformers import GPT2Config
2
3config = GPT2Config.from_pretrained("gpt2-medium")
4print(config)

the output of the above is

text

1GPT2Config {
2  "activation_function": "gelu_new",
3  "architectures": [
4    "GPT2LMHeadModel"
5  ],
6  "attn_pdrop": 0.1,
7  "bos_token_id": 50256,
8  "embd_pdrop": 0.1,
9  "eos_token_id": 50256,
10  "initializer_range": 0.02,
11  "layer_norm_epsilon": 1e-05,
12  "model_type": "gpt2",
13  "n_ctx": 1024,
14  "n_embd": 1024,
15  "n_head": 16,
16  "n_inner": null,
17  "n_layer": 24,
18  "n_positions": 1024,
19  "n_special": 0,
20  "predict_special_tokens": true,
21  "reorder_and_upcast_attn": false,
22  "resid_pdrop": 0.1,
23  "scale_attn_by_inverse_layer_idx": false,
24  "scale_attn_weights": true,
25  "summary_activation": null,
26  "summary_first_dropout": 0.1,
27  "summary_proj_to_labels": true,
28  "summary_type": "cls_index",
29  "summary_use_proj": true,
30  "task_specific_params": {
31    "text-generation": {
32      "do_sample": true,
33      "max_length": 50
34    }
35   },
36   "transformers_version": "4.47.1",
37   "use_cache": true,
38   "vocab_size": 50257
39 }

Based on the above data, answer the given subquestions.

Calculate the number of embedding parameters in the model. Enter your answer in millions, rounded to two decimal places.

Show answer

Correct answer: 51.5 (accepted within ±0.5)

Question 15 on its own page

Question 16

+3 marksOne correct option

Below is a snippet for loading the GPT-2 model configuration:

python

1from transformers import GPT2Config
2
3config = GPT2Config.from_pretrained("gpt2-medium")
4print(config)

the output of the above is

text

1GPT2Config {
2  "activation_function": "gelu_new",
3  "architectures": [
4    "GPT2LMHeadModel"
5  ],
6  "attn_pdrop": 0.1,
7  "bos_token_id": 50256,
8  "embd_pdrop": 0.1,
9  "eos_token_id": 50256,
10  "initializer_range": 0.02,
11  "layer_norm_epsilon": 1e-05,
12  "model_type": "gpt2",
13  "n_ctx": 1024,
14  "n_embd": 1024,
15  "n_head": 16,
16  "n_inner": null,
17  "n_layer": 24,
18  "n_positions": 1024,
19  "n_special": 0,
20  "predict_special_tokens": true,
21  "reorder_and_upcast_attn": false,
22  "resid_pdrop": 0.1,
23  "scale_attn_by_inverse_layer_idx": false,
24  "scale_attn_weights": true,
25  "summary_activation": null,
26  "summary_first_dropout": 0.1,
27  "summary_proj_to_labels": true,
28  "summary_type": "cls_index",
29  "summary_use_proj": true,
30  "task_specific_params": {
31    "text-generation": {
32      "do_sample": true,
33      "max_length": 50
34    }
35   },
36   "transformers_version": "4.47.1",
37   "use_cache": true,
38   "vocab_size": 50257
39 }

Based on the above data, answer the given subquestions.

What does the n_ctx parameter in the GPT-2 configuration represent?

A
The number of attention heads in the model.
B
The maximum length of the input sequence in tokens.
C
The embedding size of each token.
D
The total number of parameters in the model.

Show answer

Correct answer

B
The maximum length of the input sequence in tokens.

Question 16 on its own page