uiz Space

May 2025 term · Deep Learning Practice · BSDA5013

Deep Learning Practice Quiz 1: 13 July 2025 (May 2025 term)

The IIT Madras BS Deep Learning Practice (Deep Learning Practice) Quiz 1 paper sat on 13 Jul 2025, in the May 2025 term: 16 questions for 50 marks in 120 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.

Questions
16
Marks
50
Duration
120 min
MCQ
8
MSQ
3
Numerical
5
Take as mock testPractise with answers

Updated

Official paper: IIT M DEGREE AN EXAM QDB2 13 July 2025 · No negative marking.

Question 1

+2 marksOne correct option

A start-up is building a new language model for a low-resource language with many compound words and complex morphology. They are debating tokenization strategies. Which of the following approaches is most likely to offer the best balance between vocabulary size, handling OOV words, and capturing morphological variants effectively for this scenario?

  1. A

    Character-level tokenization

  2. B

    Word-level tokenization with a fixed vocabulary of 50,000 common words.

  3. C

    Subword tokenization (e.g., BPE or SentencePiece) trained on the available corpus.

  4. D

    Using only pre-defined special tokens and treating all other text as raw byte sequences.

Show answer

Correct answer

  • C

    Subword tokenization (e.g., BPE or SentencePiece) trained on the available corpus.

Question 1 on its own page

Question 2

+2 marksOne correct option

When fully fine-tuning a large pre-trained Transformer model (e.g., >1 Billion parameters), which of the following contributes LEAST significantly to the GPU memory bottleneck compared to the others?

  1. A

    Storing the model parameters themselves.

  2. B

    Storing the gradients for each parameter.

  3. C

    Storing the optimizer states (e.g., momentum and variance for Adam).

  4. D

    Storing the input batch data (token IDs).

Show answer

Correct answer

  • D

    Storing the input batch data (token IDs).

Question 2 on its own page

Question 3

+2 marksOne correct option

A research team wants their pre-trained language model to generate more helpful and harmless responses without extensive task-specific dataset collection. They have a collection of prompts and human-preferred responses. Which of the following techniques directly aligns with this goal and data?

  1. A

    Pre-training the model on a larger, more diverse text corpus.

  2. B

    Full fine-tuning on multiple downstream classification tasks.

  3. C

    Instruction Tuning using prompt-completion pairs or reformatting existing datasets into an instructional format.

  4. D

    Implementing Parameter-Efficient Fine-Tuning (PEFT) techniques like LoRa.

Show answer

Correct answer

  • C

    Instruction Tuning using prompt-completion pairs or reformatting existing datasets into an instructional format.

Question 3 on its own page

Question 4

+3 marksOne correct option

Consider the following Python code snippet using Hugging Face tokenizers:

python
from tokenizers import Tokenizer
from tokenizers.models import BPE
from tokenizers.pre_tokenizers import Whitespace
from tokenizers.normalizers import Lowercase
from tokenizers.processors import TemplateProcessing


tokenizer = Tokenizer(BPE(unk_token="[UNK]"))
tokenizer.normalizer = Lowercase()
tokenizer.pre_tokenizer = Whitespace()


tokenizer.post_processor = TemplateProcessing(
    single="[CLS] $A [SEP]",
    pair="[CLS] $A [SEP] $B:1 [SEP]:1",
    special_tokens=[("[CLS]", tokenizer.token_to_id("[CLS]")),
                    ("[SEP]", tokenizer.token_to_id("[SEP]"))],
)
encoded_output = tokenizer.encode("Hello World")

Which of the following attributes would be present in the encoded_output object and directly reflect the action of the TemplateProcessing post-processor for a single sequence as configured?

  1. A
  2. B
  3. C
  4. D
Show answer

Correct answer

  • D
Question 4 on its own page

Question 5

+3 marksOne correct option

Consider the following Python code snippet:

python
from datasets import Dataset


data = {"text": ["example one", "example two"]}
dataset = Dataset.from_dict(data)


def add_length(example):
  example["length"] = len(example["text"].split())
  return example


dataset = dataset.map(add_length)

After executing the code above, what will dataset.column_names return?

  1. A
  2. B
  3. C
  4. D
Show answer

Correct answer

  • C
Question 5 on its own page

Question 6

+4 marksOne or more correct options

Which of the following statements accurately describe common characteristics or goals of subword tokenization algorithms like BPE, WordPiece, or SentencePiece? (Select ALL that apply)

Select all that apply.

  1. A

    They aim to significantly reduce the vocabulary size compared to character- level tokenization.

  2. B

    They can handle out-of-vocabulary (OOV) words by breaking them into known subword units.

  3. C

    They primarily rely on merging the least frequent character or subword pairs to build the vocabulary.

  4. D

    They can represent common words as single tokens and rare words as sequences of subword tokens.

  5. E

    SentencePiece is designed to be language-agnostic, not requiring pre- segmentation based on spaces.

Show answer

Correct answers

  • B

    They can handle out-of-vocabulary (OOV) words by breaking them into known subword units.

  • D

    They can represent common words as single tokens and rare words as sequences of subword tokens.

  • E

    SentencePiece is designed to be language-agnostic, not requiring pre- segmentation based on spaces.

Question 6 on its own page

Question 7

+4 marksOne or more correct options

A team has a powerful pre-trained language model (e.g., a GPT-3 class model). They want to adapt it for a new summarization task but have very limited labeled summarization data and limited compute for full fine-tuning. Which of the following strategies could be viable and effective? (Select ALL that apply)

Select all that apply.

  1. A

    Zero-shot prompting by providing the text and an instruction like "Summarize this:".

  2. B

    Few-shot prompting (in-context learning) by providing a few examples of text and their summaries in the prompt before the target text.

  3. C

    Full supervised fine-tuning of all model parameters on the small labeled dataset.

  4. D

    Using a Parameter-Efficient Fine-Tuning (PEFT) method like LoRA on the small labeled dataset.

  5. E

    Collecting a much larger unlabeled corpus related to the summarization domain and continuing pre-training.

Show answer

Correct answers

  • B

    Few-shot prompting (in-context learning) by providing a few examples of text and their summaries in the prompt before the target text.

  • D

    Using a Parameter-Efficient Fine-Tuning (PEFT) method like LoRA on the small labeled dataset.

  • E

    Collecting a much larger unlabeled corpus related to the summarization domain and continuing pre-training.

Question 7 on its own page

Question 8

+4 marksOne or more correct options

Select all that apply.

  1. A
  2. B
  3. C
  4. D
  5. E
Show answer

Correct answers

  • B
  • D
Question 8 on its own page

Question 9

+3 marksNumerical answer

You start with an initial vocabulary consisting only of individual characters: {"a":10, "b":8, "c":5, "</w>":15}. Your corpus, after pre-tokenization and adding </w>, is:

text
a b c </w> (5 times)
a b </w> (3 times)
a a c </w> (2 times)

If you perform exactly one merge operation using the Byte Pair Encoding (BPE) algorithm (merging the most frequent adjacent pair), what will be the new size of your vocabulary (including initial characters and the newly merged token)?

Show answer

Correct answer: 5

Question 9 on its own page

Question 10

+3 marksNumerical answer

Consider the following Python code snippet:

python
from datasets import Dataset


data = {
    "text": ["short one", "a very long sentence indeed", "medium example three", "tiny"],
    "label": [0, 1, 0, 1]
}
dataset = Dataset.from_dict(data)


filtered_dataset = dataset.filter(lambda x: len(x['text'].split()) > 2)

How many samples will filtered_dataset contain after executing the code?

Show answer

Correct answer: 2

Question 10 on its own page

Question 11

+4 marksNumerical answer
Show answer

Correct answer: 4.7 (accepted within ±0.1)

Question 11 on its own page

Question 12

+3 marksOne correct option

Based on the above data, answer the given subquestions.

  1. A
  2. B
  3. C
  4. D
Show answer

Correct answer

  • B
Question 12 on its own page

Question 13

+4 marksNumerical answer

Based on the above data, answer the given subquestions.

Based on the provided configuration, calculate the total number of parameters in the model’s embedding layer (token embeddings) in millions. Enter your answer rounded to one decimal place.

Show answer

Correct answer: 32.8 (accepted within ±0.1)

Question 13 on its own page

Question 14

+3 marksOne correct option

Based on the above data, answer the given subquestions.

Based on the provided configuration, what is a primary characteristic of this language model’s architecture and training paradigm?

  1. A

    It’s an encoder-decoder model designed for sequence-to-sequence tasks like translation.

  2. B

    It’s an encoder-only model, likely using Masked Language Modeling for pre- training.

  3. C

    It’s a decoder-only model, pre-trained using a Causal Language Modeling objective.

  4. D

    It’s a small model primarily intended for edge devices due to its limited context length.

Show answer

Correct answer

  • C

    It’s a decoder-only model, pre-trained using a Causal Language Modeling objective.

Question 14 on its own page

Question 15

+4 marksNumerical answer

Based on the above data, answer the given subquestions.

Considering the Adam optimizer stores 2 floating-point values per model parameter and parameters are 32-bit floats (4 bytes), if the total number of trainable parameters in the model is exactly 350 Million, how much GPU memory (in Gigabytes, GB) would be required just for the optimizer states? (Assume 1 GB = 10⁹ bytes). Enter your answer rounded to one decimal place.

Show answer

Correct answer: 2.8 (accepted within ±0.1)

Question 15 on its own page

Question 16

+2 marksOne correct option

Based on the above data, answer the given subquestions.

The configuration states the model uses Byte Pair Encoding (BPE). What is a key implication of this choice for handling text from diverse sources during inference?

  1. A

    The model will be unable to process any words not seen during BPE vocabulary training, leading to frequent errors.

  2. B

    All input words will be tokenized into individual characters, increasing sequence length significantly.

  3. C

    Out-of-vocabulary words can be represented as sequences of known subword units, allowing the model to process them.

  4. D

    BPE ensures that every language will have roughly the same number of tokens for a text of similar semantic content.

Show answer

Correct answer

  • C

    Out-of-vocabulary words can be represented as sequences of known subword units, allowing the model to process them.

Question 16 on its own page