Figure from the original question paper

(C) Option figure from the original question paper

(A) Option figure from the original question paper

(B) Option figure from the original question paper

(D) Option figure from the original question paper

After running the following Hugging Face datasets code snippet, what will be the features of the dataset? from datasets import Dataset data = {"id": [1, 2, 3], "text": ["Review one", "Review two", "Review three"]} dataset = Dataset.from_dict(data) def classify_sentiment(example): example["sentiment"] = "positive" if "one" in example["text"] else "neutral" return example dataset = dataset.map(classify_sentiment)

(C) Option figure from the original question paper

(A) Option figure from the original question paper

(B) Option figure from the original question paper

(D) Option figure from the original question paper

September 2025 term · Deep Learning Practice · BSDA5013

Deep Learning Practice Quiz 1: 26 October 2025 (September 2025 term)

The IIT Madras BS Deep Learning Practice (Deep Learning Practice) Quiz 1 paper sat on 26 Oct 2025, in the September 2025 term: 15 questions for 50 marks in 120 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.

Questions: 15
Marks: 50
Duration: 120 min
MCQ: 7
MSQ: 5
Numerical: 3

Take as mock test Practise with answers

Updated 25 Sept 2026

Official paper: IIT M DEGREE AN EXAM QDB2 26 Oct 2025 · No negative marking.

Question 1

+2 marksOne correct option

Which of the following statements best describes the primary advantage of the SentencePiece tokenizer compared to a standard BPE implementation?

A
It is significantly faster to train because it does not need to count pairs.
B
It results in a smaller vocabulary size by always merging the shortest tokens first.
C
It is inherently language-agnostic, treating text as a raw stream of Unicode characters, which is ideal for languages without clear word delimiters like Japanese or Thai.
D
It is deterministic and always produces the same tokenization for a given string,unlike BPE which can be probabilistic.

Show answer

Correct answer

C
It is inherently language-agnostic, treating text as a raw stream of Unicode characters, which is ideal for languages without clear word delimiters like Japanese or Thai.

Question 1 on its own page

Question 2

+2 marksOne correct option

When performing full fine-tuning of a large language model (e.g., 10B parameters) using Adam, which component consumes the most GPU memory?

A
The model’s weights (parameters).
B
The gradients calculated for each parameter.
C
The optimizer states (e.g., momentum and variance).
D
The vocabulary and embedding matrix.

Show answer

Correct answer

C
The optimizer states (e.g., momentum and variance).

Question 2 on its own page

Question 3

+2 marksOne correct option

A company wants to align its chatbot with values of being helpful, harmless, and honest. Human labelers provide ideal responses and rank AI-generated outputs. Which adaptation technique is designed for this?

A
Instruction Tuning
B
Zero-shot prompting
C
Reinforcement Learning from Human Feedback (RLHF)
D
Continued pre-training

Show answer

Correct answer

C
Reinforcement Learning from Human Feedback (RLHF)

Question 3 on its own page

Question 4

+3 marksOne correct option

Show answer

Correct answer

Question 4 on its own page

Question 5

+3 marksOne correct option

After running the following Hugging Face datasets code snippet, what will be the features of the dataset?

python

1from datasets import Dataset
2
3data = {"id": [1, 2, 3], "text": ["Review one", "Review two", "Review three"]}
4dataset = Dataset.from_dict(data)
5
6def classify_sentiment(example):
7    example["sentiment"] = "positive" if "one" in example["text"] else "neutral"
8    return example
9
10dataset = dataset.map(classify_sentiment)

Show answer

Correct answer

Question 5 on its own page

Question 6

+3 marksOne correct option

What is a key implication of using a Parameter-Efficient Fine-Tuning (PEFT) method like LoRA when adapting a large language model for a new task?

A
Inference speed is 10x faster.
B
Only a small number of new parameters are trained while freezing original weights.
C
It eliminates the need for labeled data.
D
Model must be retrained from scratch.

Show answer

Correct answer

B
Only a small number of new parameters are trained while freezing original weights.

Question 6 on its own page

Question 7

+4 marksOne correct option

The WordPiece tokenization algorithm, unlike BPE, does not merge the pair with the highest frequency. Instead, it merges the pair that maximizes a likelihood score. Which of the following statements accurately describes this score and its implication?

Show answer

Correct answer

Question 7 on its own page

Question 8

+4 marksOne or more correct options

Which of the following statements accurately describes the Causal Language Modeling (CLM) objective used to pre-train models like GPT? (Select ALL that apply)

Select all that apply.

A
Predicting randomly masked tokens.
B
Auto-regressive next-token prediction.
C
Requires causal attention mask.
D
Suited for encoder-only models.
E
Maximizes joint probability of sequence.

Show answer

Correct answers

B
Auto-regressive next-token prediction.
C
Requires causal attention mask.
E
Maximizes joint probability of sequence.

Question 8 on its own page

Question 9

+4 marksOne or more correct options

A research lab has access to a powerful 175B parameter language model. They need to adapt it for a highly specialized legal text analysis task, but they only have about 500 labeled examples and limited access to high-end GPUs for fine-tuning. Which of the following are viable and computationally efficient adaptation strategies? (Select ALL that apply)

Select all that apply.

A
Full fine-tuning
B
Zero-shot prompting
C
Few-shot in-context learning
D
LoRA (PEFT)
E
Pre-training from scratch

Show answer

Correct answers

B
Zero-shot prompting
C
Few-shot in-context learning
D
LoRA (PEFT)

Question 9 on its own page

Question 10

+4 marksOne or more correct options

A team is fine-tuning a 7B parameter model on a single GPU with 24GB of memory. They are using the Adam optimizer (which stores 2 states per parameter) and 32-bit precision (4 bytes per parameter/ state/gradient). They find that they run out of memory even with a batch size of 1. Which of the following strategies could help them complete the fine-tuning process on this GPU? (Select ALL that apply)

Select all that apply.

A
Increase learning rate
B
Use LoRA
C
Use quantization (8/4-bit)
D
Switch to SGD
E
Gradient accumulation

Show answer

Correct answers

B
Use LoRA
C
Use quantization (8/4-bit)
D
Switch to SGD

Question 10 on its own page

Question 11

+4 marksOne or more correct options

The evolution of NLP models shows a distinct shift from task-specific architectures to a ”pre-train, finetune” paradigm, and now towards large-scale, general-purpose models. Which of the following accurately represents this evolution and the capabilities at each stage? (Select ALL that apply)

Select all that apply.

A
The earliest models (e.g., n-grams) were statistical, required task-specific design, and had limited generalization capacity.
B
The ”pre-train, fine-tune” era (e.g., BERT, GPT) introduced transfer learning, where a model was first trained on a general language task and then fully adapted to a specific downstream task.
C
Modern Large Language Models (LLMs like GPT-4) exhibit ”emerging abilities,” allowing them to perform new tasks with zero or few examples (in-context learning) without any weight updates.
D
Word2vec was a complete language model capable of generating text, similar to GPT.
E
The primary innovation of transformers over RNNs was the use of recurrent connections, which made them more efficient to train on parallel hardware.

Show answer

Correct answers

A
The earliest models (e.g., n-grams) were statistical, required task-specific design, and had limited generalization capacity.
B
The ”pre-train, fine-tune” era (e.g., BERT, GPT) introduced transfer learning, where a model was first trained on a general language task and then fully adapted to a specific downstream task.
C
Modern Large Language Models (LLMs like GPT-4) exhibit ”emerging abilities,” allowing them to perform new tasks with zero or few examples (in-context learning) without any weight updates.

Question 11 on its own page

Question 12

+4 marksOne or more correct options

The three main families of Transformer-based models are Encoder-only (e.g., BERT), Decoder-only (e.g.,GPT), and Encoder-Decoder (e.g., T5, BART). Match the architecture to its most suitable pre- training objective and typical use case. (Select ALL that apply)

Select all that apply.

A
Encoder-only models are best for natural language understanding tasks (like sentiment classification) and are often pre-trained with a Masked Language Modeling (MLM) objective.
B
Decoder-only models are ideal for text generation tasks and are pre-trained with a Causal Language Modeling (CLM) objective.
C
Encoder-Decoder models are most suitable for sequence-to-sequence tasks like translation or summarization.
D
All three architectures are pre-trained using the same Causal Language Modeling objective.

Show answer

Correct answers

A
Encoder-only models are best for natural language understanding tasks (like sentiment classification) and are often pre-trained with a Masked Language Modeling (MLM) objective.
B
Decoder-only models are ideal for text generation tasks and are pre-trained with a Causal Language Modeling (CLM) objective.
C
Encoder-Decoder models are most suitable for sequence-to-sequence tasks like translation or summarization.

Question 12 on its own page

Question 13

+3 marksNumerical answer

Show answer

Correct answer: 4.8

Question 13 on its own page

Question 14

+4 marksNumerical answer

A single transformer block in a GPT-style model has the following configuration: embedding dimension (d model) = 1024, num attention heads = 16. Each attention head has a dimension of d model /num attention heads. Calculate the total number of parameters (weights and biases) for the self attention mechanism (specifically the Q, K, V, and Output projection layers) within this single block. Report the answer in millions, rounded to one decimal place.(in M)

Show answer

Correct answer: 4.2

Question 14 on its own page

Question 15

+4 marksNumerical answer

You are given a GPT-style model with a vocabulary size of 50,257, a context length of 1024, and an embedding dimension of 768. The model learns its positional embeddings rather than using fixed sinusoidal ones. Calculate the total number of parameters in the model’s embedding layer, which includes both token embeddings and positional embeddings. Report your answer in millions, rounded to one decimal place.(in M)

Show answer

Correct answer: 39.4

Question 15 on its own page