Figure from the original question paper

(E) Option figure from the original question paper

(A) Option figure from the original question paper

(B) Option figure from the original question paper

(C) Option figure from the original question paper

(D) Option figure from the original question paper

Figure from the original question paper Given the following statements, select all that are correct:

(A) Option figure from the original question paper

(C) Option figure from the original question paper

(D) Option figure from the original question paper

(B) Option figure from the original question paper

Consider an RNN for a language modelling task with one hidden layer, trained with cross-entropy loss with a softmax over the logits at the output layer. - Let E be the embedding matrix that converts an input token into an embedding vector. - Let U be the matrix corresponding to the recurrent connections. - Let W be the matrix corresponding to the feed-forward connections from the embedding layer to the hidden layer. - Let V be the unembedding matrix that converts the hidden layer vectors to logits. Ignore the biases The size of the vocabulary is 5,000. The hidden layer size and the size of the word embedding are the same and both are equal to 256. Based on the above data, answer the given subquestions. Which of the following are square matrices?

(A) Option figure from the original question paper

(C) Option figure from the original question paper

(B) Option figure from the original question paper

(D) Option figure from the original question paper

Consider an RNN for a language modelling task with one hidden layer, trained with cross-entropy loss with a softmax over the logits at the output layer. - Let E be the embedding matrix that converts an input token into an embedding vector. - Let U be the matrix corresponding to the recurrent connections. - Let W be the matrix corresponding to the feed-forward connections from the embedding layer to the hidden layer. - Let V be the unembedding matrix that converts the hidden layer vectors to logits. Ignore the biases The size of the vocabulary is 5,000. The hidden layer size and the size of the word embedding are the same and both are equal to 256. Based on the above data, answer the given subquestions. Figure from the original question paper

(B) Option figure from the original question paper

(A) Option figure from the original question paper

(C) Option figure from the original question paper

(D) Option figure from the original question paper

Consider an RNN for a language modelling task with one hidden layer, trained with cross-entropy loss with a softmax over the logits at the output layer. - Let E be the embedding matrix that converts an input token into an embedding vector. - Let U be the matrix corresponding to the recurrent connections. - Let W be the matrix corresponding to the feed-forward connections from the embedding layer to the hidden layer. - Let V be the unembedding matrix that converts the hidden layer vectors to logits. Ignore the biases The size of the vocabulary is 5,000. The hidden layer size and the size of the word embedding are the same and both are equal to 256. Based on the above data, answer the given subquestions. Figure from the original question paper

(A) Option figure from the original question paper

(B) Option figure from the original question paper

(C) Option figure from the original question paper

(D) Option figure from the original question paper

Consider an RNN for a language modelling task with one hidden layer, trained with cross-entropy loss with a softmax over the logits at the output layer. - Let E be the embedding matrix that converts an input token into an embedding vector. - Let U be the matrix corresponding to the recurrent connections. - Let W be the matrix corresponding to the feed-forward connections from the embedding layer to the hidden layer. - Let V be the unembedding matrix that converts the hidden layer vectors to logits. Ignore the biases The size of the vocabulary is 5,000. The hidden layer size and the size of the word embedding are the same and both are equal to 256. Based on the above data, answer the given subquestions. At training time, consider a training sample, "this is good", where each word is a token. At the last time step (while processing "good"), the word probabilities output by the RNN for these three tokens are: Figure from the original question paper Which of the following are true?

(C) Option figure from the original question paper

(D) Option figure from the original question paper

(A) Option figure from the original question paper

(B) Option figure from the original question paper

September 2024 term · Introduction to Natural Language Processing · BSDA5005

Introduction to Natural Language Processing Quiz 2: 1 December 2024 (September 2024 term)

The IIT Madras BS Introduction to Natural Language Processing (Intro to NLP) Quiz 2 paper sat on 1 Dec 2024, in the September 2024 term: 17 questions for 50 marks in 120 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.

Questions: 17
Marks: 50
Duration: 120 min
MCQ: 6
MSQ: 5
Numerical: 6

Take as mock test Practise with answers

Updated 25 Sept 2026

Official paper: IIT M DEGREE AN EXAM QDB2 01 Dec 2024 · No negative marking.

Question 1

+2 marksOne correct option

FastText differs from traditional word embedding models like Word2Vec because it can handle out-of-vocabulary (OOV) words more effectively. Which of the following best explains how FastText achieves this capability?

A
FastText predicts context words using an entire sentence rather than just nearby words.
B
FastText generates embeddings by averaging multiple pre-trained word vectors.
C
FastText breaks words down into smaller subword units (character n-grams) and creates embeddings based on these subwords, allowing it to estimate embeddings for unseen words.
D
FastText uses neural networks with an additional layer dedicated to identifying unknown words.

Show answer

Correct answer

C
FastText breaks words down into smaller subword units (character n-grams) and creates embeddings based on these subwords, allowing it to estimate embeddings for unseen words.

Question 1 on its own page

Question 2

+3 marksOne correct option

Consider the sentences:

In a transformer model, each word receives positional encoding. How will the positional encoding of "playful" differ across these sentences?

A
It will be identical in all three sentences.
B
It will vary in each sentence based on the position of "playful" within that sentence.
C
It will vary only if "playful" has a different meaning.
D
It will be identical but scaled by the number of words in each sentence.

Show answer

Correct answer

B
It will vary in each sentence based on the position of "playful" within that sentence.

Question 2 on its own page

Question 3

+3 marksOne correct option

Show answer

Correct answer

Question 3 on its own page

Question 4

+3 marksOne correct option

In the encoder of a transformer model, what are the sources of the queries (Q), keys (K), and values (V)?

A
Q comes from the encoder output, K from the decoder input, and V from the output layer.
B
Q comes from the decoder output, while K and V come from the encoder input.
C
Q comes from the positional encoding, K from the input sequence length, and V from the output embeddings.
D
Q, K, and V all come from the input embeddings representing the tokens in the input sequence.

Show answer

Correct answer

D
Q, K, and V all come from the input embeddings representing the tokens in the input sequence.

Question 4 on its own page

Question 5

+3 marksOne or more correct options

Given the following statements, select all that are correct:

Select all that apply.

Show answer

Correct answers

Question 5 on its own page

Question 6

+3 marksOne or more correct options

Consider the following statements about statistical-based (SMT) and example-based (EBMT) machine translation techniques. Select all statements that are correct:

Select all that apply.

A
SMT uses probabilistic models to estimate the likelihood of translating one phrase to another.
B
EBMT relies on a database of example translations and reuses similar examples to translate new sentences.
C
SMT requires extensive linguistic rules and manually defined grammar to create effective translations.
D
When EBMT encounters an unseen sentence, it decomposes the sentence into smaller segments and reuses similar examples from its database to generate translations.

Show answer

Correct answers

A
SMT uses probabilistic models to estimate the likelihood of translating one phrase to another.
B
EBMT relies on a database of example translations and reuses similar examples to translate new sentences.
D
When EBMT encounters an unseen sentence, it decomposes the sentence into smaller segments and reuses similar examples from its database to generate translations.

Question 6 on its own page

Question 7

+3 marksOne or more correct options

Consider a scenario where you are fine-tuning a large language model using LoRA (Low-Rank Adaptation) and QLoRA (Quantized LoRA) techniques. Each technique has specific ways of reducing the model's memory and computational requirements. Select all statements that are correct about LoRA ans QLoRa.

Select all that apply.

A
LoRA achieves computational efficiency by performing inference with only a subset of the model's original parameters, completely ignoring any layers that are not low-rank.
B
QLoRA enables the model to dynamically switch between full and low-rank matrices during inference, improving accuracy while saving memory.
C
QLoRA applies quantization to the low-rank matrices introduced by LoRA, combining memory efficiency through quantization with a reduced number of trainable parameters from low-rank adaptation.
D
LoRA reduces the number of parameters by introducing low-rank matrices to approximate the weight updates, which minimizes the number of parameters that need to be stored and trained.

Show answer

Correct answers

C
QLoRA applies quantization to the low-rank matrices introduced by LoRA, combining memory efficiency through quantization with a reduced number of trainable parameters from low-rank adaptation.
D
LoRA reduces the number of parameters by introducing low-rank matrices to approximate the weight updates, which minimizes the number of parameters that need to be stored and trained.

Question 7 on its own page

Question 8

+3 marksNumerical answer

You are analysing a collection of customer reviews using the Bag of Words (BoW) model. Here are three sample reviews:
• Review 1: "This product is amazing, I absolutely love it!"
• Review 2: "The product is terrible, I hate it."
• Review 3: "Not the best, but it works okay."

Assume that each unique word from the vocabulary appears in at least one of the three reviews and each unique word from reviews are added in the vocabulary. A word that appears in a review contributes 1 non-zero entry to that review's BoW vector.

Show answer

Correct answer: 22

Question 8 on its own page

Question 9

+3 marksNumerical answer

Consider the following sentences:

S1: "I like studying LLMs"
S2: "I like studying big data"
S3: "I also like studying computer vision"

Assuming each word is represented by a positional encoding vector with an embedding dimension of $512$ , calculate the total number of unique elements in the positional encoding vectors of these sentences.

(Example: Number of unique elements in vectors $[3.7, 7.3]$ and $[3.4, 3.7]$ and $[7.3, 3.7]$ is $3$ )

Show answer

Correct answer: 3072

Question 9 on its own page

Question 10

+3 marksNumerical answer

Show answer

Correct answer: 950

Question 10 on its own page

Question 11

+3 marksNumerical answer

Show answer

Correct answer: 96.85 (accepted within ±0.15)

Question 11 on its own page

Question 12

+3 marksNumerical answer

Consider a reward model in RLHF that uses the Bradley-Terry model to compute pairwise preference probabilities between completions. Let $y_i$ , $y_j$ , and $y_k$ be three different completions for a given input $x$ .

$R(y_i), R(y_j)$ and $R(y_k)$ are the rewards output by the model for completions $y_i, y_j$ and $y_k$ respectively. We have:

$R(y_i) = 0.4$
$R(y_j) = 0.2$
$R(y_k) = 0.5$

Calculate the combined probability that $y_i$ is preferred over both $y_j$ and $y_k$ (assuming independence).

Use $\log_e$ (natural logarithm) for calculations and provide the final answer for the combined probability correct to two decimal places.

Show answer

Correct answer: 0.26 (accepted within ±0.04)

Question 12 on its own page

Question 13

+3 marksNumerical answer

Show answer

Correct answer: 21.5 (accepted within ±0.5)

Question 13 on its own page

Question 14

+3 marksOne or more correct options

Consider an RNN for a language modelling task with one hidden layer, trained with cross-entropy loss with a softmax over the logits at the output layer.

Let $E$ be the embedding matrix that converts an input token into an embedding vector.
Let $U$ be the matrix corresponding to the recurrent connections.
Let $W$ be the matrix corresponding to the feed-forward connections from the embedding layer to the hidden layer.
Let $V$ be the unembedding matrix that converts the hidden layer vectors to logits.

Ignore the biases

The size of the vocabulary is $5,000$ . The hidden layer size and the size of the word embedding are the same and both are equal to $256$ .

Based on the above data, answer the given subquestions.

Which of the following are square matrices?

Select all that apply.

Show answer

Correct answers

Question 14 on its own page

Question 15

+3 marksOne correct option

Consider an RNN for a language modelling task with one hidden layer, trained with cross-entropy loss with a softmax over the logits at the output layer.

Let $E$ be the embedding matrix that converts an input token into an embedding vector.
Let $U$ be the matrix corresponding to the recurrent connections.
Let $W$ be the matrix corresponding to the feed-forward connections from the embedding layer to the hidden layer.
Let $V$ be the unembedding matrix that converts the hidden layer vectors to logits.

Ignore the biases

The size of the vocabulary is $5,000$ . The hidden layer size and the size of the word embedding are the same and both are equal to $256$ .

Based on the above data, answer the given subquestions.

Show answer

Correct answer

Question 15 on its own page

Question 16

+3 marksOne correct option

Consider an RNN for a language modelling task with one hidden layer, trained with cross-entropy loss with a softmax over the logits at the output layer.

Let $E$ be the embedding matrix that converts an input token into an embedding vector.
Let $U$ be the matrix corresponding to the recurrent connections.
Let $W$ be the matrix corresponding to the feed-forward connections from the embedding layer to the hidden layer.
Let $V$ be the unembedding matrix that converts the hidden layer vectors to logits.

Ignore the biases

The size of the vocabulary is $5,000$ . The hidden layer size and the size of the word embedding are the same and both are equal to $256$ .

Based on the above data, answer the given subquestions.

Show answer

Correct answer

Question 16 on its own page

Question 17

+3 marksOne or more correct options

Consider an RNN for a language modelling task with one hidden layer, trained with cross-entropy loss with a softmax over the logits at the output layer.

Let $E$ be the embedding matrix that converts an input token into an embedding vector.
Let $U$ be the matrix corresponding to the recurrent connections.
Let $W$ be the matrix corresponding to the feed-forward connections from the embedding layer to the hidden layer.
Let $V$ be the unembedding matrix that converts the hidden layer vectors to logits.

Ignore the biases

The size of the vocabulary is $5,000$ . The hidden layer size and the size of the word embedding are the same and both are equal to $256$ .

Based on the above data, answer the given subquestions.

At training time, consider a training sample, "this is good", where each word is a token. At the last time step (while processing "good"), the word probabilities output by the RNN for these three tokens are:

Which of the following are true?

Select all that apply.

Show answer

Correct answers

Question 17 on its own page