The IIT Madras BS Introduction to Natural Language Processing (Intro to NLP) End Term paper sat on 13 Apr 2025, in the January 2025 term: 25 questions for 100 marks in 180 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.
Why might accuracy be a misleading metric for evaluating POS tagging systems in cases of class imbalance?
Because accuracy does not account for the number of true negatives.
Because accuracy gives equal weight to all classes, even if some tags are much rarer than others.
Because accuracy only measures the precision of the system, ignoring recall.
Because accuracy is not affected by the number of false positives and false negatives.
Correct answer
Because accuracy gives equal weight to all classes, even if some tags are much rarer than others.
In stochastic POS tagging, why is it necessary to calculate both individual word probabilities and tag sequence probabilities?
Individual word probabilities are sufficient for accurate tagging, and tag sequence probabilities are redundant.
Individual word probabilities are only used for rare words, while tag sequence probabilities are used for common words.
Tag sequence probabilities help in determining the most likely sequence of tags, while individual word probabilities provide local context.
Both probabilities are used to reduce the computational complexity of the tagging process.
Correct answer
Tag sequence probabilities help in determining the most likely sequence of tags, while individual word probabilities provide local context.
How does Retrieval-Augmented Generation (RAG) differ from standard parametric models like GPT-3?
RAG eliminates the need for transformer-based token generation.
RAG uses retrieval but does not update its knowledge dynamically over time.
RAG only works when trained on labeled question-answer datasets.
RAG retrieves external documents dynamically, reducing reliance on pre- trained knowledge.
Correct answer
RAG retrieves external documents dynamically, reducing reliance on pre- trained knowledge.
Which of the following architectures is most commonly used as the generator in RAG models?
GPT-3
BART
BERT
RoBERTa
Correct answer
BART
A company wants to optimize an LLM for programming tasks. Which approach is the most efficient if they have limited labeled data?
Train a new model from scratch using a large programming dataset.
Fine-tune a transformer model with instruction tuning.
Use Few-shot prompting with an existing LLM.
Apply a BERT-based model instead of transformers.
Correct answer
Use Few-shot prompting with an existing LLM.
Which of the following statements about Attention Flow in Transformer models is TRUE?
Attention flow computes the contribution of each token independently without considering other tokens.
Attention flow assigns higher scores to tokens that appear earlier in the sequence by default.
Attention flow combines attention scores across multiple layers to determine overall token influence.
Attention flow values are always normalized across all tokens in the sequence to sum to 1.
Correct answer
Attention flow combines attention scores across multiple layers to determine overall token influence.
Which of the following scenarios represents an ethical concern with LLM deployment?
A summarization model shortening news articles while maintaining factual accuracy.
A question-answering system providing different answers for different demographics when asked about salary negotiations.
A chatbot refusing to generate offensive or harmful content.
A translation model translating medical terms with higher accuracy than general terms.
Correct answer
A question-answering system providing different answers for different demographics when asked about salary negotiations.
How does an LSTM differ from a standard RNN in terms of remembering information?
It uses a larger hidden state.
It stores all past hidden states explicitly.
It introduces gates to control the flow of information.
It completely replaces hidden states with attention mechanisms.
Correct answer
It introduces gates to control the flow of information.
In the context of speculative decoding, a draft model predicts three tokens: "a", "quick", "fox". The main model assigns probabilities 0.8, 0.6, and 0.8 respectively.The draft model's probabilities for these words were 0.7, 0.9, and 0.9.
Which tokens are most likely to be accepted?
All three tokens
Only "a" and "fox"
Only "a"
None of the tokens
Correct answer
Only "a" and "fox"
Consider the following attention matrices for two layers (corresponding to tokens 1 and 2) in a transformer model:
Layer 1 Attention Matrix:
Layer 2 Attention Matrix:
What is the total attention from token 2 to token 1 after performing attention rollout?
0.57
0.33
0.39
0.26
Correct answer
0.26
Which of the following sentences contain pragmatic ambiguity due to potential sarcasm or irony?
"Wow, another Monday! Just what I needed."
"The meeting was only three hours long. So productive!"
"I love spending hours in traffic. It’s my favorite part of the day."
"The weather is perfect for a picnic today."
"She always arrives on time, unlike some people."
Correct answers
"Wow, another Monday! Just what I needed."
"The meeting was only three hours long. So productive!"
"I love spending hours in traffic. It’s my favorite part of the day."
Which of the following statements correctly describe how BERT differs from GPT and standard transformer models?
BERT is a bidirectional model, while GPT is autoregressive.
BERT uses masked language modeling (MLM), while GPT uses causal language modeling (CLM).
Standard transformers use encoder-only architectures, whereas BERT and GPT both use decoder-only architectures.
BERT is trained for sentence embeddings, while GPT is trained for next-token prediction.
Correct answers
BERT is a bidirectional model, while GPT is autoregressive.
BERT uses masked language modeling (MLM), while GPT uses causal language modeling (CLM).
BERT is trained for sentence embeddings, while GPT is trained for next-token prediction.
Which of the following statements is/are incorrect regarding Reinforcement Learning from Human Feedback (RLHF), Supervised Instruction Fine-Tuning, LoRA, and QLoRA?
RLHF helps improve model behavior by learning from human preference rankings.
LoRA (Low-Rank Adaptation) reduces the number of trainable parameters by using low-rank matrices.
Supervised Instruction Fine-Tuning involves reinforcement learning for generating human-like responses.
QLoRA applies quantization to reduce memory consumption during fine- tuning.
Correct answer
Supervised Instruction Fine-Tuning involves reinforcement learning for generating human-like responses.
During sequence-to-sequence model training, which of the following statements is/are correct regarding Teacher Forcing and Student Forcing?
Teacher Forcing is used during inference, while Student Forcing is used during training.
Teacher Forcing provides the correct previous output as input during training, whereas Student Forcing generates outputs based on its own previous predictions.
Student Forcing helps the model converge faster than Teacher Forcing.
Teacher Forcing is mainly used in reinforcement learning settings.
Correct answer
Teacher Forcing provides the correct previous output as input during training, whereas Student Forcing generates outputs based on its own previous predictions.
Which of the following statements about beam search is/are incorrect ?
Beam search balances exploration and exploitation by considering multiple sequences at each step.
A larger beam width increases computational cost but improves search quality.
Greedy search is equivalent to beam search with beam size B = 1.
Unlike greedy search, beam search guarantees finding the globally optimal sequence.
Correct answer
Unlike greedy search, beam search guarantees finding the globally optimal sequence.
Which components are essential in a RAG pipeline?
retriever model (e.g., DPR) that fetches relevant documents.
decoder-only transformer for sequence generation.
BM25-based sparse retrieval method as the primary retriever.
BERT-based query encoder for document ranking.
Correct answers
retriever model (e.g., DPR) that fetches relevant documents.
BERT-based query encoder for document ranking.
Which of the following are the examples of unintended bias in an NLP model?
A sentiment analysis model rating all political tweets as "negative".
A chatbot trained specifically for medical consultations failing on legal queries.
A text summarization model producing longer summaries for academic papers.
A machine translation system translating "doctor" to "he" and "nurse" to "she" in a gender-neutral language.
Correct answers
A sentiment analysis model rating all political tweets as "negative".
A machine translation system translating "doctor" to "he" and "nurse" to "she" in a gender-neutral language.
Which of the following factors contribute to bias amplification in NLP models?
Reinforcement learning from biased human feedback.
Training on unbalanced datasets where stereotypes exist.
Applying debiasing techniques such as zeroing out gender-related word embeddings.
Using a larger model size to improve performance.
Correct answers
Reinforcement learning from biased human feedback.
Training on unbalanced datasets where stereotypes exist.
The probability of "apple" increases further.
The probabilities of "banana" and "cherry" become closer to "apple".
The probabilities remain unchanged.
The model always picks "banana".
The model always picks "cherry".
Correct answer
The probability of "apple" increases further.
In a transition-based parser using SHIFT, LEFTARC, and RIGHTARC operations, what is the minimum number of operations needed to parse the sentence "I love coding" (excluding the ROOT node)?
Correct answer: 4
A TF-IDF model is applied to a document corpus of 50,000 documents. The term "optimization" appears in 750 documents. In a specific document, "optimization" appears 8 times, and the total number of words in that document is 1200. Using log base 10 ( log10), compute the TF-IDF score for "optimization" in this document. Enter your answer correct to three decimal places.
Correct answer: 0.015 (accepted within ±0.005)
A beam search with beam width = 4 and vocabulary size = 8 is run for 3 decoding steps. How many sequences will remain at the end of 3 steps after pruning?
Correct answer: 4
A transformer-based Retrieval-Augmented Generation (RAG) system retrieves the top-10 most relevant passages, where each passage contains 512 tokens. However, a portion of the model's 4096-token context length is reserved for the input query and special tokens (e.g., separators, instructions). If the model reserves 1024 tokens for the input query, how many full retrieved passages can fit within the remaining context window?
Correct answer: 6
Consider the following regression model:
F(x) = 500 * Age + 1000 * Income + 20000
For an input with:
Age = 30
Income = 50
Baseline Input:
Age = 0
Income = 0
What is the Integrated Gradient for the feature Income?
Correct answer: 50000
An e-commerce company predicts the total sales (in dollars) based on 3 key features:
Following are the predicted sales:
| Feature Combination | Predicted Sales (in dollars) |
|---|---|
| No features (baseline) | 200 |
| Only "Discount" | 300 |
| Only "Social Media Ads" | 280 |
| Only "Email Campaigns" | 250 |
| "Discount" + "Social Media Ads" | 450 |
| "Discount" + "Email Campaigns" | 400 |
| "Social Media Ads" + "Email Campaigns" | 370 |
| "Discount" + "Social Media Ads" + "Email Campaigns" | 600 |
What is the Shapley value of the "Discount" feature?
Correct answer: 162.5 (accepted within ±0.5)
