Quiz Space

Introduction to Natural Language Processing · Quiz 2 · 16 Mar 2025 · January 2025 term

Question 14: What happens if you increase the number of attention hea…

Question 14

+3 marksOne or more correct options

What happens if you increase the number of attention heads in a transformer model while keeping the embedding size constant?

Select all that apply.

  1. A

    Each head's dimensionality decreases.

  2. B

    The computational complexity of the attention mechanism decreases.

  3. C

    The model learns more diverse relationships in the input sequence.

  4. D

    The model processes longer sequences without truncation.

Show answer

Correct answers

  • A

    Each head's dimensionality decreases.

  • C

    The model learns more diverse relationships in the input sequence.

See the whole paperTake it as a mock test

Question 14 of 18 in the IIT Madras BS Introduction to Natural Language Processing (Intro to NLP) Quiz 2 paper sat on 16 Mar 2025, in the January 2025 term (IIT M DEGREE AN EXAM QDB2 16 Mar 2025). It carries 3 marks.

More questions from this paper

  1. Q1Consider the given Assertion and Reason. Choose the correct option.\ Assertion (A): FastText produces word embeddings o…
  2. Q2Suppose TF-IDF is applied to a corpus of medical research papers. In a document discussing COVID-19 vaccines, which of …
  3. Q3Which of the following statements is/are INCORRECT?
  4. Q4In an LSTM network, each gate has a specific function. Which of the following statements about the functions of LSTM ga…
  5. Q5Given the following corpus with word frequencies: If we apply BPE, What will be the merged token after the second merge…
  6. Q6Which of the following statements best explains why RLHF improves over instruction fine-tuning alone?
  7. Q7Match the application to the correct RNN type: Which of the following mappings is correct?
  8. Q8In an attention-based RNN, what does the context vector represent?
  9. Q9You are using QLoRA to fine-tune a neural network. The weight matrix W has dimensions 1024 x 2048, and LoRA uses rank 8…
  10. Q10Which of the following use cases can benefit from using TF-IDF?
  11. Q11You are comparing rule-based machine translation (RBMT), statistical machine translation(SMT), and example-based machin…
  12. Q12Which of the following are key components of the BERT architecture?
  13. Q13In positional encoding for a transformer model, why are sine and cosine functions used for encoding positions?
  14. Q15Which of the following is/are INCORRECT?:
  15. Q16You are analyzing a collection of movie reviews using the Bag of Words (BoW) model. Here are three sample reviews: The …
  16. Q17A simple RNN has 10 hidden units, and the input feature size is 5. If the output size is 8, what is the total number of…
  17. Q18A model has 1,000,000 parameters. LoRA is applied to a 100 x 100 parameter matrix in the model and results in 400 new L…