Quiz Space

Introduction to Natural Language Processing · End Term · 13 Apr 2025 · January 2025 term

Question 6: Which of the following statements about Attention Flow in…

Question 6

+3 marksOne correct option

Which of the following statements about Attention Flow in Transformer models is TRUE?

  1. A

    Attention flow computes the contribution of each token independently without considering other tokens.

  2. B

    Attention flow assigns higher scores to tokens that appear earlier in the sequence by default.

  3. C

    Attention flow combines attention scores across multiple layers to determine overall token influence.

  4. D

    Attention flow values are always normalized across all tokens in the sequence to sum to 1.

Show answer

Correct answer

  • C

    Attention flow combines attention scores across multiple layers to determine overall token influence.

See the whole paperTake it as a mock test

Question 6 of 25 in the IIT Madras BS Introduction to Natural Language Processing (Intro to NLP) End Term paper sat on 13 Apr 2025, in the January 2025 term (IIT M FOUNDATION AN EXAM QDF3 13 Apr 2025). It carries 3 marks.

More questions from this paper

  1. Q1Why might accuracy be a misleading metric for evaluating POS tagging systems in cases of class imbalance?
  2. Q2In stochastic POS tagging, why is it necessary to calculate both individual word probabilities and tag sequence probabi…
  3. Q3How does Retrieval-Augmented Generation (RAG) differ from standard parametric models like GPT-3?
  4. Q4Which of the following architectures is most commonly used as the generator in RAG models?
  5. Q5A company wants to optimize an LLM for programming tasks. Which approach is the most efficient if they have limited lab…
  6. Q7Which of the following scenarios represents an ethical concern with LLM deployment?
  7. Q8How does an LSTM differ from a standard RNN in terms of remembering information?
  8. Q9In the context of speculative decoding, a draft model predicts three tokens: "a", "quick", "fox". The main model assign…
  9. Q10Consider the following attention matrices for two layers (corresponding to tokens 1 and 2) in a transformer model:\ Lay…
  10. Q11Which of the following sentences contain pragmatic ambiguity due to potential sarcasm or irony?
  11. Q12Which of the following statements correctly describe how BERT differs from GPT and standard transformer models?
  12. Q13Which of the following statements is/are incorrect regarding Reinforcement Learning from Human Feedback (RLHF), Supervi…
  13. Q14During sequence-to-sequence model training, which of the following statements is/are correct regarding Teacher Forcing …
  14. Q15Which of the following statements about beam search is/are incorrect ?
  15. Q16Which components are essential in a RAG pipeline?
  16. Q17Which of the following are the examples of unintended bias in an NLP model?
  17. Q18Which of the following factors contribute to bias amplification in NLP models?
  18. Q19Figure question
  19. Q20In a transition-based parser using SHIFT, LEFTARC, and RIGHTARC operations, what is the minimum number of operations ne…
  20. Q21A TF-IDF model is applied to a document corpus of 50,000 documents. The term "optimization" appears in 750 documents. I…
  21. Q22A beam search with beam width = 4 and vocabulary size = 8 is run for 3 decoding steps. How many sequences will remain a…
  22. Q23A transformer-based Retrieval-Augmented Generation (RAG) system retrieves the top-10 most relevant passages, where each…
  23. Q24Consider the following regression model:\ F(x) = 500 Age + 1000 Income + 20000\ For an input with:\ Age = 30\ Income = …
  24. Q25An e-commerce company predicts the total sales (in dollars) based on 3 key features: Discount (e.g., 10%, 20%) Social M…