Quiz Space

Introduction to Natural Language Processing · End Term · 13 Apr 2025 · January 2025 term

Question 12: Which of the following statements correctly describe how…

Question 12

+4 marksOne or more correct options

Which of the following statements correctly describe how BERT differs from GPT and standard transformer models?

Select all that apply.

  1. A

    BERT is a bidirectional model, while GPT is autoregressive.

  2. B

    BERT uses masked language modeling (MLM), while GPT uses causal language modeling (CLM).

  3. C

    Standard transformers use encoder-only architectures, whereas BERT and GPT both use decoder-only architectures.

  4. D

    BERT is trained for sentence embeddings, while GPT is trained for next-token prediction.

Show answer

Correct answers

  • A

    BERT is a bidirectional model, while GPT is autoregressive.

  • B

    BERT uses masked language modeling (MLM), while GPT uses causal language modeling (CLM).

  • D

    BERT is trained for sentence embeddings, while GPT is trained for next-token prediction.

See the whole paperTake it as a mock test

Question 12 of 25 in the IIT Madras BS Introduction to Natural Language Processing (Intro to NLP) End Term paper sat on 13 Apr 2025, in the January 2025 term (IIT M FOUNDATION AN EXAM QDF3 13 Apr 2025). It carries 4 marks.

More questions from this paper

  1. Q1Why might accuracy be a misleading metric for evaluating POS tagging systems in cases of class imbalance?
  2. Q2In stochastic POS tagging, why is it necessary to calculate both individual word probabilities and tag sequence probabi…
  3. Q3How does Retrieval-Augmented Generation (RAG) differ from standard parametric models like GPT-3?
  4. Q4Which of the following architectures is most commonly used as the generator in RAG models?
  5. Q5A company wants to optimize an LLM for programming tasks. Which approach is the most efficient if they have limited lab…
  6. Q6Which of the following statements about Attention Flow in Transformer models is TRUE?
  7. Q7Which of the following scenarios represents an ethical concern with LLM deployment?
  8. Q8How does an LSTM differ from a standard RNN in terms of remembering information?
  9. Q9In the context of speculative decoding, a draft model predicts three tokens: "a", "quick", "fox". The main model assign…
  10. Q10Consider the following attention matrices for two layers (corresponding to tokens 1 and 2) in a transformer model:\ Lay…
  11. Q11Which of the following sentences contain pragmatic ambiguity due to potential sarcasm or irony?
  12. Q13Which of the following statements is/are incorrect regarding Reinforcement Learning from Human Feedback (RLHF), Supervi…
  13. Q14During sequence-to-sequence model training, which of the following statements is/are correct regarding Teacher Forcing …
  14. Q15Which of the following statements about beam search is/are incorrect ?
  15. Q16Which components are essential in a RAG pipeline?
  16. Q17Which of the following are the examples of unintended bias in an NLP model?
  17. Q18Which of the following factors contribute to bias amplification in NLP models?
  18. Q19Figure question
  19. Q20In a transition-based parser using SHIFT, LEFTARC, and RIGHTARC operations, what is the minimum number of operations ne…
  20. Q21A TF-IDF model is applied to a document corpus of 50,000 documents. The term "optimization" appears in 750 documents. I…
  21. Q22A beam search with beam width = 4 and vocabulary size = 8 is run for 3 decoding steps. How many sequences will remain a…
  22. Q23A transformer-based Retrieval-Augmented Generation (RAG) system retrieves the top-10 most relevant passages, where each…
  23. Q24Consider the following regression model:\ F(x) = 500 Age + 1000 Income + 20000\ For an input with:\ Age = 30\ Income = …
  24. Q25An e-commerce company predicts the total sales (in dollars) based on 3 key features: Discount (e.g., 10%, 20%) Social M…