Quiz Space

Deep Learning · End Term · 13 Apr 2025 · January 2025 term

Question 17: Which of the following is true?

Question 17

+1 markOne correct option

Based on the above data, answer the given subquestions.

Which of the following is true?

  1. A

    The mask matrix is added to the query-key product matrix before applying softmax.

  2. B

    The mask matrix is added to the query-key product matrix after applying softmax.

  3. C

    The mask matrix could be added either before or after the softmax.

Show answer

Correct answer

  • A

    The mask matrix is added to the query-key product matrix before applying softmax.

See the whole paperTake it as a mock test

Question 17 of 31 in the IIT Madras BS Deep Learning (Deep Learning) End Term paper sat on 13 Apr 2025, in the January 2025 term (IIT M DIPLOMA AN EXAM QDD3 13 Apr 2025). It carries 1 mark.

More questions from this paper

  1. Q1A neural network is being trained for a regression problem in which the target is in the interval (−1, 1). Which of the…
  2. Q2In the transformer architecture, which of the following are true regarding cross attention in the decoder?
  3. Q3Consider applying dropout to a hidden layer with p = 0.5. Which of the following are true?
  4. Q4Consider the following plot of error versus model complexity. Models of varying complexity are evaluated on the same tr…
  5. Q5Consider the following tasks which can be solved using an encoder-decoder architecture. Which of these tasks has a deco…
  6. Q6The input volume to a convolutional layer is 100×100×5. If ten kernels, each of size 9 × 9, with unit stride are applie…
  7. Q7Find the value of x at which the tower attains its maximum value.
  8. Q8Find the maximum value that the tower attains.
  9. Q9In any given head, what is the dimension of the WQ matrix?
  10. Q10In any given head, what is the dimension of the query-key product matrix?
  11. Q11Consider the multi-head self-attention mechanism in the encoder of a transformer with 8 heads. The word embedding dimen…
  12. Q12Find the value y at the end of the forward pass.
  13. Q13Based on the above data, answer the given subquestions.
  14. Q14Based on the above data, answer the given subquestions.
  15. Q15Based on the above data, answer the given subquestions.
  16. Q16Based on the above data, answer the given subquestions.
  17. Q18Based on the above data, answer the given subquestions. Find a.
  18. Q19Based on the above data, answer the given subquestions. Find b.
  19. Q20Based on the above data, answer the given subquestions. Find c.
  20. Q21Consider the CBOW model for learning word embeddings with embedding dimension two. The window size is one, that is, we …
  21. Q22Consider the CBOW model for learning word embeddings with embedding dimension two. The window size is one, that is, we …
  22. Q23For a multi-class classification problem with five classes, consider the following CNN architecture: | Layer | Specs | …
  23. Q24For a multi-class classification problem with five classes, consider the following CNN architecture: | Layer | Specs | …
  24. Q25For a multi-class classification problem with five classes, consider the following CNN architecture: | Layer | Specs | …
  25. Q26With a total of 7 time steps (T = 7), what is the total count of parameters (including bias) within the network?
  26. Q27Based on the above data, answer the given subquestions.
  27. Q28If all the parameters (including bias) in the network are initialized to zero, what will be the total loss after 10 tim…
  28. Q29Which of these is the correct ordering among the learning rates?
  29. Q30Based on the above data, answer the given subquestions.
  30. Q31Based on the above data, answer the given subquestions.