Quiz Space

Deep Learning · End Term · 13 Apr 2025 · January 2025 term

Deep Learning End Term 13 Apr 2025 — Question 11

Question 11

+1 markNumerical answer

Consider the multi-head self-attention mechanism in the encoder of a transformer with 8 heads. The word embedding dimension is 32. In a given head, the query, key and value vectors have the same dimension and each of them is 4. The sequence length is 5.
Based on the above data, answer the given subquestions.

Show answer

Correct answer: 32

See the whole paperTake it as a mock test

Question 11 of 31 in the IIT Madras BS Deep Learning (Deep Learning) End Term paper sat on 13 Apr 2025, in the January 2025 term (IIT M DIPLOMA AN EXAM QDD3 13 Apr 2025). It carries 1 mark.

More questions from this paper

  1. Q1A neural network is being trained for a regression problem in which the target is in the interval (−1, 1). Which of the…
  2. Q2In the transformer architecture, which of the following are true regarding cross attention in the decoder?
  3. Q3Consider applying dropout to a hidden layer with p = 0.5. Which of the following are true?
  4. Q4Consider the following plot of error versus model complexity. Models of varying complexity are evaluated on the same tr…
  5. Q5Consider the following tasks which can be solved using an encoder-decoder architecture. Which of these tasks has a deco…
  6. Q6The input volume to a convolutional layer is 100×100×5. If ten kernels, each of size 9 × 9, with unit stride are applie…
  7. Q7Find the value of x at which the tower attains its maximum value.
  8. Q8Find the maximum value that the tower attains.
  9. Q9In any given head, what is the dimension of the WQ matrix?
  10. Q10In any given head, what is the dimension of the query-key product matrix?
  11. Q12Find the value y at the end of the forward pass.
  12. Q13Based on the above data, answer the given subquestions.
  13. Q14Based on the above data, answer the given subquestions.
  14. Q15Based on the above data, answer the given subquestions.
  15. Q16Based on the above data, answer the given subquestions.
  16. Q17Which of the following is true?
  17. Q18Based on the above data, answer the given subquestions. Find a.
  18. Q19Based on the above data, answer the given subquestions. Find b.
  19. Q20Based on the above data, answer the given subquestions. Find c.
  20. Q21Consider the CBOW model for learning word embeddings with embedding dimension two. The window size is one, that is, we …
  21. Q22Consider the CBOW model for learning word embeddings with embedding dimension two. The window size is one, that is, we …
  22. Q23For a multi-class classification problem with five classes, consider the following CNN architecture: | Layer | Specs | …
  23. Q24For a multi-class classification problem with five classes, consider the following CNN architecture: | Layer | Specs | …
  24. Q25For a multi-class classification problem with five classes, consider the following CNN architecture: | Layer | Specs | …
  25. Q26With a total of 7 time steps (T = 7), what is the total count of parameters (including bias) within the network?
  26. Q27Based on the above data, answer the given subquestions.
  27. Q28If all the parameters (including bias) in the network are initialized to zero, what will be the total loss after 10 tim…
  28. Q29Which of these is the correct ordering among the learning rates?
  29. Q30Based on the above data, answer the given subquestions.
  30. Q31Based on the above data, answer the given subquestions.