Quiz Space

Large Language Models · Quiz 1 · 27 Oct 2024 · September 2024 term

Question 2: Choose the correct statements regarding transformer archi…

Question 2

+2 marksOne correct option

Choose the correct statements regarding transformer architecture:

  1. A

    Residual connections are practically optional in a transformer model with 10 encoder layers and 10 decoder layers.

  2. B

    Having multiple heads helps in capturing different relationships between input tokens.

  3. C

    One hot encoding is a good choice for position encoding.

  4. D

    Batch normalization can be used instead of layer normalization.

  5. E

    None of these.

Show answer

Correct answer

  • B

    Having multiple heads helps in capturing different relationships between input tokens.

See the whole paperTake it as a mock test

Question 2 of 19 in the IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 27 Oct 2024, in the September 2024 term (IIT M DEGREE AN EXAM QDB2 27 Oct 2024). It carries 2 marks.

More questions from this paper

  1. Q1Teacher forcing for training an transformer model is:
  2. Q3Assume the model has two layers (N = 2). Calculate the total number of parameters in the model (excluding the embedding…
  3. Q4How many parameters does the output layer have?
  4. Q5How many parameters does the embedding layer have?
  5. Q6At time step t = 1, the prediction probability for the words “Naan”, “transformer” and “padaththai” are 0.55,0.15 and 0…
  6. Q7Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  7. Q8Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  8. Q9Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  9. Q10Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  10. Q11Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  11. Q12Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  12. Q13Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  13. Q14Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  14. Q15Rajesh has created a GPT-like transformer model. However he doesn't have access to large compute infrastructure, so he …
  15. Q16Rajesh has created a GPT-like transformer model. However he doesn't have access to large compute infrastructure, so he …
  16. Q17Rajesh has created a GPT-like transformer model. However he doesn't have access to large compute infrastructure, so he …
  17. Q18The input embeddings for the words “learning”, “brings” and “joy” are h_1 = [0.5, 0.25, 1], h_2 = [0.1, 0.25, 0], and h…
  18. Q19Consider the embedding vector for a word, x = [0.1, 0.2, -0.3, 0.4]. Suppose the word is at position 2 in the given sen…