Quiz Space

Large Language Models · Quiz 1 · 27 Oct 2024 · September 2024 term

Question 3: Assume the model has two layers (N = 2). Calculate the to…

Question 3

+2 marksNumerical answer

Based on the above data, answer the given subquestions.

Assume the model has two layers (N = 2). Calculate the total number of parameters in the model (excluding the embedding layer and output layer). Moreover, no bias was added to the neuron in the FFNN layers.

Show answer

Correct answer: 230656

See the whole paperTake it as a mock test

Question 3 of 19 in the IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 27 Oct 2024, in the September 2024 term (IIT M DEGREE AN EXAM QDB2 27 Oct 2024). It carries 2 marks.

More questions from this paper

  1. Q1Teacher forcing for training an transformer model is:
  2. Q2Choose the correct statements regarding transformer architecture:
  3. Q4How many parameters does the output layer have?
  4. Q5How many parameters does the embedding layer have?
  5. Q6At time step t = 1, the prediction probability for the words “Naan”, “transformer” and “padaththai” are 0.55,0.15 and 0…
  6. Q7Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  7. Q8Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  8. Q9Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  9. Q10Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  10. Q11Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  11. Q12Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  12. Q13Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  13. Q14Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  14. Q15Rajesh has created a GPT-like transformer model. However he doesn't have access to large compute infrastructure, so he …
  15. Q16Rajesh has created a GPT-like transformer model. However he doesn't have access to large compute infrastructure, so he …
  16. Q17Rajesh has created a GPT-like transformer model. However he doesn't have access to large compute infrastructure, so he …
  17. Q18The input embeddings for the words “learning”, “brings” and “joy” are h_1 = [0.5, 0.25, 1], h_2 = [0.1, 0.25, 0], and h…
  18. Q19Consider the embedding vector for a word, x = [0.1, 0.2, -0.3, 0.4]. Suppose the word is at position 2 in the given sen…