Quiz Space

Large Language Models · Quiz 1 · 27 Oct 2024 · September 2024 term

Question 19: Consider the embedding vector for a word, x = [0.1, 0.2,…

Question 19

+3 marksNumerical answer

Consider the embedding vector for a word, x=[0.1,0.2,−0.3,0.4]x = [0.1, 0.2, -0.3, 0.4]. Suppose the word is at position 2 in the given sentence. Add the corresponding position embedding pp to the word embedding to get hh, i.e. the sum of the elements in h=x+ph = x + p. Use the fixed-sinusoidal position embedding vector calculated using the formula given below

PE(pos,2i)=sin(pos100002i/dmodel)PE(pos, 2i) = sin\left(\frac{pos}{10000^{2i/dmodel}}\right)

PE(pos,2i+1)=cos(pos100002i/dmodel)PE(pos, 2i+1) = cos\left(\frac{pos}{10000^{2i/dmodel}}\right)

What is h[0]+h[1]h[0] + h[1] i.e. sum of first two elements of hh?

Show answer

Correct answer: 0.8 (accepted within ±0.05)

Question 19 of 19 in the IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 27 Oct 2024, in the September 2024 term (IIT M DEGREE AN EXAM QDB2 27 Oct 2024). It carries 3 marks.

More questions from this paper

  1. Q1Teacher forcing for training an transformer model is:
  2. Q2Choose the correct statements regarding transformer architecture:
  3. Q3Assume the model has two layers (N = 2). Calculate the total number of parameters in the model (excluding the embedding…
  4. Q4How many parameters does the output layer have?
  5. Q5How many parameters does the embedding layer have?
  6. Q6At time step t = 1, the prediction probability for the words “Naan”, “transformer” and “padaththai” are 0.55,0.15 and 0…
  7. Q7Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  8. Q8Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  9. Q9Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  10. Q10Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  11. Q11Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  12. Q12Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  13. Q13Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  14. Q14Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  15. Q15Rajesh has created a GPT-like transformer model. However he doesn't have access to large compute infrastructure, so he …
  16. Q16Rajesh has created a GPT-like transformer model. However he doesn't have access to large compute infrastructure, so he …
  17. Q17Rajesh has created a GPT-like transformer model. However he doesn't have access to large compute infrastructure, so he …
  18. Q18The input embeddings for the words “learning”, “brings” and “joy” are h_1 = [0.5, 0.25, 1], h_2 = [0.1, 0.25, 0], and h…