Quiz Space

Large Language Models · Quiz 1 · 27 Oct 2024 · September 2024 term

Question 15: Rajesh has created a GPT-like transformer model. However…

Question 15

+2 marksOne correct option

Rajesh has created a GPT-like transformer model. However he doesn't have access to large compute infrastructure, so he has configured the original architecture in the following manner:

  1. He has taken a vocabulary of size 1000 words/tokens.
  2. Sequence length is 64.
  3. Embedding dimension and dmodeld_{model} is 128.
  4. There are only 4 transformer blocks (layers).
  5. Each transformer block (layer) has only 4 attention heads.
  6. FFN hidden layer size is 256.

Based on the above data, answer the given subquestions.

What will be the shape of positional embedding?

  1. A

    64 × 128

  2. B

    64 × 4

  3. C

    12 × 128

  4. D

    512 × 768

  5. E

    None of these.

Show answer

Correct answer

  • A

    64 × 128

See the whole paperTake it as a mock test

Question 15 of 19 in the IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 27 Oct 2024, in the September 2024 term (IIT M DEGREE AN EXAM QDB2 27 Oct 2024). It carries 2 marks.

More questions from this paper

  1. Q1Teacher forcing for training an transformer model is:
  2. Q2Choose the correct statements regarding transformer architecture:
  3. Q3Assume the model has two layers (N = 2). Calculate the total number of parameters in the model (excluding the embedding…
  4. Q4How many parameters does the output layer have?
  5. Q5How many parameters does the embedding layer have?
  6. Q6At time step t = 1, the prediction probability for the words “Naan”, “transformer” and “padaththai” are 0.55,0.15 and 0…
  7. Q7Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  8. Q8Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  9. Q9Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  10. Q10Consider following input matrix to a single neural network layer: X = \begin{bmatrix} 1 & 2 & 3 \ 4 & 7 & 11 \ -3 & 5 &…
  11. Q11Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  12. Q12Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  13. Q13Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  14. Q14Consider a vocabulary \mathcal{V}, \mathcal{V} =([start], breaking, ground, a, is, seismology, study, experience, earth…
  15. Q16Rajesh has created a GPT-like transformer model. However he doesn't have access to large compute infrastructure, so he …
  16. Q17Rajesh has created a GPT-like transformer model. However he doesn't have access to large compute infrastructure, so he …
  17. Q18The input embeddings for the words “learning”, “brings” and “joy” are h_1 = [0.5, 0.25, 1], h_2 = [0.1, 0.25, 0], and h…
  18. Q19Consider the embedding vector for a word, x = [0.1, 0.2, -0.3, 0.4]. Suppose the word is at position 2 in the given sen…