Quiz Space

Large Language Models · Quiz 1 · 25 Feb 2024 · January 2024 term

Question 10: A team trained a Language model using the GPT architectu…

Question 10

+4 marksOne or more correct options

A team trained a Language model using the GPT architecture using a large corpus of text. The model configurations are given below

  • Vocabulary size =30000= 30000
  • dmodel=512dmodel = 512
  • length of context window T=1024T = 1024
  • dff=4∗dmodeldff = 4 * dmodel
  • dq=dk=dv=64dq = dk = dv = 64

They trained the model for millions of steps and stopped the training after the loss had decreased significantly. Then, they used the model to generate the text by giving the special starting token [BOS] as an input to the model. Select the length of the text sequence that the model could generate.

Select all that apply.

  1. A

    512

  2. B

    1024

  3. C

    1536

  4. D

    2048

Show answer

Correct answers

  • A

    512

  • B

    1024

  • C

    1536

  • D

    2048

See the whole paperTake it as a mock test

Question 10 of 15 in the IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 25 Feb 2024, in the January 2024 term (IIT M DEGREE AN2 EXAM QDB2 25 Feb 2024). It carries 4 marks.

More questions from this paper

  1. Q1Consider a vocabulary \mathcal{V} = (A, T, C, G) and the corresponding embedding matrix E = \begin{bmatrix} 1 & 0 \ 0 &…
  2. Q2Consider a vocabulary \mathcal{V} = (A, T, C, G) and the corresponding embedding matrix E = \begin{bmatrix} 1 & 0 \ 0 &…
  3. Q3Consider a vocabulary \mathcal{V} = (A, T, C, G) and the corresponding embedding matrix E = \begin{bmatrix} 1 & 0 \ 0 &…
  4. Q4Consider a vocabulary \mathcal{V} = (A, T, C, G) and the corresponding embedding matrix E = \begin{bmatrix} 1 & 0 \ 0 &…
  5. Q5Consider the following configuration for the Vannila transformer architecture with one encoder layer and one decoder la…
  6. Q6Consider the following configuration for the Vannila transformer architecture with one encoder layer and one decoder la…
  7. Q7Figure question
  8. Q8Suppose we run the pre-trained GPT model in an autoregressive fashion for generating a text sequence. Then, the stateme…
  9. Q9Suppose we have a dataset for machine translation tasks with thousands of samples. Suppose a team considers training th…
  10. Q11Choose the correct statements
  11. Q12Figure question
  12. Q13Consider a GPT model used for Causal language modelling. We feed the input sentence “This is a cool idea” to the model …
  13. Q14Consider a GPT model used for Causal language modelling. We feed the input sentence “This is a cool idea” to the model …
  14. Q15The statement that “the Next Sentence Prediction (NSP) task requires the BERT model to run autoregressively given the f…