Quiz Space

Large Language Models · Quiz 1 · 25 Feb 2024 · January 2024 term

Question 9: Suppose we have a dataset for machine translation tasks w…

Question 9

+3 marksOne or more correct options

Suppose we have a dataset for machine translation tasks with thousands of samples. Suppose a team considers training the transformer model. The model could be trained in two approaches A  Autoregressive training
B  Teacher forcing
Choose the correct statements

Select all that apply.

  1. A

    Approach A helps the model to converge faster than approach B

  2. B

    Approach B helps the model converge faster than approach A

  3. C

    One can start the training with approach B first and then switch to approach A after some training steps

  4. D

    Once the training starts with approach B and then switching to approach A after some training steps can not be done

Show answer

Correct answers

  • B

    Approach B helps the model converge faster than approach A

  • C

    One can start the training with approach B first and then switch to approach A after some training steps

Question 9 of 15 in the IIT Madras BS Large Language Models (LLM) Quiz 1 paper sat on 25 Feb 2024, in the January 2024 term (IIT M DEGREE AN2 EXAM QDB2 25 Feb 2024). It carries 3 marks.

More questions from this paper

  1. Q1Consider a vocabulary \mathcal{V} = (A, T, C, G) and the corresponding embedding matrix E = \begin{bmatrix} 1 & 0 \ 0 &…
  2. Q2Consider a vocabulary \mathcal{V} = (A, T, C, G) and the corresponding embedding matrix E = \begin{bmatrix} 1 & 0 \ 0 &…
  3. Q3Consider a vocabulary \mathcal{V} = (A, T, C, G) and the corresponding embedding matrix E = \begin{bmatrix} 1 & 0 \ 0 &…
  4. Q4Consider a vocabulary \mathcal{V} = (A, T, C, G) and the corresponding embedding matrix E = \begin{bmatrix} 1 & 0 \ 0 &…
  5. Q5Consider the following configuration for the Vannila transformer architecture with one encoder layer and one decoder la…
  6. Q6Consider the following configuration for the Vannila transformer architecture with one encoder layer and one decoder la…
  7. Q7Figure question
  8. Q8Suppose we run the pre-trained GPT model in an autoregressive fashion for generating a text sequence. Then, the stateme…
  9. Q10A team trained a Language model using the GPT architecture using a large corpus of text. The model configurations are g…
  10. Q11Choose the correct statements
  11. Q12Figure question
  12. Q13Consider a GPT model used for Causal language modelling. We feed the input sentence “This is a cool idea” to the model …
  13. Q14Consider a GPT model used for Causal language modelling. We feed the input sentence “This is a cool idea” to the model …
  14. Q15The statement that “the Next Sentence Prediction (NSP) task requires the BERT model to run autoregressively given the f…