Here is a configuration of the GPTNeo model from the Hugging Face hub. "bos_token_id": 50256, "classifier_dropout": 0.1, "embed_dropout": 0.0, "eos_token_id": 50256, "hidden_size": 2048, "initializer_range": 0.02, "intermediate_size": null, "layer_norm_epsilon": 1e-05, "max_position_embeddings": 2048, "model_type": "gpt_neo", "num_heads": 16, "num_layers": 24, "resid_dropout": 0.0, "transformers_version": "4.44.2", "use_cache": true, "vocab_size": 50257, "window_size": 256 Figure 1: GPTNeoConfig Based on the above data, answer the given subquestions. Enter the context length.

Deep Learning Practice · Quiz 1 · 27 Oct 2024 · September 2024 term

Question 12: Here is a configuration of the GPTNeo model from the Hug…

Question 12

+3 marksNumerical answer

Here is a configuration of the GPTNeo model from the Hugging Face hub.

text

1"bos_token_id": 50256,
2"classifier_dropout": 0.1,
3"embed_dropout": 0.0,
4"eos_token_id": 50256,
5"hidden_size": 2048,
6"initializer_range": 0.02,
7"intermediate_size": null,
8"layer_norm_epsilon": 1e-05,
9"max_position_embeddings": 2048,
10"model_type": "gpt_neo",
11"num_heads": 16,
12"num_layers": 24,
13"resid_dropout": 0.0,
14"transformers_version": "4.44.2",
15"use_cache": true,
16"vocab_size": 50257,
17"window_size": 256

Figure 1: GPTNeoConfig

Based on the above data, answer the given subquestions.

Enter the context length.

Show answer

Correct answer: 2048

See the whole paper Take it as a mock test

Question 12 of 15 in the IIT Madras BS Deep Learning Practice (Deep Learning Practice) Quiz 1 paper sat on 27 Oct 2024, in the September 2024 term (IIT M DEGREE AN EXAM QDB2 27 Oct 2024). It carries 3 marks.

Question 12: Here is a configuration of the GPTNeo model from the Hug…

Question 12

More questions from this paper