Deep Learning End Term: 13 April 2025 (January 2025 term)

1. A

   ReLU

2. B

   Linear (identity)

3. C

   Tanh

1. A

   The query vectors come from the decoder stack, while the key and value vectors come from the encoder stack.

2. B

   The query and value vectors come from the decoder stack, while the key vectors come from the encoder stack.

3. C

   The query, key and value vectors come from the decoder stack.

4. D

   The query, key and value vectors come from the encoder stack.

Select all that apply.

1. A

   During training, a randomly chosen set of neurons in the layer are dropped out. The neurons to be dropped are determined dynamically during each iteration.

2. B

   During inference (testing), no neurons are dropped out. Instead, the activation of each neuron in the layer is scaled by 0.5.

3. C

   During training, a fixed set of neurons in the layer are dropped out. This set remains the same in every iteration and the neurons to be dropped are determined before the training begins.

4. D

   During inference (testing), a randomly chosen set of neurons in the layer are dropped out.