Deep Learning for Computer Vision Quiz 2: 3 August 2025 (May 2025 term)

Which of the following architectures introduced the ROI pooling layer for extracting fixed-size feature maps from variable-size region proposals?

Which object detection method first integrated a Region Proposal Network (RPN) to generate region proposals within the detection model?

Which architecture performs object detection as a single regression problem from image pixels to bounding boxes and class probabilities?

Consider the following statements P and Q regarding AlexNet and choose the correct option: (P) In AlexNet, a trainable Local Response Normalization Layers were introduced to emulate the competitive nature of real neurons, where highly active neurons suppress the activity of neighboring neurons, creating competition among different kernel outputs.
(Q) In AlexNet, a total of 8 Convolutional layers contain only about 6-7% of the total parameters hence account for the least computation.

Consider two bounding boxes in an image:
- Box A: top-left at (0, 0), bottom-right at (12, 12) - Box B: top-left at (4, 4), bottom-right at (16, 16) What is the Intersection over Union (IoU) between these two boxes ____________(in %)

Which one of the following statements is True?

Match the following:

Which of the following techniques help mitigate the vanishing gradient problem in recurrent neural networks?
1. Gradient clipping
2. Use of ReLU activation to preserve gradient flow
3. Use of LSTM units
4. Data augmentation (e.g., reversing input sequence)

Which of the following techniques help mitigate the exploding gradient problem in recurrent neural networks?
1. Gradient clipping
2. Use of Sigmoid activation to prevent gradient growth
3. Use of LSTM units
4. Data augmentation (e.g., reversing input sequence)

Which one of the following statements is false? (Pick the most appropriate one.)

Which of the following is true regarding Hard Attention and Soft Attention?

Which of the following are true:

Consider a Faster R-CNN object detection framework, which is being trained with a mini-batch size of 2. The RPN produces 512 proposals, out of which 256 and 128 of them are foreground proposals, corresponding to each image in the mini-batch. For a single mini-batch update, the bounding box regression loss will be computed_____________ times.

Suppose you want to build an encoder-decoder recurrent model with attention. The context vector at time step t is given by:

where αtj refers to the alignment score between the decoder state at time t _____________and the encoder state at time _____________. Note: Please write your answer using variables
Based on the above data, answer the given subquestions.

Decoder state at time t : _______________

Suppose you want to build an encoder-decoder recurrent model with attention. The context vector at time step t is given by:

where αtj refers to the alignment score between the decoder state at time t _____________and the encoder state at time _____________. Note: Please write your answer using variables
Based on the above data, answer the given subquestions.

Encoder state at time: _______________

Consider a Gated Recurrent Unit (GRU) cell at time step ’t’.

For each of the entities in the given subquestions, enter the appropriate number from the figure that corresponds to it:
Based on the above data, answer the given subquestions.

Hidden state ht−1______________

Consider a Gated Recurrent Unit (GRU) cell at time step ’t’.

For each of the entities in the given subquestions, enter the appropriate number from the figure that corresponds to it:
Based on the above data, answer the given subquestions.

Hidden state ht_____________

Consider a Gated Recurrent Unit (GRU) cell at time step ’t’.

For each of the entities in the given subquestions, enter the appropriate number from the figure that corresponds to it:
Based on the above data, answer the given subquestions.

Current Input xt________________

Consider a Gated Recurrent Unit (GRU) cell at time step ’t’.

For each of the entities in the given subquestions, enter the appropriate number from the figure that corresponds to it:
Based on the above data, answer the given subquestions.

Update gate zt_______________

Consider a Gated Recurrent Unit (GRU) cell at time step ’t’.

For each of the entities in the given subquestions, enter the appropriate number from the figure that corresponds to it:
Based on the above data, answer the given subquestions.

Reset gate rt ______________

Consider a Gated Recurrent Unit (GRU) cell at time step ’t’.

For each of the entities in the given subquestions, enter the appropriate number from the figure that corresponds to it:
Based on the above data, answer the given subquestions.

New hidden state content eh____________

Let input have size Df × Df × M where Df = 10 and M = 256 and output feature map (after passing input through conv layer) has Df × Df × N size where N = 512. Assume padded convolution. Let width of the square kernel in conv layer be k where k = 7 (Ignore the bias term in the calculation). Calculate the number of parameters and computational cost for this convolution layer. Based on the above data, answer the given subquestions.

Number of Parameters:_____________

Let input have size Df × Df × M where Df = 10 and M = 256 and output feature map (after passing input through conv layer) has Df × Df × N size where N = 512. Assume padded convolution. Let width of the square kernel in conv layer be k where k = 7 (Ignore the bias term in the calculation). Calculate the number of parameters and computational cost for this convolution layer. Based on the above data, answer the given subquestions.

Computational Cost:___________________

Let input have size Df × Df × M where Df = 10 and M = 256 and output feature map (after passing input through conv layer) has Df × Df × N size where N = 512. Assume padded convolution. Let width of the square kernel in conv layer be k where k = 7 (Ignore the bias term in the calculation), calculate the number of parameters and computational cost, but make use of Depthwise Seperable convolution and pointwise convolution instead of standard convolution. Based on the above data, answer the given subquestions.

Number of parameters for depthwise seperable convolution:_______________

Let input have size Df × Df × M where Df = 10 and M = 256 and output feature map (after passing input through conv layer) has Df × Df × N size where N = 512. Assume padded convolution. Let width of the square kernel in conv layer be k where k = 7 (Ignore the bias term in the calculation), calculate the number of parameters and computational cost, but make use of Depthwise Seperable convolution and pointwise convolution instead of standard convolution. Based on the above data, answer the given subquestions.

Computational Cost for depthwise seperable convolution:____________________

Let input have size Df × Df × M where Df = 10 and M = 256 and output feature map (after passing input through conv layer) has Df × Df × N size where N = 512. Assume padded convolution. Let width of the square kernel in conv layer be k where k = 7 (Ignore the bias term in the calculation), calculate the number of parameters and computational cost, but make use of Depthwise Seperable convolution and pointwise convolution instead of standard convolution. Based on the above data, answer the given subquestions.

Number of parameters for pointwise convolution:___________________

Let input have size Df × Df × M where Df = 10 and M = 256 and output feature map (after passing input through conv layer) has Df × Df × N size where N = 512. Assume padded convolution. Let width of the square kernel in conv layer be k where k = 7 (Ignore the bias term in the calculation), calculate the number of parameters and computational cost, but make use of Depthwise Seperable convolution and pointwise convolution instead of standard convolution. Based on the above data, answer the given subquestions.

Computational cost for pointwise convolution:__________________

Based on the above data, answer the given subquestions.

Number of weights in Weight Matrix U1 is:______________

Based on the above data, answer the given subquestions.

Number of weights in Weight Matrix V1 is:______________

Based on the above data, answer the given subquestions.

Number of weights in Weight Matrix U2 is:__________________

Based on the above data, answer the given subquestions.

Number of weights in Weight Matrix V2 is:___________________

Based on the above data, answer the given subquestions.

Number of weights in Weight Matrix W is:_______________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **dot-product attention**, where the attention score is computed as:

$e_{tj} = s_t^\top h_j \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

Note: Use approximate values of exponential $\exp(1) \approx 2.72$, $\exp(2) \approx 7.39$, etc.

Answer the given subquestions:

e21: _________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **dot-product attention**, where the attention score is computed as:

$e_{tj} = s_t^\top h_j \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

Note: Use approximate values of exponential $\exp(1) \approx 2.72$, $\exp(2) \approx 7.39$, etc.

Answer the given subquestions:

e22: _________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **dot-product attention**, where the attention score is computed as:

$e_{tj} = s_t^\top h_j \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

Note: Use approximate values of exponential $\exp(1) \approx 2.72$, $\exp(2) \approx 7.39$, etc.

Answer the given subquestions:

e23: _________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **dot-product attention**, where the attention score is computed as:

$e_{tj} = s_t^\top h_j \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

Note: Use approximate values of exponential $\exp(1) \approx 2.72$, $\exp(2) \approx 7.39$, etc.

Answer the given subquestions:

α21: _________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **dot-product attention**, where the attention score is computed as:

$e_{tj} = s_t^\top h_j \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

Note: Use approximate values of exponential $\exp(1) \approx 2.72$, $\exp(2) \approx 7.39$, etc.

Answer the given subquestions:

α22: _________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **dot-product attention**, where the attention score is computed as:

$e_{tj} = s_t^\top h_j \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

Note: Use approximate values of exponential $\exp(1) \approx 2.72$, $\exp(2) \approx 7.39$, etc.

Answer the given subquestions:

α23: _________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **dot-product attention**, where the attention score is computed as:

$e_{tj} = s_t^\top h_j \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

Note: Use approximate values of exponential $\exp(1) \approx 2.72$, $\exp(2) \approx 7.39$, etc.

Answer the given subquestions:

c21: _________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **dot-product attention**, where the attention score is computed as:

$e_{tj} = s_t^\top h_j \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

Note: Use approximate values of exponential $\exp(1) \approx 2.72$, $\exp(2) \approx 7.39$, etc.

Answer the given subquestions:

c22: _________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **additive attention**, where the alignment score is computed as:

$e_{tj} = v_a^\top \tanh(W_1 s_t + W_2 h_j) \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

$W_1 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}, \quad W_2 = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}, \quad v_a = \begin{bmatrix} 1 \\ -1 \end{bmatrix}$

Hint: Approximate values of $\tanh(x)$: $\tanh(0) = 0$, $\tanh(1) \approx 0.76$, $\tanh(2) \approx 0.96$, $\tanh(3) \approx 0.995$, $\tanh(4) \approx 0.9993$, $\tanh(5) \approx 0.9999$, $\tanh(6) \approx 0.99999$

Answer the given subquestions:

e21:_________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **additive attention**, where the alignment score is computed as:

$e_{tj} = v_a^\top \tanh(W_1 s_t + W_2 h_j) \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

$W_1 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}, \quad W_2 = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}, \quad v_a = \begin{bmatrix} 1 \\ -1 \end{bmatrix}$

Hint: Approximate values of $\tanh(x)$: $\tanh(0) = 0$, $\tanh(1) \approx 0.76$, $\tanh(2) \approx 0.96$, $\tanh(3) \approx 0.995$, $\tanh(4) \approx 0.9993$, $\tanh(5) \approx 0.9999$, $\tanh(6) \approx 0.99999$

Answer the given subquestions:

e22:_________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **additive attention**, where the alignment score is computed as:

$e_{tj} = v_a^\top \tanh(W_1 s_t + W_2 h_j) \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

$W_1 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}, \quad W_2 = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}, \quad v_a = \begin{bmatrix} 1 \\ -1 \end{bmatrix}$

Hint: Approximate values of $\tanh(x)$: $\tanh(0) = 0$, $\tanh(1) \approx 0.76$, $\tanh(2) \approx 0.96$, $\tanh(3) \approx 0.995$, $\tanh(4) \approx 0.9993$, $\tanh(5) \approx 0.9999$, $\tanh(6) \approx 0.99999$

Answer the given subquestions:

e23:_________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **additive attention**, where the alignment score is computed as:

$e_{tj} = v_a^\top \tanh(W_1 s_t + W_2 h_j) \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

$W_1 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}, \quad W_2 = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}, \quad v_a = \begin{bmatrix} 1 \\ -1 \end{bmatrix}$

Hint: Approximate values of $\tanh(x)$: $\tanh(0) = 0$, $\tanh(1) \approx 0.76$, $\tanh(2) \approx 0.96$, $\tanh(3) \approx 0.995$, $\tanh(4) \approx 0.9993$, $\tanh(5) \approx 0.9999$, $\tanh(6) \approx 0.99999$

Answer the given subquestions:

α21:_________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **additive attention**, where the alignment score is computed as:

$e_{tj} = v_a^\top \tanh(W_1 s_t + W_2 h_j) \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

$W_1 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}, \quad W_2 = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}, \quad v_a = \begin{bmatrix} 1 \\ -1 \end{bmatrix}$

Hint: Approximate values of $\tanh(x)$: $\tanh(0) = 0$, $\tanh(1) \approx 0.76$, $\tanh(2) \approx 0.96$, $\tanh(3) \approx 0.995$, $\tanh(4) \approx 0.9993$, $\tanh(5) \approx 0.9999$, $\tanh(6) \approx 0.99999$

Answer the given subquestions:

α22:_________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **additive attention**, where the alignment score is computed as:

$e_{tj} = v_a^\top \tanh(W_1 s_t + W_2 h_j) \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

$W_1 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}, \quad W_2 = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}, \quad v_a = \begin{bmatrix} 1 \\ -1 \end{bmatrix}$

Hint: Approximate values of $\tanh(x)$: $\tanh(0) = 0$, $\tanh(1) \approx 0.76$, $\tanh(2) \approx 0.96$, $\tanh(3) \approx 0.995$, $\tanh(4) \approx 0.9993$, $\tanh(5) \approx 0.9999$, $\tanh(6) \approx 0.99999$

Answer the given subquestions:

α23:_________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **additive attention**, where the alignment score is computed as:

$e_{tj} = v_a^\top \tanh(W_1 s_t + W_2 h_j) \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

$W_1 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}, \quad W_2 = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}, \quad v_a = \begin{bmatrix} 1 \\ -1 \end{bmatrix}$

Hint: Approximate values of $\tanh(x)$: $\tanh(0) = 0$, $\tanh(1) \approx 0.76$, $\tanh(2) \approx 0.96$, $\tanh(3) \approx 0.995$, $\tanh(4) \approx 0.9993$, $\tanh(5) \approx 0.9999$, $\tanh(6) \approx 0.99999$

Answer the given subquestions:

c21:_________________

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **additive attention**, where the alignment score is computed as:

$e_{tj} = v_a^\top \tanh(W_1 s_t + W_2 h_j) \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

$W_1 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}, \quad W_2 = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}, \quad v_a = \begin{bmatrix} 1 \\ -1 \end{bmatrix}$

Hint: Approximate values of $\tanh(x)$: $\tanh(0) = 0$, $\tanh(1) \approx 0.76$, $\tanh(2) \approx 0.96$, $\tanh(3) \approx 0.995$, $\tanh(4) \approx 0.9993$, $\tanh(5) \approx 0.9999$, $\tanh(6) \approx 0.99999$

Answer the given subquestions:

c22:_________________