Question 51: c21:_

You are given the encoder hidden states $h_1, h_2, h_3 \in \mathbb{R}^2$ and the decoder hidden state $s_2 \in \mathbb{R}^2$ at time step $t = 2$.

Use **additive attention**, where the alignment score is computed as:

$e_{tj} = v_a^\top \tanh(W_1 s_t + W_2 h_j) \quad \text{and} \quad \alpha_{tj} = \frac{\exp(e_{tj})}{\sum_{k=1}^{3} \exp(e_{tk})}$

The context vector is then computed as:

$c_t = \sum_{j=1}^{3} \alpha_{tj} h_j$

Use the following values:

$s_2 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad h_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad h_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad h_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$

$W_1 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}, \quad W_2 = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}, \quad v_a = \begin{bmatrix} 1 \\ -1 \end{bmatrix}$

Hint: Approximate values of $\tanh(x)$: $\tanh(0) = 0$, $\tanh(1) \approx 0.76$, $\tanh(2) \approx 0.96$, $\tanh(3) \approx 0.995$, $\tanh(4) \approx 0.9993$, $\tanh(5) \approx 0.9999$, $\tanh(6) \approx 0.99999$

Answer the given subquestions:

c21:_________________