Question 10: Consider the following configuration for the Vannila tra…

Consider the following configuration for the Vannila transformer architecture with one encoder layer and one decoder layer.

• Source vocabulary size $= 100$
• Target vocabulary size $= 200$
• maximum sequence length $= 32$
• length of context window $(T)$ for both encoder and decoder $= 32$
• number of heads $n_h = 4$
• $dff = 4 * dmodel$
• $dq = dk = dv = \frac{dmodel}{n_h}$

Based on the above data, answer the given subquestions.

How many parameters does the matrix WQ have in anyone head of the encoder?