Question 8: Consider the following configuration for the Vannila tran…

Consider the following configuration for the Vannila transformer architecture with one encoder layer and one decoder layer.

• Source vocabulary size $= 100$
• Target vocabulary size $= 200$
• maximum sequence length $= 32$
• length of context window $(T)$ for both encoder and decoder $= 32$
• number of heads $n_h = 4$
• $dff = 4 * dmodel$
• $dq = dk = dv = \frac{dmodel}{n_h}$

Based on the above data, answer the given subquestions.

Suppose the number of learnable parameters in the source input-embedding layer is 3200, how many parameters are there in the positional embedding layer of the source language?