Question 8: Consider following input matrix to a single neural networ…

Consider following input matrix to a single neural network layer:

$X = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 7 & 11 \\ -3 & 5 & -2 \end{bmatrix}$

It has 3 samples with 3 features each i.e. batch size is 3. Each column represents a sample. Left most column represents the first sample and topmost row represents the first feature vector.

Note: assume $\gamma = 1$, $\beta = 0$ and $\epsilon = 10^{-6}$ for both batch and layer normalization, if not mentioned explicitly. Use standard scaling.

Based on the above data, answer the given subquestions.

Say matrix F is computed by applying layer normalization on X, what will be sum of every element in first/leftmost column of F?