Question 10: Consider a vocabulary \mathcal{V}, \mathcal{V} =([start]…

Consider a vocabulary $\mathcal{V}$,

$\mathcal{V} =$([start], deep, hot, is, learning, research, topic, very, [end]).

Assume that the BART model is being trained using a denoising objective. Suppose the original sentence is “[start] deep learning is very hot research topic [end]” and the corrupted sentence is “[start] deep [mask] is [mask] very hot [mask] topic [end]”

The prediction probabilities outputted by the model are given below.

$\hat{Y} = \begin{bmatrix} 0.1 & 0.17 & 0.11 & 0.12 & 0.08 & 0.17 & 0.15 & 0.22 & 0.28 \\ 0.07 & 0.19 & 0.09 & 0.12 & 0.18 & 0.02 & 0.03 & 0.01 & 0.24 \\ 0.13 & 0.21 & 0.18 & 0.17 & 0.09 & 0.21 & 0.04 & 0.16 & 0.04 \\ 0.16 & 0.12 & 0.07 & 0.06 & 0.15 & 0.12 & 0.2 & 0. & 0.18 \\ 0.11 & 0.15 & 0.17 & 0.15 & 0.07 & 0.12 & 0.24 & 0.02 & 0.19 \\ 0.12 & 0.05 & 0.02 & 0.07 & 0.07 & 0.15 & 0.15 & 0.25 & 0.03 \\ 0.04 & 0.04 & 0.12 & 0.06 & 0.09 & 0.06 & 0.05 & 0.03 & 0.19 \\ 0.02 & 0.05 & 0.07 & 0.18 & 0.02 & 0.22 & 0.03 & 0.24 & 0.13 \end{bmatrix}$

The zeroth row of the matrix is the output probability distribution by the model given the special [Start] token as input. Following the Greedy Search decoding strategy, the subsequent rows give the conditional probability distribution conditioned over the previous tokens. What is the loss value?