Question 1

Suppose you replace the log-loss in the original GAN with a general *f*-divergence minimization framework. Which of the following cannot be directly represented as an f-divergence? Imagine in GAN, a simple setting where $p_{\text{data}} = \delta(x-2)$ and $p_g = \delta(x+4)$, i.e., both are Dirac delta functions located at different points. What would be the GAN value function $V(G)$? Dirac delta function is defined as $$\delta(x) = \begin{cases} 0, & x \neq 0 \\ \infty, & x = 0 \end{cases}$$ which satisfies $$\int_{-\infty}^{\infty} \delta(x)\, dx = 1.$$ Figure from the original question paper