Title: Quiz 12
Course: F19 Introduction to Deep Learning

Q: (True/False) One of the reasons why most papers on GAN’s are about image generation is because it is nice to have pictures to look at when reading a paper.
- True
- False
A: True

Q: (True/False) There is no generally applicable method to evaluate the quality of a GAN like the classification accuracy for a classifier.
- True
- False
A: True

Q: (True/False) The JSD is very likely to cause instability during training because in practice the supports of the generated and real distributions lie on very high dimensional manifolds. (See: Towards Principled Methods for Training Generative Adversarial Networks)
- True
- False
A: False

Q: (True/False) Since we need the discriminator to guide the generator, we can train a perfect discriminator at the beginning to avoid “Mode collapse”.
- True
- False
A: False

Q: (Select all that apply.) The Jensen-Shannon Divergence (JSD) is superior compared to the KL-divergence because...
- JSD is symmetric.
- JSD allows to calculate the distance between two continuous distributions.
- JSD has a single optimum.
- JSD is always bounded by 0 and 1.
A: JSD is symmetric.

Q: (Select all that apply.) Which statements are true about the VAEs and GANs? (see: Auto-Encoding Variational Bayes)
- GANs & VAEs both try to minimize the divergence between the generated distribution and the target distribution.
- GANs try to learn an encoder decoder pair.
- GANs & VAEs both evaluated on prior samples.
- Althrough GANs are hard to train, the results are much better than VAEs.
- VAEs use prior samples to train.
A: GANs & VAEs both evaluated on prior samples.; Althrough GANs are hard to train, the results are much better than VAEs.

Q: (Select all that apply.) Which statements are true about different type of GANs?
- A conditional GAN generates larger images conditioned on smaller images, repeated until you reach the desired size.
- A categorical GAN tries to use softmax to calculate the entropy of the true/false label.
- An InfoGAN will split the space is split into the information you care about and the noise you don’t care about based on the assumption that if it can recover the noise, then information you care about will be useful.
- A BiGANs’ discriminator tries to tell the difference between pairs of real data and encoded real data, i.e. trains a decoder and a partial encoder pair.
A: A categorical GAN tries to use softmax to calculate the entropy of the true/false label.; An InfoGAN will split the space is split into the information you care about and the noise you don’t care about based on the assumption that if it can recover the noise, then information you care about will be useful.

Q: (Select all that apply.) Which statements are true?
- If we train a GAN with an optimal discriminator, we are minimizing the JS-divergence of the distributions.
- Compared to a regular GAN, Wasserstein GAN will provide a loss with smoother gradient everywhere.
- WGAN-GP enforces the Lipschitz constraint by gradient penalty and weight clipping.
- Enforcing the Lipschitz constraint is the main difficulty in WGAN.
A: If we train a GAN with an optimal discriminator, we are minimizing the JS-divergence of the distributions.; Compared to a regular GAN, Wasserstein GAN will provide a loss with smoother gradient everywhere.; Enforcing the Lipschitz constraint is the main difficulty in WGAN.

Q: Consider \(P_0\) is a distribution defined over \((0,0)\),\((1,Z)\) (\((0,0)\) is the \((x,y)\) coordinate of one corner of the rectangle and \((1,Z)\) is the other corner), and \(P_1\) is a distribution over \((0,Z)\),\((1,9)\), \(Z \sim U[0,1]\). Which one is the EM-distance between \(P_0\) and \(P_1\)?
- 0
- 8
- -8
- log 2
A: 8