Title: Quiz 10
Course: F19 Introduction to Deep Learning

Q: When training a sequence to sequence model, where the input and output are sequences of words, which of the following is true
- The input and output sequences need not have order correspondence
- The input and output sequences both must have the same starting and end symbols
- The input and output sequences must have the same length to be able to compute the divergence
- The input and output sequences are identical
A: The input and output sequences need not have order correspondence

Q: If you are training a seq-to-seq model for translating  English to French, how can the probability distribution for the second predicted word for the French word be written, if the input English sequence is given as \(e_1\), \(e_2\), \(e_3\) and the output French words are given as  \(f_1\), \(f_2\), \(f_3\), and \(Y(f_2)\) is the second output of the network.
- \(Y(f_2)=P(f_2|e_1,e_2,e_3,f_1)\)
- \(Y(f_2)=P(f_2|e_1,e_2,e_3)\)
- \(Y(f_2)=P(e_1,e_2,e_3)P(f_2|f_1)\)
- \(Y(f_2)=P(f_2|f_1)\)
A: \(Y(f_2)=P(f_2|e_1,e_2,e_3,f_1)\)

Q: (Select all that apply.) Which of the following is true when performing machine translation using a sequence to sequence model?
- The output and input are order synchronous
- The order of the inputs to the model determines the order of the outputs
- The order of the symbols in the output sequence is not directly dependent on the order of symbols in the input
- The input may be presented either in left to right order or in right to left order, as long as the same order is used for all inputs
A: The order of the inputs to the model determines the order of the outputs; The input may be presented either in left to right order or in right to left order, as long as the same order is used for all inputs

Q: (Select all that apply.) While performing machine translation using a seq-to-seq model, which of the following may be expected to result in good translation in general:
- An RNN model that evaluates the entire input, before it begins to produce the translated output
- Processing the input with two RNNs, where one processes input left to right and the other right to left, and then performing the translation using the outputs of both RNNs
- A model that incrementally translates the input, after each word has been presented
- A model in which, each word in the translated output is computed considering both previous and subsequent words of the output for consistency
A: An RNN model that evaluates the entire input, before it begins to produce the translated output; Processing the input with two RNNs, where one processes input left to right and the other right to left, and then performing the translation using the outputs of both RNNs

Q: Say you are doing machine translation in a seq to seq model with a vocabulary of only 3 words, A, B, and C. The probability distribution while predicting 2 output words, expanded fully is given by the diagram below. The diagram also shows the probability at every step for each vocabulary word.

If using beam search with K=2, what would be the top 2 sequences retained after two steps?
- BC and AA
- AA and AB
- BC and BA
- AA and CB
A: BC and AA

Q: Attention is a probability distribution over

1. Encoder’s input values
2. Encoder’s hidden state values
3. Decoder’s hidden state values
4. Decoder’s input values
5. Encoder’s last state value
6. None of the above
- 2
- 2, 3
- 2, 4, 1
- 4, 5
- 6
A: 2

Q: A decoder in an AE can be used as a constructive dictionary for the class of data the AE is learned from
- True
- False
A: True

Q: Which of the following are true for the encoder in an autoencoder
- Produces output class probabilities for classification
- Can only be modelled by linear MLPs
- Reconstructs the input from a high-level representation
- Produce high level representation of the input for the decoder
A: Produce high level representation of the input for the decoder

Q: (Select all that apply.) Which of the following is true of an autoencoder with non-linear activation functions?
- The output of the AE will lie on a fixed manifold (surface) regardless of the location of the input
- The dimensionality of the output manifold depends only on the dimensionality of the output of the encoder
- The dimensionality of the output manifold depends on the width of the narrowest layer in the autoencoder
- The output of the autoencoder lies on a linear sub-manifold of the space
A: The output of the AE will lie on a fixed manifold (surface) regardless of the location of the input; The dimensionality of the output manifold depends on the width of the narrowest layer in the autoencoder

Q: (Select all that apply.) Which of the following is true of an autoencoder?
- The decoder architecture is required to be a mirror image of the encoder
- The depth of the decoder must be identical to that of the encoder
- There is a unique way to decompose the autoencoder network into an encoder and a decoder
- The activation functions of the decoder must be identical to that of the encoder
- Classical unregularized autoencoders minimize a reconsruction loss
- Classical unregularized autoencoders attempt to embed both the content and style of an image into a latent space
A: Classical unregularized autoencoders minimize a reconsruction loss; Classical unregularized autoencoders attempt to embed both the content and style of an image into a latent space