Title: Quiz 11
Course: F19 Introduction to Deep Learning

Q: (Select all that apply.) Which of the following is/are true of the memory capacity (stationary patterns) of a Hopfield network with \(N\) neurons?
- It can store up to \(0.14N\)0.14 N patterns by design
- It can store up to \(N\)N patterns by design
- It can have up to \(O\left(N^2\right)\)O ( N 2 ) memories
- It can have an exponential (in \(N\)N) number of memories
A: It can store up to \(N\)N patterns by design; It can have an exponential (in \(N\)N) number of memories

Q: (Select all that apply.) Which of the following is/are true of a Hopfield networks?
- It is a loopy network
- The output of a neuron connects back directly to its input
- The network is undirected
- It is a feedforward network
A: It is a loopy network; The network is undirected

Q: Which of the following is/are true of Hebbian learning of Hopfield networks
- It can perfectly remember a single pattern
- A network with \(N\) neurons trained by Hebbian learning can perfectly remember up to \(N\) patterns
- Hebbian learning can only remember orthogonal patterns
- When learning a single pattern, it will be stable and stationary
A: It can perfectly remember a single pattern; When learning a single pattern, it will be stable and stationary

Q: (Select all that apply.) Which of the following is/are true of the evolution of a Hopfield network?
- The energy function can have multiple minima over network states
- The energy function cannot have multiple minimia over network states because it is a convex quadratic and convex quadratics have a unique miniumum
- The energy function can have multiple minima because it is a non-convex quadratic
- The energy function has a unique minimum over possible network states
- The energy function of the network is always a nonconvex quadratic
- At a local energy minimum, the network state remains stationary
- Every step of the evolution will strictly decrease the energy of the network
A: The energy function can have multiple minima over network states; At a local energy minimum, the network state remains stationary; Every step of the evolution will strictly decrease the energy of the network

Q: (Select all that apply.) Which of the following is/are true of the number of parameters in a Hopfield network?
- It scales non-linearly with the number of bits in the patterns it remembers
- It scales in a linear or affine manner with the number of bits in the patterns it remembers
- It scales in a linear or affine manner with the number of patterns the network tries to remember
- It scales non-linearly with the number of patterns the network tries to remember
A: It scales non-linearly with the number of bits in the patterns it remembers

Q: (Select all that apply.) If the Hebbian learning rule is used to train a Hopfield network, then...
- The network will have "ghost" patterns with low energy states
- The energy function is convex in the length of the input pattern
- Only a single pattern can be stored in the network
- The network can evolve chaotically for certain input patterns
- The network can store multiple patterns by summing or averaging the weights calculated for each single pattern separately
A: The network will have "ghost" patterns with low energy states; The network can store multiple patterns by summing or averaging the weights calculated for each single pattern separately

Q: A vector \(a\) is a stationary state in the Hopfield network with the following weight matrix

\(\begin{bmatrix} 0 & -1 & 1 & -1 & 1 & -1\\ -1 & 0 & -1 & -1 & -1 & 1\\ 1 & -1 & 0 & 1 & -1 & -1\\ -1 & -1 & 1 & 0 & 1 & -1\\ 1 & -1 & -1 & 1 & 0 & -1\\ -1 & 1 & -1 & -1 & -1 & 0 \end{bmatrix}\)

\(a=\begin{bmatrix} 1\\1\\1\\1\\1\\1 \end{bmatrix}\)
- True
- False
A: False

Q: Hopfield assumed that less than \(0.4\%\) probability for patterns to be unstable was a good idea. This means that we can store up to \(0.14N\) (random) stable patterns when \(N\) is large enough. Following the same assumptions as Hopfield did, how many patterns can we store when we want a less than \(0.30\%\) probability for patterns to be unstable. What is the answer as a fraction of \(N\), round to 2 decimal places?
- 0.13
- 0.27
- 0.36
- 0.60
A: 0.13

Q: Given a Hopfield Network with the following weights

\(\begin{bmatrix} 0 & -1 & -1 & 1 & 1 & 1\\ -1 & 0 & 1 & -1 & -1 & -1\\ -1 & 1 & 0 & -1 & -1 & -1\\ 1 & -1 & -1 & 0 & 1 & 1\\ 1 & -1 & -1 & 1 & 0 & 1\\ 1 & -1 & -1 & 1 & 1 & 0\\ \end{bmatrix}\)

If we initialize the network with the state [1, 1, 1, 1, 1, 1], which final state will it get?
- (1,-1,-1,1,1,1)
- (1,1,1,1,1,1)
- (-1,-1,-1,-1,1,1)
- (1,-1,-1,1,-1,-1)
A: (1,-1,-1,1,1,1)

Q: Which is/are the lowest energy state(s) of the net with the given weight matrix?

\(\begin{bmatrix} 0 & -1 & -1 & 1 & 1 & 1\\ -1 & 0 & 1 & -1 & -1 & -1\\ -1 & 1 & 0 & -1 & -1 & -1\\ 1 & -1 & -1 & 0 & 1 & 1\\ 1 & -1 & -1 & 1 & 0 & 1\\ 1 & -1 & -1 & 1 & 1 & 0\\ \end{bmatrix}\)
- (1,-1,-1,1,1,1), (-1,1,1,-1,-1,-1)
- (1,1,1,-1,- 1,-1), (1,-1,-1,1,-1,-1)
- (-1,-1,-1,-1,- 1,-1), (1,-1,-1,1,1,1)
- (-1,-1,-1,-1,- 1,-1), (1,1,1,1,1,1)
A: (1,-1,-1,1,1,1), (-1,1,1,-1,-1,-1)