Opening the paper…
Deep Learning, Quiz 2
Consider four training experiments, each involving a neural network trained on the same dataset for 50 epochs using gradient descent. The loss curves for these models are shown below. The model obtained at the end of 50 epochs is retained in each of the four cases. The solid curve is the training loss and the dashed curve is the validation loss.
Now consider these four statements.
(A) Learning rate used is too high
(B) Low bias and high variance
(C) Low bias and low variance
(D) High bias and low variance
Choose the most appropriate pairing of models with the statements.
Consider four training experiments, each involving a neural network trained on the same dataset for 50 epochs using gradient descent. The loss curves for these models are shown below. The model obtained at the end of 50 epochs is retained in each of the four cases. The solid curve is the training loss and the dashed curve is the validation loss. Four plots of loss versus epochs (solid and dashed curves) for models M1, M2, M3, M4 Now consider these four statements.\ (A) Learning rate used is too high\ (B) Low bias and high variance\ (C) Low bias and low variance\ (D) High bias and low variance\ Choose the most appropriate pairing of models with the statements. Which of the following optimizers use an adaptive learning rate? An autoencoder is a feedforward neural network in which the input and the output layer have the same number of neurons. There is one hidden layer in between. The number of neurons in the hidden layer is 30% of the neurons present in the input layer. Call the two weight matrices $W_1$ and $W_2$. Ignore biases everywhere. We now consider two instances of this architecture: - Network-1: $W_1$ and $W_2$ are not tied, meaning, they are two different weight matrices with no parameters being shared. - Network-2: $W_1$ and $W_2$ are tied, meaning, $W_2 = W_1^T$ with parameters being shared. Both networks are trained on some dataset and their parameters are saved on a hard-disk with efficient storage being the focus. If the number of parameters that need to be stored in networks 1 and 2 are $n_1$ and $n_2$ respectively, find $n_1/n_2$.