Deep Learning Quiz 2: 3 December 2023 (September 2023 term)

The statement that the bias correction in ADAM optimizer is an absolute requirement for it to converge to a local minimum is

Team $A$ constructs a dataset $\mathcal{D} = \{X, y\}$ that contains $N$ samples. Each sample $x_i \in \mathbb{R}$ is uniformly sampled from the function $y = f(x) = 1 + c_1x + c_2x^2 + \cdots + c_px^p$ in the interval $-1 \le x \le 1$, with $p < 80$. The dataset is then given to Team $B$ with the information that the samples were not corrupted by noise. They split the dataset into a training set and a test (validation) set.

Team $B$ assumes a polynomial function $g(x) = k_0 + k_1x + k_2x^2 + \cdots + k_dx^d$ of unknown degree $d$. Therefore, they decided to vary the degree of the polynomial from 1 to 100. For instance, setting $d = 2$ gives the polynomial $g(x) = k_0 + k_1x_1 + k_2x^2$ and the parameters are estimated using the training set.

They measure the mean squared error for each setting. Select the true statement(s). There exists a polynomial degree $d \in \{1, 100\}$ for which

Which of the following optimization algorithms adapts the learning rate based on the current gradient and history of gradients?

In general, applying regularization to an over-fitted deep learning model

Assume a team has trained a CNN model for image classification. They observed that the model was performing poorly (that is, both training and testing errors were high). Which of the following would be a more appropriate measure that might improve the performance of the model?

Based on the above data, answer the given subquestions.

The statement that,“ The gradient of the weight w changes slowly as compared to the gradient of the bias b if we move on the contours of the loss surface in the direction parallel to the weight.” is

Based on the above data, answer the given subquestions.

The statement that,“the gradient of the bias b changes rapidly as compared to the gradient of the weight w if we move on the contours of the loss surface in the direction perpendicular to the weight” is

Based on the above data, answer the given subquestions.

The statement that,“The algorithm converged to the (local or global) minimum after 500 iterations” is

Based on the above data, answer the given subquestions.

The dimension of C1 = 101 × 101 × 100. Suppose we use 25 kernels (filters) of shape 3 × 3, with stride 1 and no zero padding, to generate the convolution layer 2. Then, enter the number of parameters (excluding bias) in the convolution layer-2 (If your answer is 1050, enter 1050)

Based on the above data, answer the given subquestions.

We know that the shape of the flattened layer is N × 1. What is the value of N ?(If your answer is 1050,enter 1050)

Based on the above data, answer the given subquestions.

Suppose we feed in the features F of dimension N × 1 to a fully connected layer with 4 neurons (assume no bias) with the softmax activation function. How many additional parameters are added to the network?

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.