Deep Learning Quiz 1: 7 July 2024 (May 2024 term)

Consider a dataset of 100 points $\mathbf{x_1}, \mathbf{x_2}, \cdots, \mathbf{x_{100}}$.

First 50 points are $\mathbf{x_1} = \mathbf{x_2} = \cdots = \mathbf{x_{50}} = \begin{bmatrix} a \\ a \end{bmatrix}$ and next 50 points are $\mathbf{x_{51}} = \mathbf{x_{52}} = \cdots = \mathbf{x_{100}} = \begin{bmatrix} -a \\ -a \end{bmatrix}$, where $a > 0$. The first 50 data points belong to the positive class (denoted as 1) and the next 50 data points belong to the negative class (denoted by 0). Suppose that the perceptron learning algorithm is used to find the decision boundary that separates these data points with the following rule,

$f(\mathbf{x}) = \begin{cases} 1 & \text{if } \mathbf{w^T x} \ge 0 \\ 0 & \text{if } \mathbf{w^T x} < 0 \end{cases}$

The algorithm checks the data points in order. How often do the weights get updated until convergence? The weights do not include bias. If the algorithm does not converge, enter the answer as $-1$

Consider a single McCulloch-Pitts (MP) neuron with four binary inputs $x_1$, $x_2$, $x_3$, and $x_4$. The neuron produces an output $y$ based on a threshold function. The MP neuron uses the following decision rule

$\hat{y} = \begin{cases} 1, & \text{if } x_1 + x_2 + x_3 + x_4 > \theta \\ 0, & \text{otherwise} \end{cases}$

Given the following input combinations and their corresponding outputs:

Inputs: $x_1 = 1, x_2 = 0, x_3 = 1, x_4 = 1$ Output: $y = 1$
Inputs: $x_1 = 0, x_2 = 1, x_3 = 1, x_4 = 0$ Output: $y = 0$
Inputs: $x_1 = 1, x_2 = 1, x_3 = 0, x_4 = 1$ Output: $y = 1$

What minimum threshold value is required for the neuron to produce an output of 1? If the threshold can not be determined using the given information, enter the answer as $-1$.

Consider a feedforward neural network with one hidden layer trained using backpropagation for a binary classification task with classes labeled as 1 and 0. The network architecture is structured as follows:

• Input layer consisting of 5 neurons
• Hidden layer containing 3 neurons
• Output layer comprising 1 neuron

During the backpropagation process, the derivative of the sigmoid activation function $\sigma(z)$ with respect to its argument $z$ is given by:

$\sigma'(z) = \sigma(z) \cdot (1 - \sigma(z))$

If the loss function utilized for binary classification is the binary cross-entropy loss, and both the hidden layer and output layer use the sigmoid activation function, the cross-entropy loss is represented by:

$L(y, \hat{y}) = -y\log_2(\hat{y}) - (1 - y)\log_2(1 - \hat{y})$

Here, $\hat{y} = P(y = 1|\mathbf{x})$.

Given that the true label $y$ for a data point $\mathbf{x}$ is 1 and the predicted value $\hat{y}$ is 0.9, and the activation at the hidden layer is represented by $h_1 = \begin{bmatrix} 1 \\ 2 \\ 1 \end{bmatrix}$, what is the value of $\frac{\partial L}{\partial W_{200}}$? Here, $W_{200}$ denotes the weight connecting the first neuron of the hidden layer to the output layer neuron.

As per gradient descent, we should move towards 180 degrees with respect to gradient direction. What will happen if we move between 90 and 180 degrees? Consider the loss function to be convex.

Which of the following best describes the objective of gradient descent in relation to the Taylor series expansion of a function f(x) around a given point x0?

Any boolean function of n inputs can be represented exactly by a network of perceptrons containing at least ____________ hidden layer(s) with at least ___________ perceptrons (each) and one output layer containing one perceptron.

Consider the plot shown below:

Which of the following could be the correct equation for this plot?

Consider the MP neuron model and its applicability to representing boolean functions. Select the correct statements:

Consider a feed-forward neural network shown below

Based on the above data, answer the given subquestions.

What is the total number of parameters (excluding bias) in the network?

Consider a feed-forward neural network shown below

Based on the above data, answer the given subquestions.

The true label for a data point $\mathbf{x}$ is given by $\begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}$. Assume that $h_1 = \begin{bmatrix} 0.5 \\ 1 \end{bmatrix}$ and $W_2 = \begin{bmatrix} 1 & 0 \\ 1 & 1 \\ 1 & 0 \end{bmatrix}$.

If the network has no bias, find the loss value.

Use natural logrithm for loss calculation.

Enter your answer correct to two decimal places.

Consider a feed-forward neural network shown below

Based on the above data, answer the given subquestions.

The true label for a data point $\mathbf{x}$ is given by $\begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}$. Assume that $h_1 = \begin{bmatrix} 0.5 \\ 1 \end{bmatrix}$ and $W_2 = \begin{bmatrix} 1 & 0 \\ 1 & 1 \\ 1 & 0 \end{bmatrix}$. If the network has no bias, compute the gradient of loss with respect to $\hat{\mathbf{y}}$ and enter the sum of the elements.

Assume you are using mini-batch gradient descent with a total dataset size of 1,000 and a batch size of 20. If you process 10,000 mini-batches, how many epochs have been completed?

If the batch size in mini-batch gradient descent is increased from 32 to 128 while keeping all other parameters constant, what is the likely impact on the variance of the gradient estimates per update?

What do contour lines on a contour plot represent?