Deep Learning Quiz 1: 13 July 2025 (May 2025 term)

Which of the following is true?

To learn the parameters of a neural network for a classification problem, mini-batch gradient descent is run on a dataset of size 1000 with batch size of 25 for 10 epochs. Find the number of times each parameter is updated.

Let $h : \{0,1\}^5 \to \{0,1\}$ be a Boolean function. Consider an MP neuron with five binary inputs, all of which are excitatory. The output of the neuron is given as follows:

$f(x_1, \cdots, x_5) = \begin{cases} 1, & x_1 + \cdots + x_5 \geqslant \theta \\ 0, & \text{otherwise} \end{cases}$

$\theta$ is a positive integer. $f(x_1, \cdots, x_5) = h(x_1, \cdots, x_5)$ for all tuples $(x_1, \cdots, x_5) \in \{0,1\}^5$.

The given sub-questions are independent of each other.

Based on the above data, answer the given subquestions.

If θ = 1, which of the following are true?

Let $h : \{0,1\}^5 \to \{0,1\}$ be a Boolean function. Consider an MP neuron with five binary inputs, all of which are excitatory. The output of the neuron is given as follows:

$f(x_1, \cdots, x_5) = \begin{cases} 1, & x_1 + \cdots + x_5 \geqslant \theta \\ 0, & \text{otherwise} \end{cases}$

$\theta$ is a positive integer. $f(x_1, \cdots, x_5) = h(x_1, \cdots, x_5)$ for all tuples $(x_1, \cdots, x_5) \in \{0,1\}^5$.

The given sub-questions are independent of each other.

Based on the above data, answer the given subquestions.

Let $h : \{0,1\}^5 \to \{0,1\}$ be a Boolean function. Consider an MP neuron with five binary inputs, all of which are excitatory. The output of the neuron is given as follows:

$f(x_1, \cdots, x_5) = \begin{cases} 1, & x_1 + \cdots + x_5 \geqslant \theta \\ 0, & \text{otherwise} \end{cases}$

$\theta$ is a positive integer. $f(x_1, \cdots, x_5) = h(x_1, \cdots, x_5)$ for all tuples $(x_1, \cdots, x_5) \in \{0,1\}^5$.

The given sub-questions are independent of each other.

Based on the above data, answer the given subquestions.

Consider a neural network for a regression problem with one input and one output. There is one hidden layer with two sigmoid neurons. The output layer is linear. Ignore biases in all units.

Based on the above data, answer the given subquestions.

Consider a neural network for a regression problem with one input and one output. There is one hidden layer with two sigmoid neurons. The output layer is linear. Ignore biases in all units.

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

Find the number of weights in the network.

Based on the above data, answer the given subquestions.

If all the weights in the network have the same value, find the cross entropy loss for an arbitrary data-point. If this can be computed, enter the value of the loss correct to two places after the decimal. If the information provided is not sufficient to compute the loss, enter −1 as the answer.

Based on the above data, answer the given subquestions.

Starting at $(x_0, y_0)$, run two steps of gradient descent with learning rate $\eta = 0.2$ to obtain $(x_2, y_2)$. If $(x_2, y_2) = \alpha \cdot (x_0, y_0)$, enter the value of $\alpha$ correct to two places after the decimal. Note that tne sequence of iterates will be: $(x_0, y_0) \to (x_1, y_1) \to (x_2, y_2)$.

Based on the above data, answer the given subquestions.

Starting at $(x_0, y_0)$, we now turn to momentum based gradient descent. The momentum updates are given below. Starting with $u_{-1} = (0,0)$ and $\eta = 0.2$, we have:

$\begin{aligned} u_t &= \beta u_{t-1} + \nabla f(x, y)\Big|_{(x_t, y_t)} \\ (x_{t+1}, y_{t+1}) &= (x_t, y_t) - \eta u_t \end{aligned}$

What should be the momentum parameter $\beta$ so that running two steps of momentum based gradient descent takes us to the global minimum? Note that the sequence of iterates will be: $(x_0, y_0) \to (x_1, y_1) \to (x_2, y_2)$, where $(x_2, y_2)$ is the global minimum of $f$. Enter your answer correct to two places after the decimal.