Deep Learning End Term: 28 April 2024, Set QDB3 (January 2024 term)

Consider the MP neuron model and its applicability to representing boolean functions. Select the correct statements:

How many sigmoid neurons do we require to construct a tower function using single hidden layer to approximate a 2 dimensional continuous function ?

Consider a feedforward neural network with one hidden layer trained using backpropagation for a binary classification task. The network has the following architecture:

• Input layer with 15 neurons
• Hidden layer with 25 neurons
• Output layer with 1 neuron

During the backpropagation process, the derivative of the sigmoid activation function $\sigma(z)$ with respect to its argument $z$ is given by:

$\sigma'(z) = \sigma(z) \cdot (1 - \sigma(z))$

If the loss function used for binary classification is the binary cross-entropy loss, and the activation fuction at hidden layer and output layer is sigmoid. The output of the neural network is denoted as $\hat{y}$, and the true label is denoted as $y$, what is the expression for $\frac{\partial L}{\partial w_j}$, where $w_j$ represents the weights connecting the $j$th neuron of hidden layer to the output layer? Assume that the output of $j$th neuron of hidden layer is $h_j$ and no biases in the network.