Question 10
Consider a fully connected neural network as follows:
•
One input neuron
•
A single hidden layer with two neurons
•
One output neuron.
The network uses the ReLU activation function in both the hidden and output layers. The loss function used is the squared error:
where
is the true target and
is the network output.
Let the initial weights of the network be
.
Based on the above data, answer the given subquestions.
Select the gradient of the loss function with respect to weight
from the following.
Note:
represents the indicator function.