Question 7
Consider a hidden layer with neurons, all initialized with identical weight vectors and identical bias . The activation function is sigmoid. Which of the following are TRUE?
After one gradient descent step, the neurons will have different weights.
For any input , all neurons will produce the same output.
During backpropagation, the gradients of all neurons are always zero because their weights are identical.
This layer is functionally equivalent to having a single neuron, because all neurons compute the same function.