Question 7
Adding more hidden layers with identity activations increases the representational capacity beyond that of a single-layer linear model.
Replacing the identity activation with a nonlinear function such as sigmoid enables the network to learn nonlinear decision boundaries.
With identity activations, the network can still learn arbitrary decision boundaries provided enough hidden neurons are used.