Question 10
A key reason CNNs are parameter-efficient compared to fully connected nets for images is:
Weight sharing and local receptive fields
They never use nonlinearities thus easier to train
They always use pooling
Compared to vanilla neural network and RNN, they are robust to gradient explosion and vanishing gradient.