Question 16
Compared with a basic recurrent update, the gating mechanisms in LSTMs and GRUs are primarily intended to:
Remove nonlinearities from the recurrent computation
Regulate information retention and update across timesteps
Make the model invariant to permutation of the sequence
Replace recurrent state with a fixed convolution kernel