Question 13
A four-dimensional input vector x = [5, 3,−1, 2] is passed to a hidden layer with a single neuron and an activation function a(.) to obtain z. Assume that the corresponding weights (from input to hidden layer) are [0.4, 0.6, 0.8,−0.2] and bias is 0.4. If the activation function a(.) is Sigmoid, Linear, Indicator function, Softplus, ReLU and Leaky-ReLU, what are the corresponding values of z? (Note that Indicator function returns 1 for a positive input, and 0 otherwise. Softplus is defined as: ln(1 + exp(z)).
0.95, 3, 1, 0.35, 3, 3
0.95, 3, 0, 3.05, 3, 3
0.92, 3, 1, 3.5, 3, 3
0.95, 3, 1, 3.05, 3, 3