Quiz Space

Deep Learning · Quiz 2 · 16 Mar 2025 · January 2025 term

Question 14: Consider a loss function, L(w) = w^3 + 2w, where w repre…

Question 14

+2 marksNumerical answer

Consider a loss function, L(w)=w3+2wL(w) = w^3 + 2w, where ww represents the model parameter. The AdaGrad optimizer is used to minimize this loss. The learning rate η\eta is set to 0.50.5, and at the initial iteration (t=0t = 0), the parameter ww has a starting value of w0=2w_0 = 2.

AdaGrad Update Rule:

vt=vt−1+(∇wt)2wt+1=wt−ηvt+ϵ∗∇wtv−1=0use ϵ=0\begin{aligned} v_t &= v_{t-1} + (\nabla w_t)^2 \\ w_{t+1} &= w_t - \frac{\eta}{\sqrt{v_t + \epsilon}} * \nabla w_t \\ v_{-1} &= 0 \\ \text{use } \epsilon &= 0 \end{aligned}

Enter your answer correct up to two decimal places for all sub-questions. For intermediate calculations, use three decimal places so that your final answer falls within the expected range.

Based on the above data, answer the given subquestions.

Show answer

Correct answer: 1.23 (accepted within ±0.15)

See the whole paperTake it as a mock test

Question 14 of 23 in the IIT Madras BS Deep Learning (Deep Learning) Quiz 2 paper sat on 16 Mar 2025, in the January 2025 term (IIT M DEGREE AN EXAM QDB2 16 Mar 2025). It carries 2 marks.

More questions from this paper

  1. Q1What is the impact of using a sigmoid activation function on the gradients when the input to the neuron is a very large…
  2. Q2Why is bias correction needed in the Adam optimizer?
  3. Q3How do skip connections in ResNet work?
  4. Q4What is the primary purpose of L2 regularization in machine learning models?
  5. Q5What is the soft target used when injecting noise at the output?
  6. Q6Consider a neural network architecture for a regression task. Three distinct subsets of the training dataset are used t…
  7. Q7The weighted moving average (WMA) is used to smooth data by giving more importance to recent values. Suppose we compute…
  8. Q8Given an input array X and a kernel/filter K as follows: X = \begin{bmatrix} 1 & 0 & -2 & 1 \ 1 & 2 & 0 & 1 \ -1 & -1 &…
  9. Q9Given an input array X and a kernel/filter K as follows: X = \begin{bmatrix} 1 & 0 & -2 & 1 \ 1 & 2 & 0 & 1 \ -1 & -1 &…
  10. Q10Given an input array X and a kernel/filter K as follows: X = \begin{bmatrix} 1 & 0 & -2 & 1 \ 1 & 2 & 0 & 1 \ -1 & -1 &…
  11. Q11Consider a loss function, L(w) = w^3 + 2w, where w represents the model parameter. The AdaGrad optimizer is used to min…
  12. Q12Consider a loss function, L(w) = w^3 + 2w, where w represents the model parameter. The AdaGrad optimizer is used to min…
  13. Q13Consider a loss function, L(w) = w^3 + 2w, where w represents the model parameter. The AdaGrad optimizer is used to min…
  14. Q15Based on the above data, answer the given subquestions.
  15. Q16Based on the above data, answer the given subquestions.
  16. Q17Based on the above data, answer the given subquestions.
  17. Q18Based on the above data, answer the given subquestions.
  18. Q19Based on the above data, answer the given subquestions.
  19. Q20Based on the above data, answer the given subquestions.
  20. Q21Based on the above data, answer the given subquestions.
  21. Q22Based on the above data, answer the given subquestions.
  22. Q23Based on the above data, answer the given subquestions.