Quiz Space

Deep Learning · End Term · 1 Sept 2024 · May 2024 term

Question 23: In a time series prediction task using a GRU (Gated Recu…

Question 23

+2 marksNumerical answer

In a time series prediction task using a GRU (Gated Recurrent Unit) network, the GRU processes input sequences where each input is represented by a 2-dimensional vector (xt∈R2x_t \in \mathbb{R}^2). The GRU uses the following formulas for the hidden state and output at time step tt:

it=σ(Wist−1+Uixt+bi)ot=σ(Wost−1+Uoxt+bo)s~t=tanh⁡(Uxt+W(ot⊙st−1)+b)st=(1−it)⊙st−1+it⊙s~ty^t=Vht+c\begin{aligned} i_t &= \sigma(W_i s_{t-1} + U_i x_t + b_i) \\ o_t &= \sigma(W_o s_{t-1} + U_o x_t + b_o) \\ \tilde{s}_t &= \tanh(Ux_t + W(o_t \odot s_{t-1}) + b) \\ s_t &= (1 - i_t) \odot s_{t-1} + i_t \odot \tilde{s}_t \\ \hat{y}_t &= Vh_t + c \end{aligned}

where ⊙\odot denotes element-wise multiplication. Assume that ht∈R3h_t \in \mathbb{R}^3 and y^t∈R2\hat{y}_t \in \mathbb{R}^2.

Based on the above data, answer the given subquestions.

Given that the GRU processes sequences of length 6 (T = 6), what is the total number of parameters (including biases) in the network?

Show answer

Correct answer: 113

See the whole paperTake it as a mock test

Question 23 of 25 in the IIT Madras BS Deep Learning (Deep Learning) End Term paper sat on 1 Sept 2024, in the May 2024 term (IIT M DEGREE AN EXAM QDB3 01 Sep 2024). It carries 2 marks.

More questions from this paper

  1. Q1Figure question
  2. Q2Consider a dataset with 150 samples and a batch size of 15. If each minibatch iteration contributes an average loss of …
  3. Q3If you use hierarchical softmax with a binary tree where each leaf node represents a word in the vocabulary, and the vo…
  4. Q4In a Skip-gram model with a vocabulary size V = 100, an embedding dimension D = 10, and a window size of 3 (on each sid…
  5. Q5Figure question
  6. Q6Consider a Transformer model with the following specifications for the decoder part: Input dimension (embedding size): …
  7. Q7Consider a feedforward neural network with the following structure: One input layer with 2 nodes One hidden layer with …
  8. Q8A neural network has the following structure: Input Layer: \mathbf{h_0} = \mathbf{x}, where \mathbf{x} \in \mathbb{R}^{…
  9. Q9Given an input array X and a kernel/filter K as follows: X = \begin{bmatrix} -1 & -1 & 0 & 2 \ -2 & 1 & 0 & 0 \ 1 & -1 …
  10. Q10What is the derivative of the ReLU activation function at x = 10?
  11. Q11In terms of convergence speed, which gradient descent method can show the most rapid progress initially but may suffer …
  12. Q12How does the use of early stopping in training a neural network affect the model’s performance on unseen data?
  13. Q13Which of the following statements is/are not true with respect to a dropout rate of 0.2?
  14. Q14In the context of unsupervised pretraining of artificial neural networks, which of the following statements accurately …
  15. Q15What are the maximum values of the derivatives of sigmoid and tanh?
  16. Q16Figure question
  17. Q17Given a matrix A with dimensions p × q, which of the following statements is NOT true regarding the rank-k approximatio…
  18. Q18Consider an encoder-decoder model trained with a batch size of 64. Each input sequence has a length of 12 tokens, and e…
  19. Q19In an encoder-decoder model, what is the significance of the context vector?
  20. Q20Given the attention weights αt,1 = 0.3, αt,2 = 0.4, αt,3 = 0.3 and the corresponding encoder hidden states h1 = [2, 1, …
  21. Q21In the Transformer model, what is the purpose of the multi-head attention mechanism?
  22. Q22In the context of the Transformer model’s encoder-decoder architecture, which of the following statements are correct?
  23. Q24In a time series prediction task using a GRU (Gated Recurrent Unit) network, the GRU processes input sequences where ea…
  24. Q25In a time series prediction task using a GRU (Gated Recurrent Unit) network, the GRU processes input sequences where ea…