Deep Learning End Term: 30 April 2023 (January 2023 term)

Suppose that we implement a three input Boolean function using the Mc-Culloch Pitts (MP) neuron. The graph below shows the Number of Correctly Classified (NCC) data points for various values of threshold θ. The threshold θ is incremented by 1 from 0 to 5. Assume that the neuron does not have any inhibitory input. This graph represents which of the following Boolean functions?

Suppose that we implement a three input Boolean function using the Mc-Culloch Pitts (MP) neuron. The graph below shows the Number of Correctly Classified (NCC) data points for various values of threshold θ. The threshold θ is incremented by 1 from 0 to 5. Assume that the neuron does not have any inhibitory input. This graph represents which of the following Boolean functions?

Consider the following two sentences
• A man was sitting at the bank of the river and gazing at stars in the sky • A man went to the bank to check his current balance
Suppose we get the word representation for the word bank in both sentences using CBOW model which was trained as shown in the image below. The model was trained by building a vocabulary that contains unique words in the sentences. Then the statement that the word representation for the word bank will be different based on its context is

Consider the following two sentences
• A man was sitting at the bank of the river and gazing at stars in the sky • A man went to the bank to check his current balance
Suppose we get the word representation for the word bank in both sentences using CBOW model which was trained as shown in the image below. The model was trained by building a vocabulary that contains unique words in the sentences. Then the statement that the word representation for the word bank will be different based on its context is

Consider a skip-gram model shown below. Each word in the vocabulary is represented as one-hot vector of size 100×1. The embedding dimension h is 5×1. Enter the number of parameters (exclude bias) in the entire network

Consider a skip-gram model shown below. Each word in the vocabulary is represented as one-hot vector of size 100×1. The embedding dimension h is 5×1. Enter the number of parameters (exclude bias) in the entire network

Based on the above data, answer the given subquestions.

Based on the above data, answer the given subquestions.

Suppose that all the elements in the input vector are zero and the corresponding true label is also 0. Further, suppose that all the parameters are initialized to zero. What is the loss value if cross entropy loss is used? Use natural logarithm ln.

Based on the above data, answer the given subquestions.

Assume that we use backpropagation to compute the gradient of the loss w.r.t. all the parameters. Then the statement that the gradients w.r.t. all the parameters are zero is

The plot below shows contours of a function f(w, b). Choose the correct statements

The plot below shows contours of a function f(w, b). Choose the correct statements

Suppose that a model produces zero training error without using any regularization technique. What happens if we use L2 regularization and retrain the model, in general?

Suppose we divide the available training samples into mini batches of size 32 to train a model with mini-batch gradient descent. Assume that we have 33 different machines to train the model. One out of 33 machines acts as a master machine. The actual weight update happens in the master machine. The master machine can send one sample for the rest of the machines along with a copy of the model in its current state to compute the gradients. We call this entire set-up parallelization. Which of the following deep learning architectures can be trained in parallel then?

A text corpus contained the following two sentences.
• In science if you know what you are doing you should not be doing it • In engineering if you do not know what you are doing you should not be doing it
Based on the above data, answer the given subquestions.

A text corpus contained the following two sentences.
• In science if you know what you are doing you should not be doing it • In engineering if you do not know what you are doing you should not be doing it
Based on the above data, answer the given subquestions.

Suppose we build a co-occurrence matrix of size m × n, where each row corresponds to a word in the vocabulary and the columns corresponds to the context of the word. Which of the following could be a valid size of the co-occurrence matrix (select all correct answers)?

Consider a simple RNN for a binary sequence classification problem.

Consider a simple RNN for a binary sequence classification problem.
Based on the above data, answer the given subquestions.

What is the loss value? Use cross entropy loss with natural logarithm.

Consider a simple RNN for a binary sequence classification problem.

Consider a simple RNN for a binary sequence classification problem.
Based on the above data, answer the given subquestions.

Consider the statement “the attention coefficients in RNN based Encoder- Decoder architecture are computed for each time step of the decoder”. The statement is