Question 3
In the Wav2Vec2 pipeline, a 1-second (16000 samples) audio clip is passed to the 'Wav2Vec2Model' and results in a 'last hidden state' of shape (1, 49, 768). What do the dimensions 49 and 768 represent?
49 = number of possible phonemes; 768 = batch size.
49 = the model’s hidden dimension; 768 = the downsampled sequence length.
49 = the downsampled sequence length (timesteps); 768 = the model’s hidden feature dimension.
49 = number of attention heads; 768 = the vocabulary size.