Question 22
Let query q = [1,2]^T and keys:k1 = [1,0]^T, k2 = [0,1]^T, k3 = [1,1]^T. Scores si = q^T ki. Use softmax over scores. Use exp(1)=2.72, exp(2)=7.39, exp(3)=20.09. What is α3 (the attention weight for k3)? Provide a value in a reasonable range.