Question 10
Consider the concept of Zero-Shot Transfer as popularized by GPT-2. Why might this be preferred over Supervised Fine-Tuning?
It allows the model to handle tasks for which no labeled training data is available.
It always achieves higher accuracy than a fine-tuned SOTA model.
It avoids the need to store a separate specialized model (checkpoint) for every downstream task.
It mimics the human ability to perform tasks based on instructions without needing thousands of examples.