Question 7
A team has a powerful pre-trained language model (e.g., a GPT-3 class model). They want to adapt it for a new summarization task but have very limited labeled summarization data and limited compute for full fine-tuning. Which of the following strategies could be viable and effective? (Select ALL that apply)
Zero-shot prompting by providing the text and an instruction like "Summarize this:".
Few-shot prompting (in-context learning) by providing a few examples of text and their summaries in the prompt before the target text.
Full supervised fine-tuning of all model parameters on the small labeled dataset.
Using a Parameter-Efficient Fine-Tuning (PEFT) method like LoRA on the small labeled dataset.
Collecting a much larger unlabeled corpus related to the summarization domain and continuing pre-training.