Question 7
When fine-tuning a model, what is a common way to prevent the new training from "overwriting" or destroying the useful features already learned during pre-training?
Applying a high dropout rate exclusively to the newly initialized classification head
Use differential learning rates (small LR for the pre-trained backbone, larger LR for the new head)
Re-initializing the weights of the last convolutional block before training
Freezing the classification head and only fine-tuning the backbone features