Question 4
Which technique most directly helps reduce catastrophic forgetting early in fine-tuning?
Use very high weight decay on the backbone
Use differential learning rates (small LR for backbone, larger LR for head)
Replace ReLU with sigmoid
Remove skip connections