Question 7
Consider a scenario where you are fine-tuning a large language model using LoRA (Low-Rank Adaptation) and QLoRA (Quantized LoRA) techniques. Each technique has specific ways of reducing the model's memory and computational requirements. Select all statements that are correct about LoRA ans QLoRa.
LoRA achieves computational efficiency by performing inference with only a subset of the model's original parameters, completely ignoring any layers that are not low-rank.
QLoRA enables the model to dynamically switch between full and low-rank matrices during inference, improving accuracy while saving memory.
QLoRA applies quantization to the low-rank matrices introduced by LoRA, combining memory efficiency through quantization with a reduced number of trainable parameters from low-rank adaptation.
LoRA reduces the number of parameters by introducing low-rank matrices to approximate the weight updates, which minimizes the number of parameters that need to be stored and trained.