Parameter-Efficient Fine-Tuning

IGU-LoRA: Adaptive Rank Allocation via Integrated Gradients and Uncertainty-Aware Scoring featured image
Group Meeting

IGU-LoRA: Adaptive Rank Allocation via Integrated Gradients and Uncertainty-Aware Scoring

An adaptive rank allocation method for LoRA that replaces instantaneous gradient sensitivity with integrated gradients to mitigate gradient saturation, and builds an …

avatar
Yuliang Yang
•
Read more
L2-LoRA: Improving Low-Rank Adaptation with Layer-Specific Regularization featured image
Group Meeting

L2-LoRA: Improving Low-Rank Adaptation with Layer-Specific Regularization

A layer-aware LoRA variant that assigns per-layer L2 regularization strength based on the layerwise knowledge distribution measured by normalized average indirect effect (NAIE) …

avatar
Yuliang Yang
•
Read more