Pruning

DLP: Dynamic Layerwise Pruning in Large Language Models featured image

DLP: Dynamic Layerwise Pruning in Large Language Models

A dynamic layerwise pruning method for LLMs that measures layer unimportance via median of weight-activation scores and derives a Relative Importance Distribution to allocate …

肖洋
•
LLM-Streamline: Streamlining Redundant Layers to Compress Large Language Models featured image

LLM-Streamline: Streamlining Redundant Layers to Compress Large Language Models

A layer pruning and lightweight replacement method for LLMs that uses cosine similarity between input and output hidden states to locate consecutive redundant layers and trains a …

肖洋
•