LLM Compression & Efficient Inference

Wed, 01 Jan 2025 00:00:00 +0000

Overview

We systematically research model compression and efficient inference techniques for resource-efficient deployment of Large Language Models (LLMs), with a focus on adaptation and optimization for domestic high-performance accelerators (e.g., Hygon DCU), providing domestic support for large-scale AI applications.

Research Directions

Post-Training Quantization (PTQ): Sensitivity-guided weight quantization methods that maintain model accuracy at low bit-widths
Sparse Gradient Communication: Low-bit sparse gradient compression algorithms with convergence guarantees for distributed training acceleration
Parameter-Efficient Fine-Tuning: PEFT strategies to reduce the cost of adapting large models
Domestic Chip Adaptation: Operator optimization and compute-power fusion for domestic accelerators such as Hygon DCU
Inference Acceleration Frameworks: Domain-Specific Language (DSL) design and compiler optimization

Key Achievements

Proposed sensitivity-guided post-training quantization method achieving excellent low-bit quantization results on mainstream LLMs
Proposed low-bit sparse gradient communication algorithm (ICPP 2026) with rigorous convergence guarantees
Proposed SelCo selective co-location multimodal training framework, significantly improving heterogeneous cluster training efficiency
Leading major projects on “Key Technologies for LLM Training Optimization on Ultra-Large-Scale Domestic Computing Clusters”
Conducting dedicated research on Hygon DCU model training optimization

LLM | Advanced Computing and Architecture

LLM Compression & Efficient Inference

Overview

Research Directions

Key Achievements