LLM Compression & Efficient Inference

Overview

We systematically research model compression and efficient inference techniques for resource-efficient deployment of Large Language Models (LLMs), with a focus on adaptation and optimization for domestic high-performance accelerators (e.g., Hygon DCU), providing domestic support for large-scale AI applications.

Research Directions

• Post-Training Quantization (PTQ): Sensitivity-guided weight quantization methods that maintain model accuracy at low bit-widths
• Sparse Gradient Communication: Low-bit sparse gradient compression algorithms with convergence guarantees for distributed training acceleration
• Parameter-Efficient Fine-Tuning: PEFT strategies to reduce the cost of adapting large models
• Domestic Chip Adaptation: Operator optimization and compute-power fusion for domestic accelerators such as Hygon DCU
• Inference Acceleration Frameworks: Domain-Specific Language (DSL) design and compiler optimization

Key Achievements

• Proposed sensitivity-guided post-training quantization method achieving excellent low-bit quantization results on mainstream LLMs
• Proposed low-bit sparse gradient communication algorithm (ICPP 2026) with rigorous convergence guarantees
• Proposed SelCo selective co-location multimodal training framework, significantly improving heterogeneous cluster training efficiency
• Leading major projects on “Key Technologies for LLM Training Optimization on Ultra-Large-Scale Domestic Computing Clusters”
• Conducting dedicated research on Hygon DCU model training optimization