Gradient Compression

L-GreCo: Layerwise-Adaptive Gradient Compression for Efficient and Accurate Deep Learning featured image

L-GreCo: Layerwise-Adaptive Gradient Compression for Efficient and Accurate Deep Learning

A layerwise-adaptive gradient compression framework that formulates per-layer compression parameter selection as a constrained optimization and solves it via dynamic programming, …

avatar
Jiaqi Li
•

Distributed Machine Learning & System Optimization

Communication optimization, heterogeneous GPU scheduling, and large-scale training acceleration for distributed deep learning.