Distributed Learning Systems with First-Order Methods

Provides a brief introduction to three distributed learning techniques that have recently been developed: lossy communication compression, asynchronous communication, and decentralized communication. These have significant impact on the work in both the system and machine learning and mathematical optimization communities.