Search

Browse

• Communities
• Authors
• Year
• Titles
• Subjects

My Space

• Login

Distributed machine learning algorithms via geometric monitoring

Konidaris Vissarion-Bertcholnt

Full record

---

Summary

Contemporary deep neural network architectures trained on massive datasets can provenly achieve state-of-the-art performance across a wide variety of domains, from image and speech recognition, to text processing, recommendation systems and fraud detection. With the explosion in the amount of data generated online entering its next phase, we are able to train bigger and deeper neural nets which can dramatically increase performance but also training time. What is more, most of the data is generated or received on different remote machines and its massive nature implies prohibitive communication costs if all data is to be collected at a single site. In view of these problems, much effort has been dedicated the past few years into parallelizing the training procedure of such complex models. We introduce a novel method for scaling up distributed training of deep neural networks using the Functional Geometric Monitoring (FGM) communication protocol, a well studied technique that is used to monitor complex continuous queries on high-volume, rapid distributed streams. This protocol is suitable for classic learning with stationary environment properties, as well as non-stationary ones with concept drift. Our goal is to minimize the prediction loss and network communication at the same time. We demonstrate empirically that the protocol achieves up to 95% less network communication than todays' cutting edge methods, while achieving high predictive performance.

Available Files

Services