Exploring Distributed Machine Learning System on Raspberry Pi Computer Cluster

Abstract

This project explored the use of Distributed Machine Learning (DML) as a potential tool in training times of Machine Learning (ML) models in lower-end computer cluster, to provide alternatives for students and scientists when implementing their ML environment without expensive/performant hardware. As part of this, an ML training environment was developed and deployed using container technology on a 4-node raspberry pi computer cluster. This cluster was used to train ML classifier models over the popular CIFAR10 dataset. Test cases were set up to analyze how the training times for models were affected when adding and removing nodes from the system and varying the number of processor cores allotted to the system. Data was recorded for each test, such as the test’s execution time, average CPU time spent when building the model, overhead, and model accuracy, among others. When analyzing this data, it was found that they were practical limits to the speedup on training times achievable when using DML for the cluster, with diminishing returns on speedup values when adding additional nodes. Meanwhile, the speedup observed when increasing processing power for the cluster displayed no such limitations, showing that DML can be used to improve training times for lower-end devices but in a limited capacity.

Key Terms: Containers, Distributed Machine Learning, Docker, Limitations Raspberry Pi.