August 16, 2018 DRAFT Efficient and Programmable Distributed Shared Memory Systems for Machine Learning Training Jinliang Wei August 16, 2018 School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Thesis Committee: Garth A. Gibson, Co-chair Eric P. Xing, Co-chair Phillip B. Gibbons Vijay Vasudevan, Google Brain Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy. Copyright © 2018 Jinliang Wei