1. SU YUXIN JAN 20, 2014 Petuum: An Iterative-Convergent Distributed Machine Learning Framework

2. Outline Introduction Implementation Questions Demo

3. Introduction to Petuum

4. Bulk Synchronous Parallel

5. Asynchronous Parameters read / update at any time

6. Stale Synchronous Parallel

7. Convergence

8. Programming read(table, row, col) inc(table, row, col, value) iteration()

9. Implementation

10. Overview in Logic

11. Overview in the Real

12. Main Components

13. Table

14. ConsistencyController::DoGet()

15. ConsistencyController::iterate()

16. Server::GetRow()

17. Least-Recently-Used(LRU) Strategy

18. Questions

19. Is Lock-Free Possible ? Data exchange in real-time ? next …

20. Is Auto-Rescheduling Possible ? sub-centralized server reduce communication cost

21. Is Auto-Partition Possible ? Run ML algorithms like that in a single thread A Solution for all ML algorithms

22. In-Memory or In-Storage ? Data capacity is greater than memory size. Memory should be a cache for disk storage. Solution for disk storage:  Hadoop  Spark  ….

23. New Schema to Reduce the Upper Bound?

24. STRADS Scheduler Variable Correlations  Auto-Parallelization Dynamic Prioritization  Monitor the contribution of variables to objective function Load-Balancing in Task

25. Demo Switch to my laptop …