Asynchronous Distributed ADMM for Consensus Optimization Ruiliang Zhang James T. Kwok Department of Computer Science and Engineering, Hong Kong University.

Slides:

Advertisements

Similar presentations

Utility Optimization for Event-Driven Distributed Infrastructures Cristian Lumezanu University of Maryland, College Park Sumeer BholaMark Astley IBM T.J.

Advertisements

© 2014 Synopsys. All rights reserved.1 Wheres my glass slipper? TAU 2014 Nanda Gopal Director R&D, Characterization.

Maximum Flow and Minimum Cut Problems In this handout: Duality theory Upper bounds for maximum flow value Minimum Cut Problem Relationship between Maximum.

Simultaneous Routing and Resource Allocation in Wireless Networks Mikael Johansson Signals, Sensors and Systems, KTH Joint work with Lin Xiao and Stephen.

Electrical and Computer Engineering Mississippi State University

Tighter and Convex Maximum Margin Clustering Yu-Feng Li (LAMDA, Nanjing University, China) Ivor W. Tsang.

Computer Science 425 Distributed Systems CS 425 / ECE 428 Consensus

by Rianto Adhy Sasongko Supervisor: Dr.J.C.Allwright

1 EL736 Communications Networks II: Design and Algorithms Class8: Networks with Shortest-Path Routing Yong Liu 10/31/2007.

Separating Hyperplanes

Javad Lavaei Department of Electrical Engineering Columbia University Various Techniques for Nonlinear Energy-Related Optimizations.

Javad Lavaei Department of Electrical Engineering Columbia University Graph-Theoretic Algorithm for Nonlinear Power Optimization Problems.

Bayesian Nonparametric Matrix Factorization for Recorded Music Reading Group Presenter: Shujie Hou Cognitive Radio Institute Friday, October 15, 2010 Authors:

TSF: Trajectory-based Statistical Forwarding for Infrastructure-to-Vehicle Data Delivery in Vehicular Networks Jaehoon Jeong, Shuo Guo, Yu Gu, Tian He,

Department of Electrical and Computer Engineering Multi-Block ADMM for Big Data Optimization in Smart Grid Department of Electrical and Computer Engineering.

Transformation Methods MOM (Method of Multipliers) Study of Engineering Optimization Guanyao Huang RUbiNet - Robust and Ubiquitous Networking Research.

1 An Asymptotically Optimal Algorithm for the Max k-Armed Bandit Problem Matthew Streeter & Stephen Smith Carnegie Mellon University NESCAI, April

Economics 214 Lecture 37 Constrained Optimization.

Flow Network Models for Sub-Sentential Alignment Ying Zhang (Joy) Advisor: Ralf Brown Dec 18 th, 2001.

Constrained Optimization Rong Jin. Outline  Equality constraints  Inequality constraints  Linear Programming  Quadratic Programming.

EE 685 presentation Optimization Flow Control, I: Basic Algorithm and Convergence By Steven Low and David Lapsley Asynchronous Distributed Algorithm Proof.

Scalable Information-Driven Sensor Querying and Routing for ad hoc Heterogeneous Sensor Networks Maurice Chu, Horst Haussecker and Feng Zhao Xerox Palo.

ISM 206 Lecture 4 Duality and Sensitivity Analysis.

A Multi-Agent Learning Approach to Online Distributed Resource Allocation Chongjie Zhang Victor Lesser Prashant Shenoy Computer Science Department University.

Constrained Optimization Rong Jin. Outline  Equality constraints  Inequality constraints  Linear Programming  Quadratic Programming.

Low-Rank Solution of Convex Relaxation for Optimal Power Flow Problem

Javad Lavaei Department of Electrical Engineering Columbia University Joint work with Somayeh Sojoudi Convexification of Optimal Power Flow Problem by.

Distributed Constraint Optimization Michal Jakob Agent Technology Center, Dept. of Computer Science and Engineering, FEE, Czech Technical University A4M33MAS.

Distributed Asynchronous Bellman-Ford Algorithm

Column Generation Approach for Operating Rooms Planning Mehdi LAMIRI, Xiaolan XIE and ZHANG Shuguang Industrial Engineering and Computer Sciences Division.

Fault Tolerance via the State Machine Replication Approach Favian Contreras.

ZONG Wen Department of Computer Science and Engineering The Chinese University of Hong Kong

Department of Information Engineering The Chinese University of Hong Kong A Framework for Monitoring and Measuring a Large-Scale Distributed System in.

Optimal Selection of Power Saving Classes in IEEE e Lei Kong, Danny H.K. Tsang Department of Electronic and Computer Engineering Hong Kong University.

CS Statistical Machine learning Lecture 18 Yuan (Alan) Qi Purdue CS Oct

Association Rule Mining in Peer-to-Peer Systems Ran Wolff Assaf Shcuster Department of Computer Science Technion I.I.T. Haifa 32000,Isreal.

Center for Reliable Engineering Computing (REC) We handle computations with care Founded 2000.

1 Chapter 10 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 Synchronization Algorithms and Concurrent Programming Synchronization.

Machine Learning Weak 4 Lecture 2. Hand in Data It is online Only around 6000 images!!! Deadline is one week. Next Thursday lecture will be only one hour.

Javad Lavaei Department of Electrical Engineering Columbia University Joint work with Somayeh Sojoudi and Ramtin Madani An Efficient Computational Method.

Javad Lavaei Department of Electrical Engineering Columbia University Convex Relaxation for Polynomial Optimization: Application to Power Systems and Decentralized.

EE 685 presentation Optimization Flow Control, I: Basic Algorithm and Convergence By Steven Low and David Lapsley.

Statistics 3502/6304 Prof. Eric A. Suess Chapter 4.

Multi-area Nonlinear State Estimation using Distributed Semidefinite Programming Hao Zhu October 15, 2012 Acknowledgements: Prof. G.

Network Lasso: Clustering and Optimization in Large Graphs

Particle Swarm Optimization † Spencer Vogel † This presentation contains cheesy graphics and animations and they will be awesome.

Instructor: Shengyu Zhang 1. Secretary hiring problem 2.

Distributed Spanning Tree Center for Information Security Department of Computer Science University of Tulsa, Tulsa, OK

1 Slides by Yong Liu 1, Deep Medhi 2, and Michał Pióro 3 1 Polytechnic University, New York, USA 2 University of Missouri-Kansas City, USA 3 Warsaw University.

Economics 2301 Lecture 37 Constrained Optimization.

Lecture notes for Stat 231: Pattern Recognition and Machine Learning 1. Stat 231. A.L. Yuille. Fall 2004 Linear Separation and Margins. Non-Separable and.

Mobile Networks and Applications (January 2007) Presented by J.H. Su ( 蘇至浩 ) 2016/3/21 OPLab, IM, NTU 1 Joint Design of Routing and Medium Access Control.

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1 In-Jeong Chung Intelligent Information System lab. Department.

The Chinese University of Hong Kong Learning Larger Margin Machine Locally and Globally Dept. of Computer Science and Engineering The Chinese University.

Seminar Announcement December 24, Saturday, 15:00-17:00, Room: A302, WNLO Title: Quality-of-Experience (QoE) and Power Efficiency Tradeoff for Fog Computing.

Zhu Han University of Houston Thanks for Dr. Mingyi Hong’s slides

Multiplicative updates for L1-regularized regression

Ph.D. Research Defense Hung Khanh Nguyen Advisor: Dr. Zhu Han

Wenkun Zhang, Hanming Zhang, Lei Li, Jinlai Ye, Bin Yan, Linyuan Wang

Decentralized Network Optimization: Algorithms and Theories

Principles of GIS Fundamental database concepts Shaowen Wang

Expectation-Maximization

Towards Next Generation Panel at SAINT 2002

Chapter 5. Sensitivity Analysis

The Lagrange Multiplier Method

CSC4005 – Distributed and Parallel Computing

A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models Jeff A. Bilmes International.

A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models Jeff A. Bilmes International.

Presentation transcript:

Asynchronous Distributed ADMM for Consensus Optimization Ruiliang Zhang James T. Kwok Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong

The Alternating Direction Method of Multipliers (ADMM) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein

Dual Ascent (1/2)

Dual Ascent (2/2) If strong duality holds,

Large sum-separable objectives, block-wise constraints

Dual ascent for scalable statistical learning

Augmented Lagrangian (for L1 penalizations)

ADMM

ADMM with asynchronous updates Asynchronous Distributed ADMM for Consensus Optimization, Ruiliang Zhang, James T. Kwok

Why do we care about asynchronous algorithms? Stragglers are very common in data centers Assume we have N machines – Only S are going to respond on time for the master to proceed with the consensus variable update Three fundamental assumptions in this paper: – Bounded delay (tau) – Identical probability of straggling across slaves – Not all machines will be stragglers

Distributed learning with a consensus variable

Instance of ADMM

Asynchronous algorithm (1/2) Master side:

Asynchronous algorithm (2/2) Slave side:

Convergence is identical to ADMM

Computation times

Communication efficiency

Scalability