1 Decentralized Jointly Sparse Optimization by Reweighted Lq Minimization Qing Ling Department of Automation University of Science and Technology of China.

Slides:



Advertisements
Similar presentations
Accelerated, Parallel and PROXimal coordinate descent IPAM February 2014 APPROX Peter Richtárik (Joint work with Olivier Fercoq - arXiv: )
Advertisements

A Hierarchical Multiple Target Tracking Algorithm for Sensor Networks Songhwai Oh and Shankar Sastry EECS, Berkeley Nest Retreat, Jan
1 Closed-Form MSE Performance of the Distributed LMS Algorithm Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE Department, University of.
Wideband Spectrum Sensing Using Compressive Sensing.
Kick-off Meeting, July 28, 2008 ONR MURI: NexGeNetSci Distributed Coordination, Consensus, and Coverage in Networked Dynamic Systems Ali Jadbabaie Electrical.
Structured Sparse Principal Component Analysis Reading Group Presenter: Peng Zhang Cognitive Radio Institute Friday, October 01, 2010 Authors: Rodolphe.
Contents 1. Introduction 2. UWB Signal processing 3. Compressed Sensing Theory 3.1 Sparse representation of signals 3.2 AIC (analog to information converter)
Graph Laplacian Regularization for Large-Scale Semidefinite Programming Kilian Weinberger et al. NIPS 2006 presented by Aggeliki Tsoli.
Wangmeng Zuo, Deyu Meng, Lei Zhang, Xiangchu Feng, David Zhang
Javad Lavaei Department of Electrical Engineering Columbia University Graph-Theoretic Algorithm for Nonlinear Power Optimization Problems.
An Introduction to Sparse Coding, Sparse Sensing, and Optimization Speaker: Wei-Lun Chao Date: Nov. 23, 2011 DISP Lab, Graduate Institute of Communication.
Compressive Oversampling for Robust Data Transmission in Sensor Networks Infocom 2010.
Volkan Cevher, Marco F. Duarte, and Richard G. Baraniuk European Signal Processing Conference 2008.
Compressed Sensing for Networked Information Processing Reza Malek-Madani, 311/ Computational Analysis Don Wagner, 311/ Resource Optimization Tristan Nguyen,
Energy-Efficient Target Coverage in Wireless Sensor Networks Mihaela Cardei, My T. Thai, YingshuLi, WeiliWu Annual Joint Conference of the IEEE Computer.
Location Estimation in Sensor Networks Moshe Mishali.
Peter Richtárik (joint work with Martin Takáč) Distributed Coordinate Descent Method AmpLab All Hands Meeting - Berkeley - October 29, 2013.
Sparsity-Aware Adaptive Algorithms Based on Alternating Optimization and Shrinkage Rodrigo C. de Lamare* + and Raimundo Sampaio-Neto * + Communications.
1 Secure Cooperative MIMO Communications Under Active Compromised Nodes Liang Hong, McKenzie McNeal III, Wei Chen College of Engineering, Technology, and.
Summary Alan S. Willsky SensorWeb MURI Review Meeting September 22, 2003.
1 Hybrid methods for solving large-scale parameter estimation problems Carlos A. Quintero 1 Miguel Argáez 1 Hector Klie 2 Leticia Velázquez 1 Mary Wheeler.
1 Unveiling Anomalies in Large-scale Networks via Sparsity and Low Rank Morteza Mardani, Gonzalo Mateos and Georgios Giannakis ECE Department, University.
Algorithms for Robot-based Network Deployment, Repair, and Coverage Gaurav S. Sukhatme Center for Robotics and Embedded.
Dynamic Clustering for Acoustic Target Tracking in Wireless Sensor Network Wei-Peng Chen, Jennifer C. Hou, Lui Sha.
Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona.
Consensus-based Distributed Estimation in Camera Networks - A. T. Kamal, J. A. Farrell, A. K. Roy-Chowdhury University of California, Riverside
A Framework for Distributed Model Predictive Control
Compressive Sensing Based on Local Regional Data in Wireless Sensor Networks Hao Yang, Liusheng Huang, Hongli Xu, Wei Yang 2012 IEEE Wireless Communications.
Aris Moustakas, University of Athens CROWN Kickoff NKUA Power Control in Random Networks with N. Bambos, P. Mertikopoulos, L. Lampiris.
Tracking with Unreliable Node Sequences Ziguo Zhong, Ting Zhu, Dan Wang and Tian He Computer Science and Engineering, University of Minnesota Infocom 2009.
November 1, 2012 Presented by Marwan M. Alkhweldi Co-authors Natalia A. Schmid and Matthew C. Valenti Distributed Estimation of a Parametric Field Using.
June 21, 2007 Minimum Interference Channel Assignment in Multi-Radio Wireless Mesh Networks Anand Prabhu Subramanian, Himanshu Gupta.
Distributed Tracking Using Kalman Filtering Aaron Dyreson, Faculty Advisor: Ioannis Schizas, Ph.D. Department of Electrical Engineering, The University.
On Energy-Efficient Trap Coverage in Wireless Sensor Networks Junkun Li, Jiming Chen, Shibo He, Tian He, Yu Gu, Youxian Sun Zhejiang University, China.
A Survey of Spectrum Sensing Algorithm for Cognitive Radio Applications YaGun Wu netlab.
Signal Processing & Communication for Smart Dust Networks Haralabos (Babis) Papadopoulos ECE Department Institute for Systems Research University of Maryland,
Distributed State-Estimation Using Quantized Measurement Data from Wireless Sensor Networks Li Chai with Bocheng Hu Professor College of.
Energy-Efficient Signal Processing and Communication Algorithms for Scalable Distributed Fusion.
1 Blind Channel Identification and Equalization in Dense Wireless Sensor Networks with Distributed Transmissions Xiaohua (Edward) Li Department of Electrical.
College of Engineering Anchor Nodes Placement for Effective Passive Localization Karthikeyan Pasupathy Major Advisor: Dr. Robert Akl Department of Computer.
Javad Lavaei Department of Electrical Engineering Columbia University Convex Relaxation for Polynomial Optimization: Application to Power Systems and Decentralized.
Secure In-Network Aggregation for Wireless Sensor Networks
PARALLEL FREQUENCY RADAR VIA COMPRESSIVE SENSING
Dr. Sudharman K. Jayaweera and Amila Kariyapperuma ECE Department University of New Mexico Ankur Sharma Department of ECE Indian Institute of Technology,
The Scaling Law of SNR-Monitoring in Dynamic Wireless Networks Soung Chang Liew Hongyi YaoXiaohang Li.
AUTOMATIC TARGET RECOGNITION AND DATA FUSION March 9 th, 2004 Bala Lakshminarayanan.
University “Ss. Cyril and Methodus” SKOPJE Cluster-based MDS Algorithm for Nodes Localization in Wireless Sensor Networks Ass. Biljana Stojkoska.
MURI Telecon, Update 7/26/2012 Summary, Part I:  Completed: proving and validating numerically optimality conditions for Distributed Optimal Control (DOC)
Multi-area Nonlinear State Estimation using Distributed Semidefinite Programming Hao Zhu October 15, 2012 Acknowledgements: Prof. G.
Network Lasso: Clustering and Optimization in Large Graphs
Rank Minimization for Subspace Tracking from Incomplete Data
Variable Bandwidth Allocation Scheme for Energy Efficient Wireless Sensor Network SeongHwan Cho, Kee-Eung Kim Korea Advanced Institute of Science and Technology.
DISTIN: Distributed Inference and Optimization in WSNs A Message-Passing Perspective SCOM Team
Adaptive Tracking in Distributed Wireless Sensor Networks Lizhi Yang, Chuan Feng, Jerzy W. Rozenblit, Haiyan Qiao The University of Arizona Electrical.
Spectrum Sensing In Cognitive Radio Networks
Resource Allocation in Hospital Networks Based on Green Cognitive Radios 王冉茵
By: Aaron Dyreson Supervising Professor: Dr. Ioannis Schizas
Chance Constrained Robust Energy Efficiency in Cognitive Radio Networks with Channel Uncertainty Yongjun Xu and Xiaohui Zhao College of Communication Engineering,
Zijian Wang, Eyuphan Bulut, and Boleslaw K. Szymanski Center for Pervasive Computing and Networking and Department of Computer Science Rensselaer Polytechnic.
Jianchao Yang, John Wright, Thomas Huang, Yi Ma CVPR 2008 Image Super-Resolution as Sparse Representation of Raw Image Patches.
StingyCD: Safely Avoiding Wasteful Updates in Coordinate Descent
Jeremy Watt and Aggelos Katsaggelos Northwestern University
Zhu Han University of Houston Thanks for Dr. Mingyi Hong’s slides
Multiplicative updates for L1-regularized regression
Computing and Compressive Sensing in Wireless Sensor Networks
Decentralized Network Optimization: Algorithms and Theories
Sudocodes Fast measurement and reconstruction of sparse signals
Signal Processing on Graphs: Performance of Graph Structure Estimation
Information Sciences and Systems Lab
Outline Sparse Reconstruction RIP Condition
Presentation transcript:

1 Decentralized Jointly Sparse Optimization by Reweighted Lq Minimization Qing Ling Department of Automation University of Science and Technology of China Joint work with Zaiwen Wen (SJTU) and Wotao Yin (RICE) 2012/09/05

2 A brief introduction to my research interest optimization and control in networked multi-agent systems autonomous agents - collect data - process data - communicate problem: how to efficiently accomplish in-network optimization and control tasks through collaboration of agents?

3 Large-scale wireless sensor networks: decentralized signal processing, node localization, sensor selection … how to fuse big sensory data? e.g. structural health monitoring how to localize blinds with anchors? blindanchor how to assign sensors to targets? difficulty in data transmission → decentralized optimization without any fusion center

4 Computer/server networks with big data: collaborative data mining new challenges in the big data era - big data is stored in distributed computers/servers - data transmission is prohibited due to bandwidth/privacy/… - computers/servers collaborate to do data mining distributed/decentralized optimization

5 Wireless sensor and actuator networks: with application in large-scale greenhouse control decentralized control system design wireless sensing - temperature - humidity - … wireless actuating - circulating fan - wet curtain - … disadvantages of traditional centralized control - communication burden in collecting distributed sensory data - lack of robustness due to packet-loss, time-delay, …

6 Recent works wireless sensor networks - decentralized signal processing with application in SHM - decentralized node localization using SDP and SOCP - decentralized sensor node selection for target tracking collaborative data mining - decentralized approaches to jointly sparse signal recovery - decentralized approaches to matrix completion wireless sensor and actuator networks - modeling, hardware design, controller design, prototype theoretical issues - convergence and convergence rate analysis

7 Decentralized Jointly Sparse Optimization by Reweighted Lq Minimization Qing Ling Department of Automation University of Science and Technology of China Joint work with Zaiwen Wen (SJTU) and Wotao Yin (RICE) 2012/09/05

8 Outline  Background  decentralized jointly sparse optimization with applications  Roadmap  nonconvex versus convex, difficulty in decentralized computing  Algorithm development  successive linearization, inexact average consensus  Simulation and conclusion

9 Background (I): jointly sparse optimization Structured signals A sparse signal: only few elements are nonzero Jointly sparse signals: sparse, with the same nonzero supports Jointly sparse optimization: to recover X from linear measurements nonzeros zeros measurement matrixmeasurement noise

10 Background (II): decentralized jointly sparse optimization Decentralized computing in a network Distributed data in distributed agents & no fusion center Consideration of privacy, difficulty in data collection, etc Goal: agent i has y (i) and A (i), to recover x (i) through collaboration Decentralized jointly sparse optimization

11 Background (III): applications Cooperative spectrum sensing [1][2] Cognitive radios sense jointly sparse spectra {x (i) } Measure from time domain [1] or frequency selective filter [2] Decentralized recovery from {y (i) =A (i) x (i) } [1] F. Zeng, C. Li, and Z. Tian, “Distributed compressive spectrum sensing in cooperative multi-hop wideband cognitive networks,” IEEE Journal of Selected Topics in Signal Processing, vol. 5, pp. 37–48, 2011 [2] J. Meng, W. Yin, H. Li, E. Houssain, and Z. Han, “Collaborative spectrum sensing from sparse observations for cognitive radio networks,” IEEE Journal on Selected Areas on Communications, vol. 29, pp. 327–337, 2011 [3] N. Nguyen, N. Nasrabadi, and T. Tran, “Robust multi-sensor classification via joint sparse representation,” submitted to Journal of Advance in Information Fusion Decentralized event detection [3] Sensors {i} sense few targets represented by jointly sparse {x (i) } Decentralized recovery from {y (i) =A (i) x (i) } Collaborative data mining, distributed human action recognition, etc

12 Roadmap (I): nonconvex versus convex Convex model: group lasso or L 21 norm minimization Nonconvex versus convex Convex: with global convergence guarantee Nonconvex: often with better recovery performance Look back on nonconvex models to recover a single sparse signal Reweighted L 1 /L 2 norm minimization [4][5] Reweighted algorithms for jointly sparse optimization? [4] E. Candes, M. Wakin, and S. Boyd, “Enhancing sparsity by reweighted L1 minimization,” Journal of Fourier Analysis and Applications, vol. 14, pp. 877–905, 2008 [5] R. Chartrand and W. Yin, “Iteratively reweighted algorithms for compressive sensing,” In: Proceedings of ICASSP, 2008 regularization parameter

13 Roadmap (II): difficulty in decentralized computing A popular decentralized computing technique: consensus common optimization variableobjective function in agent i local copy in agent ineighboring copies are equal Obviously, two problems are equivalent for a connected network Efficient algorithms (ADM, SGD, etc) for if it is convex [6] Nothing for consensus in jointly sparse optimization! Signals are different; common supports bring nonconvexity [6] D. Bertsekas and J. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, Second Edition, Athena Scientific, 1997

14 Roadmap (III): solution overview Nonconvex model + convex decentralized computing subproblem Nonconvex model -> successive linearization -> reweighted Lq Natural decentralized computing, one nontrivial subproblem Inexactly solving the subproblem still leads to good recovery

15 Algorithm (I): successive linearization Nonconvex model (q=1 or 2) regularization parameter smoothing parameter “Successive linearization” to the joint sparsity term at t Actually a majorization minimization approach

16 Algorithm (II): reweighted algorithm Centralized reweighted Lq minimization algorithm Updating weight vector weight vector u=[u 1 ; u 2 ; u N ] Updating signals From a decentralized implementation perspective … Natural decentralized computing in x-update One subproblem needs decentralized solution in u-update

17 Algorithm (III): average consensus Check u-update: average consensus problem Rewrite to more familiar forms

18 Algorithm (IV): inexact average consensus Solve the average consensus problem with ADM (time t, slot s/S) Updating weight vectors (local copies) Updating Lagrange multipliers (c is a positive constant) Exact average consensus versus inexact average consensus Exact average consensus: exact implementation of reweighted Lq Introducing inner loops: cost of coordination & communication Inexact average consensus: one iteration in the inner loop

19 Algorithm (V): decentralized reweighted Lq Algorithm outline Updating weight vectors (local copies) Updating Lagrange multipliers (c is a positive constant) Updating signals

20 Simulation (I): simulation settings Network settings L=50 agents, randomly deployed in 100×100 area Communication range=30, bidirectionally connected Measurement settings Signal dimension N=20, signal sparsity K=2 Measurement dimension M=10 Random measurement matrices and random measurement noise Parameter settings

21 Simulation (II): recovery performance

22 Simulation (III): convergence rate

23 Conclusion Decentralized jointly sparse optimization problem Jointly sparse signal recovery in a distributed network Reweighted Lq minimization algorithms Feature #1: nonconvex model <- successive linearization Feature #2: decentralized computing <- inexact average consensus Outlook: many open questions in decentralized optimization Good news and bad news Local convergence of the centralized algorithms Excellent performance of the decentralized algorithms No theoretical performance guarantee (recovery and convergence)

24 Thanks for your attention!