Scalable Multi-Stage Stochastic Programming Cosmin Petra and Mihai Anitescu Mathematics and Computer Science Division Argonne National Laboratory DOE Applied.

Slides:



Advertisements
Similar presentations
Partitional Algorithms to Detect Complex Clusters
Advertisements

Electrical and Computer Engineering Mississippi State University
MS&E 211 Quadratic Programming Ashish Goel. A simple quadratic program Minimize (x 1 ) 2 Subject to: -x 1 + x 2 ≥ 3 -x 1 – x 2 ≥ -2.
Improving Transmission Asset Utilization through Advanced Mathematics and Computing 1 Henry Huang, Ruisheng Diao, Shuangshuang Jin, Yuri Makarov Pacific.
Graph Laplacian Regularization for Large-Scale Semidefinite Programming Kilian Weinberger et al. NIPS 2006 presented by Aggeliki Tsoli.
Javad Lavaei Department of Electrical Engineering Columbia University Graph-Theoretic Algorithm for Nonlinear Power Optimization Problems.
All Hands Meeting, 2006 Title: Grid Workflow Scheduling in WOSE (Workflow Optimisation Services for e- Science Applications) Authors: Yash Patel, Andrew.
Scalable Stochastic Programming Cosmin Petra and Mihai Anitescu Mathematics and Computer Science Division Argonne National Laboratory Informs Computing.
Applications of Stochastic Programming in the Energy Industry Chonawee Supatgiat Research Group Enron Corp. INFORMS Houston Chapter Meeting August 2, 2001.
Gizem ALAGÖZ. Simulation optimization has received considerable attention from both simulation researchers and practitioners. Both continuous and discrete.
OpenFOAM on a GPU-based Heterogeneous Cluster
Stochastic Optimization of Complex Energy Systems on High-Performance Computers Cosmin G. Petra Mathematics and Computer Science Division Argonne National.
© 2003 Warren B. Powell Slide 1 Approximate Dynamic Programming for High Dimensional Resource Allocation NSF Electric Power workshop November 3, 2003 Warren.
Dual Problem of Linear Program subject to Primal LP Dual LP subject to ※ All duality theorems hold and work perfectly!
Iteration Technique toward SOC EDA Lab, Department of Computer Science and Technology, Tsinghua University
© 2005, it - instituto de telecomunicações. Todos os direitos reservados. Gerhard Maierbacher Scalable Coding Solutions for Wireless Sensor Networks IT.
Preference Analysis Joachim Giesen and Eva Schuberth May 24, 2006.
An Algebraic Multigrid Solver for Analytical Placement With Layout Based Clustering Hongyu Chen, Chung-Kuan Cheng, Andrew B. Kahng, Bo Yao, Zhengyong Zhu.
1 Combinatorial Problems in Cooperative Control: Complexity and Scalability Carla Gomes and Bart Selman Cornell University Muri Meeting March 2002.
On the convergence of SDDP and related algorithms Speaker: Ziming Guan Supervisor: A. B. Philpott Sponsor: Fonterra New Zealand.
Low-Rank Solution of Convex Relaxation for Optimal Power Flow Problem
Higher-order Confidence Intervals for Stochastic Programming using Bootstrapping Cosmin G. Petra Joint work with Mihai Anitescu Mathematics and Computer.
On parallelizing dual decomposition in stochastic integer programming
Scalable Stochastic Programming Cosmin G. Petra Mathematics and Computer Science Division Argonne National Laboratory Joint work with.
Parallel Performance of Hierarchical Multipole Algorithms for Inductance Extraction Ananth Grama, Purdue University Vivek Sarin, Texas A&M University Hemant.
Optimization Under Uncertainty: Structure-Exploiting Algorithms Victor M. Zavala Assistant Computational Mathematician Mathematics and Computer Science.
Scalable Stochastic Programming Cosmin G. Petra Mathematics and Computer Science Division Argonne National Laboratory Joint work with.
Column Generation Approach for Operating Rooms Planning Mehdi LAMIRI, Xiaolan XIE and ZHANG Shuguang Industrial Engineering and Computer Sciences Division.
Optimization for Operation of Power Systems with Performance Guarantee
Scalable Multi-Stage Stochastic Programming
Network Aware Resource Allocation in Distributed Clouds.
1 Using the PETSc Parallel Software library in Developing MPP Software for Calculating Exact Cumulative Reaction Probabilities for Large Systems (M. Minkoff.
Software Pipelining for Stream Programs on Resource Constrained Multi-core Architectures IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEM 2012 Authors:
STDM - Linear Programming 1 By Isuru Manawadu B.Sc in Accounting Sp. (USJP), ACA, AFM
Trust-Aware Optimal Crowdsourcing With Budget Constraint Xiangyang Liu 1, He He 2, and John S. Baras 1 1 Institute for Systems Research and Department.
ParCFD Parallel computation of pollutant dispersion in industrial sites Julien Montagnier Marc Buffat David Guibert.
CFD Lab - Department of Engineering - University of Liverpool Ken Badcock & Mark Woodgate Department of Engineering University of Liverpool Liverpool L69.
Statistical Sampling-Based Parametric Analysis of Power Grids Dr. Peng Li Presented by Xueqian Zhao EE5970 Seminar.
Efficient Integration of Large Stiff Systems of ODEs Using Exponential Integrators M. Tokman, M. Tokman, University of California, Merced 2 hrs 1.5 hrs.
Stochastic optimization of energy systems Cosmin Petra Argonne National Laboratory.
The swiss-carpet preconditioner: a simple parallel preconditioner of Dirichlet-Neumann type A. Quarteroni (Lausanne and Milan) M. Sala (Lausanne) A. Valli.
Javad Lavaei Department of Electrical Engineering Columbia University Joint work with Somayeh Sojoudi and Ramtin Madani An Efficient Computational Method.
Computational Aspects of Multi-scale Modeling Ahmed Sameh, Ananth Grama Computing Research Institute Purdue University.
Presented By Dr. Mohsen Alardhi College of Technological Studies, Kuwait April 19 th,2009.
Javad Lavaei Department of Electrical Engineering Columbia University Convex Relaxation for Polynomial Optimization: Application to Power Systems and Decentralized.
1 Efficient Parallel Software for Large-Scale Semidefinite Programs Makoto Tokyo-Tech Katsuki Chuo University MSC Yokohama.
1 Iterative Integer Programming Formulation for Robust Resource Allocation in Dynamic Real-Time Systems Sethavidh Gertphol and Viktor K. Prasanna University.
On implicit-factorization block preconditioners Sue Dollar 1,2, Nick Gould 3, Wil Schilders 2,4 and Andy Wathen 1 1 Oxford University Computing Laboratory,
Case Study in Computational Science & Engineering - Lecture 5 1 Iterative Solution of Linear Systems Jacobi Method while not converged do { }
October 2008 Integrated Predictive Simulation System for Earthquake and Tsunami Disaster CREST/Japan Science and Technology Agency (JST)
Multi-area Nonlinear State Estimation using Distributed Semidefinite Programming Hao Zhu October 15, 2012 Acknowledgements: Prof. G.
Nonsmooth Optimization for Optimal Power Flow over Transmission Networks GlobalSIP 2015 Authors: Y. Shi, H. D. Tuan, S. W. Su and H. H. M. Tam.
OSiL: An XML-based schema for stochastic programs H.I. Gassmann, R. Fourer, J. Ma, R.K. Martin SP XI Vienna, August 2007.
Monte Carlo Linear Algebra Techniques and Their Parallelization Ashok Srinivasan Computer Science Florida State University
A Kernel Approach for Learning From Almost Orthogonal Pattern * CIS 525 Class Presentation Professor: Slobodan Vucetic Presenter: Yilian Qin * B. Scholkopf.
Slide 1/20 Defending Against Strategic Adversaries in Dynamic Pricing Markets for Smart Grids Paul Wood, Saurabh Bagchi Purdue University
Optimally Modifying Software for Safety and Functionality Sampath Kannan U.Penn (with Arvind Easwaran & Insup Lee)
Generalization Error of pac Model  Let be a set of training examples chosen i.i.d. according to  Treat the generalization error as a r.v. depending on.
Multipole-Based Preconditioners for Sparse Linear Systems. Ananth Grama Purdue University. Supported by the National Science Foundation.
ANSYS, Inc. Proprietary © 2004 ANSYS, Inc. Chapter 5 Distributed Memory Parallel Computing v9.0.
Hybrid Parallel Implementation of The DG Method Advanced Computing Department/ CAAM 03/03/2016 N. Chaabane, B. Riviere, H. Calandra, M. Sekachev, S. Hamlaoui.
Design and Planning Tools John Grosh Lawrence Livermore National Laboratory April 2016.
A Scalable Parallel Preconditioned Sparse Linear System Solver Murat ManguoğluMiddle East Technical University, Turkey Joint work with: Ahmed Sameh Purdue.
Hui Liu University of Calgary
Block Low Rank Approximations in LS-DYNA
On Optimal Distributed Kalman Filtering in Non-ideal Situations
Parallelism in High-Performance Computing Applications
CS5321 Numerical Optimization
Hybrid Programming with OpenMP and MPI
Ph.D. Thesis Numerical Solution of PDEs and Their Object-oriented Parallel Implementations Xing Cai October 26, 1998.
Presentation transcript:

Scalable Multi-Stage Stochastic Programming Cosmin Petra and Mihai Anitescu Mathematics and Computer Science Division Argonne National Laboratory DOE Applied Mathematics Program Meeting April 3-5, 2010

Motivation  Sources of uncertainty in complex energy systems –Weather –Consumer Demand –Market prices  – Anitescu, Constantinescu, Zavala –Stochastic Unit Commitment with Wind Power Generation –Energy management of Co-generation –Economic Optimization of a Building Energy System 2

Stochastic Unit Commitment with Wind Power  Wind Forecast – WRF(Weather Research and Forecasting) Model –Real-time grid-nested 24h simulation –30 samples require 1h on 500 CPUs 3 Slide courtesy of V. Zavala & E. Constantinescu

Economic Optimization of a Building Energy System 4  Proactive management - temperature forecasting & electricity prices  Minimize daily energy costs Slide courtesy of V. Zavala

Optimization under Uncertainty  Two-stage stochastic programming with recourse (“here-and-now”)  5 subj. to. continuous discrete Sampling Inference Analysis M samples Sample average approximation (SAA)

Multi-stage SAA SP Problems – Scenario formulation s.t.  Depth-first traversal of the scenario tree  Nested half-arrow shaped Jacobian  Block separable obj. func.

Linear Algebra of Primal-Dual Interior-Point Methods 7 subj. to. Min Convex quadratic problem IPM Linear System Two-stage SP arrow-shaped linear system (via a permutation) Multi-stage SP nested

The Direct Schur Complement Method (DSC)  Uses the arrow shape of H  Solving Hz=r 8 Back substitution Implicit factorization Diagonal solveForward substitution

High performance computing with DSC 9  Gondzio (OOPS) 6-stages 1 billion variables  Zavala et.al., 2007 (in IPOPT)  Our experiments (PIPS) – strong scaling is investigated  Building energy system Almost linear scaling  Unit commitment Relaxation solved Largest instance 28.9 millions variables 1000 cores 12 x on Argonne

Scalability of DSC 10 Unit commitment 76.7% efficiency but not always the case Large number of 1 st stage variables: 38.6% efficiency on Argonne

Preconditioned Schur Complement (PSC) 11 (separate process)

The Stochastic Preconditioner  The exact structure of C is  IID subset of n scenarios:  The stochastic preconditioner (Petra & Anitescu, 2010)  For C use the constraint preconditioner (Keller et. al., 2000) 12

The “Ugly” Unit Commitment Problem 13  DSC on P processes vs PSC on P+1 process Optimal use of PSC – linear scaling Factorization of the preconditioner can not be hidden anymore. 120 scenarios

Quality of the Stochastic Preconditioner  “Exponentially” better preconditioning (Petra & Anitescu 2010)  Proof: Hoeffding inequality  Assumptions on the problem’s random data 1.Boundedness 2.Uniform full rank of and 14 not restrictive

Quality of the Constraint Preconditioner  has an eigenvalue 1 with order of multiplicity.  The rest of the eigenvalues satisfy  Proof: based on Bergamaschi et. al.,

The Krylov Methods Used for  BiCGStab using constraint preconditioner M  Preconditioned Projected CG (PPCG) (Gould et. al., 2001) –Preconditioned projection onto the –Does not compute the basis for Instead, – 16

Performance of the preconditioner  Eigenvalues clustering & Krylov iterations  Affected by the well-known ill-conditioning of IPMs. 17

A Parallel Interior-Point Solver for Stochastic Programming (PIPS)  Convex QP SAA SP problems  Input: users specify the scenario tree  Object-oriented design based on OOQP  Linear algebra: tree vectors, tree matrices, tree linear systems  Scenario based parallelism –tree nodes (scenarios) are distributed across processors –inter-process communication based on MPI –dynamic load balancing  Mehrotra predictor-corrector IPM 18

Tree Linear Algebra – Data, Operations & Linear Systems  Data –Tree vector: b, c, x, etc –Tree symmetric matrix: Q –Tree general matrix: A  Operations  Linear systems: for each non-leaf node a two-stage problem is solved via Schur complement methods as previously described. 19 subj. to. Min

Parallelization – Tree Distribution  The tree is distributed across processes.  Example: 3 processes  Dynamic load balancing of the tree –Number partitioning problem --> graph partitioning --> METIS 20 CPU 1CPU 2CPU

Conclusions  The DSC method offers a good parallelism in an IPM framework.  The PSC method improves the scalability.  PIPS – solver for SP problems.  PIPS is ready for larger problems: 100,000 cores. 21

Future work  New math / stat –Asynchronous optimization –SAA error estimate  New scalable methods for a more efficient software –Better interconnect between (iterative) linear algebra and sampling importance-based preconditioning multigrid decomposition –Target: emerging exa architectures  PIPS –IPM hot-start, parallelization of the nodes –Ensure compatibility with other paradigms: NLP, conic progr., MILP/MINLP solvers –A ton of other small enhancements  Ensure computing needs for important applications –Unit commitment with transmission constraints & market integration (Zavala) 22

Thank you for your attention! Questions? 23