Bi-Parametric Convex Quadratic Optimization Tamás Terlaky Lehigh University Joint work with Alireza Ghaffari-Hadigheh and Oleksandr Romanko RUTCOR 2009:

Slides:



Advertisements
Similar presentations
Geometry and Theory of LP Standard (Inequality) Primal Problem: Dual Problem:
Advertisements

Solving LP Models Improving Search Special Form of Improving Search
Linear Programming Problem
Linear Programming.
Advanced Topics in Algorithms and Data Structures Lecture 7.2, page 1 Merging two upper hulls Suppose, UH ( S 2 ) has s points given in an array according.
Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 123 “True” Constrained Minimization.
Introduction to multi-objective optimization We often have more than one objective This means that design points are no longer arranged in strict hierarchy.
Introduction to Algorithms
Dragan Jovicic Harvinder Singh
C&O 355 Mathematical Programming Fall 2010 Lecture 20 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A.
Automatic Control Laboratory, ETH Zürich Automatic dualization Johan Löfberg.
Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.
Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.
Introduction to Linear and Integer Programming
Multi-objective Approach to Portfolio Optimization 童培俊 张帆.
Surface to Surface Intersection N. M. Patrikalakis, T. Maekawa, K. H. Ko, H. Mukundan May 25, 2004.
ISM 206 Lecture 4 Duality and Sensitivity Analysis.
Prénom Nom Document Analysis: Linear Discrimination Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.
1 Chapter 8: Linearization Methods for Constrained Problems Book Review Presented by Kartik Pandit July 23, 2010 ENGINEERING OPTIMIZATION Methods and Applications.
Support Vector Machines Formulation  Solve the quadratic program for some : min s. t.,, denotes where or membership.  Different error functions and measures.
Problem statement; Solution structure and defining elements; Solution properties in a neighborhood of regular point; Solution properties in a neighborhood.
Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.
Lecture outline Support vector machines. Support Vector Machines Find a linear hyperplane (decision boundary) that will separate the data.
Polyhedral Containment Check for a Linear Hybrid Automata Reachability Procedure Industry mentor: Steve Vestal Honeywell, Inc. Team 5: Sonja Petrović (presenter),
Optimization of Linear Problems: Linear Programming (LP) © 2011 Daniel Kirschen and University of Washington 1.
Linear Programming.
Course: Advanced Algorithms CSG713, Fall 2008 CCIS Department, Northeastern University Dimitrios Kanoulas.
Quadratic Programming Model for Optimizing Demand-responsive Transit Timetables Huimin Niu Professor and Dean of Traffic and Transportation School Lanzhou.
S I E M E N S C O R P O R A T E R E S E A R C H 1 1 Computing Exact Discrete Minimal Surfaces: Extending and Solving the Shortest Path Problem in 3D with.
C&O 355 Lecture 2 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A.
CS 8751 ML & KDDSupport Vector Machines1 Support Vector Machines (SVMs) Learning mechanism based on linear programming Chooses a separating plane based.
STDM - Linear Programming 1 By Isuru Manawadu B.Sc in Accounting Sp. (USJP), ACA, AFM
Pareto Linear Programming The Problem: P-opt Cx s.t Ax ≤ b x ≥ 0 where C is a kxn matrix so that Cx = (c (1) x, c (2) x,..., c (k) x) where c.
Full symmetric duality in continuous linear programming Evgeny ShindinGideon Weiss.
Introduction to Operations Research
Chapter 7 Duality and Sensitivity in Linear Programming.
Department Of Industrial Engineering Duality And Sensitivity Analysis presented by: Taha Ben Omar Supervisor: Prof. Dr. Sahand Daneshvar.
Course 13 Curves and Surfaces. Course 13 Curves and Surface Surface Representation Representation Interpolation Approximation Surface Segmentation.
Hon Wai Leong, NUS (CS6234, Spring 2009) Page 1 Copyright © 2009 by Leong Hon Wai CS6234: Lecture 4  Linear Programming  LP and Simplex Algorithm [PS82]-Ch2.
Optimal Rectangular Partition of a Rectilinear Polygonal Region
MODEL FOR DEALING WITH DUAL-ROLE FACTORS IN DEA: EXTENSIONS GONGBING BI,JINGJING DING,LIANG LIANG,JIE WU Presenter : Gongbing Bi School of Management University.
Applications of Parametric Quadratic Optimization Oleksandr Romanko Joint work with Alireza Ghaffari Hadigheh and Tamás Terlaky November 1, 2004.
Learning Spectral Clustering, With Application to Speech Separation F. R. Bach and M. I. Jordan, JMLR 2006.
Chapter 4 Sensitivity Analysis, Duality and Interior Point Methods.
IE 312 Review 1. The Process 2 Problem Model Conclusions Problem Formulation Analysis.
Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010.
TU/e Algorithms (2IL15) – Lecture 12 1 Linear Programming.
Linear Programming Chap 2. The Geometry of LP  In the text, polyhedron is defined as P = { x  R n : Ax  b }. So some of our earlier results should.
Parametric Quadratic Optimization Oleksandr Romanko Joint work with Alireza Ghaffari Hadigheh and Tamás Terlaky McMaster University January 19, 2004.
Massive Support Vector Regression (via Row and Column Chunking) David R. Musicant and O.L. Mangasarian NIPS 99 Workshop on Learning With Support Vectors.
Linear Programming Piyush Kumar Welcome to CIS5930.
Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.
TU/e Algorithms (2IL15) – Lecture 12 1 Linear Programming.
1 2 Linear Programming Chapter 3 3 Chapter Objectives –Requirements for a linear programming model. –Graphical representation of linear models. –Linear.
Linear Programming for Solving the DSS Problems
Computation of the solutions of nonlinear polynomial systems
Solver & Optimization Problems
Ch 10.1: Two-Point Boundary Value Problems
Morphing and Shape Processing
Full symmetric duality in continuous linear programming
Geometrical intuition behind the dual problem
Constrained Optimization
Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization Presenter: Xia Li.
Chapter 5. Sensitivity Analysis
Chapter 1. Formulations (BW)
Enumerating All Nash Equilibria for Two-person Extensive Games
Linear Programming Problem
Part 3. Linear Programming
Linear Constrained Optimization
Multiobjective Optimization
Presentation transcript:

Bi-Parametric Convex Quadratic Optimization Tamás Terlaky Lehigh University Joint work with Alireza Ghaffari-Hadigheh and Oleksandr Romanko RUTCOR 2009: Dedicated to the 80 th Birthday of Professor András Prékopa

2

3 Outline Introduction Quadratic optimization, optimal partition Uni-Parametric quadratic optimization Bi-Parametric quadratic optimization Numerical illustration Fundamental properties Algorithm Conclusions and future work

4 General framework of parametric optimization Multidimensional parameter is introduced into objective function and/or constraints The goal is to find – optimal solution – optimal value function Generalization of sensitivity analysis Applications: multi-objective optimization Introduction: Parametric Optimization

5 Multi-objective optimization: Introduction: Multi-Objective Optimization as Parametric Problem Multi-objective optimization with weighting method: Parametric formulation: f2f2 f*2f*2 f1f1 identify Pareto frontier (all non- dominated solutions) f*1f*1 OBJECTIVE SPACE

6 Convex Quadratic Optimization (QO) problem: Introduction: Quadratic Optimization and Its Parametric Counterpart Bi-Parametric Convex Quadratic Optimization (PQO) problem: Bi-parametric QO generalizes three models: uni-parametric QO

7 Sensity Analysis: Just be careful!

8 Convex Quadratic Optimization problems: Optimal Partition for QO Optimality conditions: PrimalDual Maximally complementary solution: LO: and - strictly complementary solution QO:, but may not hold maximally complementary solution maximizes the number of non-zero coordinates in and IPMs !!!

9 An optimal solution is maximally complementary iff: Optimal Partition for QO The optimal partition of the index set {1, 2,…, n} is The optimal partition is unique!!! Example: for maximally complementary solution with:

10 Uni-Parametric Quadratic Optimization Primal and dual perturbed problems: For some we are given the maximally complementary optimal solution of and with the optimal partition. - invariancy interval The left and right extreme points of the invariancy interval: - transition points

11 Uni-Parametric QO: Optimal Partition in the Neighboring Invariancy Interval How to proceed from the current invariancy interval to the next one? (1) (2) Solve two auxiliary problems z Gengyang zz

12 Uni-param QO: Numerical Illustration type l u B N T  ( ) invariancy interval Inf transition point invariancy interval transition point invariancy interval transition point invariancy interval transition point invariancy interval transition point Solver output

13 Bi-Parametric Quadratic Optimization Primal and dual perturbed problems: Invariancy regions instead of invariancy intervals Illustrative example:

14 Bi-Parametric Quadratic Optimization Illustrative example Invariancy regions

15 Bi-Parametric Quadratic Optimization Illustrative example: Optimal value function

16 Bi-Parametric Quadratic Optimization Optimal partition is constant on invariancy regions. Invariancy regions that are transition lines or singletons are called trivial regions. Otherwise, they are called non-trivial invariancy regions. The optimal value function is continuous and piecewise bivariate quadratic The boundary of a non-trivial invariancy region consists of a finite number of line segments. The optimal value function is a bivariate quadratic function on invariancy region : Invariancy region is a convex set and its closure is a polyhedron that might be unbounded.

17 Bi-Parametric QO: Algorithm Idea: reduce bi-parametric QO problem to a series of uni-paramteric QO problems with where

18 Bi-Parametric QO: Algorithm Start from, determine the optimal partition Solve where Solve where Now, two points and on the boundary of the invariancy region are known Consider cases and Choose, and

19 Bi-Parametric QO: Algorithm Case

20 Bi-Parametric QO: Algorithm Case  : and

21 Bi-Parametric QO: Algorithm Case  : and

22 Bi-Parametric QO: Algorithm Case  : and  : back to the first or the second case

23 Bi-Parametric QO: Algorithm Invariancy region exploration

24 Bi-Parametric QO: Algorithm Enumerating all invariancy regions cell vertex edge To-be-processed queue Completed queue

25 Conclusions and Future Work Developed an IPM-based technique for solving bi-parametric problems that extends the results of the uni-parametric case allows solving both bi-parametric linear and bi-parametric quadratic optimization problems systematically explores the optimal value surface  Polynomial-time algorithm in the output size  Applications in finance, IMRT, data mining Improving the implementation Extending methodology to Parametric Second Order Conic Optimization Multi-Parametric Quadratic Optimization

26 References A. B. Berkelaar, C. Roos, and T. Terlaky. The optimal set and optimal partition approach to linear and quadratic programming. In Advances in Sensitivity Analysis and Parametric Programming, T. Gal and H. J. Greenberg, eds., Kluwer, Boston, USA, A. Ghaffari-Hadigheh, O. Romanko, and T. Terlaky. Sensitivity Analysis in Convex Quadratic Optimization: Simultaneous Perturbation of the Objective and Right-Hand-Side Vectors. Algorithmic Operations Research, Vol. 2(2), A. Ghaffari-Hadigheh, O. Romanko, and T. Terlaky. Bi- Parametric Convex Quadratic Optimization. To appear in Optimization Methods and Software, A. Ghaffari-Hadigheh, O. Romanko, and T. Terlaky. On Bi- Parametric Programming in Quadratic Optimization. P roceedings of EurOPT-2008, 2008.