SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki

Outline Introductions Course Scope Course Syllabus On Line Resources 5 minutes each Course Scope What is an optimization problem? Examplea Classification of Optimization Problems Course Syllabus Description & Pre requisites Text & References Calendar &Grading On Line Resources Course Topics Course Context DSS Example & Paradigm Career Roles & Questions

Scope

What is an Optimization Problem? Optimization problems involve the selection of values of a number of interrelated variables in such a way to optimize on one or more selection criteria designed to measure the quality of each selection

Example: Price Optimization A distributor sells 100K products to 20K customers needs to determine every month the set of prices that will maximize margin while satisfying product and customer pricing rules such as GbB (Good, better, Best), PGS (Platinum, Gold, Silver) Sources of nonlinearity Margin = (price – cost) * quantity Quantity = a – b * price

High Level Price Optimization Zilliant Fundamentals 4/23/2017 High Level Price Optimization Volume Market Response Model V* Price Price Margin $ Pm* MG* Price Increase Zone Price Decrease Zone Econ Theory suggests, and well posed price optimization problems require, that Demand (D) be inversely proportional to price (P) AND Demand goes to zero as price reaches a certain point If demand is up sloping then increasing price increases demand, revenue and margin indefinitely, not realistic Price Optimizer  Zilliant, Inc. All rights reserved.

Example: Portfolio Optimization Markowitz Model Given stocks returns and covariances, find a minimum variance portfolio that will generate a pre specified average return. That is, determine how much to invest in each stock to minimize risk at a certain level of return Sources of nonlinearity Risk = w(i) * w(j) * cov(i,j)

Example: Regression Given N(1000) observations, find the values of the (linear or nonlinear) model parameters that will minimize the distance between model and observed Sources of nonlinearity Minimum [ Y (mode) – Y (observed)]2 Y (model ) = a0 + sum ( aj * xj)

Optimization Problem Statement Select the values of a vector x in Rn From a set of feasible vectors X in Rn that satisfy a group of algebraic constraints In such a way that will optimize the value of a real-valued function f(x)

Terminology Decision Variables = x Objective Function(s) = f(x) Feasible Region = X Constraints defining X Inequality constraints : g(x)<=0 Equality constraints : h(x)=0

Optimization Problems Three Classifiers

DV Attributes Given two cities in a country with lots of hills and valleys, find the shortest road going from one city to the other. This problem is a generalization of the above, and the solution is not as obvious. Given two circles which will serve as top and bottom for a cup of given height, find the shape of the side wall of the cup so that the side wall has minimal area. The intuition would suggest that the cup must have conical or cylindrical shape, which is false. The actual minimum surface is the catenoid.

Objective Attributes

Constraints Attributes

Optimization Problems Zilliant Fundamentals 4/23/2017 Optimization Problems Objective One Many Differentiable Non Differentiable Closed Form From Simulation Linear Non linear Deterministic Stochastic Finite DV Continuous DV One Obj Decision Variables (DV) Discrete Continuous Finite Infinite Differentiable Obj Deterministic Obj Constraints Unconstrained Constrained Linear Non linear Simple bounds GUB Network Block diagonal How are these established (more later) Develop hypotheses regarding drivers of differences in pricing outcomes” Channel (OEM, CM, Disti) Customer characterisics: size, industry, end-use Etc. Work with your IT and biz experts to define data extracts / you send extracts / we QA, then begin statistical analysis Develop logic to normalize price, compute decision variable External reference price Internal reference price (list, disti book, etc.) Cost (margin, markup) Identify key variables – regression methods For each key variable, establish classification logic for normalized price (dv) – classification methods (e.g. CHAID, CART, looking at newer latent variable methods) For each key variable, establish ordering (more later) Evaluate resulting precision price segmentation definition using cross-validation methods EMPIRICAL / NON-PARAMETRIC Unconstrained Linear Constraints 15 © 2006, 2007 Zilliant, Inc. -- CONFIDENTIAL  Zilliant, Inc. All rights reserved. ZSG Implementation Methodology: 15

Course Scope Decision Variable Objective Constraints Continuous Finite dimensions Objective Single Closed form or From Simulation Linear or Nonlinear Deterministic Constraints

Exercise # 1 Identify the attributes that define the following optimization problems and provide one simple example. Display the results in table format Linear Programming Goal Programming Network Programming Integer Programming Non differentiable Optimization Global Optimization Dynamic Programming Stochastic Programming Quadratic Programming Fractional Programming Geometric Programming Multi Objective Optimization

Course Syllabus

Course Description This course discusses, presents and explains the most important methods and results used to model and solve nonlinear optimization problems.

Prerequisites Advanced calculus (partial derivatives) Linear algebra (vectors and matrices). Knowledge on linear programming is helpful but not required (we will cover what we need).

Text & References Text Book References "Nonlinear Programming: Theory and Algorithms" by, M. Bazaraa, H. Sherali and C Shetty, 3rd Edition, John Wiley, ISBN References “Handbooks in OR & MS, Volume 1, Optimization”, Nemhauser et al editors, North Holand, Chapters I and III, 1989 Dennis & Schnabel, “Numerical Methods for Unconstrained Optimization”, Prentice-Hall, 1983 Gill, Murray and Wright, “Practical Optimization”, Academic Press, 1981 D. Luenberger, “Linear and Nonlinear Programming”, 2nd Edition, Addison Wesely, 1984 D. Bertsekas, “Nonlinear Programming”, 2nd Edition, Athena Scientific, 1999

Class Calendar Class (Section001) 14 Class Periods Saturday 9 AM-11:50 PM 10 Minute break every 50 minutes Meets in 205 Junkins 14 Class Periods First Class Period: January 14 No Class: 3/10 (SB) & 4/7 (GF is on 4/6) Last Class Period: April 28

Grading In-class Exams (70%) Term Paper & Presentation (20%) Mid Term on 3/2 = (30%) Final on 4/28 = (40%) Term Paper & Presentation (20%) A nonlinear optimization topic, e.g. algorithm, application and /or software demo not covered in class Due on 3/31 Homework (10%)

On Line Resources

Mathematical Programming Glossary http://glossary.computing.society.informs.org/index.php?page=N.html   General Information - A list of dictionaries, suggested methods of citation, and contribution instructions. The Nature of Mathematical Programming - See this for basic terms and a standard form of a mathematical program that is used throughout this glossary. Notation - Read this to clarify notation. Supplements - A list of supplements that are cited by entries. Myths and Counter Examples - Some common and uncommon misconceptions. Tours - Collections of Glossary entries for a particular subject. Biographies - Some notes on famous mathematicians. Please remember this is a glossary, not a survey, so no attempt is made to cite credits.

1998 Nonlinear Programming Software Survey http://www.lionhrtpub.com/orms/surveys/nlp/nlp.html The information in this survey was provided by the vendors in response to a questionnaire developed by Stephen Nash. The survey should not be considered as comprehensive, but rather as a representation of available NLP packages. The listings are limited to products that fit the parameters of the survey as outlined in the accompanying article.

Nonlinear Programming Frequently Asked Questions http://www-unix.mcs.anl.gov/otc/Guide/faq/nonlinear-programming-faq.html Q1. "What is Nonlinear Programming?" Q2. "What software is there for nonlinear optimization?" Q4. "What references are there in this field?" Q5. "What's available online in this field?"

NEOS Guide http://www-fp.mcs.anl.gov/OTC/Guide/ The Optimization Tree. Our thumbnail sketch of optimization (also known as numerical optimization or mathematical programming) and its various sub disciplines. The Optimization Software Guide. Information on software packages from the book by Moré and Wright, updated for the NEOS Guide. Frequently Asked Questions on Linear and Nonlinear Programming. These are the FAQs initiated by John Gregory, now maintained by Bob Fourer as part of the NEOS Guide.

MIT Nonlinear Course Prof. Dimitri Bertsekas http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-252JNon-linear-ProgrammingSpring2003/CourseHome/

UCLA Nonlinear EE Course Prof. Lieven Vandenberghe http://www.ee.ucla.edu/ee236b/

Course Topics

Course Topics Chapter 1 Appendix A Chapter 2 Chapter 3 Chapter 4 Intro Appendix A Math Review Chapter 2 Convex Sets Chapter 3 Convex Functions Chapter 4 Optimality Conditions Chapter 6 Lagragian Duality Chapter 8 Unconstrained Opt Chapter 9 Penalty and Barrier Chapter 10 Methods of Feasible Directions

Exercise # 2 Plot and solve graphically: Maximize x1 Subject to h1(x1,x2) = x12 – x2 + a = 0 h2(x1,x2) = – x1 + x22 + a = 0 For the following values of a: a = 1, 0.25, 0 and -1

Course Context

DSS Example Optimization Model Simulation Model What if Objective 1 Business Rules Alternative Resources Existing Capacity Aggregate Demand Profile Cost of new Timing Constraints Capacity Configuration Feedback Simulation Model Stochastic Show-up rates Daily Demand Success Rates Resources Objective 1 Support capacity planning and machine sizing questions. In particular, to determine the number of machines needed and investigate optimum queue configuration that will enable processing all messages in a 24-hour period Objective 2 Enable what-if analysis, i.e. to evaluate impact of changes in operating parameters on system performance metrics High Level Design Optimization techniques excels at searching through numerous feasible solutions and identifying the best one but cannot handle randomness effectively Simulation techniques excels at handling randomness but cannot handle searching effectively. The proposed integrated frame work maximizes the strengths of the two techniques Operations Characteristics What if

Decision Support System Development Methodology 12. Estimate Lift (Improvement) 1. Understand Business Context and Formulate Problem Statement in English 11. Perform What-if 2. Develop DSS High Level Design 10. Prepare & analyze output. Validate results 3. Formulate Mathematical Model DV, Objective, Constraints, Goals 9. Solve Real Problems ITERATE 4. Receive, Clean and Synthesize Business Data. Create DB 8. Refine Formulation: Aggregate, Decompose, Transform 5. Estimate and/or Forecast needed data, e.g. demand 7. Prepare & Solve Small Sample Problems DV = Decision Variable DB = Database 6. Develop or Select Solver, prepare input files, connect to DB

How will the course help u with career questions? Which problems to invest $ in? How to gain tech diff over competitors? What is the expected lift? Should we productize? Develop in house or buy? How many scientists do we need? Service or license? How many person-months? What is the impact of a data change? How to prioritize tasks? How to validate results? Which solver to use? Reformulate? Which parameters? How to formulate? Which algorithm?

Questions?