SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki

Outline Course Scope Course Syllabus On Line Resources Course Topics Classification of Optimization Problems Course Syllabus Description Pre requisites Text & References Calendar Grading On Line Resources Course Topics Course Context DSS Example DSS Paradigm Career Roles & Questions

Scope

What is an Optimization Problem? Optimization problems involve the selection of values of a number of interrelated variables by focusing attention on one or more selection criteria designed to measure the quality of the selection

Optimization Problem Statement Select the values of x From a set of possible values X that satisfy a group of algebraic constraints In such a way that will optimize the value f(x)

Terminology Decision Variables = x Objective Function(s) = f(x) Feasible Region = X Constraints defining X Inequality constraints : g(x)<=0 Equality constraints : h(x)=0

Optimization Problems Three Classifiers

DV Attributes

Objective Attributes

Constraints Attributes

Optimization Problems Zilliant Fundamentals 4/21/2017 Optimization Problems Objective One Many Differentiable Non Differentiable Closed Form From Simulation Linear Non linear Deterministic Stochastic Finite DV Continuous DV One Obj Decision Variables (DV) Discrete Continuous Finite Infinite Differentiable Obj Deterministic Obj Constraints Unconstrained Constrained Linear Non linear Simple bounds GUB Network Block diagonal How are these established (more later) Develop hypotheses regarding drivers of differences in pricing outcomes” Channel (OEM, CM, Disti) Customer characterisics: size, industry, end-use Etc. Work with your IT and biz experts to define data extracts / you send extracts / we QA, then begin statistical analysis Develop logic to normalize price, compute decision variable External reference price Internal reference price (list, disti book, etc.) Cost (margin, markup) Identify key variables – regression methods For each key variable, establish classification logic for normalized price (dv) – classification methods (e.g. CHAID, CART, looking at newer latent variable methods) For each key variable, establish ordering (more later) Evaluate resulting precision price segmentation definition using cross-validation methods EMPIRICAL / NON-PARAMETRIC Unconstrained Linear Constraints 11 © 2006, 2007 Zilliant, Inc. -- CONFIDENTIAL  Zilliant, Inc. All rights reserved. ZSG Implementation Methodology: 11

Course Scope Decision Variable Objective Constraints Continuous Finite dimensions Objective Single Closed form or From Simulation Linear or Nonlinear Deterministic Constraints

Exercise # 1 Identify the attributes that define following optimization problems and prove one simple example. Display the results in table format Linear Programming Goal Programming Network Programming Integer Programming Non differentiable Optimization Global Optimization Dynamic Programming Stochastic Programming Quadratic Programming Fractional Programming Geometric Programming Multi Objective Optimization

Course Syllabus

Course Description This course discusses, presents and explains the most important methods and results used to model and solve nonlinear optimization problems.

Prerequisites Advanced calculus (partial derivatives) Linear algebra (vectors and matrices). Knowledge on linear programming is helpful but not required (we will cover what we need).

Text & References Text Book References "Nonlinear Programming: Theory and Algorithms" by, M. Bazaraa, H. Sherali and C Shetty, 3rd Edition, John Wiley, ISBN References “Handbooks in OR & MS, Volume 1, Optimization”, Nemhauser et al editors, North Holand, Chapters I and III, 1989 Dennis & Schnabel, “Numerical Methods for Unconstrained Optimization”, Prentice-Hall, 1983 Gill, Murray and Wright, “Practical Optimization”, Academic Press, 1981 D. Luenberger, “Linear and Nonlinear Programming”, 2nd Edition, Addison Wesely, 1984 D. Bertsekas, “Nonlinear Programming”, 2nd Edition, Athena Scientific, 1999

Class Calendar Class (Section001) 14 Class Periods Saturday 10 AM-12:50 PM 10 Minute break every 50 minutes Meets in 203 Junkins 14 Class Periods First Class Period: January 19 No Class: March 22 Last Class Period: April 26

Grading In-class Exams (70%) Term Paper & Presentation (20%) Mid Term on 3/8/ 08 = (30%) Final on 4/26/08 = (40%) Term Paper & Presentation (20%) A nonlinear optimization topic, e.g. algorithm, application and /or software demo not covered in class Due on 3/29/08 Homework (10%)

On Line Resources

Mathematical Programming Glossary http://glossary.computing.society.informs.org/index.php?page=N.html   General Information - A list of dictionaries, suggested methods of citation, and contribution instructions. The Nature of Mathematical Programming - See this for basic terms and a standard form of a mathematical program that is used throughout this glossary. Notation - Read this to clarify notation. Supplements - A list of supplements that are cited by entries. Myths and Counter Examples - Some common and uncommon misconceptions. Tours - Collections of Glossary entries for a particular subject. Biographies - Some notes on famous mathematicians. Please remember this is a glossary, not a survey, so no attempt is made to cite credits.

1998 Nonlinear Programming Software Survey http://www.lionhrtpub.com/orms/surveys/nlp/nlp.html The information in this survey was provided by the vendors in response to a questionnaire developed by Stephen Nash. The survey should not be considered as comprehensive, but rather as a representation of available NLP packages. The listings are limited to products that fit the parameters of the survey as outlined in the accompanying article.

Nonlinear Programming Frequently Asked Questions http://www-unix.mcs.anl.gov/otc/Guide/faq/nonlinear-programming-faq.html Q1. "What is Nonlinear Programming?" Q2. "What software is there for nonlinear optimization?" Q4. "What references are there in this field?" Q5. "What's available online in this field?"

NEOS Guide http://www-fp.mcs.anl.gov/OTC/Guide/ The Optimization Tree. Our thumbnail sketch of optimization (also known as numerical optimization or mathematical programming) and its various sub disciplines. The Optimization Software Guide. Information on software packages from the book by Moré and Wright, updated for the NEOS Guide. Frequently Asked Questions on Linear and Nonlinear Programming. These are the FAQs initiated by John Gregory, now maintained by Bob Fourer as part of the NEOS Guide.

MIT Nonlinear Course Prof. Dimitri Bertsekas http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-252JNon-linear-ProgrammingSpring2003/CourseHome/

UCLA Nonlinear EE Course Prof. Lieven Vandenberghe http://www.ee.ucla.edu/ee236b/

Course Topics

Course Topics Chapter 1 Appendix A Chapter 2 Chapter 3 Chapter 4 Intro Appendix A Math Review Chapter 2 Convex Sets Chapter 3 Convex Functions Chapter 4 Optimality Conditions Chapter 6 Lagragian Duality Chapter 8 Unconstrained Opt Chapter 9 Penalty and Barrier Chapter 10 Methods of Feasible Directions

Exercise # 2 Plot and solve graphically: Maximize x1 Subject to h1(x1,x2) = x12 – x2 + a = 0 h2(x1,x2) = – x1 + x22 + a = 0 For the following values of a: a = 1, 0.25, 0 and -1

Course Context

DSS Example Optimization Model Simulation Model What if Objective 1 Business Rules Alternative Resources Existing Capacity Aggregate Demand Profile Cost of new Timing Constraints Capacity Configuration Feedback Simulation Model Stochastic Show-up rates Daily Demand Success Rates Resources Objective 1 Support capacity planning and machine sizing questions. In particular, to determine the number of machines needed and investigate optimum queue configuration that will enable processing all messages in a 24-hour period Objective 2 Enable what-if analysis, i.e. to evaluate impact of changes in operating parameters on system performance metrics High Level Design Optimization techniques excels at searching through numerous feasible solutions and identifying the best one but cannot handle randomness effectively Simulation techniques excels at handling randomness but cannot handle searching effectively. The proposed integrated frame work maximizes the strengths of the two techniques Operations Characteristics What if

Decision Support System Development Methodology 12. Estimate Lift (Improvement) 1. Understand Business Context and Formulate Problem Statement in English 11. Perform What-if 2. Develop DSS High Level Design 10. Prepare & analyze output. Validate results 3. Formulate Mathematical Model DV, Objective, Constraints, Goals 9. Solve Real Problems ITERATE 4. Receive, Clean and Synthesize Business Data. Create DB 8. Refine Formulation: Aggregate, Decompose, Transform 5. Estimate and/or Forecast needed data, e.g. demand 7. Prepare & Solve Small Sample Problems DV = Decision Variable DB = Database 6. Develop or Select Solver, prepare input files, connect to DB

How will the course help u with career questions? Which problems to invest $ in? How to gain tech diff over competitors? Which is the expected lift? Should we productize? Develop in house or buy? How many scientists do we need? Service or license? How many person-months? What is the impact of a data change? How to prioritize tasks? How to validate results? Which solver to use? Reformulate? Which parameters? How to formulate? Which algorithm?

Questions?