1. SMU Course EMIS 8381 Nonlinear Programming
January 14, 2012
Hossam Zaki

2. Outline Introductions Course Scope Course Syllabus On Line Resources
5 minutes each
Course Scope
What is an optimization problem?
Examplea
Classification of Optimization Problems
Course Syllabus
Description & Pre requisites
Text & References
Calendar &Grading
On Line Resources
Course Topics
Course Context
DSS Example & Paradigm
Career Roles & Questions

3. Scope

4. What is an Optimization Problem?
Optimization problems involve the selection of values of a number of interrelated variables in such a way to optimize on one or more selection criteria designed to measure the quality of each selection

5. Example: Price Optimization
A distributor sells 100K products to 20K customers needs to determine every month the set of prices that will maximize margin while satisfying product and customer pricing rules such as
GbB (Good, better, Best),
PGS (Platinum, Gold, Silver)
Sources of nonlinearity
Margin = (price – cost) * quantity
Quantity = a – b * price

6. High Level Price Optimization
Zilliant Fundamentals
4/23/2017
High Level Price Optimization
Volume
Market Response Model V*
Price
Price
Margin $
Pm*
MG*
Price Increase Zone
Price Decrease Zone
Econ Theory suggests, and well posed price optimization problems require, that
Demand (D) be inversely proportional to price (P)
AND
Demand goes to zero as price reaches a certain point
If demand is up sloping then increasing price increases demand, revenue and margin indefinitely, not realistic
Price Optimizer
 Zilliant, Inc.
All rights reserved.

7. Example: Portfolio Optimization
Markowitz Model
Given stocks returns and covariances, find a minimum variance portfolio that will generate a pre specified average return. That is, determine how much to invest in each stock to minimize risk at a certain level of return
Sources of nonlinearity
Risk = w(i) * w(j) * cov(i,j)

8. Example: Regression
Given N(1000) observations, find the values of the (linear or nonlinear) model parameters that will minimize the distance between model and observed
Sources of nonlinearity
Minimum [ Y (mode) – Y (observed)]2
Y (model ) = a0 + sum ( aj * xj)

9. Optimization Problem Statement
Select the values of a vector x in Rn
From a set of feasible vectors X in Rn
that satisfy a group of algebraic constraints
In such a way that will optimize the value of a real-valued function f(x)

10. Terminology Decision Variables = x Objective Function(s) = f(x)
Feasible Region = X
Constraints defining X
Inequality constraints : g(x)<=0
Equality constraints : h(x)=0

11. Optimization Problems Three Classifiers

12. DV Attributes
Given two cities in a country with lots of hills and valleys, find the shortest road going from one city to the other. This problem is a generalization of the above, and the solution is not as obvious.
Given two circles which will serve as top and bottom for a cup of given height, find the shape of the side wall of the cup so that the side wall has minimal area. The intuition would suggest that the cup must have conical or cylindrical shape, which is false. The actual minimum surface is the catenoid.

13. Objective Attributes

14. Constraints Attributes

15. Optimization Problems
Zilliant Fundamentals
4/23/2017
Optimization Problems
Objective
One
Many
Differentiable
Non Differentiable
Closed Form
From Simulation
Linear
Non linear
Deterministic
Stochastic
Finite DV
Continuous DV
One Obj
Decision Variables (DV)
Discrete
Continuous
Finite
Infinite
Differentiable Obj
Deterministic Obj
Constraints
Unconstrained
Constrained
Linear
Non linear
Simple bounds
GUB
Network
Block diagonal
How are these established
(more later)
Develop hypotheses regarding drivers of differences in pricing outcomes”
Channel (OEM, CM, Disti)
Customer characterisics: size, industry, end-use
Etc.
Work with your IT and biz experts to define data extracts / you send extracts / we QA, then begin statistical analysis
Develop logic to normalize price, compute decision variable
External reference price
Internal reference price (list, disti book, etc.)
Cost (margin, markup)
Identify key variables – regression methods
For each key variable, establish classification logic for normalized price (dv) – classification methods (e.g. CHAID, CART, looking at newer latent variable methods)
For each key variable, establish ordering (more later)
Evaluate resulting precision price segmentation definition using cross-validation methods
EMPIRICAL / NON-PARAMETRIC
Unconstrained
Linear Constraints 15
© 2006, 2007 Zilliant, Inc. -- CONFIDENTIAL
 Zilliant, Inc.
All rights reserved.
ZSG Implementation Methodology: 15

16. Course Scope Decision Variable Objective Constraints Continuous
Finite dimensions
Objective
Single
Closed form or From Simulation
Linear or Nonlinear
Deterministic
Constraints

17. Exercise # 1
Identify the attributes that define the following optimization problems and provide one simple example. Display the results in table format
Linear Programming
Goal Programming
Network Programming
Integer Programming
Non differentiable Optimization
Global Optimization
Dynamic Programming
Stochastic Programming
Quadratic Programming
Fractional Programming
Geometric Programming
Multi Objective Optimization

18. Course Syllabus

19. Course Description
This course discusses, presents and explains the most important methods and results used to model and solve nonlinear optimization problems.

20. Prerequisites Advanced calculus (partial derivatives)
Linear algebra (vectors and matrices).
Knowledge on linear programming is helpful but not required (we will cover what we need).

21. Text & References Text Book References
"Nonlinear Programming: Theory and Algorithms" by, M. Bazaraa, H. Sherali and C Shetty, 3rd Edition, John Wiley, ISBN
References
“Handbooks in OR & MS, Volume 1, Optimization”, Nemhauser et al editors, North Holand, Chapters I and III, 1989
Dennis & Schnabel, “Numerical Methods for Unconstrained Optimization”, Prentice-Hall, 1983
Gill, Murray and Wright, “Practical Optimization”, Academic Press, 1981
D. Luenberger, “Linear and Nonlinear Programming”, 2nd Edition, Addison Wesely, 1984
D. Bertsekas, “Nonlinear Programming”, 2nd Edition, Athena Scientific, 1999

22. Class Calendar Class (Section001) 14 Class Periods
Saturday 9 AM-11:50 PM
10 Minute break every 50 minutes
Meets in 205 Junkins
14 Class Periods
First Class Period: January 14
No Class: 3/10 (SB) & 4/7 (GF is on 4/6)
Last Class Period: April 28

23. Grading In-class Exams (70%) Term Paper & Presentation (20%)
Mid Term on 3/2 = (30%)
Final on 4/28 = (40%)
Term Paper & Presentation (20%)
A nonlinear optimization topic, e.g. algorithm, application and /or software demo not covered in class
Due on 3/31
Homework (10%)

24. On Line Resources

25. Mathematical Programming Glossary
General Information - A list of dictionaries, suggested methods of citation, and contribution instructions.
The Nature of Mathematical Programming - See this for basic terms and a standard form of a mathematical program that is used throughout this glossary.
Notation - Read this to clarify notation.
Supplements - A list of supplements that are cited by entries.
Myths and Counter Examples - Some common and uncommon misconceptions.
Tours - Collections of Glossary entries for a particular subject.
Biographies - Some notes on famous mathematicians.
Please remember this is a glossary, not a survey, so no attempt is made to cite credits.

26. 1998 Nonlinear Programming Software Survey
The information in this survey was provided by the vendors in response to a questionnaire developed by Stephen Nash. The survey should not be considered as comprehensive, but rather as a representation of available NLP packages. The listings are limited to products that fit the parameters of the survey as outlined in the accompanying article.

27. Nonlinear Programming Frequently Asked Questions
Q1. "What is Nonlinear Programming?"
Q2. "What software is there for nonlinear optimization?"
Q4. "What references are there in this field?"
Q5. "What's available online in this field?"

28. NEOS Guide http://www-fp.mcs.anl.gov/OTC/Guide/
The Optimization Tree. Our thumbnail sketch of optimization (also known as numerical optimization or mathematical programming) and its various sub disciplines.
The Optimization Software Guide. Information on software packages from the book by Moré and Wright, updated for the NEOS Guide.
Frequently Asked Questions on Linear and Nonlinear Programming. These are the FAQs initiated by John Gregory, now maintained by Bob Fourer as part of the NEOS Guide.

29. MIT Nonlinear Course
Prof. Dimitri Bertsekas

30. UCLA Nonlinear EE Course
Prof. Lieven Vandenberghe

31. Course Topics

32. Course Topics Chapter 1 Appendix A Chapter 2 Chapter 3 Chapter 4
Intro
Appendix A
Math Review
Chapter 2
Convex Sets
Chapter 3
Convex Functions
Chapter 4
Optimality Conditions
Chapter 6
Lagragian Duality
Chapter 8
Unconstrained Opt
Chapter 9
Penalty and Barrier
Chapter 10
Methods of Feasible Directions

33. Exercise # 2 Plot and solve graphically: Maximize x1 Subject to
h1(x1,x2) = x12 – x2 + a = 0
h2(x1,x2) = – x1 + x22 + a = 0
For the following values of a:
a = 1, 0.25, 0 and -1

34. Course Context

35. DSS Example Optimization Model Simulation Model What if Objective 1
Business Rules
Alternative
Resources
Existing
Capacity
Aggregate
Demand
Profile
Cost of new
Timing
Constraints
Capacity
Configuration
Feedback
Simulation Model
Stochastic
Show-up rates
Daily
Demand
Success Rates
Resources
Objective 1
Support capacity planning and machine sizing questions.
In particular, to determine the number of machines needed and investigate optimum queue configuration that will enable processing all messages in a 24-hour period
Objective 2
Enable what-if analysis, i.e. to evaluate impact of changes in operating parameters on system performance metrics
High Level Design
Optimization techniques excels at searching through numerous feasible solutions and identifying the best one but cannot handle randomness effectively
Simulation techniques excels at handling randomness but cannot handle searching effectively.
The proposed integrated frame work maximizes the strengths of the two techniques
Operations
Characteristics
What if

36. Decision Support System Development Methodology
12. Estimate Lift (Improvement)
1. Understand Business Context and
Formulate Problem Statement in English
11. Perform What-if
2. Develop DSS High Level Design
10. Prepare & analyze output.
Validate results
3. Formulate Mathematical Model
DV, Objective, Constraints, Goals
9. Solve Real Problems
ITERATE
4. Receive, Clean and Synthesize Business Data. Create DB
8. Refine Formulation: Aggregate, Decompose, Transform
5. Estimate and/or Forecast needed data, e.g. demand
7. Prepare & Solve Small Sample Problems
DV = Decision Variable
DB = Database
6. Develop or Select Solver, prepare input files, connect to DB

37. How will the course help u with career questions?
Which problems to invest $ in?
How to gain tech diff over competitors?
What is the expected lift?
Should we productize?
Develop in house or buy?
How many scientists do we need?
Service or license?
How many person-months?
What is the impact of a data change?
How to prioritize tasks?
How to validate results?
Which solver to use?
Reformulate?
Which parameters?
How to formulate?
Which algorithm?

38. Questions?