1. SMU Course EMIS 8381 Nonlinear Programming
January 19, 2008
Hossam Zaki

2. Outline Course Scope Course Syllabus On Line Resources Course Topics
Classification of Optimization Problems
Course Syllabus
Description
Pre requisites
Text & References
Calendar
Grading
On Line Resources
Course Topics
Course Context
DSS Example
DSS Paradigm
Career Roles & Questions

3. Scope

4. What is an Optimization Problem?
Optimization problems involve the selection of values of a number of interrelated variables by focusing attention on one or more selection criteria designed to measure the quality of the selection

5. Optimization Problem Statement
Select the values of x
From a set of possible values X
that satisfy a group of algebraic constraints
In such a way that will optimize the value f(x)

6. Terminology Decision Variables = x Objective Function(s) = f(x)
Feasible Region = X
Constraints defining X
Inequality constraints : g(x)<=0
Equality constraints : h(x)=0

7. Optimization Problems Three Classifiers

8. DV Attributes

9. Objective Attributes

10. Constraints Attributes

11. Optimization Problems
Zilliant Fundamentals
4/21/2017
Optimization Problems
Objective
One
Many
Differentiable
Non Differentiable
Closed Form
From Simulation
Linear
Non linear
Deterministic
Stochastic
Finite DV
Continuous DV
One Obj
Decision Variables (DV)
Discrete
Continuous
Finite
Infinite
Differentiable Obj
Deterministic Obj
Constraints
Unconstrained
Constrained
Linear
Non linear
Simple bounds
GUB
Network
Block diagonal
How are these established
(more later)
Develop hypotheses regarding drivers of differences in pricing outcomes”
Channel (OEM, CM, Disti)
Customer characterisics: size, industry, end-use
Etc.
Work with your IT and biz experts to define data extracts / you send extracts / we QA, then begin statistical analysis
Develop logic to normalize price, compute decision variable
External reference price
Internal reference price (list, disti book, etc.)
Cost (margin, markup)
Identify key variables – regression methods
For each key variable, establish classification logic for normalized price (dv) – classification methods (e.g. CHAID, CART, looking at newer latent variable methods)
For each key variable, establish ordering (more later)
Evaluate resulting precision price segmentation definition using cross-validation methods
EMPIRICAL / NON-PARAMETRIC
Unconstrained
Linear Constraints 11
© 2006, 2007 Zilliant, Inc. -- CONFIDENTIAL
 Zilliant, Inc.
All rights reserved.
ZSG Implementation Methodology: 11

12. Course Scope Decision Variable Objective Constraints Continuous
Finite dimensions
Objective
Single
Closed form or From Simulation
Linear or Nonlinear
Deterministic
Constraints

13. Exercise # 1
Identify the attributes that define following optimization problems and prove one simple example. Display the results in table format
Linear Programming
Goal Programming
Network Programming
Integer Programming
Non differentiable Optimization
Global Optimization
Dynamic Programming
Stochastic Programming
Quadratic Programming
Fractional Programming
Geometric Programming
Multi Objective Optimization

14. Course Syllabus

15. Course Description
This course discusses, presents and explains the most important methods and results used to model and solve nonlinear optimization problems.

16. Prerequisites Advanced calculus (partial derivatives)
Linear algebra (vectors and matrices).
Knowledge on linear programming is helpful but not required (we will cover what we need).

17. Text & References Text Book References
"Nonlinear Programming: Theory and Algorithms" by, M. Bazaraa, H. Sherali and C Shetty, 3rd Edition, John Wiley, ISBN
References
“Handbooks in OR & MS, Volume 1, Optimization”, Nemhauser et al editors, North Holand, Chapters I and III, 1989
Dennis & Schnabel, “Numerical Methods for Unconstrained Optimization”, Prentice-Hall, 1983
Gill, Murray and Wright, “Practical Optimization”, Academic Press, 1981
D. Luenberger, “Linear and Nonlinear Programming”, 2nd Edition, Addison Wesely, 1984
D. Bertsekas, “Nonlinear Programming”, 2nd Edition, Athena Scientific, 1999

18. Class Calendar Class (Section001) 14 Class Periods
Saturday 10 AM-12:50 PM
10 Minute break every 50 minutes
Meets in 203 Junkins
14 Class Periods
First Class Period: January 19
No Class: March 22
Last Class Period: April 26

19. Grading In-class Exams (70%) Term Paper & Presentation (20%)
Mid Term on 3/8/ 08 = (30%)
Final on 4/26/08 = (40%)
Term Paper & Presentation (20%)
A nonlinear optimization topic, e.g. algorithm, application and /or software demo not covered in class
Due on 3/29/08
Homework (10%)

20. On Line Resources

21. Mathematical Programming Glossary
General Information - A list of dictionaries, suggested methods of citation, and contribution instructions.
The Nature of Mathematical Programming - See this for basic terms and a standard form of a mathematical program that is used throughout this glossary.
Notation - Read this to clarify notation.
Supplements - A list of supplements that are cited by entries.
Myths and Counter Examples - Some common and uncommon misconceptions.
Tours - Collections of Glossary entries for a particular subject.
Biographies - Some notes on famous mathematicians.
Please remember this is a glossary, not a survey, so no attempt is made to cite credits.

22. 1998 Nonlinear Programming Software Survey
The information in this survey was provided by the vendors in response to a questionnaire developed by Stephen Nash. The survey should not be considered as comprehensive, but rather as a representation of available NLP packages. The listings are limited to products that fit the parameters of the survey as outlined in the accompanying article.

23. Nonlinear Programming Frequently Asked Questions
Q1. "What is Nonlinear Programming?"
Q2. "What software is there for nonlinear optimization?"
Q4. "What references are there in this field?"
Q5. "What's available online in this field?"

24. NEOS Guide http://www-fp.mcs.anl.gov/OTC/Guide/
The Optimization Tree. Our thumbnail sketch of optimization (also known as numerical optimization or mathematical programming) and its various sub disciplines.
The Optimization Software Guide. Information on software packages from the book by Moré and Wright, updated for the NEOS Guide.
Frequently Asked Questions on Linear and Nonlinear Programming. These are the FAQs initiated by John Gregory, now maintained by Bob Fourer as part of the NEOS Guide.

25. MIT Nonlinear Course
Prof. Dimitri Bertsekas

26. UCLA Nonlinear EE Course
Prof. Lieven Vandenberghe

27. Course Topics

28. Course Topics Chapter 1 Appendix A Chapter 2 Chapter 3 Chapter 4
Intro
Appendix A
Math Review
Chapter 2
Convex Sets
Chapter 3
Convex Functions
Chapter 4
Optimality Conditions
Chapter 6
Lagragian Duality
Chapter 8
Unconstrained Opt
Chapter 9
Penalty and Barrier
Chapter 10
Methods of Feasible Directions

29. Exercise # 2 Plot and solve graphically: Maximize x1 Subject to
h1(x1,x2) = x12 – x2 + a = 0
h2(x1,x2) = – x1 + x22 + a = 0
For the following values of a:
a = 1, 0.25, 0 and -1

30. Course Context

31. DSS Example Optimization Model Simulation Model What if Objective 1
Business Rules
Alternative
Resources
Existing
Capacity
Aggregate
Demand
Profile
Cost of new
Timing
Constraints
Capacity
Configuration
Feedback
Simulation Model
Stochastic
Show-up rates
Daily
Demand
Success Rates
Resources
Objective 1
Support capacity planning and machine sizing questions.
In particular, to determine the number of machines needed and investigate optimum queue configuration that will enable processing all messages in a 24-hour period
Objective 2
Enable what-if analysis, i.e. to evaluate impact of changes in operating parameters on system performance metrics
High Level Design
Optimization techniques excels at searching through numerous feasible solutions and identifying the best one but cannot handle randomness effectively
Simulation techniques excels at handling randomness but cannot handle searching effectively.
The proposed integrated frame work maximizes the strengths of the two techniques
Operations
Characteristics
What if

32. Decision Support System Development Methodology
12. Estimate Lift (Improvement)
1. Understand Business Context and
Formulate Problem Statement in English
11. Perform What-if
2. Develop DSS High Level Design
10. Prepare & analyze output.
Validate results
3. Formulate Mathematical Model
DV, Objective, Constraints, Goals
9. Solve Real Problems
ITERATE
4. Receive, Clean and Synthesize Business Data. Create DB
8. Refine Formulation: Aggregate, Decompose, Transform
5. Estimate and/or Forecast needed data, e.g. demand
7. Prepare & Solve Small Sample Problems
DV = Decision Variable
DB = Database
6. Develop or Select Solver, prepare input files, connect to DB

33. How will the course help u with career questions?
Which problems to invest $ in?
How to gain tech diff over competitors?
Which is the expected lift?
Should we productize?
Develop in house or buy?
How many scientists do we need?
Service or license?
How many person-months?
What is the impact of a data change?
How to prioritize tasks?
How to validate results?
Which solver to use?
Reformulate?
Which parameters?
How to formulate?
Which algorithm?

34. Questions?