1. Analytics and OR
DP- summary

2. Prescriptive (Application of OR methods)
Analytics
Descriptive (IT, CS)
Prescriptive (Application of OR methods)
Predictive (STAT)
Theory and methods of OR OR

3. Thinking like an OR Analyst
Linear or non-linear
Dynamic or Static
Probabilistic or Deterministic
One time decision or sequential decisions
Objective
What is the objective of making the decision(s) (Value Modeling can be used here)
Big Data
Availability
Type (Static, dynamic (frequency of collection), discrete, continuous)
Size (Big Data)
Model
Known, unknown (Learning)
Which OR tool is appropriate
Solution Strategy
Computational complexity
Verification
Validation
Implementation
Decision making in optimization
Problem
Definition

4. The Big Picture Decisions taken over time as the system evolves
Operations
Research
(Math prog)
Static decisions
(One-time
decision)
Linear
Programming
Integer
Non-linear
Mixed Integer
Metaheuristics
Dynamic decisions
(sequential
Dynamic
programming
Deterministic DP
Finite Horizon
problems
Infinite Horizon
Stochastic
The Big Picture
Decisions taken
over time as the
system evolves
Optimal control
Continuous time
ODE
Approach
Discrete time
Large-scale
problems
Approximate
Dynamic
programming

5. Dynamic programming
What is it? Operates by finding the shortest path (minimize cost) or longest path (maximize reward) of decision making problems that are solved over time

6. Myopic vs DP thinking - find shortest path from A to B1
Cij=10 40
Cij= cost on the arc A B 2 10 20
Myopic policy: V(A)= min (Cij)
= min of (10 or 20)
leads to solution of 50 from A to 1 to B
DP policy: V(A) = min (Cij + V(next node))
= min ( , 20+10) = 30
leads to solution of 30 from A to 2 to B
Key is to find the values of node 1 and 2
How? By learning via simulation-based optimization

7. Sequential Decision Making (over time)
Dynamic Programming!!
for
Sequential Decision Making (over time)
Is based on
the idea that
We want to move from one good state of the system to another by
making an near-optimal decision
in the presence of uncertainty
It also means that the future state depends only on the current and the decision taken
in the presence of uncertainly (and not on the past – memory-less property)
The above is achieved via unsupervised learning that entails only
an interaction with the environment in a model-free setting

8. Simulation-based Optimization
In more complex problems such as helicopter control the Optimizer is an Artificial Intelligence (Learning) Agent
Environment (uncertainty)
Output
(Objective function)
System simulator built using probability distributions from
real-world data
Decisions
Dynamic Programming Optimizer
(learning agent)

9. In Summary OR OR, CS IT OR STAT IT, CS
Analytics
Descriptive (IT, CS)
Prescriptive (Application of OR methods)
Predictive (STAT)
Theory and methods of OR OR
Optimization in Prescriptive Analytics
OR Models
Computational complexity
Algorithms for solving
BIG Data and data mining
Data Visualization & Statistical analysis
Data storage
OR, CS IT OR
STAT
IT, CS

10. In Summary – Main take away
In Big Data decision making Problems (Prescriptive Analytics)
Understand characteristics of the data, linear/non-linear, deterministic or probabilistic, static or dynamic (frequency of collection)
Beware of
1. Myopic policies
2. Exhaustive enumeration of all solutions
3. Computational Complexity
Look for appropriate modeling and solution strategies that can provide near-optimal decisions (good-enough) In the long run for the problem at hand.

11. For this course - Main take away
Value function V is cumulative
When making a decision sequentially over time (dynamic programming), make sure to sum the cost/reward of making the decision with the value of the estimated future state that the decision will take you to. Then pick a decision that minimizes (if cost) or maximizes (if reward) the above sum.
DP can handle
-non-linear
-probabilistic
-Computationally hard (high dimensionality and large state space)
-dynamic (sequential) and high frequency of decision making
-unknown models
-Big data problems
In this course we solve optimally
In OR 774 and in real world we strive of near-optimal (good enough) solutions

12. Thank You