1. Dealing With Uncertainties
Chapters 19 – 20:
Dealing With Uncertainties
TPS Example: Uncertain outcome
Decision rules for complete uncertainty
Utility functions
Expected value of perfect information
Statistical decision theory
The value-of-information procedure
Fixed typo on slide /12/2015.

2. TPS Decision Problem 7 OS Development Options:
Option B-Conservative (BC)
Sure to work (cost=$600K)
Performance = 4000 tr/sec
Option B-Bold (BB)
If successful, Cost=$600K
Perf.= tr/sec
If not, Cost=$100K
Perf.= tr/sec
Which option should we choose?

3. Operating System Development Options
Option BB (Bold)
Successful
Not Successful
Option BC (Conservative)
Option A
Performance (tr/sec)
4667
4000
2400
Value ($0.3K per tr/sec)
1400
1200
720
Basic cost
600
170
Total cost
700
Net value NV
800
500
550
NV relative to Option A
250
-50 50

4. Payoff Matrix for OS Options BB, BC
State of Nature
Alternative
Favorable
Unfavorable
BB (Bold)
250
-50
BC (Conservative) 50

5. Decision Rules for Complete Uncertainty
Maximin Rule. Determine minimum payoff for each option. Choose option maximizing the minimum payoff.
Maximax Rule. Determine max-payoff for each option. Choose option maximizing max-payoff.
Laplace Rule. Assume all states of nature equally likely. Choose option with maximum expected value.
BB = 0.5(250) + 0.5(-50) = 100
BC = 0.5(50) + 0.5(50) = 50

6. Difficulty with Laplace Rule
Favorable U1 U2
Expected Value BB
250
-50 50 BC
U1 : performance failure
U2 : reliability failure

7. Breakeven Analysis Treat uncertainty as a parameter
P= Prob (Nature is favorable)
Compute expected values as f(P)
EV (BB) = P(250) + (1-P)(-50) = P
EV (BC) = P(50) + (1-P)(50) = 50
Determine breakeven point P such that
EV (BB) = EV (BC): P = 50, P=0.333
If we feel that actually P>0.333, Choose BB
P<0.333, Choose BC

8. Utility Functions Is a Guaranteed $50K The same as
A 1/3 chance for $250K
and A 2/3 chance for -$50K?

9. Utility Function for Two Groups of Managers20 10 -3 -2 -1 1 2 3 4 5
Payoff ($M)
-10
Group 1
-20
Group 2

10. Possible TPS Utility Function20
(250,20)
(170,15) 10
(50,5)
(100,10)
100
200
300
-100
-10
(-50,-20)
-20
(-80,-25)

11. Utility Functions in S/W Engineering
Managers prefer loss-aversion
EV-approach unrealistic with losses
Managers’ U.F.’S linear for positive payoffs
Can use EV-approach then
People’s U.F.’S aren’t
Identical
Easy to predict
Constant

12. TPS Decision Problem 7 Available decision rules inadequate
State of Nature
Alternative
Favorable
Unfavorable
BB (Bold)
250
-50
BC (Conservative) 50
Available decision rules inadequate
Need better information
Info. has economic value

13. Expected Value of Perfect Information (EVPI)
Build a Prototype for $10K
If prototype succeeds, choose BB
Payoff: $250K – 10K = $240K
If prototype fails, choose BC
Payoff: $50K – 10K = $40K
If equally likely,
Ev = 0.5 ($240K) ($40K) = $140K
Could invest up to $50K and do better than before
thus, EVPI = $50K

14. However, Prototype Will Give Imperfect Information
That is,
P(IB|SF) = 0.0
Investigation (Prototype) says choose bold state of nature: Bold will fail
P(IB|SS) = 1.0
Investigation (prototype) says choose bold state of nature: bold will succeed

15. Suppose we assess the prototype’s imperfections as
P(IB|SF) = 0.20, P(IB|SS) = 0.90
And Suppose the states of nature are equally likely
P(SF) = P(SS) = 0.50
We would like to compute the expected value of using the prototype
EV(IB,IC) = P(IB) (Payoff if use bold)
+P(IC) (Payoff if use conservative)
= P(IB) [ P(SS|IB) ($250K) + P(SE|IB) (-$50K) ]
+ P(IC) ($50K)
But these aren’t the probabilities we know

16. How to get the probabilities we need
P(IB) = P(IB|SS) P(SS) + P(IB|SF) P(SF)
P(IC) = 1 – P(IB)
P(SS|IB) = P(IB|SS) P(SS) (Bayes’ formula)
P(IB)
P(SF|IB) = 1 – P (SS|IB)
P(SS|IB) = Prob (we will choose Bold in a state of nature where it will succeed)
Prob (we will choose Bold)

17. Net Expected Value of Prototype
PROTO COST, $K
P (PS|SF)
P (PS|SS)
EV, $K
NET EV, $K 60 5
0.30
0.80
69.3
4.3 10
0.20
0.90
78.2
8.2 20
0.10
0.95
86.8
6.8 30
0.00
1.00 90 8 4 10 20 30
NET EV, $K
PROTO COST, $K

18. Conditions for Successful Prototyping (or Other Info-Buying)
There exist alternatives whose payoffs vary greatly depending on some states of nature.
The critical states of nature have an appreciable probability of occurring.
The prototype has a high probability of accurately identifying the critical states of nature.
The required cost and schedule of the prototype does not overly curtail its net value.
There exist significant side benefits derived from building the prototype.

19. Pitfalls Associated by Success Conditions
Always build a prototype or simulation
May not satisfy conditions 3,4
Always build the software twice
May not satisfy conditions 1,2
Build the software purely top-down
Prove every piece of code correct
May not satisfy conditions 1,2,4
Nominal-case testing is sufficient
May need off-nominal testing to satisfy conditions 1,2,3

20. Statistical Decision Theory: Other S/W Engineering Applications
How much should we invest in:
User information gathering
Make-or-buy information
Simulation
Testing
Program verification
How much should our customers invest in:
MIS, query systems, traffic models, CAD, automatic test equipment, …

21. The Software Engineering Field Exists Because Processed Information Has Value