1. Flaw of Averages
This presentation explains a common problem in the design and evaluation of systems
This problem is the pattern of designing and evaluating systems based on the “average” or “most likely” future projections
Problem derives from misunderstanding of probability and systems behavior, known as
FLAW OF AVERAGES

2. Flaw of Averages Name derives from Sam Savage
It is a pun, integrating two concepts
It refers to
A mistake => a “flaw”
The concept of the “law of averages”, that is, that things balance out “on average”
The flaw consists of assuming that design or evaluation based on “average” (or most likely) conditions give correct answers

3. The design of an oil platform and wells in Golf of Mexico (Babajide)
A motivating example
The design of an oil platform and wells in Golf of Mexico (Babajide)

4. Gulf of Mexico Platform Probability Mass Functions
Note: “Most likely” scenarios are 150 and 100

5. Combined PMF

6. Actual ENPV  Value based on Mostly Likely Conditions
Comparison of Values
Based on “most likely” estimates
Based on actual distribution of possibilities
Actual ENPV  Value based on Mostly Likely Conditions

7. Another motivating example
Decision Analysis example

8. Comparison of Results
Value based on most likely event (No Carbon Tax) = 6.00
Value based on recognizing possibility of Carbon Tax is different = 10.8

9. Why does Flaw occur? Flaw is a pattern in systems design, Why?
Several reasons converge
Difficult to evaluate system over many different possibilities – hard enough to create one design
Management fixes parameters (such as oil price) to facilitate comparisons in company
Uncertainties exist outside of technical specialty (markets, geology…) so that designers use “best estimates”

10. Mathematics of Flaw Jensen’s law:
The Average of all the possible outcomes associated with uncertain parameters
generally does not equal
the value obtained from using the average value of the parameters
E [ f(x) ]  f [ E(x)] except when f(x) linear

11. Consequences
In simple terms, this means that the answer you get from a realistic description differs – often greatly – from the answer you from using most likely estimates
This is because the gains when things do well do not balance the losses when things do not
(sometimes they’re more, sometimes less)
In short: system behavior is non-linear

12. 3 Reasons for Non-Linearity
System response is non-linear
System response involves some discontinuity (step change)
Management rationally imposes a discontinuity

13. System Response is Non-Linear
Economies of Scale: Unit costs decrease with scale of production
Large initial costs prorated over volume, so that unit costs decrease as scale increases toward capacity
Increasing marginal costs as scale increases (labor, material costs higher)
Unit
Cost
This is Usual Situation!
Scale

14. System Response involves some Discontinuity
Discontinuities = special form of non-linearity
Discontinuities are Common:
Expansion of a Project might only occur in large increments (new runways, for example)
A System may be capacity constrained, so that profitability or values increases with demand up to a point, and then levels off

15. Management Creates Discontinuity
Whenever the Managers or System Operators decide to take some major decision about a project – to enlarge it or change its function – this creates a step change in the performance of the system.
This can happen often – and does!
See “Flaw of Averages” draft chapter

16. Take-Aways Do not be a victim of Flaw of Averages
Do not value projects or make design decisions based on average or most likely forecasts.
Do consider, as best you can, the entire range of possible events and examine the entire distribution of consequences.