1. Applied Information Economics: Kickoff for Risk Return Analysis

2. Empirical Decision Theory
What is AIE?
Applied Information Economics is the practical application of scientific and mathematical methods to quantify the value of IT-enabled business investments
Economics
Decision/Game Theory
Empirical Decision Theory
Operations Research
Applied
Information
Economics
Modern Portfolio
Theory
Statistics
Information
Engineering
Information Theory
Software
Metrics

3. Real Solutions to… …the economics of information
…the economics of IT infrastructure
…the economics of risk
…the economics of labor reduction when headcount is not reduced
Bottom Line: AIE assesses and prioritizes IT investments based on quantitative and economically rational methods

4. What Do the Critics Say?
“Quantifying the risk and comparing its risk/return with other investments sets AIE apart from other methodologies. It can substantially assist in financially justifying a project -- especially projects that promise significant intangible benefits.” The Gartner Group
“AIE represents a rigorous, quantitative approach to improving IT investment decision making…..this investment will return multiples by enabling much better decision making. Giga recommends that IT executives learn more about AIE and begin to adopt its tools and methodologies, especially for large IT projects.” Giga Information Group
“AIE-like methods must become the standard way to make (IT) investment decisions.” Forrester Research, Inc.

5. Basic Risk/Return Analysis
Organization
Procedure
3. Make
Measure-
ments
4. Conduct
VIA
5. Conduct
Risk/Return
Analysis
6. Make
Recommenda-
tions
1. Describe &
Classify
2. Clarify
Decision
Model
Tools
The above overview of the RAVI method illustrates the different components:
In practice, the RAVI method describes a procedure implemented by an organization and based on spreadsheet tools. A RAVI evaluation produces a document used to decide whether or not to start up an investment.
Deploying the RAVI method in a company consists in training the different people involved in the decision-making process and in customizing the method to take account of the particular requirements of the company (structure of the project portfolio, observed project cancellation rate, reliability of development cost evaluations, corporate investment policy).
Each stage in the process generates a part of the final document which will be given to the decision-maker. Certain steps consist in gradually completing a spreadsheet which is an essential part of the evaluation deliverables. The calculations performed using the spreadsheet provide the risk and return.
The evaluation process is optimized inasmuch as it is used to take a decision as soon as sufficient information is gathered.
As far as organizational matters are concerned, the RAVI method advocates compliance with certain principles in order to increase objectivity in the evaluation process. The method sets out to improve the objectivity of the decision-making processes by implementing different roles which mitigate the consequences of any conflicts of interest.
AIE
Deliverables
AIE

6. Describe, Classify & Plan
3. Conduct
Measurements
4. Conduct
VIA
5. Conduct
Risk & Return
Analysis
6. Make
Recommendations
1. Classify &
Plan
2. Clarify
Intangibles &
CBA
Purpose: Agree on the specific investment to be analyzed, Agree on the specific question to be answered, Plan the rest of the analysis

7. Clarify The Decision Model
3. Make
Measure-
ments
4. Conduct
VIA
5. Conduct
Risk/Return
Analysis
6. Make
Recommenda-
tions
1. Describe &
Classify
2. Clarify
Decision
Model
During Clarification we translate the “Intangibles” into measurable units
These are ultimately modeled in a spreadsheet

8. Understanding Measurement (The Measurement.com approach)
Gilb’s Law “Anything can be measured in a way which is superior to not measuring it at all”
The perceived impossibility of measurement is an illusion caused by not understanding:
the Concept of measurement
the Object of measurement
the Methods of measurement
See my “Everything is Measurable” article in CIO Magazine (got to “articles” link on

9. Conduct Measurements
3. Make
Measure-
ments
4. Conduct
VIA
5. Conduct
Risk/Return
Analysis
6. Make
Recommenda-
tions
1. Describe &
Classify
2. Clarify
Decision
Model
We use the variety of measurement methods previously discussed
We usually start with what we know now (i.e. calibrated estimates)
More elaborate measurements (large controlled experiments or surveys) are only taken if we can show they are economically justified

10. Calibrated Estimates
Measuring your own uncertainty about a quantity is a general skill that can be taught with a measurable improvement
Studies show that most managers are statistically “overconfident” when assessing their own uncertainty
Training can “calibrate” people so that when they provide a 90% confidence interval, it still has a 90% chance of being right (even though it is subjective)
When asked to provide a subjective 90%
confidence interval, most managers provide
a range that only has about a 40%-50%
chance of being right
Perceived 90% Confidence Interval
Actual 90% Confidence Interval

11. Calculate the Value of Information
3. Make
Measure-
ments
4. Conduct
VIA
5. Conduct
Risk/Return
Analysis
6. Make
Recommenda-
tions
1. Describe &
Classify
2. Clarify
Decision
Model
The value of additional information can be calculated for each uncertain variable in the analysis
Measurement efforts will be more productive by focusing on variables that matter the most (results are often surprising)
This method is based on the probability of a change in a decision with additional information and the difference in the value of the decision
The value of information is derived by the following reasoning:
information reduces uncertainty,
reducing uncertainty improves the quality of the decisions,
better decisions improve performance
better performance makes for better profits.
A bad decision consists in starting up an investment for which the real outcome is unfavorable (loss), but also in not starting up an investment for which the outcome is favorable (loss of opportunity).
The entire investment project can be characterized in terms of the mean of the possible potential losses as a result of bad decisions. This mean is calculated using a spreadsheet, and simulating a large number of scenarios. For each scenario, the possible values of the different parameters are taken at random according to their probability profiles.
This criterion, the mean of the potential losses or Expected Opportunity Loss (EOL), is calculated again for a variant of the project where perfect information is supposed to be available for a particular parameter. The difference in the two values, that for the entire project, and that for this particular variant of it, is the value of having perfect information on that parameter, in other words its contribution to overall performance.
This process is repeated for each parameter to indicate the most critical parameters. One practical rule consists in assigning between 2 and 20% of the corresponding value to reduce uncertainty on the most critical parameters. A more advanced method can be used to determine the optimum value of this effort.
$$$ $

12. The Economic Value of Information
The Decision Theory Formula:
What it means:
Information reduces uncertainty
Reduced uncertainty improves decisions
Improved decisions satisfy business objectives (by definition)

13. Conduct Risk/Return Analysis
3. Make
Measure-
ments
4. Conduct
VIA
5. Conduct
Risk/Return
Analysis
6. Make
Recommenda-
tions
1. Describe &
Classify
2. Clarify
Decision
Model
Administrative Cost Reduction 5%
10%
15%
% Improvement in Customer Retention
10%
20%
30%
Total Project Cost
$2 million
$4 million
$6 million
ROI 0%
50%
100%

14. Make Recommendations The recommendations include: Deliverables include
Measure-
ments
4. Conduct
VIA
5. Conduct
Risk/Return
Analysis
6. Make
Recommenda-
tions
1. Describe &
Classify
2. Clarify
Decision
Model
The recommendations include:
To accept or reject the investment
Possible modifications to the proposed investment
Various risk management tactics
Deliverables include
The written report
The spreadsheet
The presentation

15. Overview of RRA Analysis
Classification 5%
10%
15%
20%
30%
$2 mill
$4 mill
$6 mill
Measurements
Value of Info. $
Intangibles
“Customer Satisfaction”
“Strategic Alignment”
“Technology Risk”
“Information Quality” etc.
$$$ $$
Measurables
Errors in Decision X
Change to Strategic Measure M
Productivity in Activity Y
Chance of cancellation, etc.
Each company will decide on its own what “compliance”, “strategic” and “economic” mean precisely. Also, they will determine their own boundaries on the classification chart and the specific analysis requirements in each “zone” of the chart.
Risk
Calculate Risk/Return Position
"expected" ROI
50%
100%
150%
200%
250%
-50% 0%
Probability
of a negative ROI
Probability of a positive ROI
Organization's investment limit
Acceptable region of investment
Return

16. Workshops
Much of the initial data gathering is from a series of workshops
We need to schedule 5-6 workshops for the following activities:
Define & Classify the investment
Clarify Decision Model
Measurement (initial)
Calibration
Estimation

17. Defining the Investment
What is the objective of this investment? (A one-sentence description of why)
What costs are unique to this investment?
What benefits are unique to this investment?
What are the risks of the investment?
What “decision dimensions” are important besides just an accept/reject recommendation?
Is all of the investment optional?
The decision is analyzed on behalf of which investor?

18. The Concept Of Measurement
Sometimes one believes that a thing is “immeasurable” only because they do not actually understand the concept of measurement
The “Measurement Theory” definition of measurement: “A measurement is an observation that results in information (reduction of uncertainty) about a quantity.”
Any “reduction of uncertainty” about a quantity can be of value ?

19. Real-world Measurements vs. Ideal Values
Ideal Values: Point
Real-world Meas.
Normal Distribution
Uniform Distribution
Lognormal Distribution
Hybrid
Threshold confidence
15%
85%

20. The Object of Measurement
If a thing seems like and immeasurable “intangible” it may just be ill-defined
Often, if we can define what we mean by a certain “intangible” we find ways to measure it ?

21. The Clarification Chain
AIE assumes that if a benefit or cost is defined unambiguously, then it is measurable.
If it is “Better” it is different in some relevant way...
If it is relevantly different then it is observable...
If it is observable then it is observable in some amount...
If we can observe it in some amount then it is measurable.

22. The “Thought Experiment”
Imagine that you are a scientist capable of making clones of entire companies and that you have a cloned pair of your company
Change one of the companies so that one has the stated “intangible” and the other does not
Ask what would you actually observe that would be different between the two companies

23. Examples of Clarification
The “Intangible”
Possible Meanings After Clarification
“Employee
Empowerment”
Less management overhead
Certain decisions are more accurate and faster
“Information
Availability”
Time and cost of searching is reduced
Certain costly errors are less frequent
“Customer
Relationship”
Increased repeat business
Tools like “The Clarification Chain” are used to identify unit-of-measure variables hidden beneath the “intangible” label
I offer a challenge that given any intangible, I can clarify it and identify a method of measurement within 15 minutes (I’ve never lost)