1. James W. Bilbro JB Consulting International Huntsville, AL
Using the Advancement Degree of Difficulty (AD2) as an input to Risk Management
James W. Bilbro JB Consulting International Huntsville, AL
Multi-dimensional Assessment of Technology Maturity
Technology Maturity Conference
Founders Inn & Spa, Virginia Beach, VA
September 8-12, 2008

2. Using AD2 as an Input to Risk Management
Advancement Degree of Difficulty (AD2) is a method of systematically dealing with aspects beyond TRL.
It is a “predictive” description of what is required to move a system, subsystem or component from one TRL to another.
It provides information in the form of:
Liklihood of occurrence of an adverse event. Risk
Cost to ensure that such an event does not occur.
The time required to implement the necessary action.
Impact
Academy of Program/Project & Engineering Leadership 2 2

3. Using AD2 as an Input to Risk Management
AD2 consists of a set of questions in 5 specific areas:
Design and Analysis
Manufacturing
Software Development
Test
Operations
The questions are asked about each element in the product WBS structure from the top level system down to the individual component.
The questions are not directed toward the element itself, rather toward the issue of:
Do you have the resources – people, skills, tools, facilities, etc. to design, manufacture, test and operate it?
Academy of Program/Project & Engineering Leadership 3 3

4. Using AD2 as an Input to Risk Management
The levels of risk associated with AD2 are described in terms of the experience base of the developers.
i.e., have they done this before?

5. Using AD2 as an Input to Risk Management
Requires new development outside of any existing experience base. No viable approaches exist that can be pursued with any degree of confidence. Basic research in key areas needed before feasible approaches can be defined.
Requires new development where similarity to existing experience base can be defined only in the broadest sense. Multiple development routes must be pursued.
Requires new development but similarity to existing experience is sufficient to warrant comparison in only a subset of critical areas. Multiple development routes must be pursued.
Requires new development but similarity to existing experience is sufficient to warrant comparison on only a subset of critical areas. Dual development approaches should be pursued in order to achieve a moderate degree of confidence for success. (desired performance can be achieved in subsequent block upgrades with high degree of confidence.
Requires new development but similarity to existing experience is sufficient to warrant comparison in all critical areas. Dual development approaches should be pursued to provide a high degree of confidence for success.
Requires new development but similarity to existing experience is sufficient to warrant comparison across the board. A single development approach can be taken with a high degree of confidence for success.
Requires new development well within the experience base. A single development approach is adequate.
Exists but requires major modifications. A single development approach is adequate.
Exists with no or only minor modifications being required. A single development approach is adequate. 9 8 7 1 2 3 4 5 6
90%
80%
70% 0%
10%
20%
30%
40%
50%
AD2 LEVEL
RISK

6. Using AD2 as an Input to Risk Management
Academy of Program/Project & Engineering Leadership

7. Using AD2 as an Input to Risk Management
Academy of Program/Project & Engineering Leadership

8. Using AD2 as an Input to Risk Management
Relating AD2 to Project Uncertainty: from Variation to Chaos*
Variation:
Cost, time and performance levels vary randomly, but in a predictable range.
Foreseen Certainty:
A few known factors will influence the project but in predictable ways.
Unforeseen Uncertainty:
One or more major influence factors cannot be predicted.
Chaos:
Unforeseen events completely dominate the project’s target, planning and approach.
*De Meyer, et al

9. Using AD2 as an Input to Risk Management
Academy of Program/Project & Engineering Leadership

10. Using AD2 as an Input to Risk Management
Relating AD2 to a 5X5 Risk Matrix
DOD Likelihood Descriptions
Level
Likelihood
Probability of Occurrence 1
Not Likely
~10% 2
Low Likelihood
~30% 3
Likely
~50% 4
Highly Likely
~70% 5
Near Certainty
~90%
Likelihood
Academy of Program/Project & Engineering Leadership

11. Using AD2 as an Input to Risk Management
DOD Consequence Descriptions
Level
Technical Performance
Schedule 1
Minimal or no consequence to technical performance
Minimal or no impact 2
Minor reduction in technical performance or supportability, can be tolerated with little or no impact on the program
Able to meet key dates.
Slip <*month(s)
Budget increase or unit production cost increases.
<**(1% of Budget) 3
Moderate reduction in technical performance or supportability with limited impact on program objectives
Minor schedule slip. Able to meet key milestones with no schedule float
Sub-system slip<*month(s) plus available float
<**(5% of Budget) 4
Significant degradation in technical performance or major shortfall in supportability; may jeopardize program success
Program critical path affected
<**(10% of Budget) 5
Severe degradation in technical performance. Cannot meet KPP or key technical/supportability threshold; will jeopardize program success
Cannot meet key program milestones
>**(10% of Budget)
Consequence
Academy of Program/Project & Engineering Leadership

12. Using AD2 as an Input to Risk Management
Requires new development outside of any existing experience base. No viable approaches exist that can be pursued with any degree of confidence. Basic research in key areas needed before feasible approaches can be defined.
Requires new development where similarity to existing experience base can be defined only in the broadest sense. Multiple development routes must be pursued.
Requires new development but similarity to existing experience is sufficient to warrant comparison in only a subset of critical areas. Multiple development routes must be pursued.
Requires new development but similarity to existing experience is sufficient to warrant comparison on only a subset of critical areas. Dual development approaches should be pursued in order to achieve a moderate degree of confidence for success. (desired performance can be achieved in subsequent block upgrades with high degree of confidence.
Requires new development but similarity to existing experience is sufficient to warrant comparison in all critical areas. Dual development approaches should be pursued to provide a high degree of confidence for success.
Requires new development but similarity to existing experience is sufficient to warrant comparison across the board. A single development approach can be taken with a high degree of confidence for success.
Requires new development well within the experience base. A single development approach is adequate.
Exists but requires major modifications. A single development approach is adequate.
Exists with no or only minor modifications being required. A single development approach is adequate. 9 8 7 1 2 3 4 5 6
90%
80%
70% 0%
10%
20%
30%
40%
50% 5 4 1 2 3
90%
70%
10%
30%
50%
AD2 LEVEL
Consequence
RISK

13. Using AD2 as an Input to Risk Management
5X5 Risk Matrix
Academy of Program/Project & Engineering Leadership

14. Using AD2 as an Input to Risk Management
Summary
The AD2 assessment provides the basis for the development of the Technology Development Plan and for improved accuracy of the development of program/project cost, schedule and risk.
Academy of Program/Project & Engineering Leadership 14 14

15. Using AD2 as an Input to Risk Management
Website:
Bibliography:
De Meyer, Arnould, Loch, Christoph H., and Pich Michael T., “Managing Project Uncertainty: From Variation to Chaos,” MIT Sloan Management Review, pp , Winter 2002.
Risk Management Guide for DOD Acquistion 6th Edition Version 1.0 August 2006.
Academy of Program/Project & Engineering Leadership 15 15