1. Presentation on PM2 Peculiararities
UNCLASSIFIED
Presented to:
SRE
Presentation on PM2 Peculiararities
Distribution authorized to DoD and DoD contractors only; Critical Technology; August Other requests for this document shall be referred to the U.S. Army RDECOM, ATTN: AMSRD-AMR.
Presented by:
Mark E. Sims
Reliability S&T Engineer
Aviation and Missile Research, Development and Engineering Center
DATE: 14 April 2015
UNCLASSIFIED

2. Table of Contents Finding IOT trials needed.
Discovering the problem with PM2’s DT Phase
Calculating Max Failures in IOT.
Contemplating α (the growth rate)
AMSAA PM2 Risk Assessment Matrices
Determining 70% Threshold
PM2 Equations
PM2 Factors
Questions

3. Finding IOT Trials Needed

4. Finding IOT Trials Needed
IOT Phase

5. Finding IOT Trials Needed
Max Failures
IOT Trials
PM2 Ratio 1
.09 9
.72 10
.80 19
1.51 20
1.58 36
2.86
INPUTS
R Req't
0.92
R Initial
0.91 MS
0.95
FEF
0.75
Conf
0.80 PA
0.70
Ratio needs to be < 0.80.
Trying to find the minimum trials needed in IOT.

6. Finding IOT Trials Needed
Max Failures
IOT Trials
PM2 Ratio
LCB for Req’t 1
.09
.200 9
.72
.836 10
.80
.851 19
1.51
.919 20
1.58
.923 36
2.86
.956
R Req't
0.92
R Initial
0.91 MS
0.95
FEF
0.75
Conf
0.80 PA
0.70

7. Finding IOT Trials Needed
Max Failures
IOT Trials
PM2 Ratio
LCB for Req’t 1
.09
.200 9
.72
.836 10
.80
.851 19
1.51
.919 20
1.58
.923 36
2.86
.956 37
.95
.921 2 67
0.99
.940 3 68
0.69
.9204 80
0.81
.932 82
0.83
.934 4 83
0.64
.9203
R Req't
0.92
R Initial
0.91 MS
0.95
FEF
0.75
Conf
0.80 PA
0.70

8. Problem with DT Phase DT Trials 75 IOT Trials 68 R Req't 0.9200
R Initial
0.9100 MS
0.95
FEF
0.75
Confidence
0.80
R goal in DT
0.9613
R growth potential
0.9732
(1-RGP)/(1-RG) Ratio
0.69

9. Problem with DT Phase DT Trials 5 IOT Trials 68 R Req't 0.9200
R Initial
0.9100 MS
0.95
FEF
0.75
Confidence
0.80
R goal in DT
0.9613
R growth potential
0.9732
(1-RGP)/(1-RG) Ratio
0.69

10. Problem with DT Phase DT Trials 75 IOT Trials 68 Growth rate = 0.20
R Req't
0.9200
R Initial
0.9100 MS
0.95
FEF
0.75
Confidence
0.80
R goal in DT
0.9613
R growth potential
0.9732
(1-RGP)/(1-RG) Ratio
0.69
Growth rate = 0.20

11. Problem with DT Phase DT Trials 5 IOT Trials 68 Growth rate = 0.52
R Req't
0.9200
R Initial
0.9100 MS
0.95
FEF
0.75
Confidence
0.80
R goal in DT
0.9613
R growth potential
0.9732
(1-RGP)/(1-RG) Ratio
0.69
Growth rate = 0.52

12. Problem with DT Phase DT Trials 5 IOT Trials 68
R Req't
0.9200
R Initial
0.9100 MS
0.95
FEF
0.75
Confidence
0.80
R goal in DT
0.9613
R growth potential
0.9732
(1-RGP)/(1-RG) Ratio
0.69
RGOAL,DT based on Requirement, Confidence, IOT Trials, and degradation factor.

13. Finding Max Failures in IOT (Discrete)
RLCB = BETAINV(1-Conf, S, F+1)
S = Estimated successes
F = Estimated failures

14. Finding Max Failures in IOT (Discrete)
RLCB = BETAINV(1-Conf, S, F+1)
S = Estimated successes
F = Estimated failures
Example where we have 50 trials in our demonstration and want to meet 0.91 requirement with 80% confidence.
Max Failures
RLCB
0.9863 1
0.9413 2
0.9164 3
0.8925 4
0.8692 5
0.8465 6
0.8241

15. Finding Max Failures in IOT (Discrete)
RLCB = BETAINV(1-Conf, S, F+1)
S = Estimated successes
F = Estimated failures
Example where we have 50 trials in our demonstration and want to meet 0.91 requirement with 80% confidence.
Max Failures
RLCB
0.9863 1
0.9413 2
0.9164 3
0.8925 4
0.8692 5
0.8465 6
0.8241
LCB ≥ RelReq’t
( ≥ 0.91)

16. Finding Max Failures in IOT (Continuous)
LCB

17. Finding Max Failures in IOT (Continuous)
LCB
We have 1000 hours of testing to demonstrate a 200 MTBF requirement with 80% confidence.
What are the max failures?

18. Finding Max Failures in IOT (Continuous)
LCB
We have 1000 hours of testing to demonstrate a 200 MTBF requirement with 80% confidence.
What are the max failures?
Failures
LCB
621 1
334 2
233 3
181
LCB ≥ MTBFR
(233 ≥ 200)

19. Contemplating α Ranges
GROWTH PARAMETER, α Mean Median Range One Shot Time or Distance Based
PM2 DOES NOT APPLY A GROWTH RATE IN ITS CALCULATIONS!!

20. Contemplating α Ranges
Missile System
Initial Reliability
Total Qty Tested
Growth Parameter, α
Stinger
0.85
109
0.352
Lance
0.60
122
0.427
Dragon
0.50
801
0.272
TOW (Basic)
0.45
279
0.257
TOW 2
170
0.278
HELLFIRE (Basic)
0.70
192
0.324
MLRS (Validation)
0.65
126
0.151
Shillelagh
268
0.076
Javelin
0.78
420
0.360
AMRDEC Historical Missile Reliability Growth Programs.

21. Contemplating α Ranges
Continuous Case:
Discrete Case :

22. Growth rate Calculation
Applying this example, we get a growth rate of . . .
Note: From AMSAA and Dr. Crow a growth below 0.40 is considered acceptable risk. However for PM2, I would personally keep this below 0.25.

23. PM2 Risk Assessment (Continuous)
Category
Low Risk
Medium Risk
High Risk
MG / MGP
< 70%
70 – 80%
> 80%
IOT Producer’s Risk
≤ 20% %
> 30%
IOT Consumer’s Risk MS
< 90% %
> 96%
FEF
≤ 70% %
MG / MI
< 2
2 - 3
> 3
Time to Incorporate and Validate Fixes in IOT Units Prior to Test
Adequate time and resources to have fixes implemented and verified with testing or strong engineering analysis
Time and resources for almost all fixes to be implemented & most verified w/ testing or strong engineering analysis
Many fixes not in place by IOT and limited fix verification
IOT Producer’s Risk = 1 – Probability of Acceptance
IOT Consumer’s Risk = 1 - Confidence

24. PM2 Risk Assessment (Discrete)
Category
Low Risk
Medium Risk
High Risk
(1-RGP) / (1-RG)
< 70%
70 – 80%
> 80%
IOT Producer’s Risk
≤ 20% %
> 30%
IOT Consumer’s Risk MS
< 90% %
> 96%
FEF
≤ 70% %
(1-RI) / (1-RG)
< 2
2 - 3
> 3
Time to Incorporate and Validate Fixes in IOT Units Prior to Test
Adequate time and resources to have fixes implemented and verified with testing or strong engineering analysis
Time and resources for almost all fixes to be implemented & most verified w/ testing or strong engineering analysis
Many fixes not in place by IOT and limited fix verification
MRBF = Mean Rounds Between Failures
IOT Producer’s Risk = 1 – Probability of Acceptance
IOT Consumer’s Risk = 1 - Confidence

25. PM2 Risk Assessment (Continuous / Discrete)
Category
Low Risk
Medium Risk
High Risk
Corrective Action Periods (CAPs)
5 or more CAPs which contain adequate calendar time to implement fixes prior to major milestones
3 – 4 CAPs but some may not provide adequate calendar time to implement all fixes
1 – 2 CAPs of limited duration
Reliability Increases after CAPs
Moderate reliability increases after each CAP result in lower-risk curve that meets goals
Some CAPs show large jumps in reliability that may not be realized because of program constraints
Majority of reliability growth tied to one or a couple of very large jumps in the reliability growth curve
Percent of Initial Problem Mode Failure Intensity Surfaced
Growth appears reasonable (i.e. a small number of problem modes surfaced over the growth test do not constitute a large fraction of the initial problem mode failure intensity)
Growth appears somewhat inflated in that a number of the problem modes surfaced constitute a moderately large fraction of the initial problem mode failure intensity
Growth appears artificially high with a small number of problem modes comprising a large fraction of the initial problem mode failure intensity

26. Meeting 70% of Reliability Goal (Continuous Case)
For example, if you need to meet 70% of your Requirement by some point in DT, the threshold is found using this equation.
Let MTBFREQ’T = 200

27. Meeting 70% of Reliability Goal (Discrete Case)
But what if we need to determine the 70% threshold, when the requirement is 0.91?

28. Meeting 70% of Reliability Goal (Discrete Case)
But what if we need to determine the 70% threshold, when the requirement is 0.91?
NO!!!!

29. Meeting 70% of Reliability Goal (Discrete Case)
But what if we need to determine the 70% threshold, when the requirement is 0.91?
Correct.

30. PM2 Discrete Equation R(N) = System Reliability at trial N.
RA = The portion of the system reliability not impacted by the correction action effort
RB = The portion of the system reliability addressed by the correction action effort
µ = Average Fix Effectiveness Factor (FEF)
n = Shape parameter of the beta distribution representing pseudo trials

31. PM2 Continuous Equation
β = Shape parameter

32. PM2 Factors

33. Management Strategy A-Mode: Failures that are not fixed.
The fraction of the initial system failure intensity due to failure modes that would receive corrective action if surfaced during DT.
A-Mode: Failures that are not fixed.
B-Mode: Failures that will have a fix.
λ = Failure rate.

34. Fix Effectiveness Factor
A fraction representing the reduction in the failure rate due to the implementation of a corrective action.

35. Degradation Factor (Continuous)
Adjustment Factor Value
γ = DT to IOT Degradation Factor – 10%
δ = Lag time needed before reliability fix is implemented ≥ 0
Time
MTBF
MTBFDT = 300
DT Phase
IOT Phase
Applying 10% derating factor
γ = 10%
MTBFIOT = 270

36. Degradation Factor (Discrete)
Adjustment Factor Value
γ = DT to IOT Degradation Factor – 10%
δ = Lag time needed before reliability fix is implemented ≥ 0
Trial
Reliabiility
RDT = .9711
DT Phase
IOT Phase
Applying 10% derating factor
γ = 5%
RIOT = .9696

37. Corrective Action Lag Adjustment Factor Value
γ = DT to IOT Degradation Factor – 10%
δ = The time needed before reliability fix is implemented ≥ 0
Time
MTBF
DT Phase
IOT Phase
DT2
DT1 δ
Note how the lag affects the MTBF going into each phase!!!

38. Corrective Action Lag Adjustment Factor Value
γ = DT to IOT Degradation Factor – 10%
δ = The time needed before reliability fix is implemented ≥ 0
Time
MTBF
DT Phase
IOT Phase
DT2
DT1 δ
Recommend letting lag be zero for last DT phase, so growth will be counted for the entire phase.

39. Growth Potential MGP = MTBF Growth Potential
The theoretical upper limit on MTBF
RGP = Reliability Growth Potential
The theoretical upper limit on system reliability

40. Questions??