1. 8-1 Establish Cost/Schedule Performance Impacts Program Requirements Assess Risks Evaluate Risk Handling Options Evaluate Subcontractor Risks Establish Cost Schedule/ Perf Impacts Manage Risks Identify Key Ground Rules Identify Risk Factors Estimate Effect On Cost Establish Management Reserves Estimate Effect On Schedule

2. 8-2 Changing Acquisition Environment Aeronautics Systems Center has implemented a new proposal evaluation process. What’s In - New low cost criteria termed “Most Probable Life Cycle Cost.” What’s Out - Credibility of “single point estimate” in contractor bid.

3. 8-3 Who Is “Low Cost” In the Example Below? Contractor A Contractor B Contractor C 8M* 10M 12M 15M 14M* 15M 20M 18M 16M* Contractor Bid ASC Assessment of Most Probable Cost (i.e., 50%) ASC Assessment of 90% Cost Confidence

4. 8-4 What is Most Probable Cost Based On? Answer - The 90% cumulative probability span time in the government’s Most Probable Schedule. How does the government prepare a “Most Probable Schedule”? They perform a risk assessment of our proposal and map the risk into our proposed schedule.

5. 8-5 What Are The Impacts Of the New ASC Source Selection Process? 1. Schedules will need to be prepared early in the proposal development cycle. 2. Risk should be identified and mapped to the proposed schedule. 3. A Most Probable Schedule should be developed and “pushed” to the left to help squeeze out cost.

6. 8-6 OUTLINE What is a Scheduled Risk? What is the purpose of Assessing Schedule Risk? How can an assessment of Schedule Risk be Performed? Examples of Schedule Risk Assessments(SRA) How can we use SRA during: Pre-Proposal Discussions Proposal Development Contract Implementation What are the tools with which to perform SRA?

7. 8-7 Definition What is a Schedule Risk? The likelihood of a schedule delay and the magnitude of the delay. Note:Schedule risk can be characterized as high, medium, or low depending on the level of disruption to the program schedule.

8. 8-8 Purpose of Assessing Schedule Risk Verify that schedule risk drivers have been accounted for (i.e., concurrency of design/test/production, interrelations between tasks and teams, requirements stability, etc.). Provide basis to evaluate worth of schedule risk abatement options.

9. 8-9 Common Schedule Risk Factors Concurrency (Design/Test/Production) Interrelations (between IPD Team/Functions) Funding (Timely Turn-On) Requirements Availability Requirements Stability Degree of state-of-the-art Commonality with previous systems Number/historical performance of subcontractors Lead times (materials, etc.) Amount and complexity of software required Number and complexity of engineering drawings Testing requirements (Timely Development) Tests (number of hours\ required or number of successful flights) Amount of new materials being used Facilities Availability Manpower Availability Equipment Availability Producibility Improvements Urgency/priority of the program Contractual incentives for meeting program schedule

10. 8-10 Schedule Risk Assessment Methods 1. Experience Based Reviews - Recollection of lessons learned from similar work. 2. Technical Content Assessment - Analysis of empirical data on specific tasks. 3. System Level Modeling/Simulation - Computer-based representation of schedule.

11. 8-11 Technical Content Assessment Example 1 What is the expected schedule duration for a 500 hour durability life test on an avionic subsystem given that a large number of spare subsystems are available? 500 hr 8 hr/day =62.5 days ? ?

12. 8-12 Technical Content Assessment Example 1 (Cont.) Analysis of data on a similar avionic subsystem test shows that when: Mean time between maintenance action is 3.5 hr Mean time to remove and restore is 19 hr, and Test facility availability is 80% Schedule Duration is 502 days!! What if the number of spares is limited?

13. 8-13 Technical Content Assessment Example 2 Simulation Flow Compute run time to failure  Run Times Met 500 hour req’t? Spare available ? Compute remove and restore times Compute remove repair and restore times  Down times &  # failures Running this simulation flow repeatedly and plotting the results in a histogram format is a Monte Carlo simulation technique for estimating the calendar time and generating a final schedule estimate for conducting a durability test. NY N Y Stop Start

14. 8-14 Technical Content Assessment Example 2 (Cont.) 0 120 100 80 60 40 20 # of Occur- rences 376 399 421 444 466 489 512 534 557 580 602 648 Test Duration (Days) Assumptions: 500 hour Dur. Test No Spares Limitation 3.5 hour MTBMA

15. 8-15 Technical Content Assessment Example 2 (Cont.) 800 750 700 650 600 550 500 450 400 Test Duration (Days) ($) # Equivalent Ship Sets ($) (1000 Monte Carlo runs) 0 12 3 4 5 6 7 8 90% Mean 10%

16. 8-16 Show Schedule Risk Presentation

17. 8-17 How to Leverage Applications of SRA Experienced Based Reviews Technical Content Assessment System Level Modeling/Simulation Pre-Proposal Discussions Proposal Development Contract Implementation High Risk Tasks As Required to Defuse Complex Issues Discretion of P.M.

18. 8-18

19. 8-19

20. 8-20

21. 8-21 Risk + - Add-on tool for Microsoft Project @Risk – Add-on tool for Microsoft Excel

22. 8-22 Cost-Schedule Containment Chart

23. 8-23 Trade Study Methods

24. 8-24 Types of Trade Studies Controlled Convergence - Preliminary Method Used by Engineering. Quick Method to Compare “Primitive” Design Variables Cost Effectiveness - Links Force Structure Implications to Top Level Requirements Analysis Comprehensive - Considers all Applicable Decision Criteria

25. 8-25 Time Frames For Trade Study Methods Concept & Technology Development System Development & Demonstration Production & Deployment Operations & Support A Pre Concept & Tech Dev BC -Controlled Convergence- ------Cost-Effectiveness------- --Comprehensive--

26. 8-26 Controlled Convergence Trade Study

27. 8-27 Steps in Applying Controlled Convergence Method 1. Design Alternatives to Same Level of Detail 2. Choose Comparison Criteria 3. Choose a Baseline for Comparison Purposes 4. Compare the Alternatives to the Baseline 5. Sum Pluses and Minuses 6. Can New Alternative Be Created by Changing Negative(s) of a Strong Alternative? 7. Can Weak Alternative Be Eliminated? 8. Return to Step 4 or Document Findings and Proceed

28. 8-28 Controlled Convergence Method For Preliminary Trade Studies Design Alternatives Comparison Criteria (Design Primatives) Thrust/Weight (T/W) Weight/Wing Ref. Area (W/S) Coef. of Lift (C ) Cruise Performance (Specific fuel consumption, range, speed) Observables (Shaping, materials, propulsion, etc.) Payload Capacity Agility (maneuverability & controllability) S S S S S S S – – – S S – + S – + + + + + – + – S S – – + + – + – S + TOTAL +'s TOTAL S's TOTAL –'s 0 7 0 1 2 4 5 1 1 2 1 4 4 1 2 Legend + Significantly Better S About the Same – Significantly Worse 12345 (Baseline) L...

29. 8-29 Strengths and Weaknesses of Controlled Convergence Preliminary Trade Study Method Difficult for Strong-Willed Person to Dominate Decision Making Encourages Development of Additional Design Alternatives Time to Converge Can Be Controlled Repeated Applications of This Method Will Result in “Fuzzy” Comparisons of Leading Alternatives

30. 8-30 Cost-Effectiveness Trade Study

31. 8-31 Alternative Configuration Scoring Methods

32. 8-32

33. 8-33

34. 8-34 Life Cycle Cost Composition BV41861 WEAPON SYSTEM COST Tech Data Publication Contractor Service Support Equipment Training Equipment Factory Training Management Hardware Software Nonrecurring "Start-up" Allowance for Changes FLYAWAY COST PLUS Initial Spares RDT&E Facility Construction Operations & Support (Includes Post-Produc- tion Support) Disposal PROCUREMENT COST PROGRAM ACQUISITION COST LIFE CYCLE COST

35. 8-35 Cost Estimating Methods Used During Acquisition Phases Parametric Analogy Bottom- Up Eng. Pre Concept & Tech. Dev. Early in System Dev. & Demonstration Prod. & Dep. P S N/A S P S S S P P P P = Primary S = Secondary Early in System Dev. & Demonstration Concept & Tech. Dev.

36. 8-36 Relative Values of LCC Elements (based on 100 aircraft) Life Cycle Cost RTD&E (4.3%)Procurement (49.6%) Operations & Support (46.1%) 0.30 Demo/Validation 2.12 Air Vehicle 0.13 Engine 0.22 Offensive Avionics 0.70 Launcher 0.02 Training 0.06 Special Support Eqpt 0.47 Test & Evaluation 0.15 Project Management 0.13 Data 0.59 Tooling & Engineering 31.52 Airframe 8.83 Engine 2.31 Offensive Avionics 2.18 Launcher 0.17 Training 1.94 Special Support Eqpt 0.36 Test & Evaluation 0.07 Project Management 0.15 Data 1.52 Initial Spares 1.74 Replenish Sppt Eqpt 10.72 Fuel 0.92 Base Level Maint. 11.55 Depot Maint. 3.70 Updating/Mods 0.78 Replenish Spares 0.06 Vehicular Eqpt 12.61 Military Personnel 0.46 Civilian Personnel 1.29 Support Personnel 2.23 Pipeline Costs

37. 8-37 Comprehensive Trade Study

38. 8-38 Principal Steps in Comprehensive Trade Study 1. Identify Decision Criteria within Broad Decision Categories 2. Quantify Decision Criteria for Each Configuration 3. Analyze Customer Preferences for Each Decision Criterion 4. Assign Weights to Decision Criteria 5. Score Each Configuration (Sum Weights x Preferences) 6. Perform Sensitivity Analysis on Weights If Configuration Scoring Is Close

39. 8-39

40. 8-40

41. 8-41

42. 8-42 Sample Configuration Decision Categories Air Vehicle Effectiveness CostRisk Threat Acquisition Avoidance Hit Avoidable Given Acquisition Sortie Survival Given Hit Target Acquisition Target Kill Given Acquisition Kills per Sortie Targets Killed Over Time Flyaway Weapon System Procurement Program Acquisition Life Cycle Technical Cost Schedule Producibility Supportability Management

43. 8-43 Utility Functions - Preference Indicators Utility Functions Provide a Good Technique for Translating Diverse Criteria Into a Common Scale. (i.e., Range in NMi, MTBF in Hours, etc.) Utility Scores Range From 0 to 1 With 0 Being Least Preferred and 1 Being Most Preferred. Range in MNiMTBF in hours Threshold Objective 11 Examples Utility for RangeUtility for MTBF Threshold Objective

44. 8-44 Hints for Determining the Shape of Utility Functions After Establishing the Minimum Requirements and Goal, Draw Neutral Preference Position as Shown Neutral Preference 1 2 Divide Decision Factor into Quartiles and Assess 25%, 50%, and 75% Points Relative to Neutral Preference Req Decision Factor Goal 1 1 Critical, Risk Prone Non-Critical, Risk Average

45. 8-45 Sensitivity Analysis of Configuration Preferences Select Factor of Interest Such as Performance Range Increase Weight for Factor of Interest Until the Preferred Alternative / Configuration Changes Incrementally Lower the Weight for Factor of Interest Until the Preferred Alternative / Configuration Changes

46. 8-46 Exercise Background: As system requirements are identified and flowed down form the SDR, design options for the Group A hardware must be identified and trade studies performed to determine the best design. Five design options have been developed for Group A and have been evaluated by the AFS design team. Documentation of this first pass design review by the team is presented below and must now be used to select the best design in support of entrance criteria for the program PDR. Exercise: In order to limit the scope of this Exercise, the design trade study will be restricted to the Aft Antenna and Radome assembly. Referring to the Introductory Briefing material presented on the four subsequent charts, the Statement of Customer Requirements Part 2, and the Aft Antenna/Radome Functional Requirements Baseline, evaluate the designs provided and perform a comprehensive trade study to select the best design.

47. 8-47 AJS Statement of Customer Requirements Customer: Kurdish Fighter Program (Peace Whey) Operational Need: Fighter aircraft operating in a hostile environment require extensive electronic countermeasures (ECM) to defeat air-launched and ground- launched threats to the survivability of the aircraft. These ECM systems must be capable of generating and broadcasting radio frequency (RF) energy at sufficient power levels and in appropriate patterns to defeat any threat encountered by the aircraft.

48. 8-48 AJS Statement of Customer Requirements (Cont.) Description: The AJS shall be capable of installation on a lightweight, high-speed, multi-role fighter and shall be supportable in primitive forward operating bases. The system shall be capable of transmitting radio frequency signal in the microwave frequency range at sufficient power levels and in patterns capable of successfully jamming all identified threats at the required operational range. The AJS system shall consist of the following major components: 1. Core Avionics: Shall consist of the jammer, the radar warning receiver, and the OFP software. Shall be capable of generating the required RF signal in the microwave band at required power levels and of detecting radar emissions from the threat set at the required ranges. 2. RF Switch H/I/J Band: Shall control selection of broadcast frequency bands as required. 3. Fire Control Radar Notch Filter: Shall prevent interference of the Fire Control Radar (FCR) by the AJS system. 4. Forward Transmit Antenna 5. Aft Transmit Antenna and Raydome 6. WRD-650D24 Waveguide 7. Coaxial Cable

49. 8-49 Schedule: 1. Flight Test: The Safety of Flight(SOF) unit for flight test shall be available for installation 26 months after program go-ahead. 2. First Production Delivery: The first production assembly shall be delivered 36 months after program go-ahead. 3. Delivery Rate: Delivery of AJS units shall be at the rate of 2 units per month. 4. Total Quantity: The total quantity of AJS units shall be 20. AJS Statement of Customer Requirements (Cont.) Customer Priorities: 1. Power Transmitted. 2. Weight 3. First production delivery. 4. Cost not to exceed $125,000/unit (for 20 units).

50. 8-50

51. 8-51

52. 8-52 Types of Radomes Types of Construction Uses and Advantages Solid-Wall Construction Laminated Glass Cloth/Resin or Filament Wound Sandwich-Wall Construction Laminated Glass Cloth/Resin Impregnated Skin with Various Dielectric Cores Narrow Frequency Band High Strength Optimized Electrical Performance Broad Frequency Bandwith Lightweight

53. 8-53 Extensive Testing of Antennas Confirms That Performance Will Be Achieved Parameters Tested: - Electrical Requirements: Antenna Range 1. Radiation Patterns and Gain 2. Voltage Standing Wave Ratio (VSWR) 3. RF Power Handling 4. Antenna-to-Antenna Isolation - Environmental Requirements: Engineering Test Labs 1. Vibration 2. Temperature - Altitude 3. Humidity 4. Acoustical Noise 5. Mechanical Shock

54. 8-54 Airborne Jamming System (AJS) Statement of Customer Reqt.’s: Part 2 Performance: 1. Frequency: The AJS shall provide performance over the frequency ranges and angular pattern as represented in Table 1. The low-band transmission line shall be coaxial cable. The high-band transmission line shall be double-ridge, pressurized Waveguide of type WRD-650D24. 2. RF Power Handling: The AJS, while operating in any combination of temperature and pressure consistent with the aircraft operating envelope (as shown in Figure 1), shall be capable of handling 1500 watts peak power in a continuous transmit mode. 3. Antenna Polarization: The transmit antennas shall be left-hand circularly polarized. 4. Antenna Gain: The gain for each antenna shall be as specified in Table 1 and Figure 2. The gain is defined as gain measured at the minimum level of the axial ratio and is referenced to isotropic linear polarization.

55. 8-55 Airborne Jamming System (AJS) Statement of Customer Reqt.’s: Part 2 Environmental: 1. The AJS total system shall be capable of operation at all points in the aircraft flight envelope as specified in Figure 1. 2. The antenna/radome assembly shall have a mean time between failures (MTBF) of greater than 50,000 hours.

56. 8-56

57. 8-57

58. 8-58

59. 8-59

60. 8-60

61. 8-61

62. 8-62

63. 8-63

64. 8-64 Exercise # 4 Option 1 Risk Issues Risk Issues:Very good chance additional heat sink capacity will be needed to sustain power rating. This creates.4 pound of weight risk. Schedule risk is assessed as low.

65. 8-65

66. 8-66

67. 8-67 Exercise # 4 Option 2 Risk Issues Risk Issues:Low system weight achieved through use of spiral antenna impacts power handling capability and gain. Design of antenna mounting hardware results in predicted failure of vibration and acoustic loading spec due to resonant response within frequency envelope. Structural design changes required to meet vibration and acoustic specs result in a highly likely probability that the total assembly weight will add 1 pound of weight, exceeding spec. There is also a better than even chance that two additional calendar months design/development time will impact delivery of SOF hardware.

68. 8-68

69. 8-69

70. 8-70 Exercise # 4 Option 3 Risk Issues Risk Issues:Slightly higher-than-spec gain in the high band is due to an improved dielectric currently under development. The risk of additional development and testing costs resulting in a assessment of a probable AJS system cost increase per unit of +3%. There is an unlikely probability the qual test requirements could impact the SOF hardware delivery schedule, but this is assessed as low risk.

71. 8-71

72. 8-72

73. 8-73 Exercise # 4 Option 4 Risk Issues Risk Issues:Design Option 4 includes a solid-wall radome, normally used with narrow-bandwidth systems. Potential severe internal heat loads could result from RF energy reflection from the radome. Performance risk is assessed as highly likely to reduce power handling capability by.5 watt.

74. 8-74

75. 8-75

76. 8-76 Exercise # 4 Option 5 Risk Issues Volume: Design consistent with available installation volume Predicted Unit Cost: $19,460 Risk Issues:Option 5 includes a pressurized radome to achieve an operational altitude greater than required by the specs. However, this design has a history of pressure leak problems. Loss of pressure could result in arcing and system damage impacting performance and reliability. Upgrade to seals and increased leak testing would require additional cost and test time. Assessment indicates probable additional costs would increase AJS unit cost by 10%.

77. 8-77

78. 8-78 See Trade Study Example (Excel)