1. Chuck Wagner Manufacturing Technology Division AFRL/MLM Phone # 937-904-4583 john.mistretta@wpafb.af.mil An Orientation to: Manufacturing Readiness Levels and Manufacturing Readiness Assessments

2. 2  Purpose and Background Manufacturing Readiness Levels (MRLs) IPT Tools for Evaluating MRLs Manufacturing Readiness Assessment (MRA) Process Assessment Example Discussion Session Outline

3. 3 Why MRLs? Manufacturing & Industrial Base Challenge Consensus among Congress, OSD, CSAF, GAO: “Advanced weapon systems cost too much, take too long to field, and are too expensive to sustain” Recent GAO study of 54 weapons programs: –Core set of 26 programs: RDT&E costs up by 42% and schedule slipped by 20 % $42.7B total cost growth 2.5 years slip on average –Characteristics of successful programs (GAO): Mature technologies, stable designs, production processes in control S&T organization responsible for maturing technologies, rather than program or product development manager Defense Science Board evaluated ManTech roles/impacts for AT&L –ManTech can significantly impact across all acquisition phases –Facilitates manufacturing/industrial base readiness for S&T transition and acquisition

4. 4 Up to 85% of Costs are committed during design and development – At Milestone B, up to 90% of costs could be locked in! Why MRLs? Life Cycle Cost Commitment IdeaValidationDesignDevelopmentProductionSupport 100 80 60 40 20 Cumulative Percent of Cost Life-Cycle Cost Determination Cost-Reduction Opportunities Source: DARPA Rapid Design Exploration and Optimization (RaDEO) Project 22% 35% 70% 85% 95%

5. 5 Why MRLs? Acquisition Health and Manufacturing Readiness Manufacturing risk/maturity is not the only cost/schedule/performance driver, but we need to manage manufacturing readiness integral to the overall acquisition process  Products made by immature manufacturing processes generally: -Cost more -Are prone to quality problems -May not all perform the same -Are less reliable in service -Have a hard time delivering on schedule  Establishes the Business Case: -Effects of design changes (spirals, planned upgrades) -Lot pricing agreements (long term vs single lots) -Capital investments--when does the company invest? Govt?

6. 6 Elements of Manufacturing Readiness Process planning Manufacturing instructions Process selection Manufacturing technology Quality assurance Production planning/scheduling Equipment Facilities and layout Maintenance Materials management Supply chain management Workforce development Tooling development Safety and ergonomics Product testing Product/process integration Demands a Disciplined Methodology to Assess, Track, and Manage Manufacturing Readiness

7. 7 MRL Current & Future State Current –ManTech is establishing the capability to conduct manufacturing readiness assessments for AFRL ATDs –Conducted MRAs and incorporated MRLs into approximately 10 key AFRL ATDs; 10 more planned through FY06 –Developing MRL training curriculum focused on MRL 3-6 –In partnership with JDMTP, developing DAU tools for program managers and working final MRL definitions & policy changes –Preparing to conduct MRAs on acquisition pilots Desired Future State –Manufacturing maturity considerations a normal part of acquisition milestone decision-making processes –Program managers possess a working understanding of manufacturing issues and associated risk –Acquisition plans and tech transition cost estimates incorporate manufacturing maturity considerations

8. 8 MRL Training Activities Developing training for ManTech cadre and program IPTs –Train and “certify” ManTech personnel to lead manufacturing readiness assessments (MRA) and risk mitigation plans –Train IPTs to understand and apply basic MRL concepts Current training program –1 hour Executive Orientation –8 hour ATD/IPT Introductory Training –4 day MRA Training (principally for ManTech personnel) Training program leveraging selected external activities –DAU Courses Topic introduced concurrent with existing PM and Mfg training courses More focused training under development –Univ of Tenn Industrial Preparedness Training (3 week duration) MBA style look at a company business viability and production capability Will Evolve to Encompass Full Acquisition Spectrum and Institutionalize in AFIT and DAU

9. 9 Purpose and Background  Manufacturing Readiness Levels  IPT Tools for Evaluating MRLs Manufacturing Readiness Assessment Process Assessment Example Discussion Session Outline

10. 10 Manufacturing Readiness Levels (MRLs) Common language and standard for –Assessing the manufacturing maturity of a technology or product and plans for its future maturation –Understanding the level of manufacturing risk in trying to produce a weapon system or transition the technology into a weapon system application Designed to complement TRLs Designed to help set the agenda for manufacturing risk mitigation

11. 11 Manufacturing Readiness & Technology Transition Success in manufacturing affects –Performance Success of the Technology –Defects and Reliability of the Technology Consider all these elements equally for Readiness PRODUCT MATERIAL Properties Reliability Processing Cost Quality DESIGN Loads Use Conditions Processes Available Material Selection Quantity PROCESSING Equipment Effect on Materials Configuration (layout) Tooling Quantity Quality

12. 12 Provide a common language and widely- understood standard for Assessing the performance maturity of a technology and plans for its future maturation Understanding the level of performance risk in trying to transition the technology into a weapon system application Technology Readiness Levels (TRLs)

13. 13 TRLs Leave Major Transition Questions Unanswered PERFORMANCE, SCHEDULE & COST RISK Is this level of performance reproducible in items 2 - 1000? What will these cost in production? Can these be made in a production environment by someone without a PhD? Are the key materials and components available? PERFORMANCE, SCHEDULE & COST RISK PERFORMANCE, SCHEDULE & COST RISK QUESTIONS CONCERNS TRL 6/7 – Prototype in Relevant/Service Environment

14. 14 TRL 9 Mission Proven TRL 8 System Qual TRL 7 Prototype in Ops Environmt TRL 6 Prototype in Rep Environmt TRL 5 Breadbrd in Rep Environmt TRL 4 Breadbrd in Lab TRL 3 Proof of Concept TRL 2 Concept Formulat TRL 1 Basic Principles Observed MRL 10 Lean Mfg Proc’s MRL 9 Mfg Proc’s In Place for FRP MRL 8 Mfg Proc’s In Place for LRIP MRL 7 Mfg Proc’s Maturing for LRIP MRL 6 Mfg Proc’s In Prod Rep Environmt MRL 5 Mfg Proc’s In Relevant Environmt MRL 4 Mfg Proc’s In Lab Environmt MRL 3 Mfg Concepts Identified Production & Deployment System Development & Demonstration Technology Development Concept Refinement Relationship To System Milestones Relationship To Technology Readiness Levels CBA MRL Relationships

15. 15 MRL 4 Mfg Processes Identified Key Processes Identified Producibility assessment initiated MRL 5 Mfg Processes Develop. Mfg equipment in relative environment Producibility assessment ongoing Cost drivers identified MRL 3 Mfg Concepts Identified MRL 6 Critical Mfg Processes Demo’d Mfg equipment in relevant environment Producibility assessment ongoing Cost drivers analyzed Long lead items identified MRL 7 Prototype Mfg System Mfg processes in validation Producibility improvement underway Trade studies conducted Supply chain validated Long lead plans in place MRL 8 Process Maturity Demo All materials ready for LRIP Mfg processes proven for LRIP Supply chain established MRL 9 Mfg Processes Proven Overall Mfg Process Operates At target Quality, Cost and Lead times All key Processes Meet process Control Targets MRL 10 Highest Production Readiness System in Production Or Meets Engineering Performance & Reliability Overall Mfg Process Operates At 6-Sigma Quality, and Meets Cost and Lead times Estimates AB C MRL Definitions

16. 16 MRL Evaluation Criteria Technology and Industrial Base Design Materials Cost and Funding Process Capability and Control Quality Management Manufacturing Personnel Facilities Manufacturing Management

17. 17 Evaluating MRLs The Best Way to Ensure Manufacturing Readiness: Know what Processes are Being Used for Manufacturing and Know that these Manufacturing Processes are Capable and Controlled (Stable)

18. 18 Purpose and Background Manufacturing Readiness Levels  IPT Tools for Evaluating MRLs Manufacturing Readiness Assessment Process Assessment Example Discussion Session Outline

19. 19 MRL Tools Use to Evaluate… 3 Process Flow ChartsBasic manufacturing concepts 4 Detailed Process FlowKey manufacturing processes Charts 5 Value Stream MappingMapping the current state and identifying waste 6-10 Value Stream MappingMapping the future state and eliminating waste Mapping Tools for MRL

20. 20 MRL Tools Use to Evaluate… 4-6Key CharacteristicsRequirements and Tolerances 4Process Variables MapWhich Variables to Control 5-9Process CapabilityPredictability of Process Performance 5-9Design of ExperimentsMultiple factors and levels of independent variables 6-9Failure Modes andRisks associated with failures Effects Analysis Process Control Tools for MRL

21. 21 Process Flow Charts Value Stream Maps Key Char.; Variables Mapping Process Capability; DOE FMEA Process & Sources at Deeper Levels & Custom / Tailored Tools MRL 4 Mfg Processes Identified Key Processes Identified Producibility assessment initiated MRL 5 Mfg Processes Develop. Mfg equipment in relative environment Producibility assessment ongoing Cost drivers identified MRL 3 Mfg Concepts Identified MRL 6 Critical Mfg Processes Demo’d Mfg equipment in relevant environment Producibility assessment ongoing Cost drivers analyzed Long lead items identified MRL 7 Prototype Mfg System Mfg processes in validation Producibility improvement underway Trade studies conducted Supply chain validated Long lead plans in place MRL 8 Process Maturity Demo All materials ready for LRIP Mfg processes proven for LRIP Supply chain established MRL 9 Mfg Processes Proven Overall Mfg Process Operates At target Quality, Cost and Lead times All key Processes Meet process Control Targets MRL 10 Lean System Production Meets Engineering Performance & Reliability Overall Mfg Process Operates At 6-Sigma Quality, and Meets Cost and Lead times Estimates AB C Tools for MRL Overlaid on Definitions

22. 22 Manufacturing state of the art defined Manufacturing science initiatives defined 10 9 8 7 6 5 4 1-3 SDD LRIP FRP TD CR PRE- CR Manufacturing technology voids/initiatives identified Critical manufacturing process capability needs defined. Yield/rate issues identified. Manufacturing technology development underway. Process yield/capability for all critical processes evaluated in a production representative environment. Manufacturing processes controlled to 3 sigma level. Yields and rates demonstrated to meet LRIP needs. Improvement ongoing Manufacturing processes controlled to 6 sigma level. Yields and rates meeting rate production needs. Continuous improvement on-going. Process capability demonstrated on SDD articles. Required Manufacturing technology solutions demonstrated. Process yields and capability validated as sufficient to meet LRIP requirements. Required Manufacturing technology validated. MRL Characteristics Process Control and Capability Data Captured at Each Level

23. 23 Process Flow Charts MRL 4 Mfg Processes Identified Key Processes Identified Producibility assessment initiated MRL 5 Mfg Processes Develop. Mfg equipment in relative environment Producibility assessment ongoing Cost drivers identified MRL 3 Mfg Concepts Identified MRL 6 Critical Mfg Processes Demo’d Mfg equipment in relevant environment Producibility assessment ongoing Cost drivers analyzed Long lead items identified MRL 7 Prototype Mfg System Mfg processes in validation Producibility improvement underway Trade studies conducted Supply chain validated Long lead plans in place MRL 8 Process Maturity Demo All materials ready for LRIP Mfg processes proven for LRIP Supply chain established MRL 9 Mfg Processes Proven Overall Mfg Process Operates At target Quality, Cost and Lead times All key Processes Meet process Control Targets MRL 10 Lean System Production Meets Engineering Performance & Reliability Overall Mfg Process Operates At 6-Sigma Quality, and Meets Cost and Lead times Estimates Tools for MRL 3 Data Collection

24. 24 MRL 3 – Manufacturing Concepts Identified Tools & Data that Provide Evidence Identification of current manufacturing concepts or producibility needs based on laboratory studies. Assumed that all corresponding TRL requirements are met for each MRL below. High Level Flow Charts Mfg Concepts Identified MRL 3

25. 25 Things to look for in Flow Charts at this level: Major Processes Identified Any Duplication/Repeated Processes Any Special Processes Any Uncertain/Unknown Processes Material or Design Issues Potential Equipment/Supply Issues Miscommunication/Misinterpretation MRL 3 Evidence in Flow Charts

26. 26 Process Flow Charts Key Char.; Variables Mapping MRL 4 Mfg Processes Identified Key Processes Identified Producibility assessment initiated MRL 5 Mfg Processes Develop. Mfg equipment in relative environment Producibility assessment ongoing Cost drivers identified MRL 3 Mfg Concepts Identified MRL 6 Critical Mfg Processes Demo’d Mfg equipment in relevant environment Producibility assessment ongoing Cost drivers analyzed Long lead items identified MRL 7 Prototype Mfg System Mfg processes in validation Producibility improvement underway Trade studies conducted Supply chain validated Long lead plans in place MRL 8 Process Maturity Demo All materials ready for LRIP Mfg processes proven for LRIP Supply chain established MRL 9 Mfg Processes Proven Overall Mfg Process Operates At target Quality, Cost and Lead times All key Processes Meet process Control Targets MRL 10 Lean System Production Meets Engineering Performance & Reliability Overall Mfg Process Operates At 6-Sigma Quality, and Meets Cost and Lead times Estimates A Tools for MRL 4 Data Collection

27. 27 MRL 4 – Manufacturing Processes In Lab Environment Tools & Data that Provide Evidence Conceptual design completed. Requirement validation underway and there are large numbers of engineering/design changes. Required investments, such as manufacturing technology identified. Component physical and functional interfaces have not been defined. Processes to insure producibility, manufacturability and quality are in place and are sufficient to produce technology demonstrators. Manufacturing cost drivers emerging. Producibility assessments have been initiated. Key technologies at least at TRL 4. More Detailed Flow Charts Key Characteristics Process Variables Map Manufacturing processes identified Key processes identified Producibility assessment initiated MRL 4

28. 28 Things to look for in Flow Charts at this level: Process steps identified Any training needs Duplication/repeated steps Material or design issues Unnecessary work Equipment needs Miscommunication or misinterpretations Undefined steps Technology needs MRL 4 Evidence in Flow Charts

29. 29 Milestone “ A ” Key Manufacturing Considerations Process Capability and Control –Areas where manufacturing technology advancements will be required adequately addressed in Technology Development Phase plans? Design and Technology –Key technologies at least at TRL 4 level of maturity? –Producibility challenges associated with this concept adequately addressed in plans for Technology Development Phase? Industrial Base Capability and Materiel –Industrial base gaps/risks identified for key technologies? –Single sole/source or foreign source dependencies identified? –Risks associated with exotic, critical, unproven or hazardous materials identified? –Strategies in place to mitigate? Costs/Funding and Manufacturing/Quality Management –Funding for manufacturing readiness improvement sufficient to reach MRL 6? –Potential manufacturing cost drivers identified? –Manufacturing and quality strategies adequate to produce technology demonstrators? –Unusual special tooling and test equipment requirements identified? A

30. 30 Process Flow Charts Value Stream Maps Key Char.; Variables Mapping Process Capability; DOE MRL 4 Mfg Processes Identified Key Processes Identified Producibility assessment initiated MRL 5 Mfg Processes Develop. Mfg equipment in relative environment Producibility assessment ongoing Cost drivers identified MRL 3 Mfg Concepts Identified MRL 6 Critical Mfg Processes Demo’d Mfg equipment in relevant environment Producibility assessment ongoing Cost drivers analyzed Long lead items identified MRL 7 Prototype Mfg System Mfg processes in validation Producibility improvement underway Trade studies conducted Supply chain validated Long lead plans in place MRL 8 Process Maturity Demo All materials ready for LRIP Mfg processes proven for LRIP Supply chain established MRL 9 Mfg Processes Proven Overall Mfg Process Operates At target Quality, Cost and Lead times All key Processes Meet process Control Targets MRL 10 Lean System Production Meets Engineering Performance & Reliability Overall Mfg Process Operates At 6-Sigma Quality, and Meets Cost and Lead times Estimates A Tools for MRL 5 Data Collection

31. 31 MRL 5 – Manufacturing Processes In Relevant Environment Tools & Data that Provide Evidence Most initial manufacturing requirements defined and validated (key characteristics) Manufacturing technology initiatives ongoing. Component physical and functional interfaces have not yet been defined. Significant engineering changes Producibility assessments ongoing Key materials, machines and tooling, personnel skills, and inspection and test equipment have been demonstrated in a relevant environment Most manufacturing/quality plans, processes and procedures are in development Cost drivers identified. Key Characteristics Value Stream Map Process Capability Study Design of Experiments Mfg Processes in Development Mfg equipment in relative environment Producibility assessments ongoing Cost drivers identified MRL 5

32. 32 Mapping and analyzing the entire set of activities from raw material to finished product will help you evaluate MRL 5 Final Customer Process, activity, or function Material MRL 5 and Value Stream Analysis

33. 33 More detailed map of the manufacturing process –Creates a baseline against which to measure improvements –Quantifies technical, cycle time and cost drivers Provides baseline data for assessing scale-up –Determines capacity required for expected production rates –Quantifies capacity/resource constraints Defines facilitization/equipment required Often includes supply chain processes Creates a forum for evaluating manufacturing options –Enables the redesign of an improved process Excellent Tool for any IPT VSA for MRL 5 Evidence

34. 34 Product/Product Family Collect Data/Draw Current State Opportunities & Implementation Set Goals/Draw Future State Understand how the process currently operates Design an improved process Define and make changes to realize the Future State Choose a product/family Repeat Value Stream Mapping Process

35. 35 Frequently occurring VSA opportunities: –Manual processing –High scrap/defect or low yield rates –Rework –Long setup times –Scale-up constraints - facility, training (learning curve), equipment, tooling –Product design constraints –Material cost and availability (maturity) –Inventory build-up throughout the process Opportunities/Constraints

36. Shipping I Station 43 90/60/30 day Forecasts Weekly Fax Forward fuselage 1 week 1 hour 7 hours 10 min 0.5 hours 40 min 2.7 hours 40 min 2 hours 20 min 1.0 day Production Lead time Processing Time =8.5 days =170 min Weekly Schedule Station 45 4 pcs/mo 2 shifts I C/O=0 Debur C/T=20 min Uptime=100% 1 day Staging C/O=0 Countersink C/T=40 min Uptime=100% 2 hours C/O=10 minutes Drill C/T=40 min Uptime=80% 2.7 hours C/O=10 minutes Locate C/T=10 min Uptime=100% 0.5 hours C/O= 1 hour Prep C/T=1 hour Uptime=85% 7 hours 11111 6-week Forecast Production Control MRP Forward Fuselage Current State Value Stream Map Example (Hole Drilling) Station 44

37. 37 People Equipment Material Methods Measurement Environment Information People Equipment Material Methods Measurement Environment Information Inputs Outputs Controls Process Control & Capability

38. 38 1.The features or mechanisms that control the execution of a Process (including process initiation, selection of process steps, selection of alternative steps, iteration of steps within a loop, and process termination). 2.Controlling mechanisms that ensure that a Process is conducted to maximum cost- effectiveness (including entry criteria, formal procedure specifications, and exit criteria). Degree to which a process is defined, formalized and followed will help to predict the rate of Variation and potential Defects. Process Control Defined

39. 39 Process Control monitors the extent to which our products meet specifications. Two "enemies" of product quality: –(1) deviations from target specifications –(2) excessive variability around target specifications Designed Experiments are used during the earlier stages of developing the production process to optimize these two quality characteristics Quality Control Methods are on-line or in- process quality control procedures to monitor an on-going production process. Process Control and Readiness

40. 40 “In control“ or stable process behavior is consistent over the time when compared against itself. Process capability compares the process output against a specification that assumes defect-free production Good capability needs stability ("in control") first. If the process is stable, you can compare its actual performance against the required performance and take corrective action. Example: If the process is stable but not capable you can predict the level of scrap. You also know where you are and where you need to go for acceptability. If the process is not stable, then you don't know what you will get, where you are, and where to steer, except that you need to stabilize the process first. Process Capability

41. 41 A process can be considered ‘stable’ if the observed average behavior and the observed variance is ‘consistent’ and ‘predictable’ over time. An unstable process is caused by ‘unusual’ variation. The output of an unstable process is ‘unpredictable’. Process Stability

42. 42 Purpose of capability studies –to define process capability –to help identify limiting causes –to demonstrate capability to customers –to improve process capability reduce defects, waste, cost, customer returns undertake higher spec. work –to employ statistical process controls Capability measured by Cp /Cpk Capability Studies

43. 43 Assessing Process Capability Compare the variability of the process output to the desired tolerance range You may also include a comparison between the process average and some nominal or target value Process Capability on a pilot process prior to ramp-up will show whether actual performance meets or exceeds required performance

44. 44 What is a good Cpk ratio? –Minimum normally 1.33 Cpk Based on 4 sigma spread Extra sigma compensates for –Larger spread over time & larger population –Particularly mean shift Equivalent to 63 DPM –Many companies now looking for 2.0 Cpk Consistent with 6 sigma concept Equivalent to 0 DPM –Based on centered process –Allowing up to 2 sigma shift Capability Studies

45. 45 When analyzing a process, experiments are often used to evaluate which process inputs have a significant impact on the process output, and what the target level of those inputs should be to achieve a desired result (output). Experiments can be designed in many different ways to collect this information. Design of Experiments (DOE)

46. 46 The Design of an experiment addresses the questions outlined by your organization by stipulating the following: –The factors to be tested –The levels of those factors –The structure and layout of experimental runs or conditions. A well-designed experiment is as simple as possible - obtaining the required information in a cost effective and reproducible manner. DOE Guidelines

47. 47 Process Flow Charts Value Stream Maps Key Char.; Variables Mapping Process Capability; DOE FMEA MRL 4 Mfg Processes Identified Key Processes Identified Producibility assessment initiated MRL 5 Mfg Processes Develop. Mfg equipment in relative environment Producibility assessment ongoing Cost drivers identified MRL 3 Mfg Concepts Identified MRL 6 Critical Mfg Processes Demo’d Mfg equipment in relevant environment Producibility assessment ongoing Cost drivers analyzed Long lead items identified MRL 7 Prototype Mfg System Mfg processes in validation Producibility improvement underway Trade studies conducted Supply chain validated Long lead plans in place MRL 8 Process Maturity Demo All materials ready for LRIP Mfg processes proven for LRIP Supply chain established MRL 9 Mfg Processes Proven Overall Mfg Process Operates At target Quality, Cost and Lead times All key Processes Meet process Control Targets MRL 10 Lean System Production Meets Engineering Performance & Reliability Overall Mfg Process Operates At 6-Sigma Quality, and Meets Cost and Lead times Estimates A B Tools for MRL 6 Data Collection

48. 48 MRL 6 – Manufacturing Processes In Production Representative Environment Tools & Data that Provide Evidence Majority of manufacturing requirements have been preliminarily defined and validated, to include key characteristics, but there are still significant engineering/design changes. Preliminary design of critical components completed. Component physical and functional interfaces have not yet been defined. Materials, machines and tooling, personnel skills, and Inspection and test equipment have been demonstrated in a relevant environment but most manufacturing processes and procedures are in development (or manufacturing initiatives ongoing). Producibility assessments ongoing. Production cost drivers/goals analyzed. System level DTC goals set. Long lead needs and key supply chain elements identified. Key Characteristics Value Stream Maps and Analyses Process Capability Study Design of Experiments Failure Modes & Effects Analysis Critical Mfg Processes Demo’d Mfg equipment in relevant environment Producibility assessment ongoing Cost drivers analyzed Long lead items identified MRL 6

49. 49 Consider the demand requirements for scale-up Draw the ideal process –Steps, equipment, manning Identify the constraints to realizing the ideal state Quantify the constraints and the expected impact of removing them –Time, cost, etc. Determine capacity required for expected production rates Define facilities and equipment required Include key suppliers Developing the Future State VSA

50. Station 44 Shipping I Station 43 90/60/30 day Forecasts Weekly Fax Forward fuselage 1 day 60 min.5 day Production Lead time Processing Time =1.6 days =140 min Weekly Schedule Station 45 30 pcs/mo 2 shifts I C/O=0 Inspect C/T=20 min Uptime=100%.5 day Staging C/O=0 Drill, Countersink, debur C/T=60 min Uptime=100% 45 min C/O= 1 hour Inspect, Prep, & Locate C/T=60 min Uptime=85% 45 min 13 1 6-week Forecast Production Control MRP Forward Fuselage Future State Value Stream Map Example (Hole Drilling) 45 min 60 min 45 min 20 min

51. 51 1.Identify all probable failure modes 2.Assign a value on a 1-10 scale for the Severity, Probability of Occurrence, and Probability of Detection for each of the potential failure modes. 3.After assigning a value, the three numbers for each failure mode are multiplied together to yield a Risk Priority Number (RPN). 4.The RPN becomes a priority value to rank the failure modes with the highest number demanding the most urgent improvement activity. 5.Error-proofing, or “poka-yoke” actions are often an effective response to high RPN's. How to Conduct FMEA

52. 52 Milestone “ B ” Key Manufacturing Considerations Process Capability and Control - Demonstrated yields and capabilities of key processes support the inclusion of essential technologies or design features in system design? - Essential manufacturing technology development well advanced? Design and Technology - Key technologies at least at TRL 6 level of maturity? - Known producibility challenges associated with the chosen technologies and basic design features of this system being addressed? - Is there an effective process planned or in place for assuring producibility of design? Industrial Base Capability and Materiel - Sufficient industrial base capability to support manufacture of SDD articles? - Effective plans in place to address Long-lead materials Sole or foreign source issues special handling issues Maturity of new materials Costs/Funding and Manufacturing/Quality Management - Funding and planning for manufacturing readiness improvement sufficient to reach MRL 8? - Credible production costs goals set and effective plans in place to control/reduce production costs? - Key special tooling and test equipment demonstrated in production representative environment? B

53. 53 Process Flow Charts Value Stream Maps Key Char.; Variables Mapping Process Capability; DOE FMEA Process & Sources at Deeper Levels MRL 4 Mfg Processes Identified Key Processes Identified Producibility assessment initiated MRL 5 Mfg Processes Develop. Mfg equipment in relative environment Producibility assessment ongoing Cost drivers identified MRL 3 Mfg Concepts Identified MRL 6 Critical Mfg Processes Demo’d Mfg equipment in relevant environment Producibility assessment ongoing Cost drivers analyzed Long lead items identified MRL 7 Prototype Mfg System Mfg processes in validation Producibility improvement underway Trade studies conducted Supply chain validated Long lead plans in place MRL 8 Process Maturity Demo All materials ready for LRIP Mfg processes proven for LRIP Supply chain established MRL 9 Mfg Processes Proven Overall Mfg Process Operates At target Quality, Cost and Lead times All key Processes Meet process Control Targets MRL 10 Lean System Production Meets Engineering Performance & Reliability Overall Mfg Process Operates At 6-Sigma Quality, and Meets Cost and Lead times Estimates AB Tools for MRL 7 Data Collection

54. 54 MRL 7 – Manufacturing Processes Maturing for LRIP Tools & Data that Provide Evidence Manufacturing processes in final validation test Engineering/design changes decreasing. System transitioned to formal configuration control Physical and functional interfaces clearly defined Initial producibility improvements underway. Producibility risk assessments and trade studies ongoing. All raw materials and components are fully understood, in production and available to meet planned LRIP schedule. Supply chain being validated Long lead readiness plans in place. DTC estimates and detailed production estimates being established Manufacturing/quality procedures in final validation tests Key Characteristics Value Stream Maps and Analyses Process Capability Study Design of Experiments Failure Modes and Effects Analysis Prototype Mfg System Mfg processes in validation Producibility improvement underway Trade studies conducted Supply chain validated Long lead plans in place MRL 7

55. 55 Process Flow Charts Value Stream Maps Key Char.; Variables Mapping Process Capability; DOE FMEA Process & Sources at Deeper Levels MRL 4 Mfg Processes Identified Key Processes Identified Producibility assessment initiated MRL 5 Mfg Processes Develop. Mfg equipment in relative environment Producibility assessment ongoing Cost drivers identified MRL 3 Mfg Concepts Identified MRL 6 Critical Mfg Processes Demo’d Mfg equipment in relevant environment Producibility assessment ongoing Cost drivers analyzed Long lead items identified MRL 7 Prototype Mfg System Mfg processes in validation Producibility improvement underway Trade studies conducted Supply chain validated Long lead plans in place MRL 8 Process Maturity Demo All materials ready for LRIP Mfg processes proven for LRIP Supply chain established MRL 9 Mfg Processes Proven Overall Mfg Process Operates At target Quality, Cost and Lead times All key Processes Meet process Control Targets MRL 10 Lean System Production Meets Engineering Performance & Reliability Overall Mfg Process Operates At 6-Sigma Quality, and Meets Cost and Lead times Estimates AB C Tools for MRL 8 Data Collection

56. 56 MRL 8 – Manufacturing Processes In Place for LRIP Tools & Data that Provide Evidence Design sufficiently stable to enter into low rate initial production. Physical and functional interfaces clearly defined. All materials are in production and available to meet planned LRIP schedule. Manufacturing and quality processes and procedures have been proven, are under control and ready for low rate initial production. Initial producibility risk assessments completed. Production cost estimates meet DTC goals. Supply chain established and stable. Key technologies at least at TRL 8. Key Characteristics Value Stream Maps and Analyses Process Capability Study Design of Experiments Failure Modes and Effects Analysis Process Maturity Demo All materials ready for LRIP Mfg processes proven for LRIP Supply chain established MRL 8

57. 57 Milestone “ C ” Key Manufacturing Considerations Process Capability and Control Critical manufacturing processes mature enough to produce production- representative test articles during LRIP? Key program manufacturing processes mature enough to support LRIP cost and schedule requirements? Key manufacturing technology advancements complete? Design and Technology Key technologies at least TRL 8? Design stable enough such that: -Items produced in LRIP will be production-representative? -LRIP schedule can be met? Effective trade studies been performed to assure proper balance between producibility (cost) and performance? Ability to produce all design “key characteristics” with existing processes been verified in a production-representative environment? Industrial Base Capability and Materiel Sufficient industrial base capability in place to support production of LRIP articles? Long-lead materials and sole/foreign source materiel risks adequately managed to support LRIP cost and schedule? New materials been proven to be mature enough to support LRIP and be representative of production? Costs/Funding and Manufacturing/Quality Management Funding and planning for manufacturing readiness improvement sufficient to reach MRL 9? Production cost estimates meet design to cost goals? Special tooling, test equipment and key manufacturing quality procedures validated and in place to support LRIP? C

58. 58 Process Flow Charts Value Stream Maps Key Char.; Variables Mapping Process Capability; DOE FMEA Process & Sources at Deeper Levels & Custom / Tailored Tools MRL 4 Mfg Processes Identified Key Processes Identified Producibility assessment initiated MRL 5 Mfg Processes Develop. Mfg equipment in relative environment Producibility assessment ongoing Cost drivers identified MRL 3 Mfg Concepts Identified MRL 6 Critical Mfg Processes Demo’d Mfg equipment in relevant environment Producibility assessment ongoing Cost drivers analyzed Long lead items identified MRL 7 Prototype Mfg System Mfg processes in validation Producibility improvement underway Trade studies conducted Supply chain validated Long lead plans in place MRL 8 Process Maturity Demo All materials ready for LRIP Mfg processes proven for LRIP Supply chain established MRL 9 Mfg Processes Proven Overall Mfg Process Operates At target Quality, Cost and Lead times All key Processes Meet process Control Targets MRL 10 Lean System Production Meets Engineering Performance & Reliability Overall Mfg Process Operates At 6-Sigma Quality, and Meets Cost and Lead times Estimates AB C Tools for MRL 9 Data Collection

59. 59 MRL 9 – Manufacturing Processes In Place for FRP Tools & Data that Provide Evidence During LRIP all systems engineering/design requirements are met and there are only minimal system engineering/design changes. Materials are in production and available to meet planned production schedules. Manufacturing processes and procedures are established and controlled in production to three- sigma or some other appropriate quality level. Personnel, machines, tooling and inspection and test equipment deliver three-sigma or some other appropriate quality level in production. Production risk monitoring is ongoing. LRIP actual costs meet estimates Key Characteristics Value Stream Maps and Analyses Process Capability Study Design of Experiments Failure Modes and Effects Analysis Mfg Processes Proven Overall Mfg Process Operates At target Quality, Cost and Lead times All key Processes Meet process Control Targets MRL 9

60. 60 Process Capability and Control –All key program manufacturing processes mature enough to support rate production cost and schedule requirements? Design and Technology –Key technologies at TRL 9? –Design been proven to be producible and stable? Industrial Base Capability and Materiel –Sufficient industrial base capability in place to support program FRP needs? –Long-lead materials and sole or foreign source materials issues do not pose a significant risk to FRP cost or schedule? –New materials fully mature? (controlled to specification) Costs/Funding and Manufacturing/Quality Management –Funding and planning for manufacturing readiness sufficient to reach MRL 10? –LRIP actual costs support production cost estimates? –Special tooling, test equipment plans adequate to support FRP? –Quality and manufacturing procedures adequate for FRP? Manufacturing Considerations for Full Rate Production Decision (MRL 9)

61. 61 Process Flow Charts Value Stream Maps Key Char.; Variables Mapping Process Capability; DOE FMEA Process & Sources at Deeper Levels & Custom / Tailored Tools MRL 4 Mfg Processes Identified Key Processes Identified Producibility assessment initiated MRL 5 Mfg Processes Develop. Mfg equipment in relative environment Producibility assessment ongoing Cost drivers identified MRL 3 Mfg Concepts Identified MRL 6 Critical Mfg Processes Demo’d Mfg equipment in relevant environment Producibility assessment ongoing Cost drivers analyzed Long lead items identified MRL 7 Prototype Mfg System Mfg processes in validation Producibility improvement underway Trade studies conducted Supply chain validated Long lead plans in place MRL 8 Process Maturity Demo All materials ready for LRIP Mfg processes proven for LRIP Supply chain established MRL 9 Mfg Processes Proven Overall Mfg Process Operates At target Quality, Cost and Lead times All key Processes Meet process Control Targets MRL 10 Lean System Production Meets Engineering Performance & Reliability Overall Mfg Process Operates At 6-Sigma Quality, and Meets Cost and Lead times Estimates AB C Tools for MRL 10 Data Collection

62. 62 MRL 10 – Lean Manufacturing Processes Tools & Data that Provide Evidence This is the highest level of production readiness. There are minimal engineering/design changes. System, component or item is in production or has been produced meeting all engineering, performance, quality and reliability requirements. All materials, manufacturing processes and procedures, inspection and test equipment, controlled in production to six-sigma or some other appropriate quality level in production. A proven, affordable product able to meet required schedule. Production actual costs meet estimates Key Characteristics Value Stream Maps and Analyses Process Capability Study Design of Experiments Failure Modes and Effects Analysis Lean System Production Meets Engineering Performance & Reliability Overall Mfg Process Operates At 6-Sigma Quality, and Meets Cost and Lead times Estimates MRL 10

63. 63 Purpose and Background Manufacturing Readiness Levels IPT Tools for Evaluating MRLs  Manufacturing Readiness Assessment Process Assessment Example Discussion Session Outline

64. 64 INTRODUCE TRAIN ASSESS MANAGE INCORPORATE Meet with AFRL ATD PM to get buy in and gather program info Customize MRL approach for ATD Train ATD IPT on manufacturing tools to support manufacturing maturity efforts Current MRL and TRL level Final MRL/TRL if current plan is followed (no MRL) Plan, actions, and costs to get them to MRL 5 Incorporate MRL into ATD analogous to TRL Manage overall process Manage risk identification and reduction process Manage manufacturing maturity to graduation Approach

65. 65 Assessment Lead Briefs PM on Manufacturing Assessment Efforts/Expectations Works with PM to Determine appropriate level for Manufacturing Readiness Assessment(s) (MRAs) -- System may contain several critical technologies Schedule on-site MRA with contractor(s) Send Orientation Package to contractor(s) Define Assessment Team Membership Define Deliverables of Assessment Results Conduct on-site assessment with contractor(s) Deliver final report MRL Assessment Process

66. 66 Select the company(ies) for review –Highest manufacturing risk –Highest manufacturing value added Notify companies and send orientation package –Purpose, approach, questions, strawman agenda –Address contract issues if any Select Assessment team(s) –Typically 2-6 members per team – Appropriate members (include Gov’t customer) Specialists for key technologies (if needed) Trainees (First MRA experience should not be in leading one) –May include contractor responsible product design decisions Schedule On-site assessments –Months prior to key milestone decisions to establish a baseline and allow time to develop/implement risk mitigation plans –Balance team and contractor convenience Team Orientation – Meet prior to on-site assessment –Review the logistics; the process; special issues –Review basic member responsibilities Preparations

67. 67 Introduce team and highlight specific roles –Purpose of assessment and how results will be used –Assessment process –Expectations Contractor will: –Introduce contractor participants – highlight responsibilities –Provide detailed responses to questions provided in orientation package –Indicate who is available to work with team counterparts in later meetings for detailed discussions Based on team member needs, follow-up meetings with contractor counterparts will be arranged –Assessment team members need to be ready to identify their needs/desires –Team members meet with contractor counter parts Have detailed discussions (status, plans, risks, issues) Review documents See shop-floor status Prepare feedback for contractor and identify any action items –Each team member shares insights Initial assessment of current MRL (their area or overall) Key strengths/risks/issues Key missing data (if any) Proposed action items Team lead presents feedback briefing for contractor On-Site Activity

68. 68 Gather key missing data Convene team meeting -- Typically within 2 weeks of on-site assessment –Discuss and finalize assessment –Examine current program and manufacturing risk reduction plans –Agree on likely MRL level at completion of milestone if current plan is followed Share results with contractor Identify the specific risk reduction activities necessary to reach the next milestone Identify the funding, time-phasing and approach to carrying out each activity Prepare and submit final report Follow-on Activities

69. 69 MRA Deliverables Provide briefing and/or written report Identify current MRL Identify key factors where manufacturing readiness falls short of target MRL –Define driving issues Identify programs and plans to reach target MRL Assess type and significance of risk to cost, schedule or performance Assess effectiveness of mitigation plans –Address right issues? –Timely? Adequately funded? –Probability of success? –Options for increased effectiveness Gather additional information if requested

70. 70 Purpose and Background Manufacturing Readiness Levels IPT Tools for Evaluating MRLs  Manufacturing Readiness Assessment Process  Assessment Example Discussion Session Outline

71. 71 An Assessment Example

72. 72 Shredder CONOPS

73. 73 Shredder Features

74. 74 Shredder Components

75. 75 Burst Tube Remmele Engineering Inc.

76. 76 REI Burst Tube Assembly Process Flow

77. 77 Weld BTF per AWS spec REI BTF Weldment; Cone to Mechanical Tube Control Process

78. 78 REI Burst Tube Flare (BTF) Part Preparation and Control Process

79. 79 Send BT SubAssy to Heat Treat Receive BT SubAssy from Heat Treat REI Burst Tube Sub-Assembly Control Process

80. 80 REI Finish Weld BT SubAssy to Modified ACP Process Control

81. 81 REI Adds RTV-11 to Start Bomb Body Integration Process

82. 82 REI Integrates BT Sub-Assy into Bomb Body & prepares for Finish Weld Process

83. 83 BLU-109 Bomb Body/Aft Closure Plate - Elwood National Forge

84. 84 REI Bomb Body Integration Process Flow

85. 85 White Phosphorous fill takes place in a water tank. The bomb assembly is submerged, filling the WP chamber with water. WP is piped in, forcing out the water. Excess WP is aspirated Fill port plug is pressed in place and sealed Fill Port 100% Leak Proof Tested for Safety White Phosphorus Fill Process (US Army Pine Bluff Arsenal)

86. 86 PBA Manufacturability - WP Fill Process Control Flow

87. 87 Tool DescriptionQtyTool Casting Stands4Adapt Casting Arrangement4New Handling Rings4New Mold Disassy Equip1Adapt Horizontal Lifting Lug1 setNew Horizontal Transport Skid4Adapt Weight & CG Fixture1Adapt Aft Closure Spanner Wrench1New X-Ray Rotating Cart1Adapt Fuse Cup Spanner Wrench1New Spreader Bar1As Is Casting Stand Receive/Unpack Cases Store Containers for Packout Set unit on Cart Attach Handling Ring Apply MIL-C-450C Liner to Interior Set Up Casting Pit Mold Assembly Cast Explosive Cure Explosive Test Explosive Properties Remove Tooling Set on Cart Stencil, Paint, Touchup Final Assembly X-Ray Unit Packout Store, Ship Mix Explosive Prepare Ingredients BLDG 356BLDG 378 BLDG 356 BLDG 308 BLDG 362 BLDG 356 BLDG 360 BLDG 356 BLDG 173 Research Labs BLDG 311 BLDG 373 HE Loading – ATK Tactical Systems/Allegheny Ballistic Lab.

88. 88 Loaded Shredder Warheads

89. 89 Fuse and FZU – COTS

90. 90 Interface Connector, Pigtail, & Bracket similar to BLU-116 Approach FZU cable mated to connector on bracket Flange for attaching conduit to BLU-109 body Pigtail and Connector for FMU-143 Bomb Fuze Modified Hardback External Fuze/FZU/ Aircraft Interfaces

91. Manufacturing Readiness Assessment Fabrication & Assembly Remmele Engineering Inc. (Big Lake, MN) Processes defined and sequencing established Iterative improvements documented Current capacity adequate, potential constraints known White Phosphorus Fill Army (Pine Bluff Arsenal, AR) Uses established process (under water fill) Current capacity limited to 1 unit/day High Explosive Loading ATK Tactical Systems (Rocket Center, WV) Processes steps defined Iterative improvements documented Process to apply new MIL-C-450C asphaltic liner material needs to mature Production mock-ups to identify planned improvements Remaining Items Developing yield data (processes represent current state-of-the-practice) Planning for tooling/fixtures, if necessary MRL 10 Lean Mfg Proc’s MRL 9 Mfg Proc’s In Place for FRP MRL 8 Mfg Proc’s In Place for LRIP MRL 7 Mfg Proc’s Maturing for LRIP MRL 6 Mfg Proc’s In Prod Rep Environmt MRL 5 Mfg Proc’s In Relevant Environmt MRL 4 Mfg Proc’s In Lab Environmt MRL 3 Mfg Concepts Identified Desired at completion of ATD

92. 92 Purpose and Background Manufacturing Readiness Levels IPT Tools for Evaluating MRLs Manufacturing Readiness Assessment Process Assessment Example Discussion Session Outline

93. 93 Backup BACKUP

94. 94 Scope –Primary focus in areas where new technology being transitioned –Will interview DCMA and review Company and PM management metrics/reports to see if extra effort required anywhere else –Any areas of special interest to SAF/AQR AQR notifies AFRL/MLM –Program information (Name, POC, Approaching Milestone, Special considerations, MRA due date) ManTech –Assemble team –Contact program – get/give orientation –Review program management metrics –Conduct DCMA interviews –Participate as observer in SPO or contractor reviews if applicable –Conduct on-site reviews at key sites (OEM, key suppliers) –Prepare findings Review findings with PM –Assure errors of fact, if present, are corrected –No surprises Provide results to SAF/AQR Basic MRA Process

95. 95 Provide briefing and written report to SAF/AQR Identify key factors where manufacturing readiness falls short of target MRL. Define driving issues Assess type and significance of risk to cost, schedule or performance (Nature and estimated scope of consequences if issues not effectively mitigated) Assess effectiveness of mitigation plans –Address right issues? –Timely? Adequately funded? –Probability of success? –Options for increased effectiveness Gather additional information if requested MRA Deliverables

96. 96 Commercial product developers can’t afford to be late to market or miss their product cost target Industry adopted concurrent engineering principles to consider manufacturing early in the design cycle Industry Commercial Model Manufacturing risk assessment, mitigation and capability development Technology DevelopmentProduct DevelopmentProduction Concurrent Engineering Concepts

97. 97 AFRL Technology Development Models Current AFRL Model Manufacturing risk assessment, mitigation and capability development Technology DevelopmentProduct DevelopmentProduction Future AFRL Model Technology DevelopmentProduct DevelopmentProduction Manufacturing risk assessment, mitigation and capability development AFRL fundedOther funded Co-funded Industry Commercial Model Manufacturing risk assessment, mitigation and capability development Technology DevelopmentProduct DevelopmentProduction

98. 98 Air Force fundedOther funded Current Perception of Air Force Acquisition Production Technology Development Product Development Manufacturing risk assessment, mitigation and capability development Acquisition Development Models Industry Commercial Model Manufacturing risk assessment, mitigation and capability development Technology DevelopmentProduct DevelopmentProduction