1. Dr. Thomas D. Fiorino November 2002
Engineering Manufacturing Readiness Levels (EMRLs) Fundamentals, Criteria and Metrics
Dr. Thomas D. Fiorino
November 2002

2. Purpose
Information brief on the fundamentals, criteria and metrics for Engineering Manufacturing Readiness Levels (EMRLs)

3. Need for EMRLs
Successful products require the capture of design and manufacturing knowledge early in product development
DoD acquisition policy does not require the capture of design and manufacturing knowledge or sufficiently specific criteria/metrics to enter System Integration and Demonstration (milestone B) and Production (milestone C) phases
EMRLs provide the framework with specific criteria and metrics to capture the design and manufacturing knowledge for product development, demonstration and production

4. EMRL DEFINITION
EMRL 1 - System, component or item validation in initial relevant engineering application/bread board, brass board development. Product Design/Definition phase. Technologies must have matured to at least TRL 4 or 5. Exit Criteria CKP1 (Milestone B) indicates readiness for Product Integration Phase. This is the lowest level of production readiness. Technologies must have matured to at least TRL 4 or 5. At this point all requirements have not been finalized and there are significant engineering/design changes. Component physical and functional interfaces have not been completely defined. Manufacturing processes and procedures and inspection and test equipment are in development. Overall quality and reliability levels and key characteristics initially identified, but not established. Design to Cost goals identified.
EMRL 2 - System, component or item in prototype demonstration beyond bread board, brass board development. Product Integration Phase. Technologies must have matured to at least TRL 6 or 7. Exit Criteria CKP2 (includes successful completion of Product CDR) indicates readiness for Product Demonstration Phase.
Technologies must have matured to at least TRL Systems engineering requirements are validated and defined. However, there are still many systems engineering/design changes and physical and functional interfaces are not yet fully defined. Materials are initially demonstrated in production and manufacturing processes and procedures are initially demonstrated. Investments required for machines and tooling are identified. Inspection and test equipment tested in a manufacturing environment. Quality and reliability levels and key characteristics initially identified. Design to Cost goals validated.

5. EMRL DEFINITION
EMRL 3 - System, component or item in advanced development. Product Demonstration Phase. Technologies must have matured to at least TRL 8. Exit Criteria CKP3 (Milestone C) indicates readiness for low rate initial production. Technologies must have matured to at least TRL 8. At this point engineering/design changes have decreased significantly. Physical and functional interfaces have been clearly defined. All materials are in production and readily available. Manufacturing processes and procedures should be well understood, under control and ready for low rate initial production. Machines and tooling should have completed demonstration in a production environment. All quality and reliability requirements and key characteristics should be identified though not necessarily fully capable or in control. Tooling, STE and facilities meet LRIP requirements. Design to Cost goals met for LRIP.
EMRL 4 - Similar system, component or item previously produced or in production. Or, the system, component or item is in low rate initial production. Technologies must have matured to at least TRL 9. Exit Criteria CKP4 indicates readiness for full rate production. Technologies must have matured to at least TRL 9. All systems engineering/design requirements should be met and there should be only minimal systems engineering/design changes. Materials are fully characterized, in production and readily available. Manufacturing processes and procedures are established and controlled in production to three-sigma or some other appropriate quality level. All key characteristics are controlled to three-sigma or some other appropriate quality level in production. Tooling, STE and facilities meet FRP requirements. Design to Cost goals met for FRP.
EMRL 5 - System, component or item is in full rate production with continuous product/process improvement. This is the highest level of production readiness. There are essentially no engineering/design changes. All materials, manufacturing processes and procedures, inspection and test equipment, quality, reliability and key characteristics controlled in production to six-sigma or some other appropriate quality level in production. Continuous improvement

6. EMRLs and Capability Knowledge Points (CKPs)
EMRLs describe the basic level of knowledge and capability required for a given acquisition phase. Product manager continuously assesses progress against EMRL criteria
CKPs are the exit criteria and metrics for each EMRL and indicate the degree of product maturity or readiness for entry into the next acquisition phase

7. EMRLs and Acquisition Milestones
FRP
Continuous
Improvement
Milestone C
LRIP
EMRL 5
CKP 4
Begin System Demonstration
EMRL 4
CKP 3
CDR
Begin System Integration
EMRL 3
Milestone B
CKP 2
EMRL 2
CKP 1
System Definition
EMRL 1

8. EMRL Metrics and Exit Criteria
System EMRL 1
Capability Knowledge Point 1
Milestone B
System EMRL 2
Capability Knowledge Point 2
System CDR
System EMRL 3
Capability Knowledge Point 3
Milestone C
System TRL=4 or 5-9
All system engineering/design requirements
defined; 50% validated
Component physical and functional interfaces
50% defined at system level
Materials, machines and tooling demonstrated
in a lab environment (system)
Manufacturing process and system level
integration demonstrated
Quality and reliability levels identified and
established for 50% of the system
Key characteristics identified for 50% of the
system
PDR complete at system level
90% of items and components for system/major
Subsystem are proven designs or in production;
EMRL 2-3 (hardware and software) and CDR complete
75% of major subsystems (those representing 80%
of cost) meet requirements of CKP2 and complete CDR
Failure modes and effects analysis required for all
levels
Funding identified to proceed to CKP2
Schedule and funding profile reflects achievement
of CKP2 in 1-3 years
Developmental Test Plans complete for 75% of major subsystems
Developmental test plans initiated at system level
Design to cost goals established
System TRL=6 or 7-9
All system engineering/design requirements
defined; 90% validated
Component physical and functional interfaces
90% defined at system level
System level manufacturing process and Inte-gration established in a production environment
Materials, machines and tooling established
in a production environment
Quality and reliability levels identified and
established for 90% of the system
Key characteristics identified for 90% of the
system
CDR complete at system level
All items and components in production and meet
cost, quality and reliability targets
75% of major subsystems representing 80% of
cost meet the requirements of CKP3 and are ready for or in LRIP
Failure modes and effects analysis complete on
all items and components, 75% complete at system level
Funding identified to proceed to CKP3
Schedule and funding profile reflects achievement
of CKP3 in 1-3 years
Item/components at EMRL3, CKP3
Developmental Test complete for major subsystems
Developmental Test Plans complete at System level
Design to cost goals validated
System TRL=8-9
All system engineering/design requirements
defined and validated
Minimal engineering changes
All Component physical and functional interfaces defined and validated at system level
All manufacturing processes are understood and
in control for LRIP
All Machines, tooling and test equipment purchased and ready for LRIP
in a production environment
All Quality and reliability levels established and
verified
All Key characteristics identified and verified
Follow up issues from CDR cleared
All items and components (hardware and software) are ready for on-time delivery (or delivered)
All items and components meet cost, quality and
reliability targets
Failure modes and effects analysis complete complete at system level
Reliability growth plan in place at all levels
Funding identified for reliability growth
All Reliability growth testing is in place
Funding identified for LRIP
All Developmental Test complete
All IOT&E Plans complete
Design to cost goals met for LRIP

9. Metrics Reporting and Roll Up
Color coded success/failure for each question/requirement
Meet 100% of the requirement (Green)
Requirement not met, requirement will be met within the current schedule and cost (Yellow)
Requirement not met, meeting requirement will impact current schedule or cost (Red)
Metrics are rolled up from lower level of WBS to higher level(s)
Each metric will be assigned a color based on these criteria. Emphasis should be made on the fact that the colors are based on impact to schedule. Metrics for each item, component, subsystem, etc. will be rolled up as shown on the next page.
Stress that red in this case is an indicator of POTENTIAL risk. An assessment will have to be done for any metric that is red.

10. Metric Roll up chart System Other Subsystems Subsystem Other
Components
Component
Other
Components
The roll up is a simple percentage from the lower level. For instance, if there are 15 metrics and 3 are red then 1/5 of the box for the component will be red. If the sum of all the components indicates 15% of all metrics are red then 15% of the subsystem box will be red, and so on.
Key point is that no matter what level of the WBS you present EMRLs at, you never lose track of metrics that are red.
A typical report out would have a WBS type chart similar to above with backup charts for each red item. Backups for yellow items are generally not needed since yellow indicates that the metric will be met within the course of program execution.
Metric 1
Metric 2
Metric 3
Metric 4
Metric 5

11. EMRLs and WBS
EMRLs can be tailored to apply at the item, component, subsystem and system levels
Each lower WBS product should be at a higher level EMRL (level of readiness) than the higher level product, i.e., if the missile has met the exit criteria for Milestone B, the seeker should be at or near CDR and the focal plane should be ready for LRIP.
EMRL 1
Missile
EMRL 1-2
Seeker
EMRL 2-3
FPA

12. Backups

13. TRL Definitions Technology Readiness Level Description
1. Basic Principles observed and reported.
Lowest Level of Technology Readiness. Scientific research begins to be translated into applied research and development. Examples might include paper studies of a technology’s basic properties.
2. Technology concept and/or application formulated.
Invention begins. Once basic principles are observed, practical applications can be invented. Applications are speculative and there is no proof or detailed analysis to support the assumptions. Examples are limited to analytic studies.
3. Analytical and experimental critical functions and/or characteristic proof of concept.
Active research and development is initiated. This includes analytical studies and laboratory studies to physically validate analytical predictions of separate elements of the technology. Examples include components that are not yet integrated or representative.
4. Component and/or breadboard validation in laboratory environment.
Basic technological components are integrated to establish that they will work together. This is relatively “low fidelity” compared to the eventual system. Examples include integration of “ad hoc” hardware in laboratory.
5. Component and/or breadboard validation in relevant environment.
Fetidity of breadboard technology increases significantly. The basic technological components are integrated with reasonably realistic supporting elements so that it can be tested in a simulated environment. Examples include “high fidelity” laboratory integration of components.
6. System/subsystem model or prototype demonstration in a relevant environment.
Representative model or prototype system, which is well beyond TRL 5, is tested in a relevant environment. Represents a major step up in technology’s demonstrated readiness. Examples include testing a prototype in a high fidelity laboratory environment, or in a simulated operational environment.
7. System prototype demonstration in an operational environment.
Prototype near, or at, planned operational system. Represents a major step up from TRL 6, requiring demonstration of an actual system prototype in an operational environment, such as an aircraft, vehicle, or space. Examples include testing the prototype in a test bed aircraft.
8. Actual system completed and “flight qualified” through test and demonstration.
Technology has been proven to work in its final form and under expected conditions. In almost all cases, TRL represents the end of the true system development. Examples include developmental test and evaluation of the system in its intended weapon system to determine if it meets design specifications.
9. Actual system “flight proven” through successful mission operations.
Actual Application of the technology in its final form and under mission conditions, such as those encountered in operational test and evaluation. In almost all cases, this is the end of the last “bug fixing” aspects of system development. Examples include using the system under operational mission conditions.

14. THE ‘Newer’ 5000 Model C This is new! Process entry at Milestones A,
Technology Opportunities &
B, or C (or within phases)
User Needs
“Entrance criteria” met before l
entering phase
(Program C A B
Initiation)
IOC
FOC
System Development
Production &
Concept &
Operations &
Technology
& Demonstration
Deployment
This is new!
Support
Development
Critical
FRP
Design
OT&E
Decision
Review
Review
Pre -
Systems
Systems Acquisition
Sustainment
Acquisition
All validated by
BLOCK 2
Requirements Authority
MNS
ORD
or PSA
BLOCK 3
Relationship to Requirements Process