1. Copyright © 2012 Boeing. All rights reserved. Effective Verification and Validation Testing Steve Holt Boeing Commercial Airplanes August 2013

2. Copyright © 2012 Boeing. All rights reserved. Initiatives Were Put In Place to Address:  Incomplete or unnecessary test requirements  Immature test capability  Inefficient test conduct  Success found in:  Application of systems engineering tools and methods  Teaming across design and test functions early in the design cycle  Test engineering participation in requirements and V&V development  Questioning why past requirements are needed  Questioning current methods of verification  Focusing on problematic design features and complex integration areas  Optimizing affordability to include risk mitigation (cost avoidance) as well as cost reduction Effective Verification & Validation Testing Summary - Commercial Airplane Product Development

3. Copyright © 2012 Boeing. All rights reserved. Navigate: Requirements Designers Lab Test Value Stream Flight Test Value Stream Start validating the design here! Inspection Analyses Modeling Simulation Hardware Bench Testing Integrated System Testing Full Scale Integrated Testing Verification Planning Verification & Validation Cycle Commercial Airplane Product Development

4. Copyright © 2012 Boeing. All rights reserved. Verification & Validation Target Early Verification and Risk Retirement Typical Plan Target Analyses Internal Bench Testing Supplier Bench Testing Integrated Systems Laboratory Airplane Flight Test Modeling & Simulation Requirement Verification

5. Copyright © 2012 Boeing. All rights reserved. Verification Tools Educating Design Engineering on Capability Wind Tunnel Bench Tests Full Scale Test Articles Integrated System Testing Modeling & Simulation Culture Shift

6. Copyright © 2012 Boeing. All rights reserved. Requirement Verification Plan Facility Needs Satisfying Data Test Statement of Work Test System Readiness Triggers Integrated Test Plan Data Source Determination Test Optimization Verification & Validation Governance Verification Approach Verification Culture Verification Capture Interactive Development Among Design and Test Engineers Test article readiness ensures early risk mitigation Part of test efficiency initiatives Verification & Validation Development Cross-Functional Verification, Test, and Means of Compliance

7. Copyright © 2012 Boeing. All rights reserved. Requirement Area AnalysesModeling & Simulation Laboratory Testing Integrated System Tests Flight Article Flight LoadsConcurrent High Speed Mach Buffet Multiple Failure Assessment Safety of Flight System Functionality Crew Workload Determination Pilot & Airport Operations Concurrent Spatial Integration Structural & Actuation Loads Structural Interfaces Systems Interfaces Verification Exercise – Folding Wing Tip Example of Earlier Verification and Risk Retirement

8. Copyright © 2012 Boeing. All rights reserved. A culture change is often necessary to qualify “Affordability” as cost optimization, to include investment in early verification and risk mitigation Using test experts to scrutinize design requirements and means of verification leads to a more robust and mature product Shifting verification “to the left” as much as practical helps ensure a mature and cost-optimized product Focusing on test readiness and efficiency throughout the development life cycle helps ensure verification success Complex systems yield emergent effects. Some of them increase risks, some of them increase opportunities. Effective Verification & Validation Testing Conclusions