1. May 2010Approval of Experimental Aircraft for Instrument Flight APPROVAL OF EXPERIMENTAL AIRCRAFT FOR INSTRUMENT FLIGHT Richard L (Dick) Newman (M) and Benjamin K (Ben) Johnson NAVAIR, Pax River Presented at SETP Flight Test Safety Workshop San Jose, California May 2010

2. Approval of Experimental Aircraft for Instrument Flight OUTLINE Introduction Instrument Flight Approval Pathway to Approval Lessons Learned DISCLAIMER: The material in this presentation is intended to promote safety discussion. The opinions expressed are those of the authors and not those of any organization with whom the authors may be affiliated.

3. May 2010Approval of Experimental Aircraft for Instrument Flight INTRODUCTION Background Criteria Philosophy

4. May 2010Approval of Experimental Aircraft for Instrument Flight INTRODUCTION Background –New Generation of Electronic Displays –Implicit Assumptions –Highly Integrated Avionics Systems Criteria Philosophy

5. May 2010Approval of Experimental Aircraft for Instrument Flight INTRODUCTION Background Criteria –Acceptable Level of Risk General population Other aircraft Own aircraft –Overall Mission Effectiveness Philosophy

6. May 2010Approval of Experimental Aircraft for Instrument Flight INTRODUCTION Background Criteria Philosophy –Test Pilots versus Operational Pilots –Ultimate Tester is the User –Final Operational Testing Should Use Operational Pilots

7. May 2010Approval of Experimental Aircraft for Instrument Flight INSTRUMENT FLIGHT APPROVAL Test using mission scenarios –Can not separate safety performance workload It is important to test the entire display system –Medium (the display itself) –Symbology –Software –Supporting hardware –Sensors

8. May 2010Approval of Experimental Aircraft for Instrument Flight INSTRUMENT FLIGHT APPROVAL Test using mission scenarios –Can not separate safety performance workload It is important to test the entire display system –Medium (the display itself) –Symbology –Software –Hardware –Sensors

9. May 2010Approval of Experimental Aircraft for Instrument Flight STATEMENT OF THE PROBLEM Traditional Instrument Suites Reliable Independent Known failure modes Standard displays Modern Instrument Suites Unknown reliability Highly integrated Uncertain failure modes Novel displays

10. May 2010Approval of Experimental Aircraft for Instrument Flight INSTRUMENT FLIGHT APPROVAL Approach Safety Standards Evaluation Pilots Hazard Assessment Results

11. May 2010Approval of Experimental Aircraft for Instrument Flight INSTRUMENT FLIGHT APPROVAL Approach AFFSA-style endorsement or equivalent –Representative mission tasks –Typical performance standards Safety Standards Evaluation Pilots Hazard Assessment Results

12. May 2010Approval of Experimental Aircraft for Instrument Flight INSTRUMENT FLIGHT APPROVAL Approach Safety Standards Appropriate level of safety –General population –Other aircraft –Aircraft passengers and crew Safety standards vary with aircraft type –Transport vs non-transport –GA, commercial, or military Evaluation Pilots Hazard Assessment Results

13. May 2010Approval of Experimental Aircraft for Instrument Flight INSTRUMENT FLIGHT APPROVAL Approach Safety Standards Evaluation Pilots Eval pilots should represent the users Use appropriate skill levels Include experienced and inexperienced pilots Hazard Assessment Results

14. May 2010Approval of Experimental Aircraft for Instrument Flight INSTRUMENT FLIGHT APPROVAL Approach Safety Standards Evaluation Pilots Hazard Assessment Safety standards vary with aircraft type Include reliability standards Complete software clearance –Removal of error injection routines Results

15. May 2010Approval of Experimental Aircraft for Instrument Flight INSTRUMENT FLIGHT APPROVAL Approach Safety Standards Evaluation Pilots Hazard Assessment Results All test results documented Configuration change history Final report

16. May 2010Approval of Experimental Aircraft for Instrument Flight PATHWAY TO APPROVAL Design Process Supporting Analyses Testing Buildup Interim Approvals Operating in Weather Final Approval

17. May 2010Approval of Experimental Aircraft for Instrument Flight PATHWAY TO APPROVAL

18. May 2010Approval of Experimental Aircraft for Instrument Flight PATHWAY TO APPROVAL Do we have to do these tests in order? Lab tests before sim Sim tests before flight

19. May 2010Approval of Experimental Aircraft for Instrument Flight PATHWAY TO APPROVAL Do we have to do these tests in order? Lab tests before sim Sim tests before flight Of course not, but this approach reduces program risk.

20. May 2010Approval of Experimental Aircraft for Instrument Flight PATHWAY TO APPROVAL Do we need all these preliminary software releases?

21. May 2010Approval of Experimental Aircraft for Instrument Flight PATHWAY TO APPROVAL Do we need all these preliminary software releases? Of course not, but waiting until software is fully tested would lead to unnecessary delays.

22. May 2010Approval of Experimental Aircraft for Instrument Flight TESTING BUILDUP Lab and SIL Tests

23. May 2010Approval of Experimental Aircraft for Instrument Flight TESTING BUILDUP Lab and SIL Tests Do we really need to use a SIL?

24. May 2010Approval of Experimental Aircraft for Instrument Flight TESTING BUILDUP Lab and SIL Tests Do we really need to use a SIL? No, but a SIL would save a lot of $$$$.

25. May 2010Approval of Experimental Aircraft for Instrument Flight TESTING BUILDUP Lab and SIL Tests Do we really need to use a SIL? No, but a SIL would save a lot of $$$$. You can pay me now, or You can pay me later.

26. May 2010Approval of Experimental Aircraft for Instrument Flight TESTING BUILDUP Lab and SIL Tests Simulator Tests Do we really need simulator tests?

27. May 2010Approval of Experimental Aircraft for Instrument Flight TESTING BUILDUP Lab and SIL Tests Do we really need simulator tests? Simulator Tests Again, it reduces risk later in the program.

28. May 2010Approval of Experimental Aircraft for Instrument Flight WHY AN INTERIM APPROVAL Minimize impact on schedule Allow positioning flights Need to test in weather Low visibility testing HUD/HMD testing Ice protection testing

29. May 2010Approval of Experimental Aircraft for Instrument Flight INTERIM APPROVALS Approach Safety Standards Evaluation Pilots Hazard Assessment Reasonable Restrictions

30. May 2010Approval of Experimental Aircraft for Instrument Flight INTERIM APPROVALS Approach Same approach as final approval AFFSA-style protocol Safety Standards Evaluation Pilots Hazard Assessment Reasonable Restrictions

31. May 2010Approval of Experimental Aircraft for Instrument Flight INTERIM APPROVALS Approach Safety Standards Primary emphasis on non test participants –General population –Non-participating aircraft Evaluation Pilots Hazard Assessment Reasonable Restrictions

32. May 2010Approval of Experimental Aircraft for Instrument Flight INTERIM APPROVALS Approach Safety Standards Evaluation Pilots Use project test pilots Hazard Assessment Reasonable Restrictions

33. May 2010Approval of Experimental Aircraft for Instrument Flight INTERIM APPROVALS Approach Safety Standards Evaluation Pilots Hazard Assessment Assessment appropriate for flight testing Don’t wait for full reliability/safety assessment Don’t wait for final software clearance Reasonable Restrictions

34. May 2010Approval of Experimental Aircraft for Instrument Flight INTERIM APPROVALS Approach Safety Standards Evaluation Pilots Hazard Assessment Reasonable Restrictions Use chase aircraft typical Avoid populated areas flight test Avoid high traffic areas restrictions Limit duration in clouds 

35. May 2010Approval of Experimental Aircraft for Instrument Flight LESSONS LEARNED Precede flight test with simulator test Build up Reduce incremental risk Reduce schedule risk Systems Integration Laboratory (SIL) Saving cost of SIL is false economy

36. May 2010Approval of Experimental Aircraft for Instrument Flight LESSONS BEING LEARNED There’s more to approval for flight into weather than evaluating symbology.

37. May 2010Approval of Experimental Aircraft for Instrument Flight LESSONS TO BE LEARNED For prototype airplanes with novel or non- traditional instrument suites

38. May 2010Approval of Experimental Aircraft for Instrument Flight LESSONS TO BE LEARNED For prototype airplanes with novel or non- traditional instrument suites –Consider using a simple “steam gauge” panel for the initial prototypes. –Program risk reduction

39. May 2010Approval of Experimental Aircraft for Instrument Flight LESSONS TO BE LEARNED For prototype airplanes with novel or non- traditional instrument suites –Consider using a simple “steam gauge” panel for the initial prototypes. –Program risk reduction –If there are two pilot stations, consider fitting the second pilot station with “steam gauges”

40. May 2010Approval of Experimental Aircraft for Instrument Flight QUESTIONS?