1. ACAT FRRP Automatic Collision Avoidance Technology Fighter Risk Reduction Program Copyright 2009 Lockheed Martin Corporation. Test Execution Lessons Learned 88ABW-2088ABW-2010-1914 10-1914

4. Cost of Not Fielding Auto GCAS USAF Fighter ACAT preventable losses since Auto-GCAS Maturity 19 Mar 00CFITFatalityF-16 destroyed 28 Aug 00CFITFatalityF-16 destroyed 12 Jun 01Spatial-DFatalityF-16 destroyed 06 Jul 01G-LOCFatalityF-16 destroyed 17 Jul 01Spatial-D2xFatalF-16 destroyed 09 Sep 02Spatial-DFatalityF-16 destroyed 13 Nov 02Spatial-DFatalityF-16 destroyed 09 Sep 03CFITEjectionF-16 destroyed 14 Sep 03CFITEjectionF-16 destroyed 05 Apr 06G-LOCEjectionF-16 destroyed 27 Nov 06CFITFatalityF-16 destroyed 15 Jun 07Spatial-DFatalityF-16 destroyed 15 Jan 08Spatial-DEjectionF-16 destroyed 14 Mar 08G-LOCFatalityF-16 Destroyed 25 Mar 09CFITFatalityF-22 Destroyed 22 Jun 09CFITFatalityF-16 Destroyed 28 Jul 10CFIT4xFatalC-17 Destroyed

5. Architecture Overview Recovery Request Termination Angle Simplified Terrain Profile Predicted Trajectory Aircraft Trajectory Prediction Terrain Scanning Terrain Map Aircraft State Collision Estimation Flight Control Autopilot Notify Pilot Command Fly-up Coupler Navigation Solution LN-260 EGI 3 Arc Sec SRTM+ F-22, F-35, F-16 Block 40/50 SOFTWARE ONLY DESIGN NO HARDWARE MODIFICATIONS REQUIRED

6. Top-Level Requirements 1.Do No Harm –System Wide Integrity Management Don’t fly Lead into Wingman Don’t exceed operating limits 2.Do Not Interfere –Avoid Unwarranted/Early (Nuisance) Activations 3.Avoid Collisions –Controlled Flight Into Terrain (CFIT) CANNOT SIMPLY AUTOMATE MANUAL GCAS/TAWS/EGPWS

7. Flight Test Execution Two Major Phases: –Collision Avoidance (Protection) Intentional approaches to ground to generate auto recovery –Nuisance evaluation (Interference) Application of Nuisance Criteria to Any auto recovery Operational maneuvers to assess nuisance potential (stress testing) 103 Flights / 141 hours 1670 Fly-ups

9. How High are You? Baro = 24,310’ System = 25,860’ You are 1,550’ higher than you think you are! Radar Altimeter = 22,660’

10. Where be the Ground? LESSON The Test Team MUST Validate the Terrain Assumptions Prior to conducting Collision Avoidance tests

11. Resolution Is there a way to determine DTED resolution requirements other than through trial and error? 6 Arc Sec DTED

12. DTED Resolution Specification Preliminary Data 6 Arc Sec Cell 3” 2” LESSON DTED RESOLUTION REQUIREMENTS CAN BE DETERMINED PRIOR TO FLIGHT TEST 1”

13. Tip Tilt Terrain Encoding F-22 Compressed Digital Terrain Map (CDTM) Under Development LESSON BRUTE FORCE ISN’T ALWAYS THE BEST WAY TO SOLVE A PROBLEM

14. Collision Avoidance Risk RecoveryInitiation Too Late Available Reaction Time Recovery Profile If Delayed Measure of Performance –Available Reaction Time (ART) = Time Safety Margin (TSM) –Normalizes Dependencies on Dive & Speed –Accounts for A/C Performance Minimum ART Required (MARTR) Routine: 4 sec High Risk: 1.5 - 4 sec Prohibited: <1.5 sec

15. Abort Maneuver Delayed 4 Sec

16. Range Abort Tool

17. Alert Levels Normal Flying 2.3 – 4.0 Sec Reaction Time Typical Not Acceptable for High Risk CA “Hyper Alert” <1 sec typical, <0.1 sec possible –Focused on 1 thing –No blinking allowed? –Trend data needed to anticipate –Simple Analysis/Simple Response

19. Mobile Control Room

20. Collision Avoidance

21. Nuisance Eval 2 Methodologies –Nuisance Criteria Applied to CA Runs –Operational Evaluations Identify Mission Elements Identify those most challenging to a GCAS –Low Angle Strafe: Vertical errors & uncertainties –Low Level Terrain Masking: Horizontal errors/Scanning –BFM: Longer fly-ups, Power effects, Nadir /Zenith transitions, Supersonic –AAR / Formation Flight: Midair collision Stress Test these elements with variety of pilots –Ridge Crossings, 210’ flight, Lateral Approaches, LAS to Foul Altitude LESSON NUISANCE EVAL WAS HIGHER RISK THAN COLLISION AVOIDANCE

22. Reducing RISK of Nuisance Eval Training! Crew Rest Mid Day flights (shadows, fatigue) No Tankers (fatigue) Used Auto GCAS as Backup NEVER rely on the Auto GCAS!

23. Nuisance Eval - LAS

24. Cause of Strafing Fly-ups SRTM DTED Error Continue to find holes and noise 50’ 0’ -25’ -50’ 25’ SRTM Difference from Ground (ft) WARNING! Auto GCAS CANNOT be used as a Mission Enhancement Device!!!

25. Fly-up Initiation

26. Fly-up + 10 Frames

27. Fly-up + 50 Frames

28. Darwin vs. Superman

29. Flight Test Summary Collision Avoidance –Trajectory Prediction Algorithm well tuned No ground penetrations observed (Always Conservative) Over Conservatism Corrected Nuisance Evaluation –Normal (High) Protection GLOC & higher altitude missions 500 feet low level missions nuisance free –Minimum (Low) Protection 300 feet low level missions nuisance free in aggressive terrain Air-to-Ground bombing plus Low Angle Strafe

30. Lessons Learned Challenge the Assumptions Risk is not Defined by Test Cards or Processes Stress the System early and often Understand what you know and don’t know…

31. Way Forward USAF F-16 fleet integration 2014 –F-22 operational 2016 –F-35 requirement Analog FLCS Trade Study –All non-digital FLCS F-16s –Other analog FLCS aircraft Android Phone / UAVs Commercial / General Aviation

32. Questions???