1. 1 Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin CPS V&V I&F workshop, December 11th, 2014

2. 2 Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin Verifying Pre-Existing Systems ! Verified idealized system System that actually runs on the airplane

3. 3 Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin Next-Generation Airborne Collision Avoidance System (ACAS X) Industrial system developed by the FAA replacing TCAS Designed to prevent collisions between aircraft Based on optimizing a Markov Decision Process to create a big table (several millions of entries) that is then interpolated to make decisions at runtime

4. 4 Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin COC DNC DND DES1500 CL1500 COC Next-Generation Airborne Collision Avoidance System (ACAS X) Only vertical advisories are allowed Separation property based on a puck Table in 7 dimensions with millions of entries How do we verify such a huge table?

5. 5 Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin COC DNC DND DES1500 CL1500 COC ACAS X Verification with KeYmaera ① For each action, identify a region where it is safe ② Formally prove in KeYmaera that the safe regions are correct ③ Compare the safe regions with the ACAS X decision table safe CL1500 CL1500

6. 6 Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin Computing the Safe Region: for a Climbing RA 1 2 3 4 parabola at acceleration straight up at target vertical velocity half parabola horizontal of width straight up at target vertical velocity

7. 7 Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin Comparison: ACAS X issues CL1500 Initial advisory begins to induce NMAC

8. 8 Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin Challenge: Verifying Pre-Existing Systems Solution 1: Verify the system directly Problem: its design is often ill-suited for verification Solution 2: Show that the system is subsumed by a more general, verified system Problem: we need to identify this more general system Solution 3: …

9. 9 Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin Challenge: Modeling Uncertainties Uncertainty due to uncertain parameters or unpredictable events: wind, component faults… Sensor uncertainty: sensors are never perfect, they only give values within a certain margin of error

10. 10 Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin Challenge: Human in the Loop Airplanes have pilots who follow precise procedures: in theory their behavior is easy to model However it is difficult to quantify the behavior of a human (reaction times, minimum performance,…) What about modeling reaction to unusual or stressful events?

11. 11 Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin Challenge: Numerical issues A computer cannot effectively perform real number computations Instead, computers use floats How do we transfer a proof using exact-precision real numbers to a system using limited-precision floats?

12. 12 Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin Challenge: Scalability and Automation Aerospace systems are big systems Natural approach is to verify a simplified system How do we make sure a proof on a simplified system still applies to the complete system? At some point, systems are too big and intractable for manual proofs: need proof automation

13. 13 Challenges in the Verification of Pre-Existing Aerospace Systems Jean-Baptiste Jeannin Challenges and Conclusion To bridge the gap between verified systems and implemented systems, we need to be able to: –Verify complex systems –Verify pre-existing systems To make our proofs more applicable, we need to take into account: –Uncertainties of parameters and sensors –Humans in the Loop –Numerical Issues –Scalability and Automation