1. Johann Schumann and Pramod Gupta NASA Ames Research Center schumann@email.arc.nasa.gov pgupta@email.arc.nasa.gov Bayesian Verification & Validation tools for adaptive systems

2. Motivation for NN V&V Basis for Case Study I: Neuro-adaptive control (IFCS Gen-II) Network “learns” to compensate for deviations between plant and model Previous work: SW V&V process for NN-based control “Confidence tool” for dynamic monitoring The major obstacle to the deployment of adaptive and autonomous systems is being able to verify their correct operation – In Flight Fixed gain controllers cannot deal with catastrophic changes or degradation in plant Adaptive systems (e.g., NN) can react to unexpected situations through learning Relevance and potential: IFCS NN controlled aircraft (F-15, C-17) UAV control Space exploration Any safety-critical application of NN control

3. V&V Issues & our Approach Our approach combines mathematical analysis, intelligent validation, and dynamic monitoring and supports specific software V&V process, targets multiple aspects and phases of V&V of adaptive control systems, and uses a unique combination of research in –Neural Networks –Control Theory –Numerical Methods –Bayesian Statistics Verification: how to specify an unforseen event? Validation: not possible to test all configurations While traditional V&V methods will remain useful, these methods alone are insufficient to verify and certify adaptive control systems for use in safety-critical applications

4. Our Bayesian Approach How good is the network performing at the moment? Traditional: NN as a Black Box Here: Look at probability distribution of the NN output Variance (confidence measure) depends on: –How well is the network trained? –How close are we to “well-known” areas Large variance = bad estimate; no reliable result, just a guess Small variance = good estimate Our approach, based on a Bayesian approach, provides a measure of how well the neural network is performing at the moment

5. Milestone I: Envelope Tool Basis: Adaptive NN-based controller Lyapunov error bound defines regions of eventual stability Regions where confidence is small might cause instability Informally: a safe envelope is a region where the confidence level is sufficiently high Bayesian approach combined with sensitivity analysis Challenge: methods for efficient determination of safe envelope Can help answer questions like How large is the current safe envelope? How far is the operational point from the edge? Current status: mathematical background formulated, prototypical Matlab/Simulink implementation designed, first simulation experiments

6. Confidence Envelope Confidence Surface Safety Envelope: area of good confidence airspeed The Envelope tool uses a Bayesian Approach to calculate the current safety envelope 1/confidence good bad altitude

7. Conclusions & next steps Current work as scheduled toward deliverable (9/2004) prototypical implementation in Matlab/Simulink report on mathematical background and tool Getting Case Study I ready: IFCS Gen-II simulink model Next steps in research: system identification (sysID): estimate confidence of parameters other model representations (e.g., parameter tables with polynomial interpretation) Preparation of Case Study II and III