Research Heaven, West Virginia Verification and Validation of Adaptive Systems Online Failure Detection and Identification for IFCS through Statistical Learning Bojan Cukic, Yan Liu, Srikanth Gururajan West Virginia University NASA OSMA SAS '04

Research Heaven, West Virginia 2 Problem  Adaptive systems play an increasingly important role.  Software V&V poses a major obstacle impeding the widespread use of such systems.  Abnormal data patterns have a significant impact on the system behavior.  Independent and reliable detection and identification of such conditions are vital and necessary as part of V&V of adaptive systems.

Research Heaven, West Virginia 3 Approach Investigate the role of modern AI techniques (Support Vector Machines) in failure detection and identification. –Failure Detection Designing a fast (real-time) SVDD algorithm, to detect failure conditions –Failure Identification In off-nominal flight conditions, failure results in loss of symmetry. Significant couplings between longitudinal and lateral dynamics become highly probable. These failures can be identified by studying the correlation between certain longitudinal and lateral dynamics parameters. Validate the methodology in extensive simulations.

Research Heaven, West Virginia 4 Importance/benefits  Developing a methodology suitable for independent assessment of developer’s claims on anomaly (failure) handling.  Unique and significant improvements of SVDD algorithm,  Space/time complexities have been highly reduced allowing for real time anomaly detection.  Automated correlation analysis provides accurate results as an independently developed online failure identification tool.  Approaches can be generalized for other online adaptive (flight control) systems.

Research Heaven, West Virginia 5 Relevance to NASA Verification and validation of adaptive systems is feasible. –We have been able to develop a good understanding adaptive flight control systems very well. Significant lessons learned. –Scientific, technological, organizational. –In collaboration with the ISR, Inc., these methodologies will be included in IV&V guidebooks and best practice documents. Interest to utilize this research in upcoming projects at Dryden/Ames. –Involve IV&V facility as a future partner.

Research Heaven, West Virginia 6 Accomplishments Development and experimental evaluation of real- time, robust novelty detection algorithms. Development and maintenance of a high-fidelity WVU/NASA F-15 flight simulator. –Extensive use in evaluation. Demonstrating that the developed techniques and tools are suitable for IV&V practice. Importance of a multidisciplinary research approach. –Participation of investigators/students from 4 different departments.

Research Heaven, West Virginia 7 Accomplishments - Failure Detection

Research Heaven, West Virginia 8 Accomplishments - Failure Identification

Research Heaven, West Virginia 9 Next Steps  Continue with rigorous evaluation and performance tuning.  Explore different parts of the flight envelop.  Build an SVDD database for online failure detection.  Analyze/improve the performance (false positives).  Embed the SVDD tools and cross-correlation analysis into the IFCS simulation environment for testing and demonstration.  Technology transfer.