Flight Critical Systems Software Certification Initiative A Presentation to SAE Aerospace Control and Guidance Systems Committee, Meeting 95 2 March 05 Salt Lake City, UT Mr. David Homan, Technical Area Leader Control Systems Development Branch Air Vehicles Directorate Air Force Research Laboratory Telephone: (937)

Outline Scope of the the Flight Critical V&V Problem Flight Critical System Software Initiative (FCSSI): Philosophy Strategy Programmatics

Background: Flight Safety and Manned/Unmanned Functional Migration Flight Critical Mission Critical Manned Aircraft Unmanned Aircraft Flight Mgmt Vehicle Mgmt Mission Mgmt Vehicle Mgmt On-board Off-board On-board Off-board Pilot is Integrator and Contingency Manager; FMS is mostly advisory. Flight Mgmt FMS and VMS provide Integration and Contingency Mgmt; Operator manages at high-level. Situational awareness Situational awareness? For UAVs, “Pilot Function” becomes huge design and V&V issue

Background: V&V Requirements Flight Critical Mission Critical System Focus is Performance/Security Performance Metric: Throughput and Bandwidth [event driven] Assurance Metric: Probability of Mission Success [Simplex or Back- up] Confidence Rqmt: Performance and security are validated. Consequence of Failure: Potential mission failure System Focus is Performance/Assurance Performance Metric: Sampling Rate and Latency [time triggered] Assurance Metric: Probability of Loss of Control and N x Fail Op/Fail Safe [Triplex or Quad] Confidence Rqmt: Performance and Assurance must be validated; [Failure Modes and Effects Testing] Consequence of Failure: Loss of Aircraft, potential loss of life Rule of Thumb: When you mix mission with flight criticality, the testing is held to most stringent requirement. Consequence of Failure: Loss of Aircraft, potential loss of life Developmental Timeline: Flight Critical ready by First Flight! Any changes requires Total Re-test! Flight Critical V&V isn’t just a software issue, it’s a system issue!! Failure Modes and Effects Testing

Flight/Safety Critical System Attributes Performance Capability to perform the function Quantitative Metrics: throughput, bandwidth, latency etc Assurance Capability to sustain performance during mission Quantitative Metrics: PLOC, Fail-Op rqmt, etc Confidence Capability to know that performance and assurance will execute correctly during mission. Qualitative Metric: collection of evidence that shows that the system is operating correctly Performance Assurance Confidence It’s not enough to design a flight critical system; there must be proof that it works… Confidence must be integrated into the system design to reduce the need for testing! Confidence needs some quantitativeness!

If FCV&V isn’t hard enough… New Capabilities Present New Challenges (Complexities) to V&V problem. Mixed Criticality Architecture: Non-obtrusive co-existence of mixed criticality Adaptive/Learning/Multi-Modal Functions: Indeterminate or untraceable functionality Mixed Initiative/Authority Mgmt: Human/autonomy or autonomy/autonomy interactions Multi-Entity Systems: Functions that encompass multiple platforms. Sensor Fusion/Integration: Highly confident sensor-derived information Are these new systems/capabilities affordably provable? $$

Mixed Criticality Challenge How can we separate the mission and flight critical functionality as to guarantee safety? SOA: Middleware that provides time/space partitioning (ARINC 653). Issue: Both Criticalities use common HW resources (i.e. processors, backplanes, busses etc); how do we determine PLOC and fault tolerance? Understand failure mechanisms for partitioning Non-critical function must not take out shared resources…Or the probability of its occurrence is predictable… Need guarantee on fault tolerance A A A B B C backplanes Serial bus Processors X X X Answer may reside in a SW/HW architecture specifically designed for mixed operation

Adaptive/Learning/Multimodal Challenge How can we trust functionality that we may not be able to fully test? SOA: We must try to test the complete functional envelope (till $$ runs out…)! Issue: Some new Control capabilities are untraceable and/or non-deterministic Adaptive systems Huge test space Perfect Input data Learning systems Environmental stimuli Lost memory Multi-modal systems Mode transition stability Mode synchronization Recovery mode? Answer may reside in bounding the function in run- time to known safe behavior.

Mixed Initiative Challenge How can man and autonomy safely interact? SOA: Human operator always get authority! Issue: Human operator may not have all the information or be able to comprehend situation in real-time: Situational Awareness versus Response Time Assessment of UAV mode/state/health Assessment of surrounding environment “Consequence of mishap” is a factor Complete system health is a factor Workload is a factor Answer may reside in a authority management specification that would allow the correct party to have decision authority. AF Poster Child: Auto-Aerial Refueling (AAR)

Multi-Entity Challenge How can trust systems with multiple players to safely perform cooperative functions? SOA: Keep humans away and hope for the best… Issue: Entities participating in the coordinated function may not be part of individual V&V testing: Linked Interface Control Documents? Entities with different manufacturers? System Configuration Management? Mission-specific programming? Answer may reside in a specification for contingency management, based on system degradation

High Confidence Sensing Challenge How can we trust visual/radar systems for flight critical functions? SOA: Brute force and analytic redundancy Issue: Mission-style sensors don’t have acceptable real-time methods for FDIR… Sensors will likely be multi-function! Redundant HW may not be answer, redundant information? Built-in-test may not provide good real-time coverage. Reliable signal processing/sensor fusion software Answer may reside in sensor designs that compensate for sensor degradation and plan for contingencies

Flight Critical Systems Software Initiative Understand the Problem V&V for Intelligent Adaptive Control Systems Develop a research agenda Establish Confidence as a research discipline DoD and National participation We have to define an Evolutionary Process R&D Component S&T research feeds V&V/Certification will never go away! Let’s plan for it!

DOD & FWV Level Workshop (R&D Focus) Near-Term Program Continuing S&T Investment 5 yr Continuing CRAD/IRAD R&D Investment Team A Team B Near-Term Program Team N Invite “Big Three” Airframers to discuss their ideas/approaches Boeing Lockheed Martin Northrop Grumman Develop 5 year Kick-Start R&D Program Multiple Awards (Cooperative Agreement) Collaborative Teams Govt Airframer Vendor/Suppliers Academia Programmatic Strategy This is not a “One-Shot Wonder! This feeds an evolutionary process with continuing S&T research! DoD Participants: AFRL/AFOSR ASC DARPA NAVAIR/ONR Army AATD Workshop Product New Requirement DoD buys big weapons system from these folks… Common Process with Proprietary Implementations

National Level Workshop (S&T Focus) National Attention for a Nationally Crucial Issue! Rallying S&T Community to Certification Cause! DARPA FAANSF NASA Workshop Status: Planning for NOV/DEC 05 timeframe; DC location Planning Meeting in August 05 “Research Needs for Flight/Safety Critical Systems” Workshop Product High Confidence Software and Systems Coordinating Group National Coordination Office for Information Tech R&D

Flight Critical System Software Initiative Capability Focused Tech Investment War Winning Upgrades for Today’s Platforms Superior Technologies for Future Aerospace Dominance

Flight Critical System Software Initiative To be effective assets in the force structure and mission plans, UAV’s must … Be Safe & Reliable Be Responsive & Effective Be Interoperable Not Adversely Effect Operations Capability CAO Background

Flight Critical System Software Initiative CAO Technology Goals Mixed Manned/Unmanned Teams UAV In-situ Decision Making Transparent Airspace Ops Adaptive Software V&V Reliable Unmanned Ops Same Base, Same Time, Same Tempo Based on JUCAS ICD, SAB Summer Study, Global Hawk ORD, OSD UAV Roadmap, Predator CCD

Flight Critical System Software Initiative Cooperative Airspace Ops CFTI Taxonomy Cooperative Airspace Ops Operations in manned/unmanned teams Attributes Product Safe operations from airbases and in airspace J-UCAS 4-ship flight management Multi-UAV distributed control V&V Of flight critical intelligent software Terminal area & ground ops Health mgmt integ w/adaptive control Open architecture, highly reliable VMS HALE UAV detect and avoid Non-GPS nav, landing, and ground ops Multi-vehicle see and avoid Other VA Capability’s Products Capability’s Products Other Org’s Products Future Capability

Flight Critical System Software Initiative VVIACS Objectives Enabling Technology for Certification of Emerging Intelligent & Adaptive Vehicle/Mission Management Systems Establishes Emerging Control Systems and Associate Emerging Fundamental Properties Identifies Dominant Certification Drivers for ECS and EFP Develops Certification Metrics and R&D Critical Paths

Flight Critical System Software Initiative Impact Analysis Results Significant Cost Increase Projected Primarily Due to V&V SW and Test Single-Vehicle ECS Increase V&V Costs ~2X Multiple-Vehicle ECS Increase V&V Costs ~3X Software: Single-Vehicle 100% Increase, Multiple-Vehicle 200% Increase Test: Single-Vehicle 150% Increase, Multiple-Vehicle 250% Increase V&V Costs 54% - Baseline 62% - Single-Vehicle 68% - Multiple-Vehicle

Flight Critical System Software Initiative TASS SBIR Efforts Barron Associates – Run-time monitor to check system behavior with fail-safe controller for recovery – demonstration based on UAV flight control. EDAptive – Specification and Requirements languages along with “formal methods” to streamline V&V process. Scientific Monitoring – Model-based software development used to generate and test flight critical code. WW Technology – create an embedded “fault detector” and formal transformation of control system to fail-safe control system, includes partitioning using a middleware approach.

Flight Critical System Software Initiative Certification Techniques for Advanced Flight Control Systems – CerTA FCS CerTA FCS Unique R&D push for Technology Breakthrough in Systems Certification -Certification Process Paradigms -V&V Methods/Techniques Innovations Reduced Life Cycle Costs Higher Assurance Levels for Advanced Flight Critical Software Enabling Technology  Intelligent, Adaptive, Reconfigurable  Real time Prognostics Health Info  Autonomous Operations  Multi-Vehicle Coordination Cooperative Airspace Ops Operations in manned/unmanned teams V&V Of Intelligent Software TAD: 11 VA Initial Contribution Towards Flight Critical Systems Software Initiative

Flight Critical System Software Initiative VVIACS SBIR DARPA PCES, MoBIES, SEC Flight Critical Systems Certification Initiative Technologies for Affordable & Safe Software Development (TASS) 4 SBIR Ph I V&V of Intell & Adapt Control Systems (VVIACS) 6.2 Projected TASS SBIR Ph II Current Investment Roadmap to FY10 FY04FY05FY06FY07FY08FY09PriorFY03 FY10 Runtime Monitors Specifications & Rqmts s/w w/ formal methods VA Buy Plan VA Executing Scientific Monitoring Multi-Agency Executing Barron EDAptive Model-Base s/w Development W/W Tech Middleware Fault Detection & Isolation s/w Tech Certification Techniques for Advanced FCS - CerTAFCS Integrated S/W Environment V&V for Distributed Embedded Systems (AFOSR MURI) Planned Execution

Flight Critical System Software Initiative Summary Innovations Required in Systems Certification to Enable Future Functionality of UAVs in CAO Environment VA has Established Technology Investment Area and VVIACS Study to Determine Near-Term R&D Investments SBIR Program Provides Tech Seedlings VA Investments Geared to Support Collaboration Control S&T Community “Buy In” Required NOW for Affordable and Safe Certification Practices for TOMORROW