ARO Workshop Wendy Roll - May 2004 Topic 4: Effects of software certification on the current balance between software/system modeling, analysis and testing

ARO Workshop Wendy Roll - May Context  Traditional approach for certified software development has been to implement the system and then perform rigorous testing and analysis (post-creation) – Safety critical systems are designed to reduce this testing/analysis burden  Newer model-based approaches are intended to reduce errors and support analysis (pre-creation), but…  Certification processes require the same degree of confidence in all evidence used therein – E.g., when less direct methods are to be used for certification:  Any models, modeling tools, analysis tools, etc, must be certified to the same level as the underlying system, and  Fidelity of any system representations must be similarly certified for all applicable aspects (e.g. functionality, timing, concurrency)  Area for future research – For dynamic/distributed/large-scale needs of FCS, certification will have to be reevaluated  Research into different SoS certification definition and implementation  Research into the development of software that can meet the newly defined criteria Certify?

ARO Workshop Wendy Roll - May Question 1  …What techniques are available which compellingly reduce the aggregate certification effort?  Possible areas for future research – Automated test  Generation – Including status and health monitoring collection to create inputs for test  Running - with limited configurations – COTS/Reuse  Focus on barriers to trust/assurance efforts from commercial marketplace (e.g. new business models, incentives) – Languages/Patterns that only allow safe design and/or complete analysis (E.g. SCADE/Luster)

ARO Workshop Wendy Roll - May Question 2  …What unique approaches, if any, can more efficiently certify tools and representations than the system itself?  Areas for future research – Using extensive modeling and simulation to address safety  What is the “language” for this model?  How can this be made cost efficient?  How does the model become trusted? – Certification of a process along with tool use

ARO Workshop Wendy Roll - May Question 3  …What aspects are most amenable to these approaches and techniques? – Can techniques address unique challenges resulting from mobile ad hoc networking or other system of system dynamics?  Areas for future research – How do we handle certified software that must have a dependency on an unreliable assets (network, nodes, other software components)?  New strategies to address the issue of “communication towers” being mobile, mortal soldiers - Proactive approach - Using alternate situation monitoring sources to determine failure - using the network to your advantage – Limit dynamism, but support some dynamism  Set of static choices  Assure no harm is done  Certify adaptation mechanisms – Association of certification/reliability needs with acceptable implementations  Includes characterizing those implementations

ARO Workshop Wendy Roll - May Question 4  …Are these approaches and techniques scalable to the size of FCS?  Areas of future research (might be in the context of other research areas listed) – Amount of software – Certification of individual systems does not guarantee safety of SoS – Certification of complete set of dynamic behaviors – How will solutions fit into existing development models (waterfall, spiral…) – Current certification process only addresses a subset of these issues