Lusine Sarkisian Automated Software Module Testing for FAA Certification

Outline Introduction Tools and testing process FAA Certification testing TSE Process Benefits of TSE Limitations of TSE Summary

Introduction Automated module testing process for software written in Ada using Test Set Editor (TSE) TSE consists of set of programs that works in combination with Microsoft Excel spreadsheet program and home grown test scripts written in Tcl/tk. Automation aims to achieve both reduced costs and a more thorough analysis, testing and verification.

Tools and testing process TSE helps to achieve shorter turnaround time and cut cost during regression testing. TSE is a multi-purpose test tool set designed for use with Ada. TSE includes tools To identify inputs/outputs of the module Building a driver program to apply a set of test cases to the module Measuring coverage

FAA Certification testing Testing safety-critical software begins with a set of requirements-based cases that should also achieve structural coverage. Three levels of structural coverage: statement coverage Decision coverage Modified condition-decision coverage (MCDC)

TSE Process Identifying inputs/outputs of a module Selecting test cases and tagging them with requirements Constructing a test driver and stubs for subprograms called by the module under test Compiling and running the tests Analyzing coverage

Example

Test results Test results

Report

Benefits of TSE TSE doesn’t require to be an expert programmer in Ada. Analyzes the source code and identifies inputs/outputs of a given module Coverage information is provided two ways: tabular form, HTML form Easy to maintain

Benefits of TSE (cont…) Self-verifying Repeatable Domain-independent Only requirement is to create test cases in spreadsheet file.

Limitations of TSE It is not possible to construct one generic module test to verify all instantiations. TSE driver is not suitable for testing an independent thread. Difficult for private type parameters. TSE can’t test subprograms nested deeper as independent modules.

Summary As the quality of software is becoming increasingly important, the testing of that software becomes crucial for ensuring high quality for safety-critical software. TSE is used in the structural testing of FAA certified commercial avionics software as well as the qualification of tools used in the production of such software.