Software Testing for Safety- Critical Applications Presented by: Ciro Espinosa & Daniel Llauger

Agenda  Introduction  Software and Industrial Automation  Software Testing Strategies Black-Box Testing White-Box Testing  Software Testing Processes and Steps Unit Testing Integration Testing  Top-Down Integration Testing  Bottom-Up Integration Testing  System Testing  Software Performance Testing Availability, Survivability, Flexibility, Stress, Security, Usability and Maintainability Testing  Software Maintenance  Conclusion

Introduction  Safety-critical applications in industrial automation have unique concerns for software quality (demanding requirements on system performance).  Software testing consists in functional and performance testing.

Software and Industrial Automation  The possibility for malfunction of complicated industrial automation software becomes much greater than the traditional one.  Embedded measurement and control systems in life-critical systems have unique concerns for system safety and reliability.

 Any error in the software system is fatal to the real- world applications of industrial automation software  Basic testing principles: Present expected testing results when designing test cases. Separate sw testing team from sw development team. Design invalid test cases. Perform regression testing each time sw under-test is revised. Tester should concentrate on error-prone program segments.

Software Testing Strategies  Black-Box Testing  White-Box Testing  In practical applications, the tester often combines black-box testing and white-box testing to conduct a more thorough test of the software.

Software Testing Processes and Steps  Testing real-time & embedded software is difficult  50 % of overall work in life cycle dedicated to testing and optimizing  Complexity – large number of potential paths  Real-time constraints exacerbate testing

 Unit Testing  Integration Testing: Top-Down Integration

 Bottom-Up Testing:

System testing  Find discrepancies between actual SW performance to its expected performance by comparing it w/ requirements  Test cases created based on requirement analysis description  On-site testing after SW installed with other equipment

Software Performance Testing  Real-time monitoring & control system  Sensor measures monitored values  Manner in which error between actual & target measurement to be corrected determined by microprocessor-based controller  Actuator executes command to drive control variable close to target values.

Availability testing  Applies to time-critical, on-time, & real-time applications  Real-time SW needs to be designed to meet time constraints  Must meet deadline in responding to an operation  Control correctness and real-time restrictions can be considered simultaneously in design process

Availability testing (cont.)  Example: Chemical plant  Alarm MGMT SW responsible for triggering a siren (alarm) when abnormal process occurs  If it cannot perform alarm operation quickly, then disaster or production incident occurs

Reliability testing  Measures the likelihood for failure-free SW operations  Tests ability to operate w/o failure in environment designed for  Capture any abnormal status  Why is this testing needed? Low reliability SW can destroy industrial equipment or have a loss of life To guarantee quality in manufactured products

Survivability Testing  Distributed and embedded real-time software system should be able to recover from the degraded performance.  Fault-tolerant control algorithms should be incorporated into the software design to maintain software performance in the presence of component failures.

Flexibility Testing  In the industrial automation area, the sw needs to work with heterogeneous hardware drivers and software components provided by different manufactures and vendors.  Today, open architecture-based software is gradually replacing traditional, proprietary software architecture

Stress Testing  Stress testing tests the software by pushing the system to its limits.  Stress testing may find both hardware and software defects.  Stress testing can be used to examine what types of system failures will occur when the system is heavily overloaded.

Security Testing  Security is an increasingly important issue in industrial automation software systems, especially with the proliferation of Internet-based industrial applications.  System security needs to be considered during all phases of the software life cycle.

Usability Testing  Usability tests how well the user operates the software system and likes doing so.  It is crucial to design the software with high usability  GUI design is an important component in system usability.

Maintainability Testing  High maintainability enables the released software to be revised in the presence of errors during system operations and makes the software expansion and change easy for new applications.  Maintainability should be considered from the very start of the life cycle.

 For complex software, high maintainability becomes more necessary, as it is hard to identify the faulty lines of code without well-written documentation.

Software Maintenance  Establish safe & well controlled mechanism for update and modification  ID unexposed defects after installation  Adapt to changing user requests & to different operating conditions  Corrective maintenance  Adaptive maintenance  Perfective maintenance  Preventative maintenance

Conclusion  In the testing of industrial automation software systems, performance, flexibility, reliability, survivability, and usability should also be tested thoroughly.  Testing, code inspections and reviews are needed to prevent loss of life or industrial accidents

Questions?