1 West Virginia University FY2001 University SOFTWARE INITIATIVE PROPOSAL for the NASA SOFTWARE IV&V FACILITY Initiative Title: Verification & Validation of UML Dynamic Specifications '01 Hany H. Ammar, Vittorio Cortellessa,Rania Elnaggar Lane Department of Computer Science and Electrical Engineering West Virginia University The OSMA Software Assurance Symposium September 5-7, 2001 Lakeview, Morgantown, WV Verification & Validation of UML Dynamic Specifications '01

2 West Virginia University outline Project Overview Overview, The Environment Timing Analysis Methodology The Pacemaker Example Performance Analysis Methodology The Auto teller machines Example Fault Injection Model HCS NASA case study Conclusions

3 West Virginia University Project Overview Develop an Environment for verification of performance and timing behavior of real-time applications based on dynamic specifications in UML (first year) Develop a methodology for timing analysis (first year) Develop a methodology for performance analysis (first year) Develop a methodology for fault-injection and failure propagation analysis (second year) Complexity, and Risk Assessment at the architecture level (second year) Apply the above methodologies to a NASA case study

4 West Virginia University Overview, The Environment

5 West Virginia University Component Complexity Factors Connector complexity Factors CDG “ hrf i and hrf ij unidentified” Formatted Excel charts Violation Tables UML Simulation Environment Simulation Settings Simulation Log and Violation Report Analysis Tool Timing Diag. Violation Table Analyst Rose Real Time tool Text File MS Excel Processing Macro Inspection Viewing Macro UML Model Observer Sub Run Settings Analysis Tool HRF MS Excel Risk Macro Excel sheets Severity Ranking Severity Analysis (Failure/Effect analysis) Overview, The Environment CARA Tool

6 West Virginia University Overview,The Environment: RoseRT notation It is essentially a classical software architecture: Capsules  Components Ports + Links  Connectors A1 A3 A2 C1 C2 Capsule ACapsule B Capsule C

7 West Virginia University Overview,The Environment: RoseRT notation Component nesting The internal behavior of each lowest level (primitive) capsule can be modeled by a State Charts The union of all the State Diagrams composes the model that has to be simulated

8 West Virginia University Timing Analysis Methodology

9 West Virginia University Timing Analysis Methodology

10 West Virginia University Example: Pacemaker Main Use Case Diagram

11 West Virginia University Case Study: Pacemaker Capsule Diagram

12 West Virginia University Sample Timing Diagram

13 West Virginia University Performance Analysis Methodology

14 West Virginia University Behind the Approach: RoseRt notation is well suited to also represent the hardware platform Migrating the hardware model into the same notation as the one required from the tool for the software representation and thereafter using the tool to simulate the resulting integrated model

15 West Virginia University The standard scheme Software side capsule Resource side capsule Application software architecture Main Disp CPUs Networks Disks Int Disp CPUi

16 West Virginia University Resource requests Wherever needed in the software side a resource request is originated as a demand vector The size of a demand vector depends on the number of resource types building up the platform Each cell of a demand vector represents the amount of that resource type that the software block requires to be executed (e.g., number of instructions, number of accesses to disk, etc.) Each demand vector is mostly handled, in the resource side, by the Main Dispatcher

17 West Virginia University Example: Automatic Teller Machines Observer Server SoftwareServer Resources ATM Devices ATM Software Balance Tr.AuthenticatorWithdrawal Tr. ATMsATM Resources nn

18 West Virginia University Generating and satisfying resource requests No. of times the Job is queued depends on the amount of recourses required Similarly all other Resource types are consumed

19 West Virginia University ATM resource side configuration examples CPUs: number of instructions per quantum CPUs: time per quantum (msec) Disk: time per block (msec) Exp1: 1 CPU, 1 disk Exp2: 1 CPU, 1 disk Exp3: 2 CPUs, 1 disk 10, , ,2

20 West Virginia University Some performance indices : Exp3

21 West Virginia University A Fault Injection Technique Fault Injection  Develop a Fault Model for UML dynamic specs, to perform severity analysis and test case optimization using the simulation environment

22 West Virginia University State Selection Process State faults Sate transition faults Timing Faults (Presented, Spring Showcase 2000) Fault Injection Fault Model for UML Dynamic SPECs

23 West Virginia University Order Components based on dynamic complexity Select the set of components to be injected with faults based on highest complexities. Order states and macro states in each selected component based on contribution to the component complexity. Fault Injection Fault Model for UML Dynamic Model State Selection Process

24 West Virginia University Swap the selected state with the state next in the complexity order. “state actions code fault” Swap transitions out of the selected state. If an initial state exists, force the selected state to be the initial state. If a final state exists, force the selected state to be the final state. Fault Injection Fault Model for UML Dynamic Model State faults

25 West Virginia University For the Transitions that are firing out of or into the selected state Change trigger message to null (Disable the transition) Interchange trigger message with another randomly selected message Fault Injection Fault Model for UML Dynamic Model State Transition faults

26 West Virginia University Timeouts-based Concurrency-based Performance-based Environmental-interactions Fault Injection Fault Model for UML Dynamic Model Timing faults

27 West Virginia University Hub Control Software (HCS) Case Study International Space Station

28 West Virginia University HCS Internal Architecture HCS ITCS SCITCS FRITCS LRITCS PPA mon PFMC LT PFMC MT Scheduler State manger CMD Queue O/P CMD Queue N3-1 Data Access RPCM N3-2 Data Access Other HCS sub-systems

29 West Virginia University HCS Internal Architecture ITCS: (Spec) SCITCS-> System controller FRITCS-> Fault recovery LRITCS-> Leak recovery PFMC (MT,LT) -> Pump and Fan Motor Controller PPA Monitor -> (Top Level Design) for PPA control in Spec HCS: (Top Level Design) Scheduler -> give 1 Hz interrupt State Manager -> decides if the system is in standby or operating Command Queue -> has the orders for the ITCS ( Trans to single MT,..) O/P Command Queue -> receives the orders the ITCS issues to get to other HCS components N3-1 Data Access -> Has the data of the MT Loop Valve (SFCA MT) N3-2 Data Access -> Has the data of the LT Loop Valve (SFCA LT) RPCM -> ( from Spec) open close certain switches

30 West Virginia University HCS – Observer – Fault Generator

31 West Virginia University RRT Structure Diagram for HCS

32 West Virginia University After a successful pump retry, the requirement document does not specify whether the system should return to the last operation mode (this may cause the system either to deadlock or operate without noticing there is still a problem with running that mode) the FRITCS should reconfigure the system in accordance with the current state (this is a more logical choice, in fact the simulation showed better system performance when doing so). Potential inconsistencies detected in the specs during model development

33 West Virginia University In general, preemption of commands may cause issuing new commands to the PFMC during startup and shutdown operations that are not valid according to the specs. Potential inconsistencies detected in the specs during model development

34 West Virginia University Conclusions Presented a simple methodology and an Environment for Timing and Performance Analysis of dynamic Specifications The methodology is extended to risk assessment and fault-injection analysis Illustrated the methodology using simple generic examples (the pacemaker, and ATMs) Developed a simulation model of a NASA case study (The Hub Control Software HCS, of the International Space station) Appling the methodology to the specification of HCS

35 West Virginia University Papers Ammar H.H., Cortellessa V., Report on the development of an automated simulation environment for UML dynamic specification, March 2001 deliverable. Alaa Ibrahim, Sherif M. Yacoub, Hany H. Ammar, Architectural- Level Risk Analysis for UML Dynamic Specifications, Proceedings of the 9th International Conference on Software Quality Management (SQM2001), Loughborough University, England, April 18-20, 2001, pp Ammar H.H., Cortellessa V., Ibrahim A., Modeling Resources in a UML-based simulative environment, Proc. of ACS/IEEE International Conference on Computer Systems and Applications 2001, June , Beirut (Lebanon). Cortellessa V., Ibrahim A., Ammar H.H., Simulations of distributed systems for performance analysis in UML, submitted to UML 2001 conference.

36 West Virginia University Project current work HCS timing and performance parameter collection Ammar H.H., Cortellessa V., Ibrahim A., Modeling Resources in a UML-based simulative environment, Proc. of ACS/IEEE International Conference on Computer Systems and Applications 2001, June 25-29, Beirut (Lebanon). Cortellessa V., Ibrahim A., Ammar H.H., Simulations of distributed systems for performance analysis in UML, submitted to ISPASS 2001 conference. GSM radio system timing and performance analysis Rania M. Elnaggar, Vittorio Cortellessa, Hany Ammar, A UML- based Architectural Model for Timing and Performance Analyses of GSM Radio Subsystem, 5 th World Multi-Conference on Systemics, Cybernetics and Informatics

37 West Virginia University Complexity and risk assessment Alaa Ibrahim, Sherif M. Yacoub, Hany H. Ammar, Architectural-Level Risk Analysis for UML Dynamic Specifications, Proceedings of the 9th International Conference on Software Quality Management (SQM2001), Loughborough University, England, April 18-20, 2001, pp Fault injection and Fault propagation Alaa Ibrahim., H. H. Ammar, S. Yacoub A Fault Model for Fault Injection analysis of UML Dynamic Specifications, accepted to ISSRE 2001 conference. Future Work

38 West Virginia University Integrating timing and performance to build a risk assessment approach based on the performance sensitivity of the risk factors to changes in the architecture or to fault recovery routines Semi-formal approach to identify high risk scenarios B. Cukic, H. Ammar and K. Lateef, Identifying High-Risk Scenarios of Complex Systems using Input Domain Partitioning, Proc. of ISSRE 98 Hybrid simulation/analytical validation approach Paper submitted to NASA Goddard SW Engineering Workshop, November 27-29, 2001 Development of an Analytical V&V approach Future directions