Information Software Systems 18 May 2007 Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems Vlad Volkov Program Manager Model Driven Engineering in modern Software Development Process

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 2 Contents Who we are: short overview of model driven engineering projects at ISS Problem and solution Why verification? Verification strategy Subsequent tests generation What is next?: conclusion

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 3 Who we are: short overview of model driven engineering projects at ISS : first Motorola project on automated requirements capturing conducted by research group from Glushkov Institute of Cybernetics headed b y Alexander Letichevsky and Julia Kapitonova 2000: ISS begins his operations, Motorola locates a project with a goal to create high-quality software product aimed at automated support of the initial phase of software project - - deriving of consistent and complete specifications from initial requirements 2001: In parallel, ISS is requested by Motorola to define and institutionalize SWD practices of SEI CMM Level 3

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 4 Who we are: short overview of model driven engineering projects at ISS (cont.) 2002:first product release of VRS (Verification of Requirements Specifications) tools for checking formal specifications 2003: first successful piloting of VRS tools in industrial projects of Motorola projects 2004: Developed possibilities for automated tests generation with respect to initial MSC specifications : Participation in Motorola projects on automated code C/C++ generation from models written in SDL and UML.

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 5 Software Development Catastrophes* CONFIRM travel information system: $160 million Denver Airport baggage handling: $200 million London Stock Exchanges Taurus: £400 million London Ambulance Service dispatching: £9 million Pentium FDIV bug: $500 million (found with formal analysis!) average schedule slips by 50% 25% of all large systems are cancelled 3/4 of all large systems are operating failures *(from Software Engineering Computer Science Tripos Part IA, Part II (General) Lent Term, 1997,Lawrence C Paulson Computer Laboratory University of Cambridge Copyright c 1997 by Lawrence C. Paulson)

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 6 Inevitable uncertainties and conflicts in the behavioral requirements for large and very large projects cause enormous delays in releasing high quality products and unwanted increase of Cost-Of-Quality Manual analysis of behavioral/functional requirements in large scale projects becomes a stopper Formalization of behavioral requirements in MSC/SDL/UML and their formal verification for consistency and completeness may help to improve their quality at the initial phases of a software project The problem and the solution

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 7 Cost to fix error Requirements/ Design Imple- mentation Testing [B.Boehm] * Barry W.Boehm, "Software Engineering Economics", Prentice-Hall, Inc., 1981, ** Industry post release defects analysis, 2004 Cost of fixing requirements defects increases exponentially with time * Faulty requirements have significant impact ** Requirements Management Requirements Verification Where are the costs incurred? In requirements!

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 8 40%-20%-40% Well known estimate of efforts distribution through software development phases (Design – Coding – Testing) : 40%-20%-40% - Maximum effect of efforts reduction shall be achieved on Design and Testing phases. Main investments into code/test automation and generation should be made on these phases. Now Test automation is a widely accepted practice in industry. Verification of Requirements & Specifications only started to be used.

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 9 1. Increase of product quality due to more defects found 2. Decrease of Cost-Of- Quality due to earlier finding of defects 1. Shorter time to market due to cycle time reduction Defects Time New Old Gain in σ-quality Gain in COQ & CTR Release date Business gains with VRS Conditions of efforts reduction: - Usage of formal engineering languages on the Design stage - Total efforts reduction in SW projects is achieved by reduction of design and testing manual efforts

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 10 Why verification? Program testing can be used to show the presence of bugs, but never to show their absence! (E. W. Dijkstra, 1979)

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 11 Formalization and verification is hidden from the user Input captured in engineering notations: MSC, SDL, UML Specification of system behavior Dynamic properties of the system Library of predefined properties Verifier operates fully automatically UKUSARMTR air_in taxi_in taxi_out air_out Informal Reqmts. (DOORS) Requirement Capture Specs in formal logic MSC SDL UML Properties Scenarios violating properties or traces for residual testing Verdict It should never be the case that a phone connects to a phone that is not ringing because of this call. A&~B&(c=>D) F0=>c\/~D Verification strategy

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 12 Discrepancy – document problem – wrong reference, absence of information, etc.; typos and obvious slips are not counted! Unreachability – the system will never be in that state – analog of dead code Deadlock – from this system state no further transition is possible, usually some incompleteness in requirements Transition inconsistency – non- deterministic behavior – equivalent pre- conditions, but different actions afterward Safety – violation of a specified safety (liveness) property – the timer is stopped only if it was started; or the lift door shall be always closed while the lift is moving Timing violation – mismatch of specified event ordering with respect to their timing provided in absolute or relative units Defects found in system state space basic protocols annotations documentation timing Classification of defects

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 13 Piloting VRS

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 14 Benefits of testing based on verified specification Decrease of test efforts by up to 50% 5X decrease in manual efforts for verifying the product quality Organize smooth testing life-cycle based on work with graphical specifications only Increase test suites reuse ratio (more than 50-60%) and support test suites reuse Increase confidence in absence of hard-to-find errors Increase test coverage while reducing testing time. Generated test suite will be minimal in size and maximal in coverage. Generating from correct formal specifications 100% stand-alone tests and test environment for C, C++, Java and various scripting languages and platforms.

Information Software Systems ISS Modern SW Development Practices: Processes and Technologies 18 May 2007 (c) 2007 Information Software Systems 15 Where VRS could be applied? The tool is ready for application in large-scale projects (thousands of functional requirements) from Telematics and Telecommunication domains. Additional adjustment is needed for Automotive and detection of security issues in Inet protocols.