UCM-Based Generation of Test Goals Daniel Amyot, University of Ottawa (with Michael Weiss and Luigi Logrippo) RDA Project (funded.

Slides:



Advertisements
Similar presentations
© 2005 by Prentice Hall Appendix 3 Object-Oriented Analysis and Design Modern Systems Analysis and Design Fourth Edition Jeffrey A. Hoffer Joey F. George.
Advertisements

International Telecommunication Union © ITU-T Study Group 17 Integrated Application of URN Daniel Amyot University of Ottawa, Canada
Use Case Maps Daniel Amyot, Gunter Mussbacher Introduction to Use Case Maps.
UCM Path Traversal Daniel Amyot SG17, Geneva, March 5 th, 2002 UCM Scenarios and Path Traversal.
Ch:8 Design Concepts S.W Design should have following quality attribute: Functionality Usability Reliability Performance Supportability (extensibility,
Based on Powerpoint slides by Gunter Mussbacher, Gregor v. Bochmann User Requirements Notation (URN) SEG3101 (Fall 2010)
Testing and Quality Assurance
SOFTWARE TESTING. INTRODUCTION  Software Testing is the process of executing a program or system with the intent of finding errors.  It involves any.
Daniel Amyot, PhD student, University of Ottawa Scenarios As First-Class Entities Functional Entities Mapping Scenarios on Functional Entities Network.
1 Software Processes A Software process is a set of activities and associated results which lead to the production of a software product. Activities Common.
The software process A software process is a set of activities and associated results which lead to the production of a software product. This may involve.
Software Modeling SWE5441 Lecture 3 Eng. Mohammed Timraz
Alternate Software Development Methodologies
© 2006 ITT Educational Services Inc. SE350 System Analysis for Software Engineers: Unit 9 Slide 1 Appendix 3 Object-Oriented Analysis and Design.
Practical Business Modeling in the Unified Process Tom Morgan Software Architect, Fidelity National Information Services
® IBM Software Group © 2006 IBM Corporation Rational Software France Object-Oriented Analysis and Design with UML2 and Rational Software Modeler 04. Other.
Creating Architectural Descriptions. Outline Standardizing architectural descriptions: The IEEE has published, “Recommended Practice for Architectural.
Advanced Behavioral Modeling
Architectural Design Establishing the overall structure of a software system Objectives To introduce architectural design and to discuss its importance.
1 An introduction to design patterns Based on material produced by John Vlissides and Douglas C. Schmidt.
Towards a Logic for Wide- Area Internet Routing Nick Feamster Hari Balakrishnan.
S/W Project Management Software Process Models. Objectives To understand  Software process and process models, including the main characteristics of.
Chapter 2 The process Process, Methods, and Tools
1/19 Component Design On-demand Learning Series Software Engineering of Web Application - Principles of Good Component Design Hunan University, Software.
CMSC 345 Fall 2000 Unit Testing. The testing process.
Assessing the Suitability of UML for Modeling Software Architectures Nenad Medvidovic Computer Science Department University of Southern California Los.
RUP Design RUP Artifacts and Deliverables
Copyright 2002 Prentice-Hall, Inc. Modern Systems Analysis and Design Third Edition Jeffrey A. Hoffer Joey F. George Joseph S. Valacich Chapter 20 Object-Oriented.
Jan 20-21, 2005Weiss and Amyot, MCETECH 051 Designing and Evolving Business Models with the User Requirements Notation Michael Weiss (Carleton University)
S CHOOL OF I NFORMATION T ECHNOLOGY AND E NGINEERING U NIVERSITY OF O TTAWA, C ANADA Daniel Amyot Q18/17 (URN) Rapporteur User Requirements.
Software Engineering Research paper presentation Ali Ahmad Formal Approaches to Software Testing Hierarchal GUI Test Case Generation Using Automated Planning.
Software Engineering Principles Principles form the basis of methods, techniques, methodologies and tools Principles form the basis of methods, techniques,
Abstract Use Case Map (UCM) scenarios are useful for elicitation and analysis of software requirements However, they must be used in cooperation with complementary.
1 Recent work in the area: Requirement-Driven Development of Distributed Applications Gregor v. Bochmann School of Information Technology and Engineering.
S CHOOL OF I NFORMATION T ECHNOLOGY AND E NGINEERING U NIVERSITY OF O TTAWA Feature Description and Feature Interaction Analysis with Use Case Maps and.
Review of Software Process Models Review Class 1 Software Process Models CEN 4021 Class 2 – 01/12.
1 What is OO Design? OO Design is a process of invention, where developers create the abstractions necessary to meet the system’s requirements OO Design.
Early Availability Requirements Modeling using Use Case Maps KAMARUL ZAMAN BIN PANATIK MAN
FDT Foil no 1 On Methodology from Domain to System Descriptions by Rolv Bræk NTNU Workshop on Philosophy and Applicablitiy of Formal Languages Geneve 15.
1 ECCF Training 2.0 Introduction ECCF Training Working Group January 2011.
1 Overview of the project: Requirement-Driven Development of Distributed Applications School of Information Technology and Engineering (SITE) University.
Chapter 6 CASE Tools Software Engineering Chapter 6-- CASE TOOLS
Software Architecture Evaluation Methodologies Presented By: Anthony Register.
MODEL-BASED SOFTWARE ARCHITECTURES.  Models of software are used in an increasing number of projects to handle the complexity of application domains.
Ch- 8. Class Diagrams Class diagrams are the most common diagram found in modeling object- oriented systems. Class diagrams are important not only for.
Gregor v. Bochmann, University of Ottawa Based on Powerpoint slides by Gunter Mussbacher(2009) with material from Amyot User Requirements Notation (URN)
Architecture View Models A model is a complete, simplified description of a system from a particular perspective or viewpoint. There is no single view.
Testing OO software. State Based Testing State machine: implementation-independent specification (model) of the dynamic behaviour of the system State:
Generating Software Documentation in Use Case Maps from Filtered Execution Traces Edna Braun, Daniel Amyot, Timothy Lethbridge University of Ottawa, Canada.
Topic 4 - Database Design Unit 1 – Database Analysis and Design Advanced Higher Information Systems St Kentigern’s Academy.
S CHOOL OF I NFORMATION T ECHNOLOGY AND E NGINEERING U NIVERSITY OF O TTAWA, C ANADA Daniel Amyot Q18/17 (URN) Rapporteur User Requirements.
©Ian Sommerville 2006Software Engineering, 8th edition. Chapter 4 Slide 1 Software Processes.
MICON 2000 F ormal methods for design methodology by Luigi Logrippo with D. Amyot, R. Chan, L. Charfi, N. Gorse, J.Sincennes, R. Plesa,... S CHOOL OF I.
Banaras Hindu University. A Course on Software Reuse by Design Patterns and Frameworks.
21/1/ Analysis - Model of real-world situation - What ? System Design - Overall architecture (sub-systems) Object Design - Refinement of Design.
Whole Test Suite Generation. Abstract Not all bugs lead to program crashes, and not always is there a formal specification to check the correctness of.
© 2000 D EMO D AY 2000 Page 1 Automatic Generation of Conformance Test Cases from Use Case Maps Strategic Technology Leïla Charfi, Luigi Logrippo & group.
OBJECT-ORIENTED TESTING. TESTING OOA AND OOD MODELS Analysis and design models cannot be tested in the conventional sense. However, formal technical reviews.
Reachability Testing of Concurrent Programs1 Reachability Testing of Concurrent Programs Richard Carver, GMU Yu Lei, UTA.
Testing Overview Software Reliability Techniques Testing Concepts CEN 4010 Class 24 – 11/17.
+ Informatics 122 Software Design II Lecture 13 Emily Navarro Duplication of course material for any commercial purpose without the explicit written permission.
1 Towards Integrated Tool Support for the User Requirements Notation Jean-François Roy
Objective ICT : Internet of Services, Software & Virtualisation FLOSSEvo some preliminary ideas.
Requirement Elicitation Review – Class 8 Functional Requirements Nonfunctional Requirements Software Requirements document Requirements Validation and.
Sequence diagrams Lecture 5. Main terms  Interaction  Life line  Activation  Executable behavior and derived behavior  Messages  Trajectory  Frame.
Software Testing.
Unified Modeling Language
Business System Development
Enabling UCM-based transformations
Appendix 3 Object-Oriented Analysis and Design
Presentation transcript:

UCM-Based Generation of Test Goals Daniel Amyot, University of Ottawa (with Michael Weiss and Luigi Logrippo) RDA Project (funded by NSERC) Adapted by J-Pierre Corriveau from:

2 … … … … [C1] [C2] [C3] OR-Fork & Guarding Conditions … … … … OR-Join … … … … … … … … AND-JoinAND-Fork (b) UCM Forks and Joins Start Point End Point Path …… …… Responsibility Direction Arrow …… Timestamp Point Failure Point …… Shared Responsibility …… (a) UCM Path Elements Waiting Place Trigger Path (asynchronous) Waiting Path Continuation Path Timer Release (synchronous) Waiting Path Continuation Path Timeout Path (e) UCM Waiting Places and Timers …… IN1OUT1 Static Stub & Segments ID Dynamic Stub IN1OUT1 …… S{IN1}E{OUT1} (d) UCM Stubs and Plug-ins Plug-in Map (c) UCM Components Team Agent

3 UCM-Based Testing? Can we derive test goals and test cases from UCM models?  Scenario notation  good basis!  Reuse of (evolving) requirements model for validation UCMs are very abstract:  Very little information about data and communication  Difficult to derive implementation-level test cases automatically  Deriving test goals is more realistic How much value? How much effort?

4 Test Generation Approaches Based on UCM Testing Patterns  Grey-box test selection strategies, applied to requirements scenarios  Manual Based on UCM Scenario Definitions  UCM + simple data model, initial values and start points, and path traversal algorithms  Semi-automatic Based on UCM Transformations  Exhaustive traversal  Mapping to formal language (e.g., LOTOS)  Automated

UCM Testing Patterns

6 UCMTest Goals HandleStubsHandleStartPoints HandleConstructs HandleStubs: [Causally LinkedStubs] UCM Flattened UCMs TP6 [Single Stubs] TP5 1:N [NoStubLeft] [StubsLeft] [NoLinkedStub][NoSingleStub] UCM-Oriented Testing Pattern Language TP: Test Pattern: a strategy for testing (stub choice of strategies)

7 TP5: Single Stub SP S IN1 OUT1 OUT2 EP1 EP2 IN2 OUT1 c OUT2 d Plug-in 2 IN1 OUT1 a OUT2 b Plug-in 1 Flattens one stub from a hierarchical UCM. Strategies: 5A: Static flattening (when only one plug-in in the static stub) 5B: Dynamic flattening, some plug-ins (when several plug-ins in the dynamic stub) 5C: Dynamic flattening, all plug-ins (when several plug-ins in the dynamic stub) Assuming all branches coverage for each plug-in: 5A: 2 paths {, } IF S were static 5B: 2 paths {, } or same as 5A 5C: 4 paths {,,, }

8 TP6: Causally-Linked Stubs Handles combinations of plug-ins bound to causally linked stubs.  causal: selection of plug-in for 2 nd stub depends on (is caused by) 1 st stub’s selection of plug-in Strategies:(feature functionality: affects choice of plug-in) 6A: Default behavior (when no feature is active) 6B: Individual functionalities (when one feature is active at a time) 6C: Functionality combinations (when several or all functionalities are active) SP S1 IN1 OUT1 EP1 EP3 IN1 OUT1 a OUT2 b Plug-in 2 OUT2 S2 IN2 OUT3 EP2 OUT4 IN2 OUT3 c OUT4 d Plug-in 3 Start End Plug-in 1 Default for both

9 UCMTest Goals HandleStubsHandleStartPoints HandleConstructs HandleStartPoints: [MultipleStartPoints] Flattened UCM Flat Maps with Enabled Start Points TP4 1:N [SingleStartPoint] UCM-Oriented Testing Pattern Language

10 TP4: Multiple Start Points Strategies based on necessary, redundant, insufficient, and racing subsets (8 strategies based on path sensitization) SP3 EP SP2 SP1 Case #SP1SP2SP3 SP1  (SP2  SP3) Subset 0FFFFInsufficient stimuli. Not interesting. 1FFTFInsufficient stimuli 2FTFF 3FTTTNecessary stimuli 4TFFT 5TFTTRedundant stimuli 6TTFT 7TTTTRacing stimuli

Eight strategies for start points Based on necessary, redundant, insufficient, and racing subsets of inputs : 4A: One necessary subset, one goal:  { } (if case 3 is selected) 4B: All necessary subsets, one goal:  {, } (assume interleaving) 4C: All necessary subsets, all goals:  {,, } 4D: One redundant subset, one goal:  { } 4E: All redundant subsets, one goal:  {, } 4F: One insufficient subset, one goal:  { } (rejection) 4G: All insufficient subsets, one goal:  {, } (rejection) 4H: Some racing subsets, some goals:  {, } 11

12 UCM-Oriented Testing Pattern Language HandleConstructs: Flat Maps with Enabled Start Points Test Goals [Loop] TP1 1:N TP3 TP2 [Alternative] [Concurrent] CheckConstruct [EndPoint] UCMTest Goals HandleStubsHandleStartPoints HandleConstructs

13 TP1: Alternatives a SPEP b c d 1A: All results (end points) { } 1B: All segments {, } 1C: All paths {,,, } 1D: All combinations of sub-conditions (for composite conditions, e.g., (X OR Y) AND Z )

14 TP2: Concurrency 2A: One combination (wrt ordering) { } 2B: Some combinations {, } 2C: All combinations {,, } a SPEP bc

15 TP3: Loops a SPEP b 3A: All segments 3B: At most k iterations 3C: Valid boundaries [low, high] Tests low, low+1, high-1, and high 3D: All boundaries [low, high] Tests valid ones (3C) and invalid ones (low-1 and high+1, for rejection tests)

Flattening the Loops 3A: All segments:  { } 3B: At most k iterations:  {,, } (if k = 2) 3C: Valid boundaries [low, high]: Tests low, low+1, high- 1, and high. If low = 1 and high = 5:  {,,, } 3D: All boundaries [low, high]: Tests valid ones (3C) and invalid ones (low-1 and high+1). If low = 1 and high = 5:  Accept: {,,, }  Reject: {, } 16

17 Complementary Strategies Strategies for value selection  Equivalence classes, boundary testing These are typical approaches in traditional testing Strategies for rejection test cases  Forbidden scenarios  Testing patterns  Incomplete conditions  Off-by-one value Meyer: Should we worry about invalid cases?

Appexdix 1: UCM Transformations Delegating the testing of the whole model to some tool that can test the transformed model.

Analysis Model Construction Source scenario model  Target analysis model Q1. What should the target language be?  Use Case Maps Specification  ? Q2. What should the construction strategy be?  Analytic approach build-and-test construction  Synthetic approach scenarios "compiled" into new target model interactive or automated

Specification-Validation Approach with L OTOS and UCMs Results (Coverage) Results (Coverage) Test Suite (L OTOS ) Add tests if necessary Add tests if necessary Test Cases Generation Test Cases Generation Testing Framework And Patterns Requirements Bound UCM Prototype (L OTOS ) Prototype (L OTOS ) Results (Functional) Results (Functional) Construction Modify if necessary Modify if necessary Testing Allocation Scenarios (UCM) Architecture Construction Guidelines

Complementary Yet Compatible! Use Case Maps Scenario notation, readable, abstract, scalable, loose, relatively effortless to learn Use Case Maps Scenario notation, readable, abstract, scalable, loose, relatively effortless to learn L OTOS Mature formal language, good theories and tools for V&V and completeness & consistency checking. L OTOS Mature formal language, good theories and tools for V&V and completeness & consistency checking. Both Focus on ordering of actions Have similar constructs  simpler mapping Handle specifications with or without components Have been used to describe dynamic systems in the past Have been used to detect feature interactions in the past Both Focus on ordering of actions Have similar constructs  simpler mapping Handle specifications with or without components Have been used to describe dynamic systems in the past Have been used to detect feature interactions in the past

Requirements Scenarios (UCM) Structure UCMs on Structure Prototype (L OTOS ) Prototype (L OTOS ) Results (Functions) Results (Functions) Results (Coverage) Results (Coverage) Test Suite (L OTOS ) Add tests if necessary Add tests if necessary Test Cases Generation Test Cases Generation Construction Modify if necessary Modify if necessary Testing Allocation UCM-LOTOS Construction Guidelines

UCM-LOTOS Testing Framework Requirements Scenarios (UCM) Structure UCMs on Structure Prototype (L OTOS ) Prototype (L OTOS ) Results (Functions) Results (Functions) Results (Coverage) Results (Coverage) Test Suite (L OTOS ) Add tests if necessary Add tests if necessary Test Cases Generation Test Cases Generation Construction Modify if necessary Modify if necessary Testing Allocation

Structural Coverage Requirements Scenarios (UCM) Structure UCMs on Structure Prototype (L OTOS ) Prototype (L OTOS ) Results (Functions) Results (Functions) Results (Coverage) Results (Coverage) Test Suite (L OTOS ) Add tests if necessary Add tests if necessary Test Cases Generation Test Cases Generation Construction Modify if necessary Modify if necessary Testing Allocation

25 Automatic Test Goals: LOTOS and TTCN-2 (Charfi) Path traversal algorithm  Fixed subset of testing patterns  No data model  Exhaustive coverage of UCM model  Unfeasible scenarios Ucm2LotosTest  Based on older version of UCMNav  Test goals as LOTOS processes  Converted to TTCN-2 via TGV

26 Automatic Test Goals: LOTOS and MSC (Guan) Generation of LOTOS specification and test scenarios  Fixed subset of testing patterns  Exhaustive coverage of UCM model  Unfeasible scenarios Can be verified against spec Still, no data model  Tools (in Java) independent of UCMNav  LOTOS model could be used as basis for test generation

27 Towards Test Cases Communication  Communication mechanisms between pairs of components connected by a UCM path must be specified (e.g., messages, parameters and data values, protocols). Hiding  UCM responsibilities and start/end points located inside components may be hidden and hence left out of the test goals. The interface used to test needs to be specified. Data values  Data values need to be selected such that the various conditions in the test goal are satisfied. Conventional techniques are applicable. Time information  UCM timers do not have a quantitative notion of time intervals or constraints. Set-up and clean-up  Preambles and postambles may be needed for each test case. Target  Tests need to be re-targetable and readable by test equipment, something that is supported by languages such as TTCN-3. Should some of these aspects have an impact on the source scenario language?

28 Tools UCMNav UCMExporter UCM2LOTOS Available from Daniel Amyot upon request

29 Test Case Generation: UCM/Requirements Validation Scenario Def. MSCSDL Spec UCM Test Goals Test Patterns L OTOS TestsL OTOS Spec S PEC -VALU E Scenario definition + MSC/SDL Scenario definition + L OTOS

30 Test Case Generation: Conformance Testing UCM direct SDL direct (conventional) SDL Spec TTCN (or other) L OTOS Spec Scenario Def. UCM MSC L OTOS Tests L OTOS direct (conventional) UCM-SDL indirect (validated UCM tests) UCM- L OTOS indirect (validated UCM tests)

Appendix 2: Why UCMs?

Requirements Engineering Issues Early focus on low-level abstractions Requirements and high-level decisions buried in the details Evolution of functionalities difficult to handle (feature interactions, V&V, adaptability to legacy architectures...) Delay introduction of new services

Software Engineering Issues Requirements/analysis models need to support new types of dynamic systems  Run-time modification of system structure  Run-time modification of behaviour Need to go from a requirements/analysis model to design models in a seamless way We propose Use Case Maps (UCMs)!

Use Case Maps (UCMs) The Use Case Maps notation allows illustrating a scenario path relative to optional components involved in the scenario (gray box view of system) UCMs are a scenario-based software engineering technique for describing causal relationships between responsibilities of one or more use cases UCMs show related use cases in a map- like diagram

Why Use Case Maps? Bridge the modeling gap between requirements (use cases) and design  Link behaviour and structure in an explicit and visual way  Provide a behavioural framework for making (evaluating) architectural decisions at a high level of design  Characterize the behaviour at the architecture level once the architecture is decided Convey a lot of information in a compact form Use case maps integrate many scenarios - enables reasoning about potential undesirable interactions of scenarios

Why Use Case Maps? Provide ability to model dynamic systems where scenarios and structures may change at run-time  E-commerce applications  Telecommunication systems based on agents Simple, intuitive, low learning curve Document while you design Effective learning tool for people unfamiliar with the domain May be transformed (e.g. into MSC/sequence diagrams, performance models, test cases)