Presentation is loading. Please wait.

Presentation is loading. Please wait.

BAiT: Adaptive Test Case Execution in Practice Jens R. Calamé Centrum voor Wiskunde en Informatica, Amsterdam.

Published byAlisha Walker Modified over 10 years ago

Similar presentations

Presentation on theme: "BAiT: Adaptive Test Case Execution in Practice Jens R. Calamé Centrum voor Wiskunde en Informatica, Amsterdam."— Presentation transcript:

1 BAiT: Adaptive Test Case Execution in Practice Jens R. Calamé Centrum voor Wiskunde en Informatica, Amsterdam

2 13-09-2007DWFTT 2007, Enschede2 Agenda 1.Introduction 2.The Toolset for BAiT 3.An Exemplary Case: The ATM 4.Structure of BAiT 5.Conclusion

3 13-09-2007DWFTT 2007, Enschede3 Introduction BAiT: Behavior Adaptation in Testing Toolset to generate test cases and test oracles as well as execute tests on nondeterministic systems with data

4 13-09-2007DWFTT 2007, Enschede4 Agenda 1.Introduction 2.The Toolset for BAiT 3.An Exemplary Case: The ATM 4.Structure of BAiT 5.Conclusion

5 13-09-2007DWFTT 2007, Enschede5 BAiT: Test Case & Oracle Generation Test generation with TGV (enumerative approach) Chaotic data abstraction for input and output parameters of events Test oracle generation for Eclipse Prolog constraint solver in order to select test data Automatic generation of variable names for the test case

6 13-09-2007DWFTT 2007, Enschede6 BAiT: Test Proxy Generation Test Proxy for Java-based SUTs Instantiation of test object within test proxy Procedure-based bidirectional communication (publisher- subscriber pattern) Mapping of datatypes from specification language to target platform

7 13-09-2007DWFTT 2007, Enschede7 BAiT: Test Execution Test execution of pre- calculated test trace (behavior and data) Adaptation to the IUT's reactions, rather than immediate assignment of verdict BAiT framework provides adapters for user-defined –trace selection and –test data selection algorithms

8 13-09-2007DWFTT 2007, Enschede8 Agenda 1.Introduction 2.The Toolset for BAiT 3.An Exemplary Case: The ATM 4.Structure of BAiT 5.Conclusion

9 13-09-2007DWFTT 2007, Enschede9 An Exemplary Case: The ATM ATM specification (partially)ATM test purpose (completely)

10 13-09-2007DWFTT 2007, Enschede10 An ATM Test Case Photos: Wikipedia and ECB !getPin(1234)?pinCorrect !getAmount(100) !initBalance(1000)?retFifty(2) ?retFifty(1)?retTwenty(2)?retTen(1) ? !initPin(1234)

11 13-09-2007DWFTT 2007, Enschede11 An ATM Test Case (2) May 25, 2007 3:06:19 PM nl.cwi.sen2.bait.steps.TraceImpl merge FINE: Pruning planned test trace. May 25, 2007 3:06:19 PM nl.cwi.sen2.bait.RunTest findTrace FINER: Examining: [init],initPin(Pin),getPin(PinUser),pinCorrect, initBalance(Bal),getAmount(Amt),[tau],retFifty(Fif),emitBankNotes May 25, 2007 3:06:19 PM nl.cwi.sen2.bait.steps.TraceImpl solve FINEST: init(G1),initPin(G1,G2,lparam(nat(Pin))),[...] [Pin in {[-Infinity.. Infinity]} => 1234] May 25, 2007 3:06:21 PM nl.cwi.sen2.bait.steps.TraceImpl solve FINEST: init(G1),initPin(G1,G2,lparam(nat(1234))),[...] [...] May 25, 2007 3:06:26 PM nl.cwi.sen2.bait.RunTest executeTest INFO: Trying: [init],initPin(1234),getPin(1234),pinCorrect, initBalance(1000),getAmount(100),[tau],retFifty(2),emitBankNotes May 25, 2007 3:06:26 PM nl.cwi.sen2.bait.RunTest executeTest FINE: initPin(1234) -> OK [...] May 25, 2007 3:06:26 PM nl.cwi.sen2.bait.RunTest executeTest FINE: retFifty(2) -> NOT OK May 25, 2007 3:06:26 PM nl.cwi.sen2.bait.RunTest executeTest INFO: Expected: retFifty(2), received: retFifty(1) -> Trace failed, trying alternative. 1. Precalculation of possible test trace 2. Iterative data selection 3. Stepwise execution of the trace… …until the IUT deviates from the precalculated trace.

12 13-09-2007DWFTT 2007, Enschede12 An ATM Test Case (3) May 25, 2007 3:06:26 PM nl.cwi.sen2.bait.RunTest executeTest INFO: Trace deviated; executed so far: [init],initPin(1234),getPin(1234),pinCorrect,initBalance(1000), getAmount(100),[tau],retFifty(1),retTwenty(2),retTen(1), emitBankNotes May 25, 2007 3:06:26 PM nl.cwi.sen2.bait.steps.TraceImpl merge FINE: Pruning planned test trace. May 25, 2007 3:06:26 PM nl.cwi.sen2.bait.steps.TraceImpl merge FINE: Adding retFifty(1) to trace stub. [...] May 25, 2007 3:06:26 PM nl.cwi.sen2.bait.steps.TraceImpl merge FINE: Adding emitBankNotes to trace stub. May 25, 2007 3:06:27 PM nl.cwi.sen2.bait.RunTest executeTest INFO: Test finished; executed: [init],initPin(5),getPin(5), pinCorrect,initBalance(1000),getAmount(100),[tau],retFifty(1), retTwenty(2),retTen(1),emitBankNotes May 25, 2007 3:06:27 PM nl.cwi.sen2.bait.RunTest executeTest INFO: The test case ended with verdict PASS. 5. Appending received system output to trace. 6. Solve test trace with pending system output; assign verdict or calculate new trace. 4. Pruning not yet executed part of test trace.

13 13-09-2007DWFTT 2007, Enschede13 Agenda 1.Introduction 2.The Toolset for BAiT 3.An Exemplary Case: The ATM 4.Structure of BAiT 5.Conclusion

14 13-09-2007DWFTT 2007, Enschede14 Structure of BAiT: The Tools BAiT: Behavior Adaptation in Testing Toolset to generate test cases and test oracles as well as execute tests on nondeterministic systems with data

15 13-09-2007DWFTT 2007, Enschede15 Structure of BAiT: Execution Part BAiT Execution Core Behavior Mapping Constraint Solver Behavior Mapping Impl. Under Test Data Mapping Constraint Solver Data Mapping Impl. Under Test Impl. Under Test Constraint Solver Trace Selection Data Selection Trace class: trace execution management Step class: communication with IUT Trace selection from complete test graph (IOLTS with all possible test cases for a given test purpose) Test Oracle Test Case(s) Variable class: Codec of data to constraint solver, data instantiator and IUT Data instantiation algorithm(s)

16 13-09-2007DWFTT 2007, Enschede16 Agenda 1.Introduction 2.The Toolset for BAiT 3.An Exemplary Case: The ATM 4.Structure of BAiT 5.Conclusion

17 13-09-2007DWFTT 2007, Enschede17 Conclusion Behavior Adaptation in Testing helps finding more reliable verdicts in testing non-deterministic systems A prototypical framework for BAiT has been developed procedure-based setting with atomic data Further work: –Extension of framework for other settings –Extension of framework to structured datatypes –Case Study: Mozilla Firefox CSS subsystem

18 13-09-2007DWFTT 2007, Enschede18 Further Information Background information Calamé, Jens R.; Ioustinova, Natalia; Pol, Jaco van de; Sidorova, Natalia; Data Abstraction and Constraint Solving for Conformance Testing (APSEC 2005). Calamé, Jens R.; Ioustinova, Natalia; Pol, Jaco van de; Towards Automatic Generation of Parameterized Test Cases from Abstractions. Technical Report, 2006. The toolset/framework itself Calamé, Jens R.; Adaptive Test Case Execution in Practice. Technical Report, 2007. Calamé, Jens R.; The Internals of Adaptive Test Case Execution. Technical Report, to appear. All downloadable from www.calame.de/works,published.bib,date,.htmlwww.calame.de/works,published.bib,date,.html Contact: Jens R. Calamé CWI, Amsterdam jens.calame@cwi.nl

Download ppt "BAiT: Adaptive Test Case Execution in Practice Jens R. Calamé Centrum voor Wiskunde en Informatica, Amsterdam."

Similar presentations

Chapter 4 Loops Liang, Introduction to Java Programming, Eighth Edition, (c) 2011 Pearson Education, Inc. All rights reserved. 0132130807.

Chapter 4 Loops Liang, Introduction to Java Programming, Eighth Edition, (c) 2011 Pearson Education, Inc. All rights reserved
BPEL4WS Business Process Execution Language for Web Services Jim Clark eBusiness Strategist jimclark@microsoft.com.

BPEL4WS Business Process Execution Language for Web Services Jim Clark eBusiness Strategist
Spanish Professional Localization Pack Extension Module Webinar November 26th, 2009.

Spanish Professional Localization Pack Extension Module Webinar November 26th, 2009.
Thursday, November 1, 2001(c) 2001 Ibrahim K. El-Far. All rights reserved.1 Enjoying the Perks of Model-based Testing Ibrahim K. El-Far Florida Institute.

Thursday, November 1, 2001(c) 2001 Ibrahim K. El-Far. All rights reserved.1 Enjoying the Perks of Model-based Testing Ibrahim K. El-Far Florida Institute.
Requirements Engineering Processes – 2

Requirements Engineering Processes – 2
Embedded Systems & Parallel Programming. - 2 - P. Marwedel, Univ. Dortmund/Informatik 12 + ICD/ES, 2007 Universität Dortmund A view on embedded systems.

Embedded Systems & Parallel Programming P. Marwedel, Univ. Dortmund/Informatik 12 + ICD/ES, 2007 Universität Dortmund A view on embedded systems.
Chapter 7 Constructors and Other Tools. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 7-2 Learning Objectives Constructors Definitions.

Chapter 7 Constructors and Other Tools. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 7-2 Learning Objectives Constructors Definitions.
Multicriteria Decision-Making Models

Multicriteria Decision-Making Models
Cultural Heritage in REGional NETworks REGNET Project Meeting Content Group 2002-06-24 - 2002-06-26 Part 2: Functional Testing.

Cultural Heritage in REGional NETworks REGNET Project Meeting Content Group Part 2: Functional Testing.
Quality Education Investment Act of 2006 (QEIA) 1 Quality Education Investment Act (QEIA) of 2006 County Superintendents Oversight and Technical Assistance.

Quality Education Investment Act of 2006 (QEIA) 1 Quality Education Investment Act (QEIA) of 2006 County Superintendents Oversight and Technical Assistance.
Career Newsletter by Dan Florescu BEST Corporate Relations Coordinator BEST Partners Round Table - Tallinn 2008.

Career Newsletter by Dan Florescu BEST Corporate Relations Coordinator BEST Partners Round Table - Tallinn 2008.
European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies Grid.

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies Grid.
NexSAT NexSAT Steering Group Meeting - 8 June 2004 © 2004 European Organisation for the Safety of Air Navigation (EUROCONTROL) 1 Welcome to the 4th meeting.

NexSAT NexSAT Steering Group Meeting - 8 June 2004 © 2004 European Organisation for the Safety of Air Navigation (EUROCONTROL) 1 Welcome to the 4th meeting.
(c) 2007 Mauro Pezzè & Michal Young Ch 17, slide 1 Test Execution.

(c) 2007 Mauro Pezzè & Michal Young Ch 17, slide 1 Test Execution.
Testing Workflow Purpose

Testing Workflow Purpose
Chapter 5 Plan-Space Planning.

Chapter 5 Plan-Space Planning.
# 1 06.09.07 The Core Cadastre Domain Model Survey Package and Model Driven Architecture Master Thesis Project Approach The Core Cadastre Domain Model.

# The Core Cadastre Domain Model Survey Package and Model Driven Architecture Master Thesis Project Approach The Core Cadastre Domain Model.
Heppenheim Producer-Archive Interface Specification Status of standardisation project Main characteristics, major changes, items pending.

Heppenheim Producer-Archive Interface Specification Status of standardisation project Main characteristics, major changes, items pending.
Coursework.  5 groups of 4-5 students  2 project options  Full project specifications on 3 rd March  Final deadline 10 th May 2011  Code storage.

Coursework.  5 groups of 4-5 students  2 project options  Full project specifications on 3 rd March  Final deadline 10 th May 2011  Code storage.
31242/32549 Advanced Internet Programming Advanced Java Programming

31242/32549 Advanced Internet Programming Advanced Java Programming

Similar presentations

Ads by Google