1. © Telelogic AB UML Testing Profile Graphical Testing With UML The UML Testing Profile Eric Samuelsson eric.samuelsson@telelogic.com

2. © Telelogic AB UML Testing Profile Agenda Background Key concepts Advanced concepts Summary

3. © Telelogic AB UML Testing Profile Background Test technologies are not on par with modern design technologies such as model-driven architectures (MDA) Theres a need for a widespread graphical test language OMG realized this and issued and RFP for a graphical testing profile to UML –Using the UML ensures widespread use across a wide range of application areas –Numerous tool vendors support UML –UML is a proven de facto standard –But UML is focused on definition of system structure and behavior and has no built-in testing constructs...

4. © Telelogic AB UML Testing Profile UML Testing Profile RFP Issued by OMG in July 2001 LOIs submitted by –Telelogic, Rational, SofTEAM, FOKUS, Ericsson, Motorola, IBM One joint submission Initial submission made in April 2002 Revised initial submission made in June 2002 –Terminology changes –Elaboration of Test Data Final submission in January 2003

5. © Telelogic AB UML Testing Profile Requirements and restrictions Define structural and behavioral test concepts Conformance testing only –Black-box functional testing Based upon the UML meta-model –UML 2.0 as proposed by the U2 PartnersU2 Partners –Inter-operation with existing testing technologies –TTCN 3 –Junit –...

6. © Telelogic AB UML Testing Profile Current proposal A graphical testing language based the current UML 2 metamodel Reusing as many UML concepts as possible –Recognition important for UML users –No need to re-invent the wheel Designed with TTCN 3 in mind –Semantics basically follows TTCN 3 semantics –But caters for other technologies like JUnit as well

7. © Telelogic AB UML Testing Profile Current proposal Defined as a UML profile –Will be recognized by UML users Basically a number of stereotypes –Easy to implement for tool vendors Models will be interchangeable –Any tool implementing XMI will be able to exchange models «profile» Testing Profile «import» UML «stereotype» TestCase «metaclass» BehavioralFeature

8. © Telelogic AB UML Testing Profile Key Concepts Test Architecture –Test Context, Test Configuration –Test Component, SUT –Arbiter Test Behavior –Test Case –Verdict, Validation Action –Defaults Test Data –Logical partitions, Pattern matching Test Deployment

9. © Telelogic AB UML Testing Profile Test Objective An informal description of what to test in one test case Commonly expressed using natural language Can also be expressed using other diagrams, such as use case diagrams and sequence diagrams Verify that if a user inserts and authorizes a valid card correctly, he is able to withdraw money if he has sufficient funds. Client Withdrawal InvalidPin Tester >

10. © Telelogic AB UML Testing Profile Test Case A specification of one test, including –Test objective –Test behavior Defined as a BehavioralFeature in UML Its method is the behavior of the test case «testCase» +validWithdrawal() : Verdict «testContext» ATMContext «testCase» +validWithdrawal() : Verdict +invalidPIN() : Verdict -authorizeCard() : Verdict

11. © Telelogic AB UML Testing Profile Test Behavior Describes the expected behavior of a test case Defined using any type of UML behavior –Interactions, statemachines and activities Facilitates specification of arbitrarily complex behavior Idle WaitCode

12. © Telelogic AB UML Testing Profile Test Architecture A complete specification of all types used in the tests –An ordinary UML package –Contains Test Contexts, Test Components, Ports, Interfaces, etc. «testArchitecture» ATMTest

13. © Telelogic AB UML Testing Profile Test Architecture Example «testArchitecture» ATMTest «testContext» ATMContext «testCase» +validWithdrawal() : Verdict +invalidPIN() : Verdict -authorizeCard() : Verdict Account balance : Integer number : String credit(a : Integer) debit(a : Integer) * accounts ATM::CardData pinCode : Integer number : String isPinCorrect(c: Integer) : Boolean * cards «testComponent» BankEmulator IAccount bePort -pinOk : Boolean -enteredPIN : String -message : String «testComponent» HWEmulator hwCom IScreen, ICardMachine, IMoneyBox IATM

14. © Telelogic AB UML Testing Profile Test Component Implements (parts of ) the behavior of one or more test cases Each test case is normally executed by several components Ordinary classes or components -pinOk : Boolean -enteredPIN : String -message : String «testComponent» HWEmulator hwCom ATM::CardData pinCode : Integer number : String isPinCorrect(c: Integer) : Boolean 0..1 -currentCard IScreen, ICardMachine, IMoneyBox IATM

15. © Telelogic AB UML Testing Profile Test Context A classifier that contains –a number of test cases and their behavior and objectives –default behavior used by the test cases –configuration of test components –test control behavior Can be a class, a component or a collaboration «testContext» ATMContext «testCase» +validWithdrawal() : Verdict +invalidPIN() : Verdict -authorizeCard() : Verdict

16. © Telelogic AB UML Testing Profile Test Configuration A specification of the structural aspects of test cases Defines the parts participating in the test case and their interconnections –Test Components, SUT, Ports, Interfaces and Connectors Defined as the internal structure of the Test Context « sut » atm : BankATM hwe : HWEmulator be : BankEmulator atmPort hwCom bePortnetCom accnts : Account [0..*]

17. © Telelogic AB UML Testing Profile Test Control Test execution can be controlled is two ways –By defining the classifier behavior of a test context –Or by invoking the test cases from outside of the context

18. © Telelogic AB UML Testing Profile Test Control by ClassifierBehavior sd ATMContext invalidPIN() ref [verdict == fail] [verdict == pass] validWithdrawal() ref «testContext» ATMContext «testCase» +validWithdrawal() : Verdict +invalidPIN() : Verdict -authorizeCard() : Verdict

19. © Telelogic AB UML Testing Profile Test Control by Invocation * -context MyClass -verdict : Verdict «testContext» MyTestContext «testCase» +TC1() : Verdict +TC2() : Verdict -TC3() : Verdict ad myOp verdict := context.TC1() [else] [verdict == pass] verdict := context.TC2() +myOp()

20. © Telelogic AB UML Testing Profile Test Data Parameters, default values, return values are reused from UML without changes Logical partitions Pattern matching TDB

21. © Telelogic AB UML Testing Profile Test Deployment Test Deployment is important, but there is probably no difference between deploying test systems compared to ordinary UML systems. We will try to reuse the existing UML deployment concepts without changes Under investigation

22. © Telelogic AB UML Testing Profile Advanced Concepts Test Evaluation –Verdict –Validation action –Arbiter Default behavior Structural and behavioral reuse

23. © Telelogic AB UML Testing Profile Test Evaluation A number of concepts are used to evaluate test results –Verdicts –Validation actions –Arbiter

24. © Telelogic AB UML Testing Profile Verdicts and Validation Actions Each test component is associated with a verdict Validation Actions are used to set the verdict A Validation Action sets the verdict of a test component and reports the verdict to the Arbiter Client « sut » ATM storeCardData(cardData) isPinCorrect(pin) true display(Enter PIN) alt false setverdict(fail)

25. © Telelogic AB UML Testing Profile Arbiter A test component that knows how to evaluate the test behavior Separates test evaluation from test case specification –One behavior can yield different results Each test context has a default arbiter –Implicit connections to all test components –TTCN 3 semantics Advanced arbiters can be defined –Caters for more advanced semantics «arbiter» myArbiter setVerdict(v : Verdict)

26. © Telelogic AB UML Testing Profile Default behavior Defines behavior for unexpected observations Can be expressed using any UML behavior Can be applied to several different levels: –Test component –Message reception –Alternative statement –... Client « sut » ATM storeCardData(cardData) isPinCorrect(pin) true display(Enter PIN) sd authorizeCard enterPinDefault

27. © Telelogic AB UML Testing Profile Default expressed as interaction... sd enterPinDefault self alt getStatus statusOk ejectCard() display(Connection lost) setverdict(pass) setverdict(inconc) setverdict(fail)

28. © Telelogic AB UML Testing Profile...and as statemachine The representations are semantically equivalent enterPinDefault * getStatus / ^statusOk, setverdict(pass); display(Connection lost) / setverdict(inconc) ejectCard / setverdict(fail)

29. © Telelogic AB UML Testing Profile Meta Default Each test component has a meta default to cater for differences in semantics Can be redefined MetaDefault * * / defer

30. © Telelogic AB UML Testing Profile Reuse of Test Structure and Behavior

31. © Telelogic AB UML Testing Profile «testContext» ATMContext « sut » atm : BankATM hwe : HWEmulator be : BankEmulator atmPort hwCom bePortnetCom accnts : Account [0..*] ATM Client ac: AuthorizationContext

32. © Telelogic AB UML Testing Profile sd ValidWithdrawal hwe « sut » atm be ref ac.authorizeCard() selectOperation(withdrawal) true withdraw(amount) netCom findAccount(cardData) account display(Take cash) deliverMoney(amount) true debitAccount(account, amount) true atmPort

33. © Telelogic AB UML Testing Profile Summary The testing profile enables complete graphical specification of tests for conformance testing of UML models –Defined as a UML profile –Reuses existing UML concepts whenever possible –Designed for inter-operation with TTCN-3 and other technologies Due date –January 6th, 2003 More information –Testing Profile ConsortiumTesting Profile Consortium –OMGs Testing Profile pageOMGs Testing Profile page