1. SDS Foil no 1 V&V&S Verification, Validation and Synthesis: doing away with defects Verification, Validation and Synthesis: doing away with defects

2. SDS Foil no 2 Verification and Validation Barry Boehm (Boehm, 1981): Verification: To establish the truth of correspondence between a software product and its specification (from the Latin veritas, “truth”). Validation: To establish the fitness or worth of a software product for its operational mission (from the Latin valere, “to be worth”). Verification: are we building the product right? Validation: are we building the right product?” Barry Boehm (Boehm, 1981): Verification: To establish the truth of correspondence between a software product and its specification (from the Latin veritas, “truth”). Validation: To establish the fitness or worth of a software product for its operational mission (from the Latin valere, “to be worth”). Verification: are we building the product right? Validation: are we building the right product?”

3. SDS Foil no 3 3 Recall the important errors we make: We plan a functionality that is not in harmony with the purpose of the system. We define the functionality in a way that is internally inconsistent. We implement a different functionality than the one we planned. We plan a functionality that is not in harmony with the purpose of the system. We define the functionality in a way that is internally inconsistent. We implement a different functionality than the one we planned.

4. SDS Foil no 4 4 Verification and validation Validate that: the functional requirements are right the functional design is right the execution is right Verify that: that the design satisfy requirements that the implementation satisfies the design Check for consistency and properties Validate that: the functional requirements are right the functional design is right the execution is right Verify that: that the design satisfy requirements that the implementation satisfies the design Check for consistency and properties Validate verify Validate Developers check

5. SDS Foil no 5 Quality and quality assurance methods Quality: process quality = meeting the specification. system quality = playing the role required by the environment. Methods: 1.Constructive methods that aim to generate the right results in the first place =>Model driven development, design synthesis, readable models. 2.Corrective methods that aim to detect errors and make corrections. =>Verification and validation. Quality: process quality = meeting the specification. system quality = playing the role required by the environment. Methods: 1.Constructive methods that aim to generate the right results in the first place =>Model driven development, design synthesis, readable models. 2.Corrective methods that aim to detect errors and make corrections. =>Verification and validation.

6. SDS Foil no 6 Model, Check, Synthesize -design -implement. Requirements Design Implementation Results Reviews Analysis Testing Corrective methods (V&V) Defects and errors Constructive methods Techniques

7. SDS Foil no 7 Techniques for checking and analyzing models Syntax analysis: by model editor Static semantic analysis: by model analyzer/inspector Static interface compatibility Dynamic semantic analysis: by tools for reachability analysis (also called modelchecking) Dynamic interface compatibility Execution: by simulation and testing Verification: by manual inspection and checking formal relationships between models Validation: by manual inspection, by simulation, testing and interface compatibility checks Syntax analysis: by model editor Static semantic analysis: by model analyzer/inspector Static interface compatibility Dynamic semantic analysis: by tools for reachability analysis (also called modelchecking) Dynamic interface compatibility Execution: by simulation and testing Verification: by manual inspection and checking formal relationships between models Validation: by manual inspection, by simulation, testing and interface compatibility checks

8. SDS Foil no 8 Static interfaces Java Interfaces – One-way method invocations with parameters CORBA Interfaces –defined using IDL Web services – One way remote invocations with parameters - defined using WSDL APIs – One-way invocations with parameters. UML Interfaces – One-way invocations or messages with parameters SDL Interfaces – Two way asynchronous messages with parameters UML Collaborations – Two way invocations or messages with parameters Java Interfaces – One-way method invocations with parameters CORBA Interfaces –defined using IDL Web services – One way remote invocations with parameters - defined using WSDL APIs – One-way invocations with parameters. UML Interfaces – One-way invocations or messages with parameters SDL Interfaces – Two way asynchronous messages with parameters UML Collaborations – Two way invocations or messages with parameters

9. SDS Foil no 9 Static interface checks illustrated using SDL (1) Derive valid input and output signal sets from process graph Check signal sets against signal lists on each connection: P2in = L2 + L3 + L5 P2out = L1 + L4 + L6 Check signal destinations: P2 out-to-p1 = L1 P2 out-to-p3 = L4 P2 out-to-env = L6 Derive valid input and output signal sets from process graph Check signal sets against signal lists on each connection: P2in = L2 + L3 + L5 P2out = L1 + L4 + L6 Check signal destinations: P2 out-to-p1 = L1 P2 out-to-p3 = L4 P2 out-to-env = L6

10. SDS Foil no 10 Static interface checks (2) Valid input and output signal sets against gate signal lists once for each type Signal lists against each other for each connection Signal destinations Valid input and output signal sets against gate signal lists once for each type Signal lists against each other for each connection Signal destinations

11. SDS Foil no 11 Static interface checks (3) Find logical signal routes by signal list partitioning conserving flows Signal sets against signal lists Signal destinations Find logical signal routes by signal list partitioning conserving flows Signal sets against signal lists Signal destinations Who talks together here?

12. SDS Foil no 12 Liveness = something good will eventually happen, e.g. sometimes a service is successful (system specific) safety = nothing bad will ever happen, e.g. no deadlocks (system independent) 1. Reachability analysis Full (up to 10**5 states) Controlled partial (up to 10**8 states) Random simulation (larger) 2. Structural reasoning Invariants Logical inference on equations, rules, axioms Model checking 3. Execution Simulation Testing Liveness = something good will eventually happen, e.g. sometimes a service is successful (system specific) safety = nothing bad will ever happen, e.g. no deadlocks (system independent) 1. Reachability analysis Full (up to 10**5 states) Controlled partial (up to 10**8 states) Random simulation (larger) 2. Structural reasoning Invariants Logical inference on equations, rules, axioms Model checking 3. Execution Simulation Testing Dynamic analysis

13. SDS Foil no 13 Reachability analysis Iteratively generate all reachable system states Analyse each state for deadlocks; incompleteness; other errors. Analyse the generated graph partial deadlock; liveness; etc. Iteratively generate all reachable system states Analyse each state for deadlocks; incompleteness; other errors. Analyse the generated graph partial deadlock; liveness; etc.

14. SDS Foil no 14 Are there any problems here?

15. SDS Foil no 15 Step 1: Make transition charts

16. SDS Foil no 16 Step 2: Generate reachability graph Find all possible behaviours considering that a signal transfer takes two steps: send consume Find all possible behaviours considering that a signal transfer takes two steps: send consume

17. SDS Foil no 17 Findings The errors: Unspecified receptions and deadlocks may occur The reason: mixed initiatives not properly handled The errors: Unspecified receptions and deadlocks may occur The reason: mixed initiatives not properly handled Graph “explodes” because independent behaviours are interleaved and signals are queued Many paths lead to the same error Graph “explodes” because independent behaviours are interleaved and signals are queued Many paths lead to the same error Is there a simpler way??

18. SDS Foil no 18 Analyzing role behavior directly in SDL Association role behaviour = the visible behaviour of one object when observed from another (associated) object. Mixed initiative state = a state with inputs from two independent sources (i.e. association roles) Association role behaviour = the visible behaviour of one object when observed from another (associated) object. Mixed initiative state = a state with inputs from two independent sources (i.e. association roles) invisible from B mixed initiative state

19. SDS Foil no 19 Role behaviour and input consistent roles Deriving a role behaviour: Replace invisible inputs by “none” (or just make a mental note) Remove invisible outputs (or just ignore them) The result is a process graph with only visible inputs and outputs. (or just a mental view on the original graph) Input consistency: An invisible transition is a transition with a none input. An invisible transition is input consistent iff the next-state explicitly accepts all the visible inputs of the (present) state. The role is input consistent iff every invisible transition in the role is input consistent. Deriving a role behaviour: Replace invisible inputs by “none” (or just make a mental note) Remove invisible outputs (or just ignore them) The result is a process graph with only visible inputs and outputs. (or just a mental view on the original graph) Input consistency: An invisible transition is a transition with a none input. An invisible transition is input consistent iff the next-state explicitly accepts all the visible inputs of the (present) state. The role is input consistent iff every invisible transition in the role is input consistent. 1 none Sb 3 Gb 5 none Qb 1 Sa Ga 8 Qa 1 A invisible transitions Are there input inconsistent transitions here?

20. SDS Foil no 20 Role behaviour and input consistent roles Deriving a role behaviour: Replace invisible inputs by “none” (or just make a mental note) Remove invisible outputs (or just ignore them) The result is a process graph with only visible inputs and outputs. (or just a mental view on the original graph) Input consistency: An invisible transition is a transition with a none input. An invisible transition is input consistent iff the next-state explicitly accepts all the visible inputs of the (present) state. The role is input consistent iff every invisible transition in the role is input consistent. Deriving a role behaviour: Replace invisible inputs by “none” (or just make a mental note) Remove invisible outputs (or just ignore them) The result is a process graph with only visible inputs and outputs. (or just a mental view on the original graph) Input consistency: An invisible transition is a transition with a none input. An invisible transition is input consistent iff the next-state explicitly accepts all the visible inputs of the (present) state. The role is input consistent iff every invisible transition in the role is input consistent. 1 none Sb 3 Gb 5 none Qb 1 Sa Ga 8 Qa 1 A invisible transitions 1 and 3 are not input consistent because 3 does not accept Sa 5 and 1 are input consistent because 1 accepts more than 5

21. SDS Foil no 21 How to ensure input consistency ? When checking - for each role: Find invisible transitions Check that every invisible transition is input consistent Consider transitions from mixed initiative states When designing: for each mixed initiative state, check for each role that all transitions are input consistent. When checking - for each role: Find invisible transitions Check that every invisible transition is input consistent Consider transitions from mixed initiative states When designing: for each mixed initiative state, check for each role that all transitions are input consistent. 1 none Sb 3 Gb 5 none Qb 1 Sa Ga 8 Qa 1 A Mixed initiative state 1 and 3 are not input consistent 5 and 1 are input consistent The intuition behind this is that the associated process will not immediately observe an invisible transition and therefore may send the same output in both states of the invisible transition. invisible transition

22. SDS Foil no 22 S-Rule: Input consistent process Design SDL processes such that all their role behaviours, i.e. the behaviour visible to processes in their environment, are input consistent. Check the mixed initiative states

23. SDS Foil no 23 Dynamic Interfaces Not only static lists of operations/messages But also the the dynamic behavior that must be obeyed Defined by (dual) interface role behaviors That are projections of local behavior onto interfaces Not only static lists of operations/messages But also the the dynamic behavior that must be obeyed Defined by (dual) interface role behaviors That are projections of local behavior onto interfaces 1 none Sb 3 Gb 5 none Qb 1 Sa Ga 8 Qa 1 A 1a 1 B 1b 1

24. SDS Foil no 24 Compatibility and interface roles Interface roles are ”projected and distilled” behaviours visible on interfaces, or associations. Given two state machines A and B: 1.Derive the interface role behaviours ­Project: hide and transform ­Distill: gather, minimize and merge ­Detect inconsistencies 2.Correct inconsistencies and repeat from 1 3.Validate role compatibility Interface roles are ”projected and distilled” behaviours visible on interfaces, or associations. Given two state machines A and B: 1.Derive the interface role behaviours ­Project: hide and transform ­Distill: gather, minimize and merge ­Detect inconsistencies 2.Correct inconsistencies and repeat from 1 3.Validate role compatibility A B 1a 1b 1 3 1 2 2 session

25. SDS Foil no 25 Two interface roles (a-roles) in state transition form Interface roles (also called a-roles) define distilled interface behavior dual roles can be obtained by: miroring and conflict resoulution Here mixed initiatives show up as nodes with both output and input transitions any conflicts here? Interface roles (also called a-roles) define distilled interface behavior dual roles can be obtained by: miroring and conflict resoulution Here mixed initiatives show up as nodes with both output and input transitions any conflicts here? a1 a2 a4 a3 !m1 ?m2 ?m3 !m4 b1 b2 b4 b3 ?m1 !m2 !m3 ?m4 a5 b4 ab

26. SDS Foil no 26 conflict resolution after mixed initiatives Two kinds of behaviour: service behaviour and conflict reolution behaviour The service/feature behaviour of a dual roles can normally be obtained by mirroring Conflict resolution in general, can not be obtained by mirroring. Hence, a dual role may contain two parts: a mirrored part providing the core service logic and a conflict resolution part. In this example a and b are dual! Two kinds of behaviour: service behaviour and conflict reolution behaviour The service/feature behaviour of a dual roles can normally be obtained by mirroring Conflict resolution in general, can not be obtained by mirroring. Hence, a dual role may contain two parts: a mirrored part providing the core service logic and a conflict resolution part. In this example a and b are dual! a1 a2 a4 a3 !m1 ?m2 ?m3 !m4 b1 b2 b4 b3 ?m1 !m2 !m3 ?m4 a b a5 ?m2 !m4 b6 a1 a2 a4 a3 !m1 ?m2 ?m3 !m4 ?m2 b1 b2 b4 b3 ?m1 !m2 !m3 ?m4 ?m1 a5 b4 ?m1 a4 b4

27. SDS Foil no 27 Correcting for input inconsistency = resolving mixed initiatives Decide on resolution strategy: select the winning initiative if they have different goals – typically using priority Add transitions for all inputs that are not handled Check again to ensure that result is input consistent Decide on resolution strategy: select the winning initiative if they have different goals – typically using priority Add transitions for all inputs that are not handled Check again to ensure that result is input consistent

28. SDS Foil no 28 The taxi central with interface collaborations op[n] Ii[m]ACD TaxiDispatch taxi[l] TaxiCentral line Interface taxi Interface taxi Dispatch operator Interface acdTd acdLiacdOp td li td taxi op Interface behavior is here explicitly defined by the collaborations

29. SDS Foil no 29 Sequence diagrams for Interface behavior Global behavior Good overview Normally incomplete Roles, not complete objects Global behavior Good overview Normally incomplete Roles, not complete objects op acdOp operatorInterface ref msc operatorInterface opacdOp available call(caller) answer end loop tourOrder(TourInfo) confirm msc operatorInterface

30. SDS Foil no 30 Compatibility with sequence diagrams What does it mean? Normally that roles exhibit at least the specified sequences! Is this sufficient? What about the unspecified sequences? What does it mean? Normally that roles exhibit at least the specified sequences! Is this sufficient? What about the unspecified sequences? op acdOp operatorInterface ref msc operatorInterface opacdOp available call(caller) answer end loop tourOrder(TourInfo) confirm msc operatorInterface

31. SDS Foil no 31 Components and interactions, e.g. SDL and MSC The state machine behaviour shall “contain” all the interaction behaviours in addition the state machine behaviour may contain: ­ cases not treated by the MSCs ­ behaviour on other interfaces ­ behaviour in other services The state machine behaviour shall “contain” all the interaction behaviours in addition the state machine behaviour may contain: ­ cases not treated by the MSCs ­ behaviour on other interfaces ­ behaviour in other services SDL (object) behaviour tree MSCs MSC (instance) behaviour tree SDL structure “contains”

32. SDS Foil no 32 Verifying SDL generate all possible traces defined by the MSCs check that all traces may execute in the global behaviour tree for the corresponding SDL (using the Validator) generate all possible traces defined by the MSCs check that all traces may execute in the global behaviour tree for the corresponding SDL (using the Validator) SDL behaviour tree MSCsMSC behaviour tree SDL

33. SDS Foil no 33 process taxi confirm none 1 33 2 tourOrder(ti) none 1 5 2 DCL ti TourInfo; finished availableunavailable available pickup tourOrder(ti) 3 pickup tourOrder(ti) available 1 pickup 4 3 2 none confirm finished 4 unavailable 1 5 available 2 process td DCL ti TourInfo; pickup 4 taxi td taxiInterface Role state machines for complete interface behvior With resolution of mixed initiatives A class/type playing the taxi role shall have projected interface behavior equivalent to the taxi role defined here With resolution of mixed initiatives A class/type playing the taxi role shall have projected interface behavior equivalent to the taxi role defined here

34. SDS Foil no 34 Recall compatibility and interface roles Interface roles are ”projected and distilled” behaviours visible on interfaces, or associations. Given two state machines A and B: 1.Derive the interface role behaviours ­Project: hide and transform ­Distill: gather, minimize and merge ­Detect inconsistencies 2.Correct inconsistencies and repeat from 1 3.Validate role compatibility Interface roles are ”projected and distilled” behaviours visible on interfaces, or associations. Given two state machines A and B: 1.Derive the interface role behaviours ­Project: hide and transform ­Distill: gather, minimize and merge ­Detect inconsistencies 2.Correct inconsistencies and repeat from 1 3.Validate role compatibility A B 1a 1b 1 3 1 2 2 session

35. SDS Foil no 35 A-roles and collaborations for semantic interfaces... a basis for incremental and scalable compatibility checks... as well as for service discovery and service selection... a basis for incremental and scalable compatibility checks... as well as for service discovery and service selection If Class_A is typed with Call.a (alternatively by A alone) and Class_B is typed with Call.b (alternatively by B alone) Consistency of links can be checked statically and Service opportunities discovered and selected dynamically Call a b Class_A S-roleA Call B A 1 a Class_B Call S-roleB B A 1 b X:Class_A 1 y:Class_B 1

36. SDS Foil no 36 Dual roles are fully compatible a-roles: 1.iff the original state machine is input consistent, then the dual a-role may be constructed, 2.and used to discover, compare and select compatible services, 3.and to partially synthesise state machines. 1.iff the original state machine is input consistent, then the dual a-role may be constructed, 2.and used to discover, compare and select compatible services, 3.and to partially synthesise state machines. A 1a 1b 2 session B 1c C 1d 1 2 D 1b 2b 3b 1 1 1 3 3 3

37. SDS Foil no 37 Using collaborations to type interfaces Compatibility of collaboration roles modelchecked at design time Conformance with contract checked for each interface at design time Simple compatibility assurance at runtime Compatibility of collaboration roles modelchecked at design time Conformance with contract checked for each interface at design time Simple compatibility assurance at runtime Application Stock Info Server Sq:StockQuote requestor provider Sq:StockQuote requestorprovider

38. SDS Foil no 38 Stock quote Sq:StockQuote requestorprovider

39. SDS Foil no 39 Conformance with interface roles 1.Project component behavior to interface role behavior 2.Compare interface role behavior with collaboration role behavior: are they equivalent or substitutable? 1.Project component behavior to interface role behavior 2.Compare interface role behavior with collaboration role behavior: are they equivalent or substitutable? Application Stock Info Server Project Compare Sq:StockQuote

40. SDS Foil no 40 Equivalent state machines Informally: Two states are equivalent if they have the same set of transitions and the next states are equivalent. Two state machines are equivalent if their initial states are equivalent Equivalent state machines cannot be distinguished by an external observer, and can replace each other A role state machine can be replaced by a subtype (with the extension guarded by inputs) Informally: Two states are equivalent if they have the same set of transitions and the next states are equivalent. Two state machines are equivalent if their initial states are equivalent Equivalent state machines cannot be distinguished by an external observer, and can replace each other A role state machine can be replaced by a subtype (with the extension guarded by inputs)

41. SDS Foil no 41 Interfaces in Access control 41

42. SDS Foil no 42 1.Validate each collaboration separately 2.Check that both sides play compatible roles 3.Validate the ordering (flows) Collaborations as semantic interfaces panel userAgt StoreCID Enter Card Enter PIN Deny- eject Pass- eject Card Entry ceu Accept au Deny du PIN Entry peu Card Entry cep PIN Entry pep Deny dp Accept ap Validate vu Open ou

43. SDS Foil no 43 Arctis ESMs: dynamic interfaces of building blocks One ESM per building block Not one per interface The inner behavior of a building block must be compatible with its ESM When analyzing a composition of blocks, the inner behavior of each block is replaced by its ESM One ESM per building block Not one per interface The inner behavior of a building block must be compatible with its ESM When analyzing a composition of blocks, the inner behavior of each block is replaced by its ESM

44. SDS Foil no 44 Types of interfaces and roles Card Entry ceu cep A-role role Internal IF, ESM Internal IF, ESM Internal IF, ESM Semantic Interface Collaboration activity

45. SDS Foil no 45 Synthesizing state machines from sequence diagrams 1.Direct realization: A projection of the global behavior onto the component state machines. 1.Find realization problems: Event orderings that are possible in the direct realization, but not specified. Caused by mixed initiatives, weak sequencing, non-causal ordering 1.Resolve the realization problems 1.For incomplete sequence diagrams: Add the omitted behavior 1.Direct realization: A projection of the global behavior onto the component state machines. 1.Find realization problems: Event orderings that are possible in the direct realization, but not specified. Caused by mixed initiatives, weak sequencing, non-causal ordering 1.Resolve the realization problems 1.For incomplete sequence diagrams: Add the omitted behavior