SDS Foil no 1 V&V&S Verification, Validation and Synthesis: doing away with defects Verification, Validation and Synthesis: doing away with defects

SDS Foil no 2 Quality and quality assurance methods Quality: process quality = meeting the specification. system quality = playing the role required by the environment. Methods: 1.Constructive methods that aim to generate the right results in the first place =>The SDL methodology, program generation. 2.Corrective methods that aim to detect errors and make corrections. =>Verification and validation. Quality: process quality = meeting the specification. system quality = playing the role required by the environment. Methods: 1.Constructive methods that aim to generate the right results in the first place =>The SDL methodology, program generation. 2.Corrective methods that aim to detect errors and make corrections. =>Verification and validation.

SDS Foil no 3 Verification and Validation Barry Boehm (Boehm, 1981): Verification: To establish the truth of correspondence between a software product and its specification (from the Latin veritas, “truth”). Validation: To establish the fitness or worth of a software product for its operational mission (from the Latin valere, “to be worth”). Verification: Are we building the product right? Validation: are we building the right product?” Barry Boehm (Boehm, 1981): Verification: To establish the truth of correspondence between a software product and its specification (from the Latin veritas, “truth”). Validation: To establish the fitness or worth of a software product for its operational mission (from the Latin valere, “to be worth”). Verification: Are we building the product right? Validation: are we building the right product?”

SDS Foil no 4 Verification and Validation in TIMe imple- mentation instance system domain config. speci- fication Application design Framework design Architecture design needs Developers Validate validate Market and users verify

SDS Foil no 5 Synthesize -design -implement. Requirements Design Implementation Results Reviews analysis Testing Corrective methods (V&V) Defects and errors Constructive methods Techniques

SDS Foil no 6 Analysi s Syntax analysis: by editor Static semantic analysis: by analyzer Dynamic semantic analysis: by simulation and reachability analysis (modelchecking) Verification: by manual inspection and checking formal relationships between models Validation: by manual inspection, by testing and formal compatibility checks Syntax analysis: by editor Static semantic analysis: by analyzer Dynamic semantic analysis: by simulation and reachability analysis (modelchecking) Verification: by manual inspection and checking formal relationships between models Validation: by manual inspection, by testing and formal compatibility checks

SDS Foil no 7 Static signal check 1 Derive valid input and output signal sets from process graph Check signal sets against signal lists on each connection: P2in = L2 + L3 + L5 P2out = L1 + L4 + L6 Check signal destinations: P2 out-to-p1 = L1 P2 out-to-p3 = L4 P2 out-to-p3 = L6 Derive valid input and output signal sets from process graph Check signal sets against signal lists on each connection: P2in = L2 + L3 + L5 P2out = L1 + L4 + L6 Check signal destinations: P2 out-to-p1 = L1 P2 out-to-p3 = L4 P2 out-to-p3 = L6

SDS Foil no 8 Static signal check 2 Valid input and output signal sets against gate signal lists once for each type Signal lists against each other for each connection Signal destinations Valid input and output signal sets against gate signal lists once for each type Signal lists against each other for each connection Signal destinations

SDS Foil no 9 Static signal check 3 Find logical signal routes by signal list partitioning conserving flows Signal sets against signal lists Signal destinations Find logical signal routes by signal list partitioning conserving flows Signal sets against signal lists Signal destinations Who talks together here?

SDS Foil no 10 Liveness = something good will eventually happen (system specific) safety = nothing bad will ever happen (system independent) 1. Reachability analysis Full (up to 10**5 states) Controlled partial (up to 10**8 states) Random simulation (larger) 2. Structural reasoning Invariants Logical inference on equations, rules, axioms Model checking 3. Execution Simulation Testing Liveness = something good will eventually happen (system specific) safety = nothing bad will ever happen (system independent) 1. Reachability analysis Full (up to 10**5 states) Controlled partial (up to 10**8 states) Random simulation (larger) 2. Structural reasoning Invariants Logical inference on equations, rules, axioms Model checking 3. Execution Simulation Testing Dynamic analysis

SDS Foil no 11 Reachability analysis Iteratively generate all reachable system states Analyse each state for deadlocks; incompleteness; other errors. Analyse the generated graph partial deadlock; liveness; etc. Iteratively generate all reachable system states Analyse each state for deadlocks; incompleteness; other errors. Analyse the generated graph partial deadlock; liveness; etc.

SDS Foil no 12 Are there any problems here?

SDS Foil no 13 Step 1: Make transition charts

SDS Foil no 14 Step 2: Generate reachability graph Find all possible behaviours considering that a signal transfer takes two steps: send consume Find all possible behaviours considering that a signal transfer takes two steps: send consume

SDS Foil no 15 Findings The errors: Unspecified receptions and deadlocks may occur The reason: mixed initiatives not properly handled The errors: Unspecified receptions and deadlocks may occur The reason: mixed initiatives not properly handled Graph “explodes” because independent behaviours are interleaved and signals are queued Many paths lead to the same error Graph “explodes” because independent behaviours are interleaved and signals are queued Many paths lead to the same error Is there a simpler way??

SDS Foil no 16 Working directly in SDL Association role behaviour = the visible behaviour of one object when observed from another (associated) object. Mixed initiative state = a state with inputs from two independent sources (i.e. association roles) Association role behaviour = the visible behaviour of one object when observed from another (associated) object. Mixed initiative state = a state with inputs from two independent sources (i.e. association roles) invisible from B mixed initiative state

SDS Foil no 17 Role behaviour and input consistent roles Deriving a role behaviour: Replace invisible inputs by “none” (or just make a mental note) Remove invisible outputs (or just ignore them) The result is a process graph with only visible inputs and outputs. (or just a mental view on the original graph) Input consistency: An invisible transition is a transition with a none input. An invisible transition is input consistent iff the next-state explicitly accepts all the visible inputs of the (present) state. The role is input consistent iff every invisible transition in the role is input consistent. Deriving a role behaviour: Replace invisible inputs by “none” (or just make a mental note) Remove invisible outputs (or just ignore them) The result is a process graph with only visible inputs and outputs. (or just a mental view on the original graph) Input consistency: An invisible transition is a transition with a none input. An invisible transition is input consistent iff the next-state explicitly accepts all the visible inputs of the (present) state. The role is input consistent iff every invisible transition in the role is input consistent. 1 none Sb 3 Gb 5 none Qb 1 Sa Ga 8 Qa 1 A invisible transitions 1 and 3 are not input consistent because 3 does not accept Sa 5 and 1 are input consistent because 1 accepts more than 5

SDS Foil no 18 How to ensure input consistency ? When checking - for each role: Find invisible transitions Check that every invisible transition is input consistent Consider transitions from mixed initiative states first. When designing: for each state, in particular mixed initiative states, check for each role that all transitions are input consistent. When checking - for each role: Find invisible transitions Check that every invisible transition is input consistent Consider transitions from mixed initiative states first. When designing: for each state, in particular mixed initiative states, check for each role that all transitions are input consistent. 1 none Sb 3 Gb 5 none Qb 1 Sa Ga 8 Qa 1 A Mixed initiative state 1 and 3 are not input consistent 5 and 1 are input consistent The intuition behind this is that the associated process will not immediately observe an invisible transition and therefore the same visible outputs should be accepted in both states invisible transition

SDS Foil no 19 S-Rule: Input consistent process Design SDL processes such that all their role behaviours, i.e. the behaviour visible to processes in their environment, are input consistent. In particular check the vicinity of mixed initiative states

SDS Foil no 20 SDL and MSC The SDL behaviour shall “contain” all the MSC behaviours in addition the SDL behaviour may contain: ­ cases not treated by the MSCs ­ behaviour on other interfaces ­ behaviour in other services The SDL behaviour shall “contain” all the MSC behaviours in addition the SDL behaviour may contain: ­ cases not treated by the MSCs ­ behaviour on other interfaces ­ behaviour in other services SDL (object) behaviour tree MSCs MSC (instance) behaviour tree SDL structure “contains”

SDS Foil no 21 Verifying SDL generate all possible traces defined by the MSCs check that all traces may execute in the global behaviour tree for the corresponding SDL (using the Validator) generate all possible traces defined by the MSCs check that all traces may execute in the global behaviour tree for the corresponding SDL (using the Validator) SDL behaviour tree MSCsMSC behaviour tree SDL

SDS Foil no 22 Synthesizing behaviour 1 process ACSystem User AC System Code OK msc User_accepted Unlock Card out Idle Door unlocked UserAC System Code NOK msc User_rejected Card out Idle Door unlocked Code CardOut OK Unlock Code CardOut NOK How to correct this?

SDS Foil no 23...by combining and branching Idle Door unlocked Code CardOut OK Unlock Code CardOut NOK Idle Door unlocked CardOut OK Unlock CardOut NOK Code Validate OKNOK

SDS Foil no 24 Validate PIN wait for PIN Idle Process Type UserServer DCL Cardid, PIN, zone integer; Synthesizing behaviour 2 MSC EnterZone User UserServer Authorizer Card [cardid] ValidatePIN [cardid,PIN] Authenticator Givepin [PIN] Enter PIN exc NOK Illegal PIN OK ValidateRights [cardid,zone] exc NOK AccessDenied OK AccessGranted ValidatePIN (cardid, PIN) GivePIN (PIN) Enter PIN Card (Cardid)

SDS Foil no 25 Synthesizing behaviour 3 MSC EnterZone User UserServer Authorizer Card [cardid] ValidatePIN [cardid,PIN] Authenticator Givepin [PIN] Enter PIN exc NOK Illegal PIN OK ValidateRights [cardid,zone] exc NOK AccessDenied OK AccessGranted Idle ValidateRights Validate PIN Process Type UserServer Idle AccessGranted OK AccessDenied NOK ValidateRights (cardid, zone) OK IllegalPIN NOK

SDS Foil no 26 Synthesizing behaviour 4 General approach: Split each MSC into instances, and make corresponding SDL fragments. Combine all the SDL fragments for a given process into a more complete behaviour Remove non-determinism by combining and adding decisions or states Add unspecified behaviour details Add exceptions Consider all possibilities in each state: ensure that all roles are input consistent! General approach: Split each MSC into instances, and make corresponding SDL fragments. Combine all the SDL fragments for a given process into a more complete behaviour Remove non-determinism by combining and adding decisions or states Add unspecified behaviour details Add exceptions Consider all possibilities in each state: ensure that all roles are input consistent!

SDS Foil no 27 Specification and design Specification Design Verify MSC Synthesize Verify SDL Decompose Validate interface

SDS Foil no 28 Validating interfaces Static signal checks: are all output signals received? Aligning MSC: do both sides align with the same MSCs? Comparing role behaviours manually: are required roles contained in provided roles? Simulating with SDT: will the interface work for all simulated cases? Reachability analysis with SDT: will the validator find errors using state space exploration? Static signal checks: are all output signals received? Aligning MSC: do both sides align with the same MSCs? Comparing role behaviours manually: are required roles contained in provided roles? Simulating with SDT: will the interface work for all simulated cases? Reachability analysis with SDT: will the validator find errors using state space exploration? structure Type

SDS Foil no 29 Verify Aspects: verifying MSC decomposition verifying SDL behaviour: object behaviour composition behaviour Aspects: verifying MSC decomposition verifying SDL behaviour: object behaviour composition behaviour context structure behaviour Verifying SDL behaviour verifying MSC decomposition Principle: every trace defined by the MSCs shall be possible in the SDL Principle: every trace defined by the MSCs shall be possible in the SDL aligning

SDS Foil no 30 MSC and testing system MSC system under test system under test application infrastructure support sw hw Abstract test suite Executable test suite Tester verdict

SDS Foil no 31 Testing what? Application (SDL). Application support: Protocols. Input-output modules. General support: Routing. Operating system. SDL runtime support. Error handling. Test and debug facilities. Performance (non-functional): Real time response, delays. Traffic handling capacity. Load control. Application (SDL). Application support: Protocols. Input-output modules. General support: Routing. Operating system. SDL runtime support. Error handling. Test and debug facilities. Performance (non-functional): Real time response, delays. Traffic handling capacity. Load control.