1. Fraunhofer Institute for Open Communication Systems | Kaiserin-Augusta-Allee 31 | 10589 Berlin, Germany

2. From textual requirements to test models - Coping with natural language ambiguities for testing Marc-Florian Wendland | RCIS 2013 Industrial Day | 31 st May, 2013 | Paris, France

3. Agenda  Motivation  Requirements and model-based testing  Conclusion

4. Agenda  Motivation  Requirements and model-based testing  Conclusion

5. req. spec. designimpl. in field - bugs introduced - bugs found [4] Costs per bug found From textual requirements to test models Some facts about requirements [1] Costs of fixing software defects: logical architecture1.000 € technical architecture4.000 € realisation12.000 € acceptance / rollout48.000 € in field90.000 € [2] As much as 60% of all defects in a systems lifetime originate from deficient requirements [3] 5% 55% 10% 30% 40% 15% 45%

6. Test AnalysisReq. Engineering Test Design System Dev. From textual requirements to test models Impact network of requirements Requirements Test Requirements Test Case > 1..* System/Software Requirements Specification (SRS) System/Software Specification > Implementation > System/Software Test Specification > Test Exec. >

7. From textual requirements to test models Requirements specification techniques Unrestricted NL Restricted NL Visual Notation Formal language Formalism and precision Granted freedom to engineers We need an approach that copes with natural language ambiguity and imprecision. Suitable for automated transformation Acceptance in industry

8. From textual requirements to test models Satisfaction with requirements engineering tasks Requirements elicitation Requirements analysis Requirements specification Requirements validation Requirements management high medium low 50% of all respondents sees unsufficiently specified requirements as biggest challenge for testing [5]

9. Introduction State of the art in test design – Traditional testing Implicit knowledge Requirements Manual derivation Test Plan Test specification Implicit knowledge (mental model) of the test basis and system under test (SUT) Quality of test specification depends on the ingenuity and experiences of the tester Time consuming and prone to errors Not repeatability, lack of systematics Often not documented Loss of knowledge possible Intellectual task Clerical task

10. Agenda  Motivation  Requirements and model-based testing  Conclusion

11. From textual requirements to test models State of the art in automated test design – Model-Based Testing Implicit knowledge Requirements Formalisation Test Plan Test Model TC SUT TC SUT TC SUT Implicit/imperfect knowledge is made explicit and (semi-)perfect Test design is done on the model Repeatable, comprehensible and systematic Prevents loss of knowledge Model is self-documented Quality of test model depends on experiences and ingenuity of the tester Automated clerical task Intellectual task

12. From textual requirements to test models The MBT Dilemma System Model-based -Requires well-formed system model -Missing independency of testing side -Error propagation into the test model --> lacks validation potential Test Model-based -Binds resources -Modeling is error prone -Requires modeling experiences /education on testing side -->costly approach

13. From textual requirements to test models Requirements models as starting point We need a methodology, which ensures the requirements model is complete, unambiguous, consistent, correct and testable (IEEE 830)  Introduce a dedicated requirements model  Formalize the externally observable behavior of the system based on its requirements  Requirements model can be exploited to partially facilitate activities both system and testing side  Shared source of information for further development

14. From textual requirements to test models What is Behavior Engineering? Requirements Engineering (RE) Requirements Elicitation Requirements Identification Requirements Specification Behavior Engineering (BE) Requirements Formalisation Requirements Integration Requirements Refinement/Completion Requirements Validation & Verification Document-centric Model-centric Phases of BE Formalisation (RBT) Integration (pIBT) Validation (IBT)

15. From textual requirements to test models Requirements Formalisation – Translation Requirement 1 There is a single control button available for the use of the oven. If the oven door is closed and the user push the button, the oven will start cooking (that is, energize the power-tube) for one minute. RBT CT Translate each single requirement independently User

16. From textual requirements to test models Requirements Integration Precondition Axiom Every constructive, implementable, individual functional requirement of a system, expressed as a behavior tree, has associated with it a precondition that needs to be satisfied in order for the behavior encapsulated in the functional requirement to be applicable." Interaction Axiom For each individual functional requirement of a system, expressed as a behavior tree, the precondition it needs to have satisfied in order to exhibit its encapsulated behavior, must be established by the behavior tree of at least one other functional requirement that belongs to the set of functional requirements of the system."

17. From textual requirements to test models Requirements Integration Precondition Axiom Every constructive, implementable, individual functional requirement of a system, expressed as a behavior tree, has associated with it a precondition that needs to be satisfied in order for the behavior encapsulated in the functional requirement to be applicable." Interaction Axiom For each individual functional requirement of a system, expressed as a behavior tree, the precondition it needs to have satisfied in order to exhibit its encapsulated behavior, must be established by the behavior tree of at least one other functional requirement that belongs to the set of functional requirements of the system."

18. From textual requirements to test models (Test) Analysis model as result of BE IBTCT Fokus!BE BE with UML Fokus!BE BE with UML

19. Agenda  Motivation  Requirements and model-based testing  Conclusion

20. From textual requirements to test models Conclusion  Requirements are still mostly specified textually with unrestricted natural language  Requirements validation techniques need to cope with unrestricted natural language  BE is a methodology that is aiming at validating the requirements by integration  BE alleviates the transition from a requirements model to further models  Testing is highly depending on the quality of the requirements  MBT is highly depending on the quality of the model that model the requirements  BE is predestined to tackle current model-based testing challenges  BE with UML seems wise since system/test models are often represented in UML/UTP

21. Thanks for your attention. Questions?

22. References [1] Boehm, Berry: Software Engineering Economics, Englewood Cliffs, NJ : Prentice-Hall, 1981. ISBN 0-13-822122-7 [2] Spillner, A: SQS Empirische Daten aus 3000 IT-Projekten. 1st Testing Day Franken, Nürnberg, 10-March-2009 [3] Berry, D. M.: Formal methods: the very idea - some thoughts about why they work when they work. Sci. Comput. Program., 42(1):11–27, 2002 [4] Standish Group: ChAOS Reports. URL: http://www.standishgroup.com. Last visit: January 05, 2012 [5] SwissQ, Trends und Benchmark Report Schweiz, Wo stehen wir, wo geht es hin? Testing 2013, http://de.slideshare.net/swissq/testing-trends-und- benchmarks-2013-webhttp://de.slideshare.net/swissq/testing-trends-und-