Component-Based Software Engineering Testing Ivica Crnkovic Mälardalen University, Department of Computer Engineering Sweden ivica.crnkovic@mdh.se http://www.idt.mdh.se/~icc Page 1, 16 September 2018

Västerås Stockholm Page 2, 16 September 2018

Ivica Crnkovic Professor at Mälardalen Högskola, Västerås Industrial Software Engineering Computer Science Laboratory chair Many years at ABB CM and Development Environments Development and management Interests Component-based Software Engineering Software engineering in general Software Configuration Management Software Architecture Software Engineering of RT and embedded systems Page 3, 16 September 2018

Building reliable Component-based Software Systems Lecture based on: Building reliable Component-based Software Systems Artech House Publisher, Boston, London Ivica Crnkovic, Magnus Larsson – editors Many known authors from CBSE http://www.idt.mdh.se/cbse-book Related chapters 1,2, - Component specification 5 – development process (10) – system trustworthiness 14 – testing reusable software components Page 4, 16 September 2018

Topic overview 1 The challenges of SW- how can CBD help? What is a software component? Component models Component-based Software Development Process Component- based testing References Page 5, 16 September 2018

Part 1 The challenges of software development - how can component software help? Page 6, 16 September 2018

SW Productivity Gap (ITEA) Page 7, 16 September 2018 www.itea-office.org

Software paradigm shift Productivity © 2001 ITEA Office Assoc ITEA-Softec project Agent-based development Component-based development Object-Oriented Development Structured programming Time Design method SA/SO UML ADL’s ??? We have been moving system organisation since the 70-ies from mainframe to client server to the current N-tier distributed systems in development, moving towards self organising SW systems. We started software engineering technologies with structured programming (Structured Analysis and Organisation) and moved through Object-Oriented development now to Component-Based Technologies. The next step maybe Agent-based. Aspect based OO and CBT allows us to improve the requirements and system modeling, to set up SW development platforms based on re-use and COTS, using Architecture Design Languages at design level We are working at technologies to support distributed SW development. However, the Component based development is not yet well understood and broadly used. Large problems remain, e.g. including non-functional requirements at the design phase, scalability of architectures and design patterns, testing and validation in particular in a distributed environment, reconfiguration. At the same time we need to work towards the next step to allow connectivity for pervasive computing, dynamic reconfigurability, to create self-organising and self-correcting software. We could call this “agent-based development” We can be confident that the Information Society will come over us. How beneficial it will be and when the various steps will be made, depends on how we will address the challenges ahead. The above considerations will certainly shape the planning currently ongoing for the sixth FP. COBOL, C FORTRAN, ADA Smalltalk C++ C++ Java Agglets ? Language System organization Mainframe Client-server N-tier distributed Self-organizing 1970 1990 2000 2005 Page 8, 16 September 2018

Also, applications are not adaptable to changes SW problems and CBD SW is late SW is buggy SW is expensive … because too often applications are created instead of constructed, requiring re-invention re-coding re-testing Component-based software engineering can help because it focuses on reuse of subsystems infrastructure Also, applications are not adaptable to changes Page 9, 16 September 2018

Component-based software construction (1) components Component #1 construction application Component Component Component #1 #2 #4 Component #2 Component Component Component #3 #3 #4 Page 10, 16 September 2018

Component-based software construction (2) Application 1 Component #1 #2 #3 #4 components Component construction Application 2 #1 Component #1 #5 #3 #6 Page 11, 16 September 2018

Component-based software construction (3) application Component Component component #1 #2 New Component #4 Component Component #3 #4 update Page 12, 16 September 2018

Concentration on the business parts “30 % of SW development effort is spent on infrastructure that adds no value” Application specific Business issues GUI GUI Standard Reusable parts Communication Data model Deployment - - - - - - INFRASTRUCTURE Page 13, 16 September 2018

Part 2 What is a component? Page 16, 16 September 2018

Different understandings… Architectural point of view The software architecture of a program or computing system is the structure or structures of the system, which comprise software components [and connectors], the externally visible properties of those components [and connectors] and the relationships among them.” Bass L., Clements P., and Kazman R., Software Architecture in Practice, Page 17, 16 September 2018

Is a unix command a component? Example - Unix A general purpose OS, used as a platform for dedicated purposes Standard API - POSIX Commands uses as components in a shell-process Example: sort out words from text files: $ cat file1 file2 file3 ... | sed ’s/ /\ /g’ | sort -u >words.txt TOO SIMPLE INTERFACE (stream) Page 18, 16 September 2018

Frameworks - building “the real components” Component Object Management - COM, Active X Enterprise JavaBeans CORBA components .NET Late binding - easy replacement Word document Excel document My_application Excel document component Page 19, 16 September 2018

Component Definition A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third party. Clemens Szyperski Page 20, 16 September 2018

Composition unit How much costs the glue code? A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third party. How much components fit together? How much costs the glue code? System Glue code Components Page 21, 16 September 2018

What is a contract? More discussion later A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third party. Contract - A specification attached to an interface that mutually binds the clients and providers of the components. Functional Aspects (API) Pre- and post-conditions for the operations specified by API. Specification of invariants More discussion later Page 22, 16 September 2018

What is an explicit context dependency? A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third party. Explicit specification of Provided interface Required interface Subcomponent Interfaces CC C1 s m C2 C3 Page 23, 16 September 2018

What does it mean deployed independently? A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third party. Late binding - dependencies are resolved at load or run-time. Replacing of a component independent of the client (main application) if the contract is not broken. Delegation - interaction with a weak coupling (for example no inheritance). Which problems can occur in relation to late binding? How can we guarantee that a replacement of a component will not affect other parts of the system? Page 24, 16 September 2018

Components and Interfaces - UML definition Component – a set of interfaces required (in-interfaces) provided (out-interfaces) Interface – set of operations Operations – input and output parameters of certain type UML metamodel of the concepts used in syntactic specification of software components Page 26, 16 September 2018

IDL Example interface ISpellCheck : IUnknown { HRESULT check([in] BSTR *word, [out] bool *correct); };   interface ICustomSpellCheck : IUnknown HRESULT add([in] BSTR *word); HRESULT remove([in] BSTR *word); library SpellCheckerLib coclass SpellChecker [default] interface ISpellCheck; interface ICustomSpellCheck; Page 27, 16 September 2018

Substitution Substituting a component Y for a component X is said to be safe if: All systems that work with X will also work with Y From a syntactic viewpoint, a component can safely be replaced if: The new component implements at least the same interfaces as the older components From semantic point of view? Page 28, 16 September 2018

Specifying the Semantics of Components Current component technologies assume that the user of a component is able to make use of such semantic information. Extension of Interface (adding semantics) a set of interfaces that each consists of a set of operations. a set of preconditions and postconditions is associated with each operation. A set of invariants Also called: Contractually specified interfaces (it comes from component definition) Page 29, 16 September 2018

Precondition, Postconditions, Invariants an assertion that the component assumes to be fulfilled before an operation is invoked. Will in general be a predicate over the operation’s input parameters and this state Postcondition An assertion that the component guarantees will hold just after an operation has been invoked, provided the operation’s pre-conditions were true when it was invoked. Is a predicate over both input and output parameters as well as the state just before the invocation and just after Invariant Is a predicate over the interface’s state model that will always hold Page 30, 16 September 2018

Semantic Specification in a UML metamodel Page 31, 16 September 2018

Semantic Interface Specification context ISpellCheck::check(in word : String, out correct : Boolean): HRESULT pre: word <> “” post: SUCCEEDED(result) implies correct = words->includes(word)   context ICustomSpellCheck::add(in word : String) : HRESULT SUCCEEDED(result) implies words = words@pre->including (word) context ICustomSpellCheck::remove(in word : String) : HRESULT SUCCEEDED(result) implies words = words@pre->exluding(word) Page 32, 16 September 2018

Part 3 Component models Page 33, 16 September 2018

Component models (technologies) There are many component technologies In-house developed Domain specific Platform specific Language-related Standards and de facto standards COM .NET JavaBeans, Enetrpise JavaBeans CORBA, CCM …… Page 34, 16 September 2018

Part 4 Component-based software development process Page 35, 16 September 2018

Time to Market – “Classical” Development Process? Product Lifecycle Problems: Time To Market High Costs Meeting deadlines Visibility Requirements Specification Design Implementation Test Operation & Maintenance TIME Page 36, 16 September 2018

Development process COTS and outsourcing require different development processes Requirements Specification Design Implementation Test Find & Select Adapt Deploy Page 37, 16 September 2018

Development process – emphasize reuse Managing COTS in the early stage of the development process Requirements Specification Design Implementation Test Find & Select Test Adapt Deploy Page 38, 16 September 2018

CBD – separation of development processes Requirements Application development Specification Find & Select Design Test Implementation Adapt Test Deploy Operation & Maintenance Component development Page 39, 16 September 2018

Component Selection and Evaluation The components selected must therefore be evaluated: Technically Non-technically Page 40, 16 September 2018

Technical and Non-technical Evaluation Technical aspects of evaluation include Integration Validation Verification Examples of non-technical issues include: The marketing position of the component supplier Maintenance support provided Alternative solutions, etc Page 41, 16 September 2018

Evaluation Methods Procurement-oriented requirements engineering: Identify several component candidates. Investigate these. Reject those not compliant with the main requirements. Continue with the evaluation of the reduced number of candidate components. If necessary refine the requirements, and repeat the evaluation process. Page 42, 16 September 2018

Component-based testing Page 43, 16 September 2018

Different levels of testing Integration Testing Component test Component Development Integration Testing ? Integration Testing Component test What is the system? Integration Testing Standard techniques for testing Providing extensive test documentation, test code, test environment Page 44, 16 September 2018

Different levels of testing Integration Testing Component test System Development Integration Testing Component test ? What is a component? Page 45, 16 September 2018

Testing Component test – a typical black box testing Interface testing What does interface testing include? Parameter interface ?? -Shared memory interface??? -Procedural Interface?? -Message passing Interface? Is that enough? Functional/procedural/messaging interface Protocol testing Contact-based testing What is with non-functional attributes? Page 46, 16 September 2018

Component testing What about contractual interface? Contractually Specified output input Input not specified by the contract Input causing erroneous output (how to find it)? Fault Injection testing Page 47, 16 September 2018

Component testing Internal state testing A component – a grey box Documentation State diagram Protocol documentation Test results a 2 a,b 1 b 4 b a a 3 5 b Page 48, 16 September 2018

Evaluation and test of the Assembly It is necessary to evaluate and test the assembly; Test of the individual components is not sufficient. This implies that an investigation of the integration procedure may be a part of an evaluation. Assembly Page 49, 16 September 2018

Why do we need test assemblies? PA C1 P1 PA = F(P1, P2) C2 P2 Assembly Properties Determined by Component Properties Page 50, 16 September 2018

Assembly Properties Problems of “hidden interface” PH1 A PA C1 P1 PA = F(P1, P2 PH1, PH2, f(PH1,PH2)) C2 P2 PH1 Page 51, 16 September 2018

System Testing Do we need test component-based systems when components (and assemblies) have been tested? Page 52, 16 September 2018

Re-testing cannot be eliminated in general. Examples of correct components in a new environment Ariane 5 example Changes in System environment Can cause problems Stress testing Fault injection testing system Page 53, 16 September 2018

Reuse and Exhaustive Testing How to minimize testing? Provide evidence based on the component’s: Contracts, Experience accumulated Requirements for the systems (component environments) Page 54, 16 September 2018

Assuming that the environment is invariant First Use New Environment is not covered by the first use Page 55, 16 September 2018

Overlapping Input Domain 0 -1027 P-G First case -27 -1 A-G Second case Covered by us1 and use 2 Page 56, 16 September 2018

Reliability and Confidence for a Input Domain R(c) C(c) 1027 I(c) A graph representing the reliability and the confidence for a input domain Page 57, 16 September 2018

Lower Reliability Requirements R(c) C(c) 1027 I(c) A component reused in a context with lower reliability requirements   Page 58, 16 September 2018

Reaching Desired Reliability R(c) C(c) 1027 I(c) The component must be run for a longer time to reach the desired reliability Page 59, 16 September 2018

Previously Experienced Reliability R(c) C(c) 1027 I(c) Previously experienced reliability cannot be utilized if input domains are outside historical use of the component Page 60, 16 September 2018

Example: Deadline Requirements If we reuse a component with only a deadline requirement in a new environment in which the execution time is shorter, the component can be reused without re-testing. Page 61, 16 September 2018

Deadline Requirements R(c) C(c) Worst case execution time New old The deadline requirement is still fulfilled since the new execution time is shorter Page 62, 16 September 2018

Summary In component-based approach testing plays even more important role Separation of component from system testing Not sufficient test components Testing on assembly and system level is necessary There are methods for reducing testing Page 64, 16 September 2018

Books Desmond Francis D'Souza, Alan Cameron Wills: Objects, Components, and Frameworks With Uml : The Catalysis Approach Clemens Szyperski: Component Software : Beyond Object-Oriented Programming: (1998) Peter Herzum and Oliver Sims: Business Component Factory : A Comprehensive Overview of Component-Based Development for the Enterprise Alan W. Brown: Large-Scale Component-Based Development Betrand Meyer: Object-Oriented Software Construction, 2nd G.T. Heineman, W. Councill: CBSE Putting the Pieces Together J. Cheesmam, J. Daniels: UML Components K. Wallnau: Building Systems form Commercial Components Ivica Crnkovic & Magnus Larsson: CBSE - Building reliable component--based systems Page 65, 16 September 2018

References Page 66, 16 September 2018

Journals IEEE Computer IEEE Software IEEE Internet Computing IEEE Transactions on Software Engineering IEEE Transactions on Computers ACM Transactions on Programming Languages and Systems languages and programming systems. ACM Transactions on Software Engineering and Methodology ACM Transactions on Computer Systems Software Development (www.sdmagazine.com) MSDN Magazine (msdn.microsoft.com/msdnmag) … all major SW development magazines Page 67, 16 September 2018

Conferences International Workshop on Component-Based Software Engineering, ICSE, CBSE http://www.icse-conferences.org/2002/ http://www.sei.cmu.edu/pacc/workshop_call.html Symposium on Engineering of Computer-Based Systems (ECBS), CBSE workshop http://www.cigital.com/conferences/ecbs02/ Euromicro CBSE track http://www.euromicro.org/Upcoming.html http://www.idt.mdh.se/ecbse/ Page 68, 16 September 2018

Conferences ACM Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA) (oopsla.acm.org) International Workshop on Component-Oriented Programming (WCOP) Symposium on Generative and Component-Based Software Engineering Technology of Object-Oriented Languages and Systems (TOOLS) (www.tools-conferences.com) International Conference on Software Reuse (ICSR) Page 69, 16 September 2018