Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan and Karo Mazidzhyan

Breakdown  Introduction  Related Research  Research Process  Analysis Results Process Improvement Propositions  Conclusion

Introduction  The objective of this study was to understand how software testing is conducted by observation.  From observations propose improvement to the testing process.  Improvements by reducing development and testing costs, and improving quality.

Software Costs and Quality  Software Engineering strives to reduce development costs and improving quality.  Software Process Improvements (SPI) are the means to reaching these goals.  Commitment to SPI’s by all from all organizational levels is key to success  Quality can be tested into products or developed and built into products

Software Costs and Quality Cont.  External events such as deadlines affect software quality.  The cost of software testing is high, therefore SPI’s are necessary to reduce cost.

Related Research  Involvement of testing during development occurs when testers develop test for developers to analyze  The complexity of testing increases as a function of the complexity of the systems under testing.  Testing strategy defines the contents of testing.

Related Research Cont.  Communication and interaction between development and testing processes requires cooperation and coordination.  The use of software components are increasing rapidly  Design outsourcing and distributed development increase the use of components.  Cost of Quality is “Free”, but being late with products may be more costly than fixing faults.

Research Process  This study consisted of Organizational Units (OU) which develop and test technical software for automation or telecommunication in Finland.  Initial Sample included 26 OU’s, from which 5 were used as case studies.  Cases were chosen to show polar types

Research Process Cont.  Data for the research was collected by a series interviews.  Each interview had a different theme in mind and possibly a different interviewee in mind.  The interviews took place during five rounds, based on the theme.

Research Process Cont.  Interview Rounds 1. Development and Testing Managers were asked to define their testing process. 2. Managers of Testing were asked to define their testing process in depth. 3. Testers were interviewed. 4. Systems Analysts interviewed.

Research Process Cont. Case Breakdowns

Research Process Cont.  Data Analysis Information gathered from these interviews were then categorized. Categories were then analyzed to see how they were connected. The categories were then used to identify factors which affected testing.

Analysis Results  Description of Cases: Case A - Developed and tested Manufacturing Execution Systems :  Turnover 50% product, 50% service  Services included systems integration and customization  Testing against requirements was a challenge because customers had special in-house requirements standards  Developers and testers worked physically close to each other  Time allocated to testing was consistent, although over time it has been reduced  Use of components low, hinders testing

Analysis Results Cont. Case B - Tested in house products and provided testing services for external customers:  Turnover 75% service, 25% product  Majority of requirements specifications were based on standards.  Delays in development allowed for fewer time allocated for testing  Communication flexible, developers talked face-to- face with testers  Use of components high, testability of components must be considered

Analysis Results Cont. Case C – Customized Software Development:  Turnover 2/3 service, 1/3 product  Testers were involved early, involved in development process  Testers often had issues due to lack of advisement from developers  Delays in development does not often result in reduction of testing time  Developers and testers communicate face-to-face  Use of components low  Testing of software components seen as difficult b/c of different implementation.

Analysis Results Cont. Case D - Electronics:  Turnover 100% product  High product orientation required high quality because recalls are very expensive  Avoided testing of unfinished product  Testing tasks clear and well documented  Testing involved in development late but planning of testing automation provided information on upcoming tests  Communication between developers and tester planned, formal and transparent  Use of components high  Components tested initially by suppliers then again at system testing.

Analysis Results Cont. Case E – Software Testing Services:  Turnover 100% service  Working as an external testing organization required the adaptation of the process of the customer  Early involvement of testing was necessary for the testing company to increase the testability of the software  Sometimes testing was involved late  Budgets for testing affected testing time  Communication was handle through a contact person, but was active and clear  Use of components depends on customer  As an external testing organization it was hard to receive information about customers purchased components.

Analysis Results Cont.

 Cause and effect

Analysis Result Cont.  Cause and Effect Business Orientation  Directly associated with use of components and testing schedules Business Model – value adding process  Purely Service Oriented  System integration  Customizing  Consulting  Customers directly affect development and testing process  Purely Product Oriented  Product Development  Marketing  Customers do not directly affect development and testing process

Analysis Results Cont.  Process Improvement Propositions Testing ought to be adjusted to business orientation  Product oriented should adopt formal planned testing process  Service oriented should adopt a flexible testing process Enhanced Testability of software  Consider testability when selecting components  Review testing process of suppliers

Analysis Results Cont. Effective Communication and interaction between development and testing Early involvement of testing and planning of testing Use of risk based testing

Conclusion  Proposals by observing best practices using grounded theory  Better documentation improved testability of software and components  Efficient communication between development and testing improved quality  When time is a major issue, risk based testing is the best solution  Business orientation affects: Use of components Amount and quality of communication Allocated testing time and the planning of testing