Software testing techniques Testing Maturity Model Presentation on the seminar Kaunas University of Technology

Introduction For the past decade, the software industry has put substantial effort in improving the quality of its products. This has been a difficult job, since the size and complexity of software increases rapidly while customers and users are becoming more and more demanding. Despite encouraging results with various quality improvement approaches, the software industry is still far from zero defects. To improve product quality, the software industry has much focused on improving their development processes. The TMM is a detailed model for test process improvement and is positioned as being complementary to the CMM.

Scope of the TMM Software and System Engineering Test levels TMM and CMM Assessments Improvement Approach

Software and System Engineering The TMM is intended to support testing activities and test process improvement in both the systems engineering and software engineering discipline. Systems engineering covers the development of total systems, which may or may not include software. Software engineering covers the development of software systems.

Test levels Whereas some models for test process improvement focus mainly on high-level testing or address only one aspect of structured testing e.g. the test organization, The TMMi addresses all test levels (including static testing) and aspects of structured testing. With respect to dynamic testing, both low-level testing and high-level testing are within the scope of the TMMi. Studying the model more in detail one will learn that the model addresses all four cornerstones for structured testing (lifecycle, techniques, infrastructure and organization).

TMM and CMM It is also important to note that TMM is positioned as a complementary model to the CMM. In many cases a given TMM level needs specific support from process areas at its corresponding CMM level or from lower CMM levels. In exceptional cases there is even a relationship to higher CMM levels. Process areas and practices that are elaborated within the CMM are mostly not repeated within TMM; they are only referenced. For example the process area configuration management, which is of course also applicable to test (work) products / test ware, is not elaborated upon in detail within the TMM; the practices from CMM are referenced and implicitly re-used.

Assessments Many organizations find value in benchmarking their progress in test process improvement for both internal purposes and for external customers and suppliers. Test process assessments focus on identifying improvement opportunities and understanding the organization’s position relative to the selected model or standard. The TMM provides an excellent reference model to be used during such assessments. Assessment teams use TMM to guide their identification and prioritization of findings. These findings with guidance of TMM practices are used to plan improvements for the organization. The assessment framework itself is not part of the TMM.

Improvement Approach The TMMi provide a full framework to be used as a reference model during test process improvement. It does not provide an approach for test process improvement such as the IDEAL4 (Initiating, Diagnosing, Establishing, Acting, And Learning) model. Practical experiences have shown that the most powerful initial step to test process improvement is to build strong organizational sponsorship before investing in test process assessments. Given sufficient senior management sponsorship, establishing a specific, technically competent test process group that represents relevant stakeholders to guide test process improvement efforts had proved to be an effective approach.

TMM Maturity Levels

Level 1 Initial Testing is chaotic and undefined process Testing is often considered a part of debugging The organization usually does not provide a stable environment to support the processes

Level 2 Managed The process areas at TMM level 2 are: 2.1 Test Policy and Strategy 2.2 Test Planning 2.3 Test Monitoring and Control 2.4 Test Design and Execution 2.5 Test Environment

Level 3 Defined The process areas at TMM level 3 are: 3.1 Test Organization 3.2 Test Training Program 3.3 Test Lifecycle and Integration 3.4 Non-Functional Testing 3.5 Peer Reviews

Level 4 Management and Measurement The process areas at TMMi level 4 are: 4.1 Test Measurement 4.2 Product Quality Evaluation 4.3 Advanced Peer Reviews

Level 5 Optimization Process areas at level 5 are: 5.1 Defect Prevention 5.2 Test Process Optimization 5.3 Quality Control

Conclusion Test process improvement is a journey. The TMM just makes that journey easier. Test process improvement should not be considered as being in addition to any other process improvement initiative, but should work in conjunction with that initiative. The TMM is laid out in incremental steps because that is the best way to make the improvements. Very few companies have the resources to try to do everything at one time. Experience has shown the incremental approach is the most successful. Just because it is laid out incrementally does not mean that you cannot start working on an improvement at the next level.

Literature TMMi® TMMi® Reference Model ( pdf) pdf Introduction to the Testing Maturity Model Enhanced (TMMe) ( 0to%20the%20Testing%20Maturity%20Model%20Enhance.pdf) 0to%20the%20Testing%20Maturity%20Model%20Enhance.pdf

Thank you.