1. SE 450 Software Processes & Product Metrics 1 Quality Systems Frameworks

2. SE 450 Software Processes & Product Metrics 2 Some Major Quality Frameworks ISO 9000 Family of Standards A general international standard for organizational quality systems. Oriented towards assessment and certification. Malcolm-Baldrige Assessment Discipline A set of criteria for the (US) Malcolm-Baldrige Quality Award. Designed to encourage and recognize excellence. SEI CMM (Capability Maturity Model) A software-specific model for improving the maturity of software development practices. Oriented towards self-assessment and improvement. CMMI Total Quality Management (TQM) A philosophy and practices for improving quality. Focuses on building an organizational quality culture.

3. SE 450 Software Processes & Product Metrics 3 Quality vs. Quality Frameworks A major point to note is that all these are about Quality Systems and not directly about the actual quality of the product. The difference between excellence in quality control for an assembly line car and producing a handmade Rolls-Royce (work of art) is significantly different! The principle is that an organization with a culture of focusing on quality and on continuous improvement will consistently produce good product output. Remember also “continually optimize achievement of multiple objectives” – that is what systems do.

4. SE 450 Software Processes & Product Metrics 4 Value of the Frameworks “Optimize across all project and organizational objectives” is too open- ended. Frameworks provide models of what needs to be addressed. Primary value from these frameworks includes: Defining the specific set of areas to address. Defining specific criteria for determining whether the areas are being addressed well. Providing basic structures to ensure continuing focus. Defining appropriate processes and metrics Mechanisms for continuous improvement, so that processes keep improving and evolving as needs change Assessment mechanisms, to check that all this is happening

5. SE 450 Software Processes & Product Metrics 5 Which Framework to Use? Different frameworks address different needs. Also, there are many other frameworks, and many additions/variations to each. Organizations design their own quality management approaches (or it just evolves without design!), possibly using one or more frameworks as a starting point. Anyway, frameworks only supply goals, and suggest some ways to achieve goals. Each organization needs to adapt the framework(s) to their needs, and decide how to achieve the goals. It is common for organizations to go through multiple different assessments, for different purposes. If used well, any of the frameworks are good enough. If used poorly, none of them will help. (In fact, they will hurt!)

6. SE 450 Software Processes & Product Metrics 6 Why Do You Care About All this? General knowledge. As a software engineer, people will expect you to know a little bit about these models Understanding the big picture helps, before we start to focus on specific metrics and techniques. Understanding the philosophy and limitations helps you to get a more balanced picture of the quality area.

7. SE 450 Software Processes & Product Metrics 7 Malcolm-Baldrige An award for companies that excel in quality management and quality achievement. Broad areas, organization-level focus: Looks at approach, deployment, results. Quality assurance of products and services. The value is in its assessment of quality focus and excellence in all aspects of an organization, not just engineering. Leadership Quality results Human resource utilization Customer satisfaction Information and Analysis Strategic quality planning

8. SE 450 Software Processes & Product Metrics 8 ISO 9000 A standard for certifying that organizations follow procedures for ensuring quality. Heavy focus on processes and evidence of compliance (documentation). Some focus on statistical techniques and processes for improvement ISO 9000 is about procedures: “assure minimum standards of operation” “existence of quality systems and commitment to them” Complementary to other quality management frameworks – limited value in itself

9. SE 450 Software Processes & Product Metrics 9 Total Quality Management (TQM) Not really a framework, more a philosophy and set of practices. Very strongly oriented towards building a “quality culture”. Major points of emphasis: Human aspects: “Quality is the responsibility of every employee” Empowering employees to take actions to improve quality Management commitment Metrics & analysis Continuous improvement of process & quality Customer needs, customer satisfaction Most successful quality management programs incorporate all these aspects.

10. SE 450 Software Processes & Product Metrics 10 CMM Created by the SEI (Software Engineering Institute) to drive process improvement in software. Defines a series of “maturity levels” so that organizations can incrementally put software quality systems in place. Self-assessment – the idea is to help organizations create the most appropriate quality system for themselves. Focused (relatively) narrowly on the software development process. Defines a set of “Key process areas” and goals for each area. There is also a CMMI, that expands CMM to cover “system engineering” aspects. Useful when project scope includes system design.

11. SE 450 Software Processes & Product Metrics 11 Frameworks as Knowledge Bases It is really a knowledge base: “What areas do we need to address if we want projects to be successful?” “How do we keep everyone aware of good ways to accomplish tasks?” (define processes and practices!) “What are common sources of problems? What structures can we put in place to reduce the chance that they will occur?” “What structures do we need to ensure that the organization will keep trying to improve its processes and practices?” “How do we ensure that good processes lead to good results?” (define appropriate metrics) “How we can we figure out when things aren’t working and how to fix them?” (metrics interpretation, causal analysis, prevention) An organization’s quality management system is its own knowledge base of the best answers to these questions!)

12. SE 450 Software Processes & Product Metrics 12 Limitations of CMM It is heavily oriented towards “optimization of repetitive tasks”, especially at higher maturity levels. NOT appropriate if projects differ significantly from one another. There is a fundamental assumption that being highly structured is a good thing But structure is not free! (Effort, time, flexibility) This is where agile development methodologies came in. It might run counter to the desired organizational culture. Organizations need to decide how much maturity is right for them in each process area. The more recent “continuous” model addresses this. It only works well if the organization has an underlying commitment to quality and structure as the road to results.

13. SE 450 Software Processes & Product Metrics 13 CMMI for Process Improvement Use CMMI in process improvement activities as a : Collection of best practices. Framework for organizing and prioritizing activities. Support for the coordination of multi-disciplined activities that might be required to successfully build a product. Means to emphasize the alignment of the process improvement objectives with organizational business objectives. (Look at the CMMI Overview presentation in the Resources section) Source: Carnegie Mellon SEI

14. SE 450 Software Processes & Product Metrics 14 Assessments Assessments are massive exercises Value: Feedback on what’s working, opportunities for improvement Cross-fertilization of ideas Problems Easy to “create evidence for the assessment” Passing means at best that systems are in place, not that results are superior Assessments easily become exercises in PR (public relations) Over-focus on “avoiding mistakes” can take energy away from excellence It would be a mistake to read too much into the results Being assessed at high maturity levels or receiving a quality award does NOT guarantee that the organization will be more successful or produce better products – it just means that they have structures in place to keep trying to do better

15. SE 450 Software Processes & Product Metrics 15 Some Thoughts About Quality Frameworks Culture is always the best approach. Systems have their place and value. “Less is more” Small organizations may not need very much formal quality management Know the theory. As problems are perceived, incrementally put in only what is obviously useful When designing a system, think carefully about what the needs of the organization are and what is appropriate (“start with the objectives and preferences”) Processes tend to grow with time. Quality people should spend as much energy “deleting” unnecessary process as adding process

16. SE 450 Software Processes & Product Metrics 16 Conclusion There are many quality systems frameworks, appropriate to different needs. They provide a good starting point for creating quality systems. Most organizations use a combination of quality systems.