1. Quality Systems Frameworks
RIT Software Engineering

2. Some Major Quality Frameworks
ISO 9000 family of standards
A general international standard for organizational quality systems
Oriented towards assessment and certification
Malcolm-Baldrige assessment discipline
A set of criteria for the (US) Malcolm-Baldrige quality award
Designed to encourage and recognize excellence
SEI CMM (Capability Maturity Model)
A software-specific model for improving the maturity of software development practices
Oriented towards self-assessment and improvement
Total Quality Management (TQM)
A philosophy and practices for improving quality
Focuses on building an organizational quality culture
RIT Software Engineering

3. Quality vs. Quality frameworks
A major point to note is that all these are about Quality Systems, and not directly about the actual quality of the product
The difference between excellence in quality control for an assembly line car and producing a handmade Rolls-Royce (work of art)!
The principle is that an organization with a culture of focusing on quality and on continuous improvement will consistently produce good output
Remember also “continually optimize achievement of multiple objectives” – that is what systems do
RIT Software Engineering

4. RIT Software Engineering
“Systems don’t produce quality, People do”
Would you agree?
What does this say about the need for quality systems?
RIT Software Engineering

5. Value of the frameworks
“Optimize across all project and organizational objectives” is too open-ended
Frameworks provide models of what needs to be addressed
Primary value from these frameworks is
Defining the specific set of areas to address
Defining specific criteria for determining whether the areas are being addressed well
Providing basic structures to ensure continuing focus
Defining appropriate processes and metrics
Mechanisms for continuous improvement, so that processes keep improving and evolving as needs change
Assessment mechanisms, to check that all this is happening
RIT Software Engineering

6. RIT Software Engineering
Which framework to use?
Different frameworks address different needs
Also, there are many other frameworks, and many additions/variations to each
Organizations design their own quality management approaches (or it just evolves without design!), possibly using one or more frameworks as a starting point
Anyway, frameworks only supply goals, and suggest some ways to achieve goals
Each organization needs to adapt the framework(s) to their needs, and decide how to achieve the goals
It is common for organizations to go through multiple different assessments, for different purposes
If used well, any of the frameworks are good enough
If used poorly, none of them will help! (In fact, they will hurt)
“Just another tool”
RIT Software Engineering

7. Why do you care about all this?
General knowledge
As a software engineer, people will expect you to know a little bit about these models
Understanding the big picture helps, before we start to focus on specific metrics and techniques
Understanding the philosophy and limitations helps you to get a more balanced picture of the quality area
And yes, it will be on the exam!
But only the concepts, not the specifics of each model
RIT Software Engineering

8. RIT Software Engineering
Malcolm-Baldrige
An award for companies that excel in quality management and quality achievement
Broad areas, organization-level focus
Looks at: Approach, Deployment, Results
The value is in its assessment of quality focus and excellence in all aspects of an organization
Not just engineering
Leadership
Quality results
Information and Analysis
Strategic quality planning
Human resource utilization
Customer satisfaction
Quality assurance of products and services
RIT Software Engineering

9. RIT Software Engineering
Do you think it helps to have such awards?
Preparing for a Malcolm-Baldrige evaluation takes a company dozens, maybe hundreds of staff months. Is it energy well-spent?
RIT Software Engineering

10. RIT Software Engineering
ISO 9000
A standard for certifying that organizations follow procedures for ensuring quality
Heavy focus on processes and evidence of compliance (documentation)
Some focus on statistical techniques and processes for improvement
ISO 9000 is about procedures
“assure minimum standards of operation”
“existence of quality systems and commitment to them”
Complementary to other quality management frameworks – limited value in itself
RIT Software Engineering

11. RIT Software Engineering
What are the advantages & disadvantages of a quality system that is heavily procedures and documentation-oriented?
RIT Software Engineering

12. RIT Software Engineering
TQM
Not really a framework, more a philosophy and set of practices
Very strongly oriented towards building a “quality culture”
Major points of emphasis
Human aspects:
“Quality is the responsibility of every employee”
Empowering employees to take actions to improve quality
Management commitment
Metrics & analysis
Continuous improvement of process & quality
Customer needs, customer satisfaction
Most successful quality management programs incorporate all these aspects
RIT Software Engineering

13. RIT Software Engineering
The culture approach is not really about “compliance” at all
It is about building an environment where focusing on project and organizational objectives (“Q”) is part of what it means to “get work done” and to “deliver”
Is such an approach good? Is it workable?
What are the barriers?
RIT Software Engineering

14. RIT Software Engineering
CMM
Created by the SEI (Software Engg Institute) to drive process improvement in software
Defines a series of “maturity levels” so that organizations can incrementally put software quality systems in place
Self-assessment – the idea is to help organizations create the most appropriate quality system for themselves
Focused (relatively) narrowly on the software development process
Defines a set of “Key process areas” and goals for each area
There is now also a CMMI, that expands CMM to cover “system engineering” aspects
Useful when project scope includes system design
RIT Software Engineering

15. Frameworks as Knowledge Bases
(Look at the CMM 1.1 doc on the course website)
It is really a knowledge base
“What areas do we need to address if we want projects to be successful?”
“How do we keep everyone aware of good ways to accomplish tasks?” (define processes and practices!)
“What are common sources of problems? What structures can we put in place to reduce the chance that they will occur?”
“What structures do we need to ensure that the organization will keep trying to improve its processes and practices?”
“How do we ensure that good processes lead to good results?” (define appropriate metrics)
“How we can we figure out when things aren’t working and how to fix them?” (metrics interpretation, causal analysis, prevention)
An organization’s quality management system is its own knowledge base of the best answers to these questions!)
RIT Software Engineering

16. RIT Software Engineering
Limitations of CMM
It is heavily oriented towards “optimization of repetitive tasks”, especially at higher maturity levels
NOT appropriate if projects differ significantly from one another
There is a fundamental assumption that being highly structured is a good thing
But structure is not free! (Effort, time, flexibility)
This is where agile development methodologies came in
It might run counter to the desired organizational culture
Organizations need to decide how much maturity is right for them in each process area
The more recent “continuous” model addresses this
It only works well if the organization has an underlying commitment to quality and structure as the road to results
RIT Software Engineering

17. RIT Software Engineering
Assessments
Assessments are massive exercises
Value: Feedback on what’s working, opportunities for improvement
Cross-fertilization of ideas
Problems
Easy to “create evidence for the assessment”
Passing means at best that systems are in place, not that results are superior
Assessments easily become exercises in PR (public relations)
Over-focus on “avoiding mistakes” can take energy away from excellence
It would be a mistake to read too much into the results
Being assessed at high maturity levels or receiving a quality award does NOT guarantee that the organization will be more successful or produce better products – it just means that they have structures in place to keep trying to do better
RIT Software Engineering

18. RIT Software Engineering
Do you think quality management frameworks and quality management systems are necessary?
Do they add more value than they cost?
Which is better: the “culture” route, the “systems” route, or is it possible to get the best of both worlds?
RIT Software Engineering

19. RIT Software Engineering
My opinions
Culture is always the best approach
But systems have their place and value
“Less is more”
Small organizations may not need very much formal quality management
Know the theory. As problems are perceived, incrementally put in only what is obviously useful
When designing a system, think carefully about what the needs of the organization are and what is appropriate (“start with the objectives and preferences”)
Processes tend to grow with time. Quality people should spend as much energy “deleting” unnecessary process as adding process
RIT Software Engineering

20. RIT Software Engineering
Conclusion
There are many quality systems frameworks, appropriate to different needs
They provide a good starting point for creating quality systems
Most organizations use a combination of quality systems
RIT Software Engineering