SPIN Montreal Feb 23, 2005 Prepared by Nicolae Giurescu The Philip Crosby Philosophy on Quality Applied to Software Development SPIN Montreal Feb 23, 2005 Prepared by Nicolae Giurescu

Presentation Outline Introduction The Four Absolutes of Quality Management Teamwork Measurements Problem Analysis and Resolution Conclusion

Introduction

“Quality is free. What costs money are the unquality things – all the actions that involve not doing jobs right the first time” Philip Crosby

Views on Quality P. Crosby W. Deming J.M Juran Orientation Motivational Technical Process Definition Conformance to requirements Nonfaulty systems Fitness for use; freedom from trouble Responsible Management Goal Zero defects Meet/exceed customer needs Please customer Method 14 Step Quality Improvement Process Statistical; continual improvement Quality trilogy: plan, control, improve

The Four Absolutes of Quality Management

Four Absolutes of Quality Management™ The definition of quality is conformance to requirements, not goodness. Quality is achieved through prevention, not appraisal. The quality performance standard is zero defects, not acceptable quality levels. Quality is measured by the price of nonconformance, not indexes.

Process Model Worksheet PROCESS NAME PROCESS FLOW PROCESS SCOPE R E Q U I M N T S OUTPUT PERFORMANCE ST AND ARDS WHO DEFINES REQ UIREMENTS INPUTS SUPPLIERS F A CILITIES & EQ UIPMENT PR O VIDES OCEDURES OUTPUTS CUST OMERS TRAINING & KNO WLEDGE INITIAL ACTIVITYA FINAL ACTIVITYA Materials Inf ormation Quality Cost Sc hedule © Proudfoot

1st Abs: Requirements C S All work is a process Personal processes Organization’s processes Requirements are “filters”

1st Abs: Identify Requirements Sources: External customers; marketing; system design Quality control; validation Design standards Development process Define: Desired requirements Needed requirements Mandated requirements Implied requirements PROCESS NAME PROCESS FLOW PROCESS SCOPE R E Q U I M N T S OUTPUT PERFORMANCE ST AND ARDS WHO DEFINES REQ UIREMENTS INPUTS SUPPLIERS F A CILITIES & EQ UIPMENT PR O VIDES OCEDURES OUTPUTS CUST OMERS TRAINING & KNO WLEDGE INITIAL ACTIVITYA FINAL ACTIVITYA Materials Inf ormation Quality Cost Sc hedule © Proudfoot

1st Abs: Requirements Traceability Love / hate relationship Bi-directional traceability: Requirements  Test cases Requirements  Architectural components Requirements  Design Requirements  Implementation Keep traceability matrix updated Consider using specialized tools

1st Abs: Requirements Change Customers / Market Organization’s needs Technology Suppliers Government regulations Competition People Process

1st Absolute of Quality Management Conformance to requirements Goodness

2nd Abs: Prevention Preventing defects to occur. When the non-conformances are detected? Requirements review Architecture review Design review Code inspection Unit testing Integration testing Validation Process review

2nd Abs: Planning Prevention Establish policies for improvement Defect-free software releases Develop systems for preventing problems Reviews (formal/informal; peers) Code inspections (manual/automatic) Process performance reviews

2nd Abs: Implementing Prevention Identify process scope Four step implementation: Define the output Define the process (new or modified) Take into consideration the controlling inputs: Facilities and equipment Training and knowledge Procedures Performance standards Proof Operate and manage PROCESS NAME PROCESS FLOW PROCESS SCOPE R E Q U I M N T S OUTPUT PERFORMANCE ST AND ARDS WHO DEFINES REQ UIREMENTS INPUTS SUPPLIERS F A CILITIES & EQ UIPMENT PR O VIDES OCEDURES OUTPUTS CUST OMERS TRAINING & KNO WLEDGE INITIAL ACTIVITYA FINAL ACTIVITYA Materials Inf ormation Quality Cost Sc hedule © Proudfoot

2nd Abs: Proof Often skipped due to lack of time or overconfidence Steps: Define criteria for success Perform process trial Evaluate process outcome

2nd Abs: Operate and Manage Process Control Process with Requirement Measurement Action Comparison © Proudfoot

2nd Absolute of Quality Management Prevention Appraisal

3rd Abs: Performance Standards Type: Quality Schedule Cost Who defines Management Requirements Meeting requirements On-time Within budget

3rd Abs: That’s Close Enough vs. ZD Acceptable percent nonconforming < 10% known problems still not solved Acceptable non-conformances per unit < 10 major defects per software component Acceptable non-conformances per time unit < 10 system crash per year Zero defects: a personal commitment Understand the process and product requirements Work to meet them Recognize and expose the non-conformances

3rd Absolute of Quality Management Zero defects Close enough

4th Abs: Measuring Quality Costs Get management attention Prioritize problems Show improvement

4th Abs: Price of Nonconformance Error-Free Costs Price of Conformance What it costs to get things done right the first time Price of Nonconformance

4th Abs: Calculation Techniques Whole account Warranty costs Whole person Maintenance team Labor/resource claiming Debugging Unit pricing Average cost to solve a defect Deviation from the ideal

4th Abs: Collecting Data Gather actual expenditures Use surveys Include ripple costs

4th Absolute of Quality Management Price of nonconformance Indexes

Teamwork

Working on Teams Advantages Disadvantages More knowledge and skills Better communication Different approaches to challenges (e.g. problems) Disadvantages Commitment of time and money Presence of “strong” personalities Imposing decisions without considering alternatives Not supporting team decisions

Team Design Type of teams Leadership skills Member selection Life span: temporary, permanent Function: department, project, QIT Skills: specialized, cross-functional Leadership skills Member selection Member education

Team Meetings Mechanics Environment Agenda / minutes Meeting procedure Clear objectives, preparation Consensus Conflict resolution Action assignment Environment Listening Openness / trust

Better Teams Foster Trust Feelings and values Ideas Facts and information

Measurements

Measure a Process to Improve PROCESS NAME PROCESS FLOW PROCESS SCOPE R E Q U I M N T S OUTPUT PERFORMANCE ST AND ARDS WHO DEFINES REQ UIREMENTS INPUTS SUPPLIERS F A CILITIES & EQ UIPMENT PR O VIDES OCEDURES OUTPUTS CUST OMERS TRAINING & KNO WLEDGE INITIAL ACTIVITYA FINAL ACTIVITYA Materials Inf ormation Quality Cost Sc hedule © Proudfoot

Measurement System Identify Record Communicate Start with identified problems and their root cause Learn from others Identify correlations Record Use dedicated tools (buy/make) Determine correct sampling time Communicate Reports, management briefings Use charts

Charts

Measurements: Product Defects Identify and record defects “Traditionally” during maintenance (reactive) Identified during validation testing Recorded into a formal defect database Eliminated by maintenance, not development team During requirement reviews (proactive) During development (proactive) Identified during design reviews, code inspections, unit testing and (partial) integration testing

Collect Defect Data Record defects into dedicated databases Defect record State machine States Transition rights Fields When and where was found, and by whom Heading, description, what release, … Severity, priority Resolution, workaround, review comments, … Who worked on it Root cause

Reporting Defects, Based On… Selected characteristics: Number of defects per severity/priority levels Distribution: Number of defects per component Root cause: Number of defects per root cause class Dynamics: Number of defects submitted, closed, …, per time unit Combinations of criteria

Measurements: Product Design (OO) Number of model elements Gives info on size, evolution Size of model elements Lines of code, number of functions, … Level of complexity Levels of inheritance Levels of coupling Number of changes Gives info on component stability

Measurements: Product Verification Review Number of reviews per component Number of defects / changes identified per review Testing (unit, integration, validation) Number of tests that failed / passed Test plans Number of completely specified test cases Number of tests

Measurements: Project Schedule Size: Number of tasks, number of resources, duration Tasks: Number of tasks experiencing delays Number of new tasks per time unit Number of tasks on the critical path Resources: Number of over / under allocated resources Number of unavailable resources

Problem Analysis and Resolution

Eliminating Non-conformances Define the problem Focus on data; determine PONC, not cause, not blame Quick fix (not a permanent solution) Identify the root cause(s) Threads of similarities Opportunities for error Cause and effect diagram Identify solution and take corrective action Generate, select, plan, implement Evaluate and follow up Reviews, surveys, audits

Cause and Effect Diagram Inputs (Material) Performance Standards Procedures EFFECT Inputs (Information) Facilities & Equipment Training & Knowledge

Pareto Analysis

Broken Traceability Matrix Problem: X % of bad links from requirements to design in subsequent documentation baselines Quick fix: verify all and correct bad links Root causes: Links to old versions Lack of time to properly update the traceability Solution Migrate from Word to DOORS Store in the same database all engineering specs

“Forgotten” Requirement Changes Problem: X% of requirement changes not reflected into the design and test cases Quick fix: Rework if possible Report the change to future versions Root causes: Changes not communicated Changes occurred to late to be considered Solution: Automatic e-mail notification Joint requirement reviews

No Builds are Alike Problem: Building the same version from different computers gives different results Quick fix: Define standard development environment Audit environment on all development computers Root causes: Different versions of the development tools Development tools installed with different options “Local” environment variables Solution: Recreate the standard environment for every build

Unmanaged Defect Reports Problem: defects discovered during the development phase were solved in an ad-hoc way Quick fix: none considered Root causes: No system to manage defects during development Solution: Start managing defects during development, same way as during maintenance

Being Late Problem: X% of missed schedule milestones Quick fix: Revised schedule Root causes: Imposed schedule milestones Ever changing WBS Wrong task estimations Insufficient resources Solution: Incremental integration and development plan

Conclusion

Individual’s Role in Quality Improvement Motivation Respond to organization’s need and commitment to improve Fulfill individual need and commitment to improve Actions Take requirements and quality improvement seriously Have long-term objectives Seek agreement on objectives Set specific, measurable goals Develop schedules Take action

14 Step Quality Improvement Process Get management commitment Form cross-functional quality improvement teams Measure quality Estimate costs of quality Raise the organization’s quality awareness Take corrective action at lower levels Establish ad-hoc committee for the Zero Defects Program Train employees Establish zero defects as the performance standard Set quality improvement goals Error-Cause Removal Recognize the meeting of goals and outstanding acts Establish a quality council Repeat the program

Thank you!