________________________________________________________________________ Jonsson School of Engineering and Computer Science Dr. Mark C. Paulk 2015 ASEE Annual Conference, 21 February 2015 Adoption of Agile Methods by High Maturity Organizations

Software CMM v1.1 ( ) Competent people (and heroics) Defect Prevention Technology Change Management Process Change Management Continuous process improvement Product and process quality Engineering processes and organizational support Project management processes Quantitative Process Management Software Quality Management Organization Process Focus Organization Process Definition Training Program Integrated Software Management Software Product Engineering Intergroup Coordination Peer Reviews Requirements Management Software Project Planning Software Project Tracking & Oversight Software Subcontract Management Software Quality Assurance Software Configuration Management LevelFocusKey Process Areas Initial Optimizing 1 Repeatable 2 3 Managed 4 5 Defined 2

Implications of Maturity Initial Repeatable Defined Process is informal and unpredictable Project management system in place; performance is repeatable Software engineering and management processes defined and integrated Product and process are quantitatively controlled Time/$/... Optimizing Process improvement is institutionalized Time/$/... LevelProcess CharacteristicsPredicted Performance Managed Better predictability… Less variability… Improved performance… 3

Empirical Data on Improvement By Maturity Level Quality (reliability, defects) improves by roughly a factor of 2 (or more) -C. Jones, “Software Benchmarking,” IEEE Computer, October K.D. Williams, "The Value of Software Improvement… Results! Results! Results!" SPIRE97, June R. Yacobellis, “Panel: Does SEI Level 5 Lead to High Quality Software?” COMPSAC Effort for a given product decreases 15-21%, productivity increases, cycle time decreases -B.K. Clark, “Quantifying the Effects on Effort of Software Process Maturity,” IEEE Software, November/December D.E. Harter, M.S. Krishnan, and S.A. Slaughter, “Effects of Process Maturity on Quality, Cycle Time, and Effort in Software Product Development,” Management Science, April L.H. Putnam, “Linking the QSM Productivity Index with the SEI Maturity Level,” QSM,

CMMI-DEV v1.3 Process is unpredictable, poorly controlled, and reactive Process is characterized for projects and is often reactive Process is characterized for the organization and is proactive Process is measured and controlled Focus is on quantitative continuous process improvement Level Process Characteristics Requirements Management Project Planning Product & Process Quality Assurance Configuration Management Project Monitoring & Control Supplier Agreement Management Quantitative Project Management Organizational Process Performance Causal Analysis & Resolution Process Areas Requirements Development Technical Solution Product Integration Validation Verification Organizational Process Focus Integrated Project Management 1 Initial 2 Managed 3 Defined 4 Quantitatively Managed 5 Optimizing Measurement & Analysis Organization Process Definition Organizational Training Risk Management Decision Analysis & Resolution Organizational Performance Management 5

A Scrum Adoption Survey 184 distinct organizations responded to the 2011 Scrum adoption survey 24 ML5 2 ML4 44 ML3 61 ML2 125 ML1 and unknown 128 projects adopting Scrum M.C. Paulk, “A Scrum Adoption Survey,” ASQ Software Quality Professional, March

Organizational Size 7

Organizational Size by Level Org SizeML5ML4ML3ML2Unknown ≤ >

Software Engineering Methods 9

Methods by Level A high percentage of high maturity organizations use Scrum. Many high maturity organizations use XP, FDD, and the Unified Process. 10 MethodML5ML4ML3ML2Unknown Scrum Extreme Programming (XP) Feature Driven Development (FDD) Crystal methods (including Crystal Clear)00105 Team Software Process (TSP)10210 Unified Process (including RUP, AUP, OUP) Other methods

Requirements Volatility Per Month 11

Requirements Volatility by Level 12 Requirements VolatilityML5ML4ML3ML2Unknown <1% per month % per month % per month % per month % per month % per month >50% per month50117

Use of Scrum 13

Use of Scrum by Level Few high maturity organizations are unaware of Scrum or uninterested in it. 14 Scrum AdoptionML5ML4ML3ML2Unknown We never heard of Scrum before We are aware that Scrum exists We are currently piloting Scrum We have piloted Scrum but no decision about adoption has been made We are currently deploying Scrum across the organization Scrum is one of the standard methods we use Scrum is the normal way we build software

Scrum Team Size by Level Team SizeML5ML4ML3ML2Unknown ≤ to to to to ≥

Quality of the Software 16

Quality by Level QualityML5ML4ML3ML2Unknown much lower00001 lower10123 about the same higher much higher Quality tends to be higher for organizations adopting Scrum in general.

Cost 18

Cost by Level CostML5ML4ML3ML2Unknown much lower00003 lower about the same higher00119 much higher Cost does not appear to be significantly affected for most organizations adopting Scrum. lower for a noticeable percentage

Meeting Schedule Expectations 20

Schedule Expectations by Level Schedule ExpectationsML5ML4ML3ML2Unknown much lower00004 lower20013 about the same higher much higher Meeting schedule expectations appears to be higher for organizations adopting Scrum in general.

Customer Satisfaction 22

Customer Satisfaction by Level Customer SatisfactionML5ML4ML3ML2Unknown much lower00001 lower00022 about the same10149 higher much higher Customer satisfaction tends to be higher for organizations adopting Scrum in general.

Concluding Thoughts It appears that most high maturity organizations are aware of, have piloted, and have adopted agile methods. Most particularly Scrum. With good success in terms of customer satisfaction, quality, and meeting schedule expectations… and some impact on improving cost. This survey is only a first step in exploring the adoption of agile methods by high maturity organizations… 24

25 Questions and Answers