CS 577b Software Engineering II -- Introduction 8 September 2018 Technical Debt CS 577 Software Engineering Barry Boehm Supannika Koolmanojwong © 2002-6 USC Center for Software Engineering

What are the cost of a software? Think about the whole life cycle

Total cost of ownership a financial estimate whose purpose is to help consumers and enterprise managers determine direct and indirect costs of a product or system. including the costs to research, develop, acquire, own, operate, maintain, and dispose of a system

Potential total cost of ownership Computer hardware and programs Operation expenses Long term expenses HW and SW Network Server Workstation Installation & integration Purchasing research Warranties and licenses License tracking - compliance Migration expenses Risks: susceptibility to vulnerabilities, availability of upgrades, patches and future licensing policies, etc. Infrastructure Electricity Testing costs Downtime, outage and failure expenses Diminished performance Security (including breaches, loss of reputation, recovery & prevention) Backup and recovery process Technology training Audit Insurance IT personnel Replacement Future upgrade or scalability expenses Decommissioning

Relative* Total Ownership Cost (TOC) For single system life cycle (TOC-SS) ~5% architecture investment ~5% architecture investment ~25% architecture investment * Cumulative architecting and rework effort relative to initial development effort

Technical Debt Code Debt Deficit programming Software Decay CodeSmell coined by Ward Cunningham Design Debt Technical Inflation

Examples of Technical Debt

What is Technical Debt? "Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.” — Ward Cunningham, 1992

Outline Examples of technical debt situations Major causes of technical debt Ways to identify technical debt Ways to address and avoid technical debt

Technical Debt Example 1 “I don’t know what happened, I just changed one line” “We can’t upgrade, It will break” “We can’t upgrade the code, we don’t have time” “We can’t upgrade the code, no one understands it” “Just put in the comment XXX, we will do it later” “Just put in the TODO comment” http://petdance.com/perl/technical-debt

Technical Debt Example 2 Wrong architecture or NDI choices Lack of needed scalability, reliability, performance Incompatible Non-Developmental Items (NDI) Architectural style clashes cause crashes You don’t find out until too late (operations phase) No choice but to start again or rewrite big chunk to keep it working Read more: http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUV

Technical Debt Example 3 Error-prone code “ .. If I change X, it is going to break Y, I think ..” “ Don’t touch that code, last time we did, we spent a week fixing it…” 20% of the code where 80% of bugs are found Hard to understand Dangerous to change because done poorly one in the first place Not rewriting this code is one of the most expensive mistakes that developers make Read more: http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUV http://petdance.com/perl/technical-debt/technical-debt.007.html

Technical Debt Example 4 Not easily tested “ .. I thought we had a test for that ..” Don’t have good automated tests Tests keep falling apart when you change the code Testing costs tend to go up over time as you write more code Read more: http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUV

Technical Debt Example 4 Code that mysteriously works Nobody is sure how or why Might be written by the geek who left the company If nobody on the team understands it, it’s a time bomb Read more: http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUV

Technical Debt: More Examples Forward and backward compatibility Short term debt Duplicate, copy-and-paste code How many ? Trackable ? Hard coding Out of date documentation “We just lost the drive, where are the backups?” If nobody is using it, get rid of it. If people are using it, why isn’t it up to date? Read more: http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUV

Development Cost(Perfect World) % Effort per Phase Analysis Design Implementation Test Integration

Classification of Defects Analysis Design Implementation Test Integration Defect Insertion Defect detection & Removal Defects Typical Defect Profiles Implementation Defects

Development Cost(Real World) % Effort per Phase Analysis Design Implementation Test Integration

Development Cost % Effort per Phase Analysis Design Implementation Test Integration Real world Perfect World

Development Cost % Effort per Phase Technical Debt? Technical Debt? % Effort per Phase Technical Debt? Not enough information in this diagram to determine whether these are technical debt. The dip in the middle could represent technical debt from trying to move things along, but paying for it in integration and test. Technical Debt? Analysis Design Implementation Test Integration Real world Perfect World

Development Cost % Effort per Phase Technical Debt? Technical Debt? Analysis Design Implementation Test Integration Real world Perfect World

COTS Integration % Effort per Phase This one shows that a decision must have been made that there was no real need to perform the System Definition activities. This represents Technical Debt. Technical Debt? Analysis Design Implementation Test Integration Real world Perfect World

Outline Examples of technical debt situations Major causes of technical debt Ways to identify technical debt Ways to address and avoid technical debt

Major Causes of Technical Debt Conspiracy of Optimism Neglect of ICSM Principles Stakeholder value-based guidance Incremental commitment and accountability Concurrent system engineering Evidence and risk-driven decisions Business pressures Easiest-first; neglecting rainy day use cases Delayed Refactoring http://en.wikipedia.org/wiki/Technical_debt

The Conspiracy of Optimism and The Cone of Uncertainty Copyright © USC-CSSE October 16, 2012

Assumption of Stability vs Assumption of Stability vs. Rapid Change – Need evolutionary/incremental vs. one-shot development Uncertainties in competition, technology, organizations, mission priorities There is Another Cone of Uncertainty: Shorter increments are better Uncertainties in competition and technology evolution and changes in organizations and mission priorities, can wreak havoc with the best of system development programs. In addition, the longer the development cycle, the more likely it will be that several of these uncertainties or changes will occur and make the originally-defined system obsolete. Therefore, planning to develop a system using short increments helps to ensure that early, high priority capabilities can be developed and fielded and changes can be more easily accommodated in future increments. Copyright © USC-CSSE October 16, 2012

Major Causes of Technical Debt Conspiracy of Optimism Neglect of ICSM Principles Stakeholder value-based guidance Incremental commitment and accountability Concurrent system engineering Evidence and risk-driven decisions Business pressures Easiest-first; neglecting rainy day use cases Delayed Refactoring http://en.wikipedia.org/wiki/Technical_debt

http://pkruchten. files. wordpress http://pkruchten.files.wordpress.com/2011/10/kruchten-111027-techdebt.pdf

Cost Growth of Delaying Rework 06/25/07 ©USC-CSSE

Outline Examples of technical debt situations Major causes of technical debt Ways to identify technical debt Ways to address and avoid technical debt

Ways to Identify Technical Debt Top 10 inhibitors Not addressing major causes of technical debt Cost, schedule overrruns Neglected stakeholders Lack of feasibility evidence Delayed refactoring Neglecting rainy day use cases Technical debt tools: SONAR, CAST, SQALE

Sonar Mainly manage the software quality SQALE – Software Quality Assessment based on Lifecycle Expectations

Nemo - An open source tool – by sonar http://nemo.sonarqube.org

Sonar tool Bad Distribution of Complexity Duplications http://docs.codehaus.org/display/SONAR/Developers%27+Seven+Deadly+Sins

Sonar tool Lack of Unit Tests No Coding Standards http://docs.codehaus.org/display/SONAR/Developers%27+Seven+Deadly+Sins

Sonar tool Not Enough or Too Many Comments (time consuming maintenance) Documented complex or not straightforward pieces of code Readable unit tests scenarios that will help the developers understand the code Documented API that will help external developers understand how to use those API Respect of coding standard such as naming conventions Potential Bugs http://docs.codehaus.org/display/SONAR/Developers%27+Seven+Deadly+Sins

Top 10 enablers – Similarities causes of technical debt A New Single System An Existing Single System A System of Systems Requirements Volatility Lack of Interoperability Unprecedentedness High numbers of external interfaces Lack of / incompatible standard & protocol Delayed authority to proceed/start with fixed milestone Infeasible schedule/staffing profile Vague Requirements Lack of Domain Experience Embedded poor quality software Technology Volatility Conflicting Stakeholders Inability to test across systems Under average people / Personnel Capability Technical debt Technology Immaturity Interoperability / compatibility

Top 10 enabler– Differences A New Single System An Existing Single System A System of Systems Requirements Volatility Lack of Interoperability Unprecedentedness High numbers of external interfaces Lack of / incompatible standard & protocol Delayed authority to proceed/start with fixed milestone Infeasible schedule/staffing profile Vague Requirements Lack of Domain Experience Embedded poor quality software Technology Volatility Conflicting Stakeholders Inability to test across systems Under average people / Personnel Capability Technical debt Technology Immaturity Interoperability / compatibility

Technical Debt Sources, Cost: SONAR (in man days) cost_to_fix_duplications +  cost_to_fix_violations +  cost_to_comment_public_API +  cost_to_fix_uncovered_complexity +  cost_to_bring_complexity_below_threshold +  cost_to_cut_cycles_at_package_level Average $3.61 to $5.42 per line of code. http://docs.codehaus.org/display/SONAR/Technical+Debt+Calculation

Technical Debt Observations “Agile Project Management”, Jim Highsmith, second edition

Outline Examples of technical debt situations Major causes of technical debt Ways to identify technical debt Ways to address and avoid technical debt

Fixing technical debt Big Bang Dedicated Team Boy Scout no new features for a year? Really?  spend some time cleaning up the mess Good ? Dedicated Team Have another team dedicated Good ? 80/20 rule ? Boy Scout remove technical debt little and often If no tests, add some. If poor test, improve them. If bad code, refactor it The boy scout rule – leave the camp cleaner than you found it http://www.javacodegeeks.com/2011/11/dealing-with-technical-debt.html#ixzz1pjQ8bQpF

Ways to Address and Avoid Technical Debt Top 10 enablers Following ICSM Principles Stakeholder value-based guidance Incremental commitment and accountability Concurrent system engineering Evidence and risk-driven decisions Use technical debt tools: SONAR, CAST, SQALE

Top 10 Inhibitors- Similarities Help mitigating technical debt A New Single System An Existing Single System A System of Systems Rapid Prototyping Target hardware lab / test like you fly & simulation Customer /tech requirements flexibility Incremental test and feedback Incremental Delivery & feedback Flexible / Tailorable rules Agile/lean approach Common standard and protocol Decision making authority Reusing assets Best people / personnel capability Common standard, interface Tools and automation Less context switching when doing multiple projects Domain knowledge Best people / Personnel Capability Understanding of the existing system and interfaces Team cohesion

Top 10 Enablers – Differences Help mitigating technical debt A New Single System An Existing Single System A System of Systems Rapid Prototyping Target hardware lab / test like you fly & simulation Customer /tech requirements flexibility Incremental test and feedback Incremental Delivery & feedback Flexible / Tailorable rules Agile/lean approach Common standard and protocol Decision making authority Reusing assets Best people / personnel capability Common standard, interface Tools and automation Less context switching when doing multiple projects Domain knowledge Best people / Personnel Capability Understanding of the existing system and interfaces Team cohesion

Addressing Enablers and Inhibitors Best people Under average people Decision making authority Lack of decision making authority Team Cohesion Conflicting stakeholders Incremental Test and Feedback Inability to test across systems

The Bottom Line Technical debt often a good practice Meeting market windows Prototyping to determine user needs, satisfaction Short-term fixes, targets of opportunity But needs pay-down later To avoid mounting debt and interest Need balanced investment in fixes, new features

FCR ARB Post all materials to your team website before your presentation All team members must be available Don’t forget to dry run Print out 2 copies of your presentation for your clients (2 slides per page) Check EC-17 for more info No class on 10/12, 10/14, and 10/16   Mon 10/12 Wed 10/14 Fri 10/16 Mon 10/19 12:30 pm – 1:50 pm Team 06 2:00 PM - 3:20 PM Team 04 3:30 PM - 4:50 PM Team 05 Team 07 5:00 PM - 6:20 PM Team 02 Team 03 Team 01 (5:30PM)

References http://pkruchten.files.wordpress.com/2011/10/kruchten-111027-techdebt.pdf

Technical Debt Cost Estimation Principal = Repair Effort Interest = Maintenance Effort [Nugroho et al 2011] [Curtis et al 2012] Debt (in man days) cost_to_fix_duplications +  cost_to_fix_violations +  cost_to_comment_public_API +  cost_to_fix_uncovered_complexity +  cost_to_bring_complexity_below_threshold +  cost_to_cut_cycles_at_package_level Ref: Nugroho et al 2011, An Empirical Model of Technical Debt and Interest, MTD 2011 Ref: Curtis e al 2011, Estimating the Principal of an Application's Technical Debt. IEEE Software

No sample less than 10KSLOC 700 applications 158 organizations 357 MLOC No sample less than 10KSLOC Conservative - Assume all violations would be fixed within one hour Weighted - varied the hours needed for fixing within each severity category, More realistic - based on data observed in several IT organizations. Ref: Curtis e al 2011, Estimating the Principal of an Application's Technical Debt. IEEE Software

Average Technical Debt per LOC = $3.61 Ref: Curtis e al 2011, Estimating the Principal of an Application's Technical Debt. IEEE Software

Ref: Curtis e al 2011, Estimating the Principal of an Application's Technical Debt. IEEE Software

Technical Debt within each technology http://www.castsoftware.com/research-labs/technical-debt-estimation

Sonar Mainly manage the software quality SQALE – Software Quality Assessment based on Lifecycle Expectations

Nemo - An open source tool – by sonar http://nemo.sonarqube.org

Sonar – SQALE plug-in http://www. sonarsource

Developers’ 7 Deadly Sins Bad Distribution of Complexity Duplications Lack of Unit Tests No Coding Standards Not Enough or Too Many Comments Potential Bugs Spaghetti Design http://docs.codehaus.org/display/SONAR/Developers%27+Seven+Deadly+Sins

Developers’ 7 Deadly Sins Bad Distribution of Complexity Duplications http://docs.codehaus.org/display/SONAR/Developers%27+Seven+Deadly+Sins

Developers’ 7 Deadly Sins Lack of Unit Tests No Coding Standards http://docs.codehaus.org/display/SONAR/Developers%27+Seven+Deadly+Sins

Developers’ 7 Deadly Sins Not Enough or Too Many Comments (time consuming maintenance) Documented complex or not straightforward pieces of code Readable unit tests scenarios that will help the developers understand the code Documented API that will help external developers understand how to use those API Respect of coding standard such as naming conventions Potential Bugs http://docs.codehaus.org/display/SONAR/Developers%27+Seven+Deadly+Sins

Developers’ 7 Deadly Sins Spaghetti Design Hunting for cycles Checking your own architecture rules Managing your libraries Checking the single responsibility principle with LCOM4 (Lack of Cohesion of Methods) RFC (Response for Class) – checking coupling (complexity of class in terms of method calls) http://docs.codehaus.org/display/SONAR/Developers%27+Seven+Deadly+Sins

x = = x http://www.moovweb.com/blog/technical-debt-infographic/

In-Class 5 From your team project Be Specific Identify 1 technical debt that your team has What is the enabler of this technical debt? How do you plan to mitigate this technical debt? Be Specific Individual work, do not discuss with others Off-campus : Due Monday Oct 12. D2L