1. University of Southern California Center for Systems and Software Engineering Technical Debt CS 577 Software Engineering Barry Boehm Supannika Koolmanojwong

2. University of Southern California Center for Systems and Software Engineering What is Technical Debt? "Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.”interestobject-oriented — Ward Cunningham, 1992Ward Cunningham 2

3. University of Southern California Center for Systems and Software Engineering Outline Examples of technical debt situations Major causes of technical debt Ways to identify technical debt Ways to address and avoid technical debt 3

4. University of Southern California Center for Systems and Software Engineering Technical Debt Example 1 “I don’t know what happened, I just changed one line” “We can’t upgrade, It will break” “We can’t upgrade the code, we don’t have time” “We can’t upgrade the code, no one understands it” “Just put in the comment XXX, we will do it later” “Just put in the TODO comment” http://petdance.com/perl/technical-debt 4

5. University of Southern California Center for Systems and Software Engineering Technical Debt Example 2 Wrong architecture or NDI choices Lack of needed scalability, reliability, performance Incompatible Non-Developmental Items (NDI) Architectural style clashes cause crashes You don’t find out until too late (operations phase) No choice but to start again or rewrite big chunk to keep it working Read more: http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUVhttp://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUV 5

6. University of Southern California Center for Systems and Software Engineering Technical Debt Example 3 Error-prone code “.. If I change X, it is going to break Y, I think..” “ Don’t touch that code, last time we did, we spent a week fixing it…” 20% of the code where 80% of bugs are found Hard to understand Dangerous to change because done poorly one in the first place Not rewriting this code is one of the most expensive mistakes that developers make Read more: http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUVhttp://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUV http://petdance.com/perl/technical-debt/technical-debt.007.html 6

7. University of Southern California Center for Systems and Software Engineering Technical Debt Example 4 Not easily tested “.. I thought we had a test for that..” Don’t have good automated tests Tests keep falling apart when you change the code Testing costs tend to go up over time as you write more code Read more: http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUVhttp://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUV 7

8. University of Southern California Center for Systems and Software Engineering Technical Debt Example 4 Code that mysteriously works Nobody is sure how or why Might be written by the geek who left the company If nobody on the team understands it, it’s a time bomb Read more: http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUVhttp://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUV 8

9. University of Southern California Center for Systems and Software Engineering Technical Debt: More Examples Forward and backward compatibility –Short term debt Duplicate, copy-and-paste code –How many ? Trackable ? Hard coding Out of date documentation –“We just lost the drive, where are the backups?” –If nobody is using it, get rid of it. If people are using it, why isn’t it up to date? Read more: http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUVhttp://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUV 9

10. University of Southern California Center for Systems and Software Engineering Outline Examples of technical debt situations Major causes of technical debt Ways to identify technical debt Ways to address and avoid technical debt 10

11. University of Southern California Center for Systems and Software Engineering Major Causes of Technical Debt Conspiracy of Optimism Neglect of ICSM Principles –Stakeholder value-based guidance –Incremental commitment and accountability –Concurrent system engineering –Evidence and risk-driven decisions Business pressures Easiest-first; neglecting rainy day use cases Delayed Refactoring http://en.wikipedia.org/wiki/Technical_debt 11

12. University of Southern California Center for Systems and Software Engineering The Conspiracy of Optimism and The Cone of Uncertainty Copyright © USC-CSSE 12 October 16, 2012

13. University of Southern California Center for Systems and Software Engineering Copyright © USC-CSSE13 Effects of First Budget Shortfall: Added Cost of Weak System Engineering Calibration of COCOMO II Architecture and Risk Resolution (RESL) factor to 161 project data points October 16, 2012 Tech Debt

14. University of Southern California Center for Systems and Software Engineering 06/25/07 ©USC-CSSE14 How Much Architecting is Enough? Percent of Project Schedule Devoted to Initial Architecture and Risk Resolution Added Schedule Devoted to Rework (COCOMO II RESL factor) Total % Added Schedule 10000 KSLOC 100 KSLOC 10 KSLOC Sweet Spot Sweet Spot Drivers: Rapid Change: leftward High Assurance: rightward

15. University of Southern California Center for Systems and Software Engineering 1/13/2014 © USC-CSSE15 ICSM Principles Counterexample: Bank of America Master Net

16. University of Southern California Center for Systems and Software Engineering 16 October 16, 2012 Assumption of Stability vs. Rapid Change – Need evolutionary/incremental vs. one-shot development Uncertainties in competition, technology, organizations, mission priorities Copyright © USC-CSSE

17. University of Southern California Center for Systems and Software Engineering 1/13/2014 ICSM Activity Levels for Complex Systems 17© USC-CSSE

18. University of Southern California Center for Systems and Software Engineering 1/13/2014 Anchor Point Feasibility Evidence Descriptions Evidence provided by developer and validated by independent experts that: If the system is built to the specified architecture, it will –Satisfy the requirements: capability, interfaces, level of service, and evolution –Support the operational concept –Be buildable within the budgets and schedules in the plan –Generate a viable return on investment –Generate satisfactory outcomes for all of the success-critical stakeholders All major risks resolved or covered by risk management plans Serves as basis for stakeholders’ commitment to proceed Synchronizes and stabilizes concurrent activities Can be used to strengthen current schedule- or event-based reviews 18© USC-CSSE

19. University of Southern California Center for Systems and Software Engineering Major Causes of Technical Debt Conspiracy of Optimism Neglect of ICSM Principles –Stakeholder value-based guidance –Incremental commitment and accountability –Concurrent system engineering –Evidence and risk-driven decisions Business pressures Easiest-first; neglecting rainy day use cases Delayed Refactoring http://en.wikipedia.org/wiki/Technical_debt 19

20. University of Southern California Center for Systems and Software Engineering http://pkruchten.files.wordpress.com/2011/10/kruchten-111027-techdebt.pdf 20

21. University of Southern California Center for Systems and Software Engineering 06/25/07 ©USC-CSSE21 Cost Growth of Delaying Rework

22. University of Southern California Center for Systems and Software Engineering Outline Examples of technical debt situations Major causes of technical debt Ways to identify technical debt Ways to address and avoid technical debt 22

23. University of Southern California Center for Systems and Software Engineering Ways to Identify Technical Debt Top 10 inhibitors Not addressing major causes of technical debt –Cost, schedule overrruns –Neglected stakeholders –Lack of feasibility evidence –Delayed refactoring –Neglecting rainy day use cases Technical debt tools: SONAR, CAST, SQALE 23

24. University of Southern California Center for Systems and Software Engineering Top 10 Inhibitors – Similarities 24 A New Single SystemAn Existing Single SystemA System of Systems Requirements Volatility Lack of Interoperability UnprecedentednessHigh numbers of external interfaces Lack of / incompatible standard & protocol Delayed authority to proceed/start with fixed milestone UnprecedentednessRequirements Volatility Infeasible schedule/staffing profileVague RequirementsUnprecedentedness Lack of Domain ExperienceEmbedded poor quality softwareHigh numbers of external interfaces Technology VolatilityConflicting StakeholdersInfeasible schedule/staffing profile High numbers of external interfaces Delayed authority to proceed/start with fixed milestone Inability to test across systems Vague RequirementsInfeasible schedule/staffing profile Delayed authority to proceed/start with fixed milestone Under average people / Personnel Capability Technical debtLack of Domain Experience Technology ImmaturityInteroperability / compatibilityTechnology Volatility

25. University of Southern California Center for Systems and Software Engineering Top 10 Inhibitors – Differences 25 A New Single SystemAn Existing Single SystemA System of Systems Requirements Volatility Lack of Interoperability UnprecedentednessHigh numbers of external interfaces Lack of / incompatible standard & protocol Delayed authority to proceed/start with fixed milestone UnprecedentednessRequirements Volatility Infeasible schedule/staffing profileVague RequirementsUnprecedentedness Lack of Domain ExperienceEmbedded poor quality softwareHigh numbers of external interfaces Technology VolatilityConflicting StakeholdersInfeasible schedule/staffing profile High numbers of external interfaces Delayed authority to proceed/start with fixed milestone Inability to test across systems Vague RequirementsInfeasible schedule/staffing profile Delayed authority to proceed/start with fixed milestone Under average people / Personnel Capability Technical debtLack of Domain Experience Technology ImmaturityInteroperability / compatibilityTechnology Volatility

26. University of Southern California Center for Systems and Software Engineering Technical Debt Sources, Cost: SONAR Debt (in man days) cost_to_fix_duplications + cost_to_fix_violations + cost_to_comment_public_API + cost_to_fix_uncovered_complexity + cost_to_bring_complexity_below_threshold + cost_to_cut_cycles_at_package_level http://docs.codehaus.org/display/SONAR/Technical+Debt+Calculation 26 Average $3.61 to $5.42 per line of code.

27. University of Southern California Center for Systems and Software Engineering Technical Debt Observations “Agile Project Management”, Jim Highsmith, second edition 27

28. University of Southern California Center for Systems and Software Engineering Outline Examples of technical debt situations Major causes of technical debt Ways to identify technical debt Ways to address and avoid technical debt 28

29. University of Southern California Center for Systems and Software Engineering Fixing technical debt Big Bang –no new features for a year? Really? –spend some time cleaning up the mess –Good ? Dedicated Team –Have another team dedicated –Good ? 80/20 rule ? Boy Scout –remove technical debt little and often –If no tests, add some. If poor test, improve them. If bad code, refactor it –The boy scout rule – leave the camp cleaner than you found it http://www.javacodegeeks.com/2011/11/dealing-with-technical-debt.html#ixzz1pjQ8bQpF 29

30. University of Southern California Center for Systems and Software Engineering Ways to Address and Avoid Technical Debt Top 10 enablers Following ICSM Principles –Stakeholder value-based guidance –Incremental commitment and accountability –Concurrent system engineering –Evidence and risk-driven decisions Use technical debt tools: SONAR, CAST, SQALE 30

31. University of Southern California Center for Systems and Software Engineering Top 10 Enablers - Similarities 31 A New Single SystemAn Existing Single SystemA System of Systems Rapid Prototyping Target hardware lab / test like you fly & simulation Customer /tech requirements flexibility Target hardware lab / test like you fly & simulation Incremental test and feedbackRapid Prototyping Customer /tech requirements flexibilityIncremental Delivery & feedback Target hardware lab / test like you fly & simulation Incremental test and feedbackFlexible / Tailorable rulesIncremental test and feedback Incremental Delivery & feedbackAgile/lean approachCommon standard and protocol Decision making authorityRapid PrototypingReusing assets Best people / personnel capabilityCommon standard, interfaceTools and automation Agile/lean approachCustomer /tech requirements flexibilityCommon standard, interface Less context switching when doing multiple projects Domain knowledgeBest people / Personnel Capability Tools and automation Understanding of the existing system and interfaces Team cohesion

32. University of Southern California Center for Systems and Software Engineering Top 10 Enablers - Differences 32 A New Single SystemAn Existing Single SystemA System of Systems Rapid Prototyping Target hardware lab / test like you fly & simulation Customer /tech requirements flexibility Target hardware lab / test like you fly & simulation Incremental test and feedbackRapid Prototyping Customer /tech requirements flexibilityIncremental Delivery & feedback Target hardware lab / test like you fly & simulation Incremental test and feedbackFlexible / Tailorable rulesIncremental test and feedback Incremental Delivery & feedbackAgile/lean approachCommon standard and protocol Decision making authorityRapid PrototypingReusing assets Best people / personnel capabilityCommon standard, interfaceTools and automation Agile/lean approachCustomer /tech requirements flexibilityCommon standard, interface Less context switching when doing multiple projects Domain knowledgeBest people / Personnel Capability Tools and automation Understanding of the existing system and interfaces Team cohesion

33. University of Southern California Center for Systems and Software Engineering Addressing Enablers and Inhibitors 33 EnablersInhibitors Best peopleUnder average people Decision making authorityLack of decision making authority Team CohesionConflicting stakeholders Incremental Test and FeedbackInability to test across systems

34. University of Southern California Center for Systems and Software Engineering The Bottom Line Technical debt often a good practice –Meeting market windows –Prototyping to determine user needs, satisfaction –Short-term fixes, targets of opportunity But needs pay-down later –To avoid mounting debt and interest Need balanced investment in fixes, new features 34

35. University of Southern California Center for Systems and Software Engineering References http://pkruchten.files.wordpress.com/2011/ 10/kruchten-111027-techdebt.pdfhttp://pkruchten.files.wordpress.com/2011/ 10/kruchten-111027-techdebt.pdf 35