COSC 4506/ITEC 3506 Software Engineering Case Study in Software Evolution

The work of software engineers [CapersJones98] 1950 is baseline (100%) If we look at historical data regarding SE projects we can see a trend towards more enhancements and repairs and less new projects. When you see these numbers, you might wonder [next slide] Currently > 60% of software engineers work on enhancement and repair of existing systems in 2020, only 30% will work on new software

What causes those numbers? … what causes them. ANY IDEAS? [changing expectations, changing requirements, changing environment, …] This a phenomenon dubbed “software aging”

Parnas: Software Aging [Parnas ICSE94] Software systems deteriorate over time Aging is caused by: expectations that change – new demands new environments – emergent properties Examples: MS-Word generates HTML, OLE, network drives, XML as a RDBMS, etc… It’s inevitable in successful products (long life) large software systems are major assets of organisations Software Aging was reported by Parnas’ invited talk at ICSE 1994, part of the reading material of this course. In biology, aging caused by cell decay, in software there is not really ‘bit rot’. The causes are different: … Eick et al invesigated these claims on software aging in the data in change management systems (e.g. CVS, subversion,…) and found proof of its existence.

Why replacement doesn’t work Significant business risk in simply scrapping a legacy system and replacing it with a new system: Legacy systems rarely have a complete specification. During their lifetime they have undergone major changes which may not have been documented Business processes are reliant on the legacy system The system may embed business rules that are not formally documented elsewhere New software development is risky and may not be successful Now you might think that it’s possible to prevent such aging problems by replacing the system when it has become too old Unfortunately, things are not that simple…

Problems of waterfall Requirements volatility (Cusumano and Selby) 30% or more requirements may change during development this is the direct result of the team’s learning process and software interoperability Maintenance phase is not uniform frequency of the changes in long lived systems peaks, then declines

Standish group report In 1995 31% of all software projects were cancelled 53% were “challenged” (completed only with great difficulty, with large cost or time overruns) only 16% could be called successful. Obviously, the waterfall metaphor did not solve the problems of software development

Instances of the new paradigm New life-cycle models emphasize software evolution Staged model of software lifecycle Based on data from long-lived systems Iterative development Rational Unified Process Agile development SCRUM Extreme programming

minicycle of an incremental change Change request, concept extraction Location Locate the latent concept in the code Actualization Implement the concept explicitly and fully Incorporation "plug-in” the new code into the old code Propagation propagate changes through the old code

change request formulate the change request analyze the change request extract the concepts Point-of-Sale: introduce credit card payment

domain concepts play key role in incremental change concepts are present in the code New concept primitive concepts have to be enriched Point-of-Sale: introduce credit card payment old code: payment represented as just one number

Concept location Concept location finds where a change is to be made Change requests are most often formulated in terms of domain concepts Example: “Correct error that arises when trying to paste a text” the programmer must find in the code the locations where concept “paste” is located this is the start of the change

concept location finds code that implements concept part of program comprehension concept location methodologies human knowledge "grep" (pattern matching) dynamic search (execution traces) static search through dependency graph traceability tools

actualization, incorporation new implementation of the concept(s) standard software design methodologies incorporation replace the old code by program dependencies between the old and the new instance - definitions function call data flow

change propagation visit components one-by-one if the visited component is modified, it may no longer fit secondary changes must be made in interacting (“neighboring”) components secondary changes may trigger additional changes “ripple effect” software is inconsistent during propagation

Visit to a class (graphically) Change propagates Change does not propagate

Example Software Evolution Point-Of-Sale application initial development from scratch implements a working program keeps an inventory, receives delivery, sells products, and supports a cash register. concepts are closely tied together and must be present in any meaningful program classes designed and implemented at a minimal level of functionality

UML class diagram

adding support for credit cards and checks location “payment” in class “sale” actualization hierarchy of classes base class payment subclasses for cash and authorized payments further subclasses for checks and credit cards

credit cards and checks, cont. incorporation int “payment” in class “sale” replaced by an instance of class “payment” propagation classes sale and register must be changed

next incremental change support for price fluctuations products can have different prices at different times limited time sales location “price” in class “Item” actualization new classes price and promoPrice

price fluctuation, cont. incorporation int “price” in class “item” replaced by an instance of class “price” propagation classes item, store, saleLineItem, and sale required a change

UML class diagram

next incremental change complexities of sale taxes different products may have different sales tax, depending on state law location “tax” in class “item” actualization new class taxCategory

incorporation register store item price taxCategory sale saleLineItem promoPrice payment authorizedPayment cash check charge

change propagation register store item price sale saleLineItem promoPrice taxCategory payment authorizedPayment cash check charge

change propagation register store item price sale saleLineItem promoPrice taxCategory payment authorizedPayment cash check charge

change propagation register store item price sale saleLineItem promoPrice taxCategory payment authorizedPayment cash check charge

change propagation register store item price sale saleLineItem promoPrice taxCategory payment authorizedPayment cash check charge

change propagation register store item price sale saleLineItem promoPrice taxCategory payment authorizedPayment cash check charge

order of implementation The concepts must be implemented in the order of their dependency “tax” is dependent on “item” tax without an item is meaningless the implementation of “item” must precede the implementation of “tax” mutually dependent concepts must be implemented at the same time example: store and register

observations overhead of change concept location and change propagations small changes have the same overhead decreased efficiency of the work increased likelihood of errors big changes overload the cognitive capabilities of the programmers

some numbers and issues all incremental changes: introduced 1 .. 5 new classes into the program visited 3 .. 7 old classes modified 1 .. 4 of them when a class was visited but not changed, should its neighbors have been visited also? we inspected more classes than absolutely necessary the decision whether to propagate is based on semantics