1. Formal Foundations for Software Evolution Programming Technology Lab Tom Mens tommens@vub.ac.be http://prog.vub.ac.be/~tommens

2. January 2001© Programming Technology Lab 2 Need better tool support for  version control  e.g. upgrading application frameworks  collaborative software development  software merging  change management  change propagation  change impact analysis  ripple effect ...  evolution at a high level of abstraction  evolution of design patterns  architectural evolution ... object-oriented software evolution

3. January 2001© Programming Technology Lab 3 Need better tool support for  co-evolution between different phases  maintaining consistency  checking compliance between architecture and design, design and implementation,...  re-engineering of legacy systems  program comprehension  reverse engineering  migration ...  empirical research on software evolution  based on change metrics  predictive models ...

4. January 2001© Programming Technology Lab 4 Tool support must be  scalable  applicable to large-scale software systems  “A major challenge for the research community is to develop a good theoretical understanding an underpinning for maintenance and evolution, which scales to industrial applications.” [Bennett&Rajlich 2000]  language-independent  independent of the programming or modelling language  generally applicable in all phases of the software life- cycle

5. January 2001© Programming Technology Lab 5 Reuse Contracts Use graph rewriting to provide a formal foundation for software evolution based on reuse contracts

6. January 2001© Programming Technology Lab 6 Benefits of Reuse Contracts 1.Document reuse and evolution 2.Deal with upgrade problems 3.Provide support for software merging 4.Provide support for framework refactoring 5.Are independent of kind of software artifact  analysis, architecture, design, implementation

7. January 2001© Programming Technology Lab 7 1. Documenting Reuse DesktopFolder position contents move: add: addMany: SizedFolder add: item size reuse Extension(size, attribute) Refinement(add, size, update) size =+ item.size

8. January 2001© Programming Technology Lab 8 DesktopFolder position contents move: add: addMany: 1. Documenting Evolution evolution Coarsening(addMany, add, calls) DesktopFolder position contents move: add: addMany:

9. January 2001© Programming Technology Lab 9 2. Dealing with Upgrade Problems DesktopFolder position contents move: add: addMany: evolution DesktopFolder position contents move: add: addMany: reuse size not updated when adding many items SizedFolder add: item size size =+ item.size SizedFolder add: item size size =+ item.size

10. January 2001© Programming Technology Lab 10 2. Dealing with Upgrade Problems DesktopFolder position contents move: add: addMany: evolution DesktopFolder position contents move: add: addMany: SizedFolder add: size reuse Coarsening(addMany, add, calls) Extension(size, attribute) Refinement(add, size, update) inconsistent operation conflict

11. January 2001© Programming Technology Lab 11 Conflict Table 2. Dealing with Upgrade Problems extension refinement coarsening extensionrefinementcoarsening no conflicts inconsistent operations interface conflicts operation capture, unanticipated recursion operation capture, inconsistent operations no conflicts operation capture, inconsistent operations  extension/cancellation: adding/removing an operation or attribute  refinement/coarsening: adding/removing invocation or attribute access  abstraction/concretisation: making an operation abstract/concrete

12. January 2001© Programming Technology Lab 12 3. Support for Software Merging DesktopFolder position contents move: add: addMany: evolution DesktopFolder v1 contents add: addMany: evolution Extension(position, attribute) Extension(move:, method) Coarsening(addMany, add, calls) Extension(size, attribute) Refinement(add, size, update) inconsistent operation conflict DesktopFolder v2a contents size add: addMany:

13. January 2001© Programming Technology Lab 13 4. Support for FW Refactoring Refactoring conflict ! After merging, size in NestingFolder accidentally overrides size in Folder, which has a different behaviour! CreateSubclass FolderFile totalSizesize NestingFolder size size calculates number of items in a folder FolderFile totalSizesize both calculate item size in bytes PullUpVariable FolderFile DesktopItem size

14. January 2001© Programming Technology Lab 14 5. Independent of software artifact «interaction» linkResolving :Browser:Document browser {resolveLink invokes} getURL «provider» WebNavigation «interaction» mouseClicking :Browser:Document doc {handleClick invokes} mouseClick self {mouseClick invokes} resolveLink «collaboration» «interface» Browser handleClick getURL «interface» Document mouseClick resolveLink doc browser {participant extension} {participant refinement} «client» PDFNavigation «collaboration» «interface» Document gotoPage«added» {resolveLink invokes} gotoPage «added» «interaction» linkResolving :Browser:Document browser {resolveLink invokes} getURL «removed» {participant coarsening}

15. January 2001© Programming Technology Lab 15 Problems with reuse contracts  How to scale up to higher-level transformations  e.g. dealing with class collaborations  How to apply to other domains  e.g. software architectures  Solution: Provide a formal foundation for reuse contracts

16. January 2001© Programming Technology Lab 16 Reuse Contract Formalism  Represent software artifacts by graphs  Represent software evolution by graph rewriting  Domain-independent formalism  Independent of the target language  Independent of the phase in the life-cycle  Lightweight formalism to facilitate tool support  Formal characterisation of evolution conflicts  Prototype tool implemented in Prolog

17. January 2001© Programming Technology Lab 17 Graphs G Triangle «class» Circle «class» «is-a» intersects «operation» «assoc» center radius «attribute» «has-a» vertices {3} Point «class»Shape «class» area «operation» perimeter «operation» x «attribute» distanceTo «operation» y «attribute» «is-a»  Internal Graph Representation +intersects(c: Circle) -radius Circle +distanceTo(p: Point) -x -y Point Triangle +area() +perimeter() Shape center vertices3  Example: UML class diagram  Node types:  «class»  «attribute»  «operation»  «interface»  Edge types:  «assoc»  «has-a» (aggregation)  «is-a» (generalisation)  «implements»

18. January 2001© Programming Technology Lab 18 Type Graph v e node type edge type implements nested operation attribute interface class assoc, has-a, is-a is-a uses invokes nested  Used to specify domain-specific constraints

19. January 2001© Programming Technology Lab 19 P m R area «operation» radius «attribute» «uses» G Circle «class» area «operation» «uses» perimeter «operation» radius «attribute» H Circle «class» area «operation» «uses» perimeter «operation» radius «attribute» «uses» L area «operation» radius «attribute» Graph Rewriting  Used to specify software evolution graph productionleft-hand side right-hand side (negative) precondition (positive) postcondition graph rewriting embedding in context result graphinitial graph

20. January 2001© Programming Technology Lab 20  Use restricted set of graph productions  AddNode  DropNode  AddEdge  DropEdge  RetypeNode  RetypeEdge  RelabelNode  RelabelEdge Primitive Graph Productions AddEdge (area,radius,«uses») R area «operation» radius «attribute» «uses» L area «operation» radius «attribute» DropNode (Triangle.area,«operation») R Triangle L «operation» area Triangle

21. January 2001© Programming Technology Lab 21 Syntactic Conflicts P1P1 P 2 = DropNode(area,«operation») P 1 = AddEdge(area,radius,«uses») P2P2 P1P1 P2P2 Undefined source conflict Syntactic conflict if P 1 and P 2 are not parallel independent G Circle «class» area «operation» perimeter «operation» radius «attribute» «uses» > G2G2 Circle «class» perimeter «operation» radius «attribute» «uses» G1G1 Circle «class» area «operation» perimeter «operation» radius «attribute» «uses»

22. January 2001© Programming Technology Lab 22 Syntactic Conflict Table Complete fine-grained characterisation of syntactic conflicts AC 3 AddN (v,  ) DropN (v,  ) AddE (e,v,w,  ) AddE (e,u,v,  ) DropE (e,v,w,  ) DropE (e,u,v,  ) RetypeN (v, ,   ) RetypeE (e,v,w, ,   ) RetypeE (e,u,v, ,   ) AddNode (v,  ) AC 1  DropNode (v,  )  AC 2 AC 3 AC 4  AC 9  AddEdge (e,v,w,  )  AC 5  AddEdge (e,u,v,  )  AC 4  AC 5  DropEdge (e,v,w,  )  AC 6  AC 10 if  =   DropEdge (e,u,v,  )  AC 6  AC 10 if  =  RetypeNode (v, ,   )  AC 9  AC 7  RetypeEdge (e,v,w, ,   )  AC 10 if  =   AC 8 if  =   RetypeEdge (e,u,v, ,   )  AC 10 if  =   AC 8 if  = 

23. January 2001© Programming Technology Lab 23  Based on the formal notion of  pushouts and pullbacks  Fine-grained conflict characterisation  By detecting occurrence of graph patterns in result graph Semantic Conflicts addManyaddsize « updates » « calls » {added during reuse} {removed during evolution} inconsistent method conflict « class » ? « class » ? « inherits » {added by developer 2} {added by developer 1} cyclic inheritance

24. January 2001© Programming Technology Lab 24 Structural Conflicts InsertClass (Shape,Quadrangle, [Square,Rectangle]) AddSubclass(Shape,Parallellogram) AddSubclass(Shape,Triangle) Shape SquareRectangle QuadrangleCircle Shape SquareRectangleCircle Shape SquareRectangleCircleTriangleParallellogram More difficult to detect in a general way

25. January 2001© Programming Technology Lab 25 Using Assertions  Problem  Using a predefined set of graph productions is not generic  Introducing new productions requires changes to conflict table  Does not scale up to composite graph productions  Solution  make formalism independent of chosen productions  define productions based on assertions only  preconditions, postconditions, invariants

26. January 2001© Programming Technology Lab 26 Example

27. January 2001© Programming Technology Lab 27 Example ctd. surface attribute a L area operation a radius attribute c f uses R preconditions postconditions invariants

28. January 2001© Programming Technology Lab 28 Syntactic Conflict Table  Detect syntactic merge conflicts  in terms of transformation preconditions  compare breaches of application conditions  Advantages  more general  does not rely on predefined graph produtions  more scalable  can be used directly for composite or domain-specific graph productions

29. January 2001© Programming Technology Lab 29 Syntactic Conflict Table -i+isource (e,v) target (e,v) label (i,L) type (i,T) -source (e,v) -target (e,v) -j  if i=j  if j=e  if j=v  if j=e  if j=v  if i=j  if j=v +j C1 if i=j C2 if e=j  if v=j C3 if e=j  if v=j C4 if i=jC5 if i=jC6 if j=v C7 if j=e C8 if j=v C9 if j=e source (f,w) C10 if (e,v)=(f,w)  if v=w  if (e,v)=(f,w)  target (f,w) C11 if (e,v)=(f,w)  if v=w  if (e,v)=(f,w) label (j,L2) C12 if i=j  type (j,T2) C13 if i=j  -source (f,w) C14 if e=f and v  w  -target (f,w) C15 if e=f and v  w

30. January 2001© Programming Technology Lab 30 Using dependencies  Dependencies can be defined between productions  based on relations between assertions strong satisfaction dependency strong capture dependency weak satisfaction dependency derived assertions

31. January 2001© Programming Technology Lab 31 Using dependencies  Dependencies can be used to address scalability 1.Reordering productions in a sequence 2.Defining “atomic” composite productions from a sequence of productions 3.Removing redundancy in a production sequence 4.Factoring out commonalities from parallel production sequences 5.Parallellising subsequences

32. January 2001© Programming Technology Lab 32 1. Reordering productions

33. January 2001© Programming Technology Lab 33 2. Composite productions

34. January 2001© Programming Technology Lab 34 3. Removing redundancy  Allows us to simplify a production sequence AddN(b) AddN(a)AddN(b)Rena(c,d)AddE(a,b)Rety(d)DropE(a,b)DropN(a) AddN(a)AddN(b)DropN(a) AddN(a)AddN(b)DropN(a) redundant pair reorder redundant pair

35. January 2001© Programming Technology Lab 35 3. Removing Redundancy

36. January 2001© Programming Technology Lab 36 4. Factoring out commonalities  Find commonalities in two parallel sequences, and factor out  facilitates merging and conflict detection

37. January 2001© Programming Technology Lab 37 4. Factoring out: Example

38. January 2001© Programming Technology Lab 38 4. Factoring out: Example

39. January 2001© Programming Technology Lab 39 5. Parallellising subsequences RenameN(a,surface,area) AddN(b,perimeter,attribute) RetypeN(a, attribute,operation) RetypeN(b, attribute,operation) AddN(c,radius,attribute) AddE(e,b,c,uses,uses) AddE(f,a,c,uses,uses) DropE(e,b,c) DropN(b)

40. January 2001© Programming Technology Lab 40 5. Parallellising subsequences RenameN(a,surface,area) AddN(b,perimeter,attribute) RetypeN(a, attribute,operation) RetypeN(b, attribute,operation) AddN(c,radius,attribute) AddE(e,b,c,uses,uses)AddE(f,a,c,uses,uses) DropE(e,b,c) DropN(b)

41. January 2001© Programming Technology Lab 41 Validation of Reuse Contracts  Industrial case  One base release line with many customisations for different customer applications  Collaborative software development with parallel changes to base release and customisations 7.2 7.4 10.x 11 12 NDR DR WDR 0.1 VTM TV2 WDR 1.0 WDR 2.0

42. January 2001© Programming Technology Lab 42 Validation of Reuse Contracts  Use reuse contracts to document  evolution of base release line  customisation to different customer applications  evolution of customer applications  Use conflict detection to support  upgrades of customer application to more recent base release  refactoring of base release for easier future customisation  Provide help with consolidation  detect commonalities between different customisations  apply commonalities to base release

43. January 2001© Programming Technology Lab 43 To Do  User-friendly tool support  Perform large-scale experiments  Validate scalability  Look at conflict resolution techniques  Deal with problem of co-evolution

44. January 2001© Programming Technology Lab 44 Co-Evolution  Underlying idea  Keep representation of same software artifact at different levels of abstraction (e.g. design and implementation) synchronised during evolution refinement abstract layer concrete layer consistent??? evolution

45. January 2001© Programming Technology Lab 45 Co-Evolution  needed for collaborative development  when different persons make changes to same software concept at different levels of abstraction  facilitates delta analysis  relate changes at analysis level back to the code and keep them synchronised  address architectural drift and software erosion  use a compliance checking algorithm  detect conflicts between changes at different levels