Formal Foundations for Software Evolution Programming Technology Lab Tom Mens

January 2001© Programming Technology Lab 2 Need better tool support for  version control  e.g. upgrading application frameworks  collaborative software development  software merging  change management  change propagation  change impact analysis & effort estimation  ripple effect ...  evolution at a high level of abstraction  evolution of design patterns, refactorings  architectural evolution ... object-oriented software evolution

January 2001© Programming Technology Lab 3 Need better tool support for  co-evolution between different phases  maintaining consistency  checking compliance between architecture and design, design and implementation,...  re-engineering of legacy systems  program comprehension  reverse engineering  migration ...  empirical research on software evolution  based on change metrics  predictive models ...

January 2001© Programming Technology Lab 4 Tool support must be  scalable  applicable to large-scale software systems  “A major challenge for the research community is to develop a good theoretical understanding an underpinning for maintenance and evolution, which scales to industrial applications.” [Bennett&Rajlich 2000]  domain-independent  independent of the programming or modelling language  generally applicable in all phases of the software life- cycle

January 2001© Programming Technology Lab 5 Strategy  Use lightweight formalisms  declarative reasoning  graph rewriting  Tackle problems from formal as well as practical side  Bottom-up approach: rely on existing formalisms to provide a clear and unambiguous definition of evolution problems  Top-down approach: look at real-world situations to ensure usefulness and scalability

January 2001© Programming Technology Lab 6 General Setup declarative representation declarative reasoning engine (for software evolution) high-level descriptions and transformations (e.g. design patterns and refactoring) implementation (source code repository) formal definitions and properties representational mapping

January 2001© Programming Technology Lab 7 Presentation overview  Bottom-up approach  Graph rewriting as a formal foundation for software evolution  Based on the idea of reuse contracts  Top-down approach  Support evolution of design patterns  Use declarative metaprogramming as a technique  Future Research Intrests  uniform use of graph rewriting or declarative programming  co-evolution support ...

January 2001© Programming Technology Lab 8 Bottom-up approach: Reuse Contracts Use graph rewriting to provide a formal foundation for software evolution based on reuse contracts

January 2001© Programming Technology Lab 9 Reuse Contracts Overview 1.Benefits of reuse contracts (illustrated with a simple example) 1.Document reuse and evolution 2.Deal with upgrade problems 3.Provide support for software merging 4.Provide support for framework refactoring 2.Generalising reuse contracts (using formalism of graph rewriting) 1.Apply reuse contracts to different domains  analysis, architecture, design, implementation  Scale up reuse contracts to high-level transformations

January 2001© Programming Technology Lab Documenting Reuse DesktopFolder position contents move: add: addMany: SizedFolder add: item size reuse Extension(size, attribute) Refinement(add, size, updates) size =+ item.size

January 2001© Programming Technology Lab 11 DesktopFolder position contents move: add: addMany: 1. Documenting Evolution evolution Coarsening(addMany, add, invokes) DesktopFolder position contents move: add: addMany:

January 2001© Programming Technology Lab Dealing with Upgrade Problems DesktopFolder position contents move: add: addMany: evolution DesktopFolder position contents move: add: addMany: reuse size not updated when adding many items SizedFolder add: item size size =+ item.size SizedFolder add: item size size =+ item.size

January 2001© Programming Technology Lab Dealing with Upgrade Problems DesktopFolder position contents move: add: addMany: evolution DesktopFolder position contents move: add: addMany: SizedFolder add: size reuse Coarsening(addMany, add, invokes) Extension(size, attribute) Refinement(add, size, updates) inconsistent operation conflict

January 2001© Programming Technology Lab 14 Conflict Table 2. Dealing with Upgrade Problems extension refinement coarsening extensionrefinementcoarsening no conflicts inconsistent operations interface conflicts operation capture, unanticipated recursion operation capture, inconsistent operations no conflicts operation capture, inconsistent operations  extension/cancellation: adding/removing an operation or attribute  refinement/coarsening: adding/removing invocation or attribute access  abstraction/concretisation: making an operation abstract/concrete

January 2001© Programming Technology Lab Support for Software Merging DesktopFolder position contents move: add: addMany: evolution DesktopFolder v1 contents add: addMany: evolution Extension(position, attribute) Extension(move:, method) Coarsening(addMany, add, invokes) Extension(size, attribute) Refinement(add, size, updates) inconsistent operation conflict DesktopFolder v2a contents size add: addMany:

January 2001© Programming Technology Lab Support for FW Refactoring Refactoring conflict ! Bank Account Loan handles Company SplitClass (Bank,[Bank,Agency]) Agency Account Loan handles BankCompany represents AddClass(Safe) AddAssociation(Bank,Safe) Bank Account Loan handles Company Safe  Need for higher-level evolution transformations  Need for user-defined conflict resolution

January 2001© Programming Technology Lab 17 Reuse Contract Formalism  Domain-independent formalism  Independent of the target language  Independent of the phase in the life-cycle  Lightweight formalism to facilitate tool support  Represent software artifacts by graphs  Represent software evolution by graph rewriting  Formal characterisation of evolution conflicts  Prototype tool implemented in Prolog

January 2001© Programming Technology Lab 18 Graphs G Triangle «class» Circle «class» «inherits» intersects «operation» «assoc» center radius «attribute» «aggregation» vertices {3} Point «class»Shape «class» area «operation» circumference «operation» x «attribute» distanceTo «operation» y «attribute» «inherits»  Internal Graph Representation +intersects(c: Circle) -radius Circle +distanceTo(p: Point) -x -y Point Triangle +area() +circumference() Shape center vertices3  Example: UML class diagram  Node types:  «class»  «attribute»  «operation»  «interface»  Edge types:  «assoc»  «aggregation»  «inherits»  «implements»

January 2001© Programming Technology Lab 19 Type Graph v e node type edge type implements nested operation attribute interface class assoc, aggregation, inherits inherits updates, accesses invokes nested  Used to specify domain-specific constraints Specify extra constraints declaratively

January 2001© Programming Technology Lab 20 P m R area «operation» radius «attribute» «accesses» G Circle «class» area «operation» «accesses» circumference «operation» radius «attribute» H Circle «class» area «operation» «accesses» circumference «operation» radius «attribute» «accesses» L area «operation» radius «attribute» Graph Rewriting  Used to specify software evolution

January 2001© Programming Technology Lab 21  Use restricted set of graph productions  AddNode  DropNode  AddEdge  DropEdge  RetypeNode  RetypeEdge  RelabelNode  RelabelEdge Primitive Graph Productions AddEdge (area,radius,«accesses») R area «operation» radius «attribute» «accesses» L area «operation» radius «attribute» DropNode (Triangle.area,«operation») R Triangle L «operation» area Triangle

January 2001© Programming Technology Lab 22 Primitive Graph Productions evolution reuse DropEdge(addMany, add, invokes) AddNode(size, attribute) AddEdge(add, size, updates) inconsistent operation conflict SizedFolder «class» size «attribute» add: «operation» DesktopFolder «class» position «attribute» contents «attribute» move: «operation» add: «operation» addMany: «operation» DesktopFolder «class» position «attribute» contents «attribute» move: «operation» add: «operation» addMany: «operation» «invokes» «updates»

January 2001© Programming Technology Lab 23 Syntactic Conflicts P1P1 P 2 = DropNode(area,«operation») P 1 = AddEdge(area,radius,«uses») P2P2 P1P1 P2P2 Undefined source conflict Syntactic conflict if P 1 and P 2 are not parallel independent G Circle «class» area «operation» perimeter «operation» radius «attribute» «uses» > G2G2 Circle «class» perimeter «operation» radius «attribute» «uses» G1G1 Circle «class» area «operation» perimeter «operation» radius «attribute» «uses»

January 2001© Programming Technology Lab 24 Syntactic Conflicts  Use a syntactic conflict table  Detect syntactic merge conflicts  in terms of transformation preconditions  compare breaches of application preconditions  Advantages general  does not rely on predefined graph produtions scalable  can be used directly for composite or domain-specific graph productions

January 2001© Programming Technology Lab 25 Syntactic Conflicts -i+isource (e,v) target (e,v) label (i,L) type (i,T) -source (e,v) -target (e,v) -j  if i=j  if j=e  if j=v  if j=e  if j=v  if i=j  if j=v +j C1 if i=j C2 if e=j  if v=j C3 if e=j  if v=j C4 if i=jC5 if i=jC6 if j=v C7 if j=e C8 if j=v C9 if j=e source (f,w) C10 if (e,v)=(f,w)  if v=w  if (e,v)=(f,w)  target (f,w) C11 if (e,v)=(f,w)  if v=w  if (e,v)=(f,w) label (j,L2) C12 if i=j  type (j,T2) C13 if i=j  -source (f,w) C14 if e=f and v  w  -target (f,w) C15 if e=f and v  w

January 2001© Programming Technology Lab 26  Based on the formal notion of  pushouts and pullbacks  Fine-grained conflict characterisation  By detecting occurrence of graph patterns in result graph Semantic Conflicts addManyaddsize « updates » « invokes » {added during reuse} {removed during evolution} inconsistent method conflict « class » ? « class » ? « inherits » {added by developer 2} {added by developer 1} cyclic inheritance

January 2001© Programming Technology Lab 27 Structural Conflicts Refactoring conflict ! Bank Account Loan handles Company SplitClass (Bank,[Bank,Agency]) Agency Account Loan handles BankCompany represents AddClass(Safe) AddAssociation(Bank,Safe) Bank Account Loan handles Company Safe More difficult to detect in a general way  Customise conflict table with user-defined and domain-specific conflict resolution rules

January 2001© Programming Technology Lab 28 Using dependencies  Dependencies can be used to address scalability 1.Defining “atomic” composite productions from a sequence of productions 2.Reordering productions in a sequence 3.Removing redundancy in a production sequence 4.Factoring out commonalities from parallel production sequences 5.Parallellising subsequences

January 2001© Programming Technology Lab Composite productions 1. Define dependencies between productions  based on relations between pre/postconditions and assertions 2. Calculate pre/postconditions of composite in terms of this information

January 2001© Programming Technology Lab Factoring out commonalities  Find commonalities in two parallel sequences, and factor out  facilitates merging and conflict detection

January 2001© Programming Technology Lab 31 Validation of Reuse Contracts  Industrial case  One base release line  many customisations for  customer applications  Collaborative software development  parallel changes to base release and customisations  Provide support for  customisation, refactoring, upgrading, consolidation x NDR DR WDR 0.1 VTM TV2 WDR 1.0 WDR 2.0

January 2001© Programming Technology Lab 32 Top-down approach: Declarative metaprogramming  Goal: use declarative reasoning as a technique for managing evolution  Managing refactoring  Managing design patterns  Managing co-evolution ...

January 2001© Programming Technology Lab 33 What is DMP?  combines declarative meta language with object-oriented base language  e.g. Prolog (SOUL) and Smalltalk (Squeak)  explicit symbiosis between meta language and base language  representational mapping: base-level programs can be manipulated as terms, facts or rules in the meta level  allows declarative reasoning over, and manipulation of, object-oriented programs

January 2001© Programming Technology Lab 34 Current Focus  Use DMP for supporting 1.Evolution conflicts (detection and resolution)  use DMP instead of graph rewriting to support reuse contracts  (High-level) refactoring transformations  Evolution of design patterns  Design pattern variants  Design pattern detection  Design pattern generation  Design pattern evolution  Co-evolution between design and implementation

January 2001© Programming Technology Lab 35 Evolving Design Patterns: Dealing with variants  Abstract Factory variant 1  uses factory methods

January 2001© Programming Technology Lab 36 Evolving Design Patterns: Dealing with variants  Abstract Factory variant 2  uses prototype and singleton pattern

January 2001© Programming Technology Lab 37 Evolving Design Patterns: Dealing with variants  Declarative specification of Abstract Factory common part: patternInstance(AF1,abstractFactory). role(AF1,abstractFactory,Look). role(AF1,concreteFactory,MSLook). role(AF1,concreteFactory,MacLook). role(AF1,genericProduct,Widget). role(AF1,abstractProduct,Window). role(AF1,abstractProduct,Button). role(AF1,concreteProduct,[MSWindow,Window]). role(AF1,concreteProduct,[MSButton,Button]). role(AF1,concreteProduct,[MacWindow,Window]). role(AF1,concreteProduct,[MacButton,Button]). role(AF1,absRelation,[Look,Window]). role(AF1,absRelation,[Look,Button]). role(AF1,conRelation,[MSLook,MSWindow]). role(AF1,conRelation,[MacLook,MacWindow]). role(AF1,conRelation,[MSLook,MSButton]). role(AF1,conRelation,[MacLook,MacButton]).

January 2001© Programming Technology Lab 38 Evolving Design Patterns: Dealing with variants  Declarative specification of Abstract Factory variant 1 pattern(absFacVariant1,abstractFactory, [abstractFactoryMethod,concreteFactoryMethod]). patternInstance(AF2,absFacVariant1). role(AF2,abstractFactoryMethod,[newWindow,Look]). role(AF2,abstractFactoryMethod,[newButton,Look]). role(AF2,concreteFactoryMethod,[newWindow,MSLook]). role(AF2,concreteFactoryMethod,[newWindow,MacLook]). role(AF2,concreteFactoryMethod,[newButton,MSLook]). role(AF2,concreteFactoryMethod,[newButton,MacLook]).

January 2001© Programming Technology Lab 39 Evolving Design Patterns: Dealing with variants  Declarative specification of Abstract Factory variant 2 pattern(absFacVariant2,abstractFactory, [productDictionary,addProductMethod,productCreationMethod]). patternInstance(AF2,absFacVariant2). role(AF2,productDictionary,[products,Look]). role(AF2,addProductMethod,[addProduct:named:,Look]). role(AF2,productCreationMethod,[create:,Look]).

January 2001© Programming Technology Lab 40 Evolving Design Patterns: Dealing with variants  Declarative constraints for Abstract Factory variant 1 rule patternConstraint(?I,abstractFactory) if  ! ?AF such that role(?I,abstractFactory,?AF) % there must be exactly one abstract factory rule patternConstraint(?I,abstractFactory) if  ?AF such that role(?I,abstractFactory,?AF):  ?CF such that role(?I,concreteFactory,?CF): hierarchy(?AF,?CF) % all concrete factories must be descendants of the abstract factory  ?C such that concreteSubclass(?AF,?C): role(?I,concreteFactory,?C) % all concrete subclasses of the abstract factory must be concrete factories...

January 2001© Programming Technology Lab 41 Evolving Design Patterns: Evolution transformations  Abstract Factory evolution 1: addConcreteFactory(LinuxLook)

January 2001© Programming Technology Lab 42 Evolving Design Patterns: Evolution transformations rule addConcreteFactory(abstractFactory,?I,[?CF]) if addRole(?I,concreteFactory,?CF),  ?AP such that role(?I,abstractProduct,?AP): input(' Name of concrete ?AP created by concrete ?CF factory ',[?AP,?CF],?CP), addRole(?I,concreteProduct,[?CP,?AP]), addRole(?I,concreteRelation,[?CF,?CP]),  ?Method such that role(?I,abstractFactoryMethod,[?Method,_]): addRole(?I,concreteFactoryMethod,[?Method,?CF]) query addConcreteFactory(abstractFactory,AF,[LinuxLook]) User input: Name of concrete Window created by concrete LinuxLook factory: LinuxWindow Name of concrete Button created by concrete LinuxLook factory: LinuxButton

January 2001© Programming Technology Lab 43 Evolving Design Patterns: Evolution transformations  Abstract Factory evolution 2: addAbstractProduct(Menu)

January 2001© Programming Technology Lab 44 Evolving Design Patterns: Evolution conflicts  Straighforward merging of both evolved versions does not work:  the Menu class does not contain a LinuxMenu subclass  the LinuxLook class does not contain a newMenu factory method  Solution:  specify the design pattern transformations declaratively  apply these transformations one after the other

January 2001© Programming Technology Lab 45 Evolving Design Patterns: Evolution conflicts End result after having applied both transformations:

January 2001© Programming Technology Lab 46 Evolving Design Patterns: Evolution conflicts  Composite design pattern version 1:  composite method generate with internal action and postorder traversal

January 2001© Programming Technology Lab 47 Evolving Design Patterns: Evolution conflicts  Composite version 2a:  generate action is externalised

January 2001© Programming Technology Lab 48 Evolving Design Patterns: Evolution conflicts  Composite version 2b:  composite method serialise is added

January 2001© Programming Technology Lab 49 Evolving Design Patterns: Evolution conflicts  Merging version 2a and 2b gives a problem:  serialise action must be externalised first

January 2001© Programming Technology Lab 50 Evolving Design Patterns: Evolution conflicts  Visitor design pattern version 1

January 2001© Programming Technology Lab 51 Evolving Design Patterns: Evolution conflicts  Two parallel evolutions:  Add a TypeChecker visitor  Factor out the traversal algorithm from the visitor using a Strategy pattern  Behavioural conflict  The parallel evolutions cannot be merged correctly:  For type checking, accept in CompoundExpr needs to be overwritten in a subclass Division  For the refactoring, accept in CompoundExpr must be reimplemented to take into account the traversal strategy  Solution: Use conflict table approach a la reuse contracts?

January 2001© Programming Technology Lab 52 Evolving Design Patterns: Evolution conflicts  Merging both parallel evolutions gives a behavioural conflict

January 2001© Programming Technology Lab Co-Evolution  Underlying idea  Keep representation of same software artifact at different levels of abstraction (e.g. design and implementation) synchronised during evolution refinement abstract layer concrete layer consistent??? evolution

January 2001© Programming Technology Lab 54 WHAT NEXT ???  Possible ways of cooperation?  Concrete experiments?  Applying FAMOOS tools in the context of software evolution?  Applying reuse contract ideas in FAMOOS context?