More Flexible Software By Favoring Implicit Calls and Implicit Communication by Karl Lieberherr Joint work with Bryan Chadwick, Ahmed Abdelmeged and Therapon Skotiniotis 2/23/2019 PPL

What we want well-accepted Data Abstraction Principle: the implementation of objects can be changed without affecting clients provided the interface holds. Adaptive Programming Principle: the interface of objects can be changed without affecting clients provided adaptive constraints hold. 2/23/2019 PPL

can abstract traversal code non-cyclic objects explicit and implicit cyclic objects cause problems 2/23/2019 PPL

Abstraction boundaries M consists of (interfaceM, implementationM) so that interfaceM(implementationM). implementationM can be changed to implementationM’ provided interfaceM(implementationM’). M2 uses M1. 2/23/2019 PPL

Abstraction boundaries M consists of (faceM, mentM) so that faceM(mentM). mentM can be changed to mentM’ provided faceM(mentM’). (faceM2,mentM2) uses (faceM1,mentM1). mentM2 relies on faceM1 which changes to faceM1’. Implementation of M2 defines constraints on M1. 2/23/2019 PPL

Module Consists of Interface and Implementation. Implementation imports other modules. AdaptiveConstraints: Constraints on imported modules. 2/23/2019 PPL

Module = Interface + Implementation uses module M2 module M1 faceM2 mentM2 faceM1 mentM1 uses faceM1 mentM1’ faceM1’ mentM12 faceM1 mentM1’’ faceM1’’ mentM13 change implementation change interface 2/23/2019 PPL

Module = Interface + Implementation + AdaptiveConstraints uses (imports) module M2 module M1 faceM2 mentM2 AdapConM1 faceM1 mentM1 uses (imports) faceM1 mentM1’ module M1 faceM1’ mentM12 faceM1 mentM1’’ faceM1’’ mentM13 change implementation change interface AdapConM1(M1) 2/23/2019 PPL

What Hinders Creation of Flexible Software Rigidly following rules like: Follow the structure, follow the grammar. Actively call traversal methods (explicit traversal problem). Also leads to manual passing of arguments (explicit argument problem). 2/23/2019 PPL

Alternative 1: AP-F Think of computation as moving or pushing up information in an object (combine methods). Allow transformation at each node (apply methods). Send information down as needed (update methods). Default behavior: copy object. 2/23/2019 PPL

Alternative 2: AP-P Think of computation as traversing the object and collecting information in a visitor object. Use both regular visitor variables as well as interposition variables. Interposition variables facilitate implicit communication. Default behavior: traverse object. 2/23/2019 PPL

AP-F implemented in Java by DemeterF Examples plan apply only apply and combine apply and combine and update 2/23/2019 PPL

Type Unifying: information flow X update(A a, X x) T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) X update(B b, X x) B T apply(T t) T combine(C c, T t1) C D T apply(D d, X x) T apply(E e, X x) E 2/23/2019 PPL

Type Unifying: information flow no update T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) B T apply(T t) T combine(C c, T t1) C D T apply(D d) T apply(E e) E 2/23/2019 PPL

Type Unifying: information flow no update T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) B T apply(T t) T combine(C c, T t1) C D* T apply(D d) T apply(E e) E* 2/23/2019 PPL

Type Unifying: information flow no update T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) B T apply(T t) T combine(C c, T t1) C D T apply(D d) T apply(E e) E 2/23/2019 PPL

Type Unifying: information flow no update T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) B T apply(T t) T combine(C c, T t1) C D T apply(D d) T apply(E e) E 2/23/2019 PPL

Type Unifying: information flow no update T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) B T apply(T t) T combine(C c, T t1) C D T apply(D d) T apply(E e) E 2/23/2019 PPL

Type Unifying: information flow no update T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) B T apply(T t) T combine(C c, T t1) C D T apply(D d) T apply(E e) E 2/23/2019 PPL

Type Unifying: information flow no update T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) B T apply(T t) T combine(C c, T t1) C D T apply(D d) T apply(E e) E 2/23/2019 PPL

Type Unifying: information flow combine(a,apply(combine(b,apply(combine(c,apply(e,x))), apply(d,x)))) Type Unifying: information flow X update(A a, X x) T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) X update(B b, X x) B T apply(T t) T combine(C c, T t1) C D T apply(D d, X x) T apply(E e, X x) E 2/23/2019 PPL

Type Preserving: information flow X update(A a, X x) A combine(A a, B b) A object graph B apply(B b) B combine(B b, C c, D d) X update(B b, X x) B C apply(C c) C combine(C c, E e) C D D apply(D d, X x) E apply(E e, X x) E 2/23/2019 PPL

Type Preserving: information flow combine(a,apply(combine(b,apply(combine(c,apply(e,x))), apply(d,x)))) Type Preserving: information flow X update(A a, X x) A combine(A a, B b) A object graph B apply(B b) B combine(B b, C c, D d) X update(B b, X x) B C apply(C c) C combine(C c, E e) C D D apply(D d, X x) E apply(E e, X x) E 2/23/2019 PPL

semantics: apply-combine expression Tree = <n> Node [<left> Tree] [<right> Tree]. Node = [IdentityApply] <o> Object. apply(combine(this, apply(combine(left,…)), apply(combine(right,…)) )) Translate tree object into apply-combine expression. 2/23/2019 PPL

Traversal semantics 2/23/2019 PPL

Default Transformer copy object after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

Parameterize Default Transformer PathSpec apply(J j) { return new Complement(j); } after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

Parameterize Default Transformer PathSpec combine(J j, Boolean fn, Boolean sn) { return fn && sn; } PathSpec combine(Object j, Boolean fn, Boolean sn) { return fn && sn; } after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

Simple application Program transformation Old E : Num | Var | Op | Call … Op : Plus | Equals. Equals = “=“. New E : … | Bool. Bool : True | False. class BoolTrans extends IDf { static E newtrue = Call.parse(“(= 1 1)”), static E newfalse= Call.parse(“(= 1 0) “); E apply(True t) {return newtrue; } E apply(False t) {return newfalse; } } apply for transformation of result returned by builder 2/23/2019 PPL

de Bruijn indices Old New Var : Sym. Sym = Ident. Var : Sym | Addr. for later de Bruijn indices Old Var : Sym. Sym = Ident. New Var : Sym | Addr. Addr = Integer. class AddrTrans extends IDf { Var apply(Var var, SymList senv) { return new Addr(senv.lookup(var));} } class SymExtender extends IDa { SymList update(Lambda l, SymList senv) { return senv.push(l.formals); } 2/23/2019 PPL

The default Builder for PathSpec after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

well-formed movie show how the default builder is modified to combine Boolean objects. 2/23/2019 PPL

The default Builder for PathSpec well-formed specialization 1 #t: true #f: false after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J #t :J 5 2 :S :S :S :S #t 7 #t 8 #t 3 4 #t 2/23/2019 PPL

The default Builder for PathSpec well-formed specialization 2 #t: true #f: false after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J #t :J #t 5 2 :S :S :S :S #t 7 #t 8 #t 3 4 #t 2/23/2019 PPL

The default Builder for PathSpec well-formed specialization 3 #t: true #f: false after blue arrow copy is built (like after) #t 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S #t 7 #t 8 #t 3 4 #t 2/23/2019 PPL

The default Builder for PathSpec well-formed specialization 4 #t: true #f: false after blue arrow copy is built (like after) #t 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. #t 6 :J :J 5 2 :S :S :S :S 3 #t 7 #t 8 #t 4 #t 2/23/2019 PPL

The default Builder for PathSpec well-formed specialization 5 #t: true #f: false after blue arrow copy is built (like after) #t 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. #t 6 :J #t :J 5 2 :S :S :S :S 3 #t 7 #t 8 #t 4 #t 2/23/2019 PPL

The default Builder for PathSpec well-formed specialization 6 #t: true #f: false after blue arrow copy is built (like after) #t 10 #f :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 3 #t 7 #t 8 #t 4 #t 2/23/2019 PPL

The default Builder for PathSpec well-formed specialization 7 #t: true #f: false after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S #t 7 #t 8 #t 3 4 #t 2/23/2019 PPL

The default Builder for PathSpec well-formed specialization #t: true #f: false after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S #t 7 #t 8 #t 3 4 #t 2/23/2019 PPL

The default Builder for PathSpec NOT_JOIN_MERGE specialization after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

The default Builder for PathSpec NOT_JOIN_MERGE specialization means a copy after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

The default Builder for PathSpec NOT_JOIN_MERGE specialization after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

The default Builder for PathSpec NOT_JOIN_MERGE specialization after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

The default Builder for PathSpec NOT_JOIN_MERGE specialization after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

The default Builder for PathSpec NOT_JOIN_MERGE specialization after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

The default Builder for PathSpec NOT_JOIN_MERGE specialization after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

The default Builder for PathSpec NOT_JOIN_MERGE specialization after blue arrow copy is built (like after) 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

The default Builder for PathSpec NOT_JOIN_MERGE specialization insert NOT after blue arrow copy is built (like after) :N 10 :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

The default Builder for PathSpec NOT_JOIN_MERGE specialization insert NOT :M after blue arrow copy is built (like after) :N 10 :N :M 1 9 Count only upon first visit (red) and upon final visit (blue). For leaf nodes, count only in red. 6 :J :J 5 2 :S :S :S :S 7 8 3 4 2/23/2019 PPL

Illustration of combine for capacity constraint violation 19 :C (w1+w2+w3+w4,2) :Cons (w1+w2+w3+w4,1) :E (w4,0) 1 20 2 18a 3 18 3a 13a :Cons (w1+w2+w3,1) after blue arrow combine is active (like after) 17a :C (w2+w3,1) 4 12a 17 :Cons (w1,0) 5 13 14 12 16a :Cons (w2+w3,0) 15a 6 :E (w1,0) :Empty (0,0) 11a 7a 15 16 11 :E (w3,0) :Cons (w2,0) 8 both containers (C) violate capacity constraints 7 10a 9a :E (w2,0) :Empty (0,0) 9 2/23/2019 PPL 10

Illustration of combine for capacity constraint violation (w1+w2+w3+w4,2) :Cons (w1+w2+w3+w4,1) :E (w4,0) :Cons (w1+w2+w3,1) :C (w2+w3,1) :Cons (w1,0) :Cons (w2+w3,0) :E (w1,0) :Empty (0,0) :E (w3,0) :Cons (w2,0) both containers (C) violate capacity constraints :E (w2,0) :Empty (0,0) 2/23/2019 PPL

Theory t[f,b](d) => d’, where d’=f(d), d is atomic On left side of => the term c(…) only indicates a compound object. Theory f = apply b = combine t[f,b](d) => d’, where d’=f(d), d is atomic t[f,b](c(d0, … ,dn)) => f(b(c(d0, … ,dn), d’0, … ,d’n)), where d’i = t[f,b](di) Default functions: id[f](d) => d id[b](c(d0, … ,dn), d’0, … ,d’n) => c(d0, … ,dn) b[c](c(d0, … ,dn), d’0, … ,d’n) => c(d’0, … ,d’n) 2/23/2019 PPL

Theory f is a polymorphic function that takes a single argument and returns a result. b is a function object that is responsible for reconstruction of data types. 2/23/2019 PPL

Traversal Component Approach implemented in DemeterF Traversal with 3 components: Builder (combine), Transformer (apply), Augmentor (update) 2/23/2019 PPL

Augmentors / update methods so far: we covered combine and apply methods combine: to combine information up the object apply: to transform before the information is sent up. up refers to the traversal: when a traversal has finished visiting an object, it goes up. add: update, to send information down the object if it's not used/needed, it does not need to be mentioned, since the traversal will do the passing around. 2/23/2019 PPL

motivating example: Address computation Var : Sym | Addr. Addr = Integer. class AddrTrans extends IDf { Var apply(Var var, SymList senv) { return new Addr(senv.lookup(var)); } } class SymExtender extends IDa { SymList update(Lambda la, Symlist senv){ return senv.push(la.formals); } } 2/23/2019 PPL

Optional argument The argument passed *into* the traversal (by the programmer) is available everywhere (in every apply/combine/update function) but it may be needed only in some objects. In the typechecker case is only needed when looking up a variable use, or modifying the type-environment with a binding.   2/23/2019 PPL

Like a let for subtraversals An update argument can be viewed (almost) as a 'let' for sub-traversals: before traversing sub-terms we do a recalculation of the traversal argument. Update is called before traversing sub terms, and the modified value is only available for sub terms. 2/23/2019 PPL

Structure-shy passing of argument If you look at the Scheme code you would write to traverse the same structure; you would pass along an argument to the functions, all the way through recursive calls until it is needed. At a lambda the recursive call on the body would look something like:   (type-Lambda l tenv)     (cases Exp l        (Lambda (arg body)          (type-Exp body (tenv-extend tenv arg)))        (Call (op arglist)           (let ((ret (type-Exp op tenv)) ...)        (... other cases...))) 2/23/2019 PPL

Default: passing it along: no code needed with DemeterF Note that we don't change the traversal argument in most cases, only when binding an argument to a type. Usually we just keep passing it along throughout the traversal functions. The augmentor/update methods encapsulate (only) the changes of this argument, so we simply write: TEnv update(Lambda l, TEnv te) {return te.extend(l.formal);} The traversal takes care of the default case when we don't need to change the argument. 2/23/2019 PPL

All arguments are optional All arguments are now optional... thus the most general method is:     Object combine(){ return ...; } Since it is applicable in all cases (all arguments optional, including the 'traversal argument'). (the traversal argument is the one updated by update methods. There is only one traversal argument) 2/23/2019 PPL

AST illustration Call     |--- Lambda     |      |--- Arg     |      |     |--- Type     |      |     |     |--- int     |      |     |     |      |     |--- Sym     |      |           |--- 'a'     |      |     |      |--- Call     |            |--- Plus     |            |--- Sym     |            |     |--- 'a'     |            |     |            |--- Num     |                  |--- 2     |---  Num            |--- 4 Update is called at each label and the Object returned is then available to all terms 'connected' and to the right of that label. 2/23/2019 PPL

Technical Details Methods you want to call determine from where you inherit: ID (all), IDa (update), IDb (combine), IDba (combine, update), IDf (apply), IDfa (apply, update), IDfb(apply, combine). 2/23/2019 PPL

Motivation Showing a full scheme type-checking function, and highlighting the points where the type-environment is passed, used, and extended.  The number of cases where it is just passed along motivates the want/need to put the modifications in one place, and being able to ignore the argument when it's not really needed. 2/23/2019 PPL

Traversal a function: 2/23/2019 PPL

What is shy code? Shy code shouldn’t reveal too much of itself and shouldn’t be too nosy into others’ affairs. Law of Demeter: only talk to your friends 2/23/2019 PPL