Karl Lieberherr CCIS/PRL Northeastern University Correlations in Computer Science: an approach based on Noise Elimination Karl Lieberherr CCIS/PRL Northeastern University 11/28/2018 Correlations

Technique for building a correlation paradigm Noise Elimination Requirements-level techniques to separate crosscutting concerns: reductions Specker Derivative Game (SDG) Programming-level techniques to separate crosscutting concerns: AP-F/ DemeterF 11/28/2018 Correlations

Connections to Quantum Computing Phase transitions are important to play the game well. SDG phase transitions separate P from NP. Entanglement is connected to phase transitions. SDG deals with hiding secrets. Entanglement is used to hide secrets. 11/28/2018 Correlations

Kochen-Specker Theorem / Specker Derivative Game It is the same Ernst Specker. The Kochen-Specker theorem is an important, but subtle, topic in the foundations of quantum mechanics. The theorem provides a powerful argument against the possibility of interpreting QM in terms of hidden variables. 11/28/2018 Correlations

Outline Specker Derivative Game (SDG) history top-down derivatives, raw materials, finished products example, bottom up Risk analysis for a derivative Problem reductions – noise elimination SDG(MAX-SAT): risk analysis using polynomials 11/28/2018 Correlations

History Around 1975: I was working on non-chronological backtracking for MAX-SAT for my PhD at ETH Zurich with Erwin Engeler. With Ernst Specker (co-advisor) I analyzed MAX-SAT which lead to the Golden Ratio Result: joint FOCS 79 and JACM 1981 paper. Ideas applicable to MAX-CSP. J. Algorithms 1982. 2006: sabbatical at Novartis reactivated my interest in MAX-CSP. 2007: Turned Golden Ratio Result into a game SDG(Max) parameterized by a maximization problem Max. 2007/2008: Taught SDG to students who had a lot of fun trying to produce a winning robot (class size 30). 11/28/2018 Correlations

The SDG Game: Two high level views Financial: Implement trading robots that survive in an artificial derivative market through offering derivatives, and buying and processing derivatives produced by other trading robots. Biological: Implement organisms that survive in an artificial world through offering outsourced services, and consuming and processing outsourced services produced by other organisms. 11/28/2018 Correlations

Buyer makes profit of 0.8 - 0.7 = 0.1 (p2,0.9,s) Buyer b Seller s O 0.8 O 0.8 R R (p1,0.7,s) sold Derivatives R satisfies p1 0.7 11/28/2018 Correlations

Derivative: (pred, p, s) bought by b Max is an NP-hard combinatorial maximization problem with objective function range [0,1]. Buyer b buys derivative at price p. Seller s delivers raw material R (instance of Max) satisfying predicate pred. Raw material R is finished by buyer with outcome O of quality q and seller pays q to buyer. Buyer only buys if she thinks q > p. Uncertainty for buyer: which raw material R will I get? Only know the predicate! What is the quality of the solution of Max I can achieve for R? 11/28/2018 Correlations

Survive in an artificial market Each robot contains a: Derivative buying agent Derivative offering agent Raw material production agent Finished product agent (solves Max) Game is interesting even if robots are far from perfect. Focus today: how to play the game perfectly (never losing) 11/28/2018 Correlations

SDG Example derivative based on simplified 3-SAT: S3-SAT = all clauses of length 3, except that all clauses of length 1 contain one positive literal. derivative(S3-SAT, 0.70) = derivative(CNF{(3,0),(3,1),(3,2),(3,3),(2,0),(2,1),(2,2),(1,1)}, 0.70) 4 variables maximum would you like to buy it? you will get two rights you will receive a CNF R of the given type. if you can satisfy the fraction q of clauses in R, I will pay back q to you. CNF = SEQUENCE of clauses 11/28/2018 Correlations

SDG Example Noise Elimination derivative(CNF{(3,0),(3,1),(3,2),(3,3),(2,0),(2,1),(2,2),(1,1)}, p) “=“ derivative(CNF{(2,0),(1,1)}, p) If the seller acts rationally, only clause types {(2,0),(1,1)} will be used in the raw material. CNF = SEQUENCE of clauses 11/28/2018 Correlations

CNF{(2,0),(1,1)} raw materials 2: b 2: c 1: !a !b 1: !b !c 2: a 2: b 3: c 1: d 3: !a !b 9: !a !c 7: !b !c 1: !a !d 6: !b !d 6: !c !d 2: a 3: b 1: c 8: !a !b 6: !b !c Find best satisfaction ratio 1: a 1: b 8: !a !b 11/28/2018 Correlations

CNF{(2,0),(1,1)} raw materials, finished products 2: b 2: c 1: !a !b 1: !b !c 2: a 2: b 3: c 1: d 3: !a !b 9: !a !c 7: !b !c 1: !a !d 6: !b !d 6: !c !d 6/8=0.75 2: a 3: b 1: c 8: !a !b 6: !b !c 17/20=0.85 price of 0.7 seems fair!? 1: a 1: b 8: !a !b 9/10=0.9 35/40 = 0.875 11/28/2018 Correlations

Oops!! Our analysis was not thorough enough! 2 kinds of uncertainty: worst formula? best assignment? 11/28/2018 Correlations

Playing with the weights x: a x: b x: c x: d y: !a !b y: !a !c y: !b !c y: !a !d y: !b !d y: !c !d x =1, y=1 best assignment a=1, b=0, c=0, d=0: (1+6)/10=7/10=0.7 x = 2, y=1 a=1, b=1, c=0, d=0: (4+5)/14=9/14=0.64 derivative(CNF{(2,0),(1,1)}, 0.70) LOSS: 0.06 11/28/2018 Correlations

To play well: solve min max maximum solutions instances selected by predicate (an infinite set) 0.91 0.62 0.8 0.618 0.619 minimum Analysis for one Derivative 11/28/2018 Correlations

Concerns Win the game. Seller: Buyer offering: too expensive or too cheap? raw material construction how well can I hide the best assignment? how can I keep the satisfaction ratio low? efficiency Buyer buying how well will the best assignment be hidden? how low can the satisfaction ratio be? finishing: efficiency 11/28/2018 Correlations

Concerns: SDG(Max) tpred = lim n -> ∞ min all raw materials rm of size n satisfying predicate pred max all finished products fp produced for rm q(fp) The better the Seller hides the maximum, the harder the Buyer has to work to find it. Seller approximates minimum efficiently Buyer approximates Max efficiently 11/28/2018 Correlations

To play well: solve min max maximum solutions instances selected by predicate (an infinite set) 0.91 0.62 0.8 0.62 0.618 0.619 minimum small subset of raw materials guaranteed to contain minimum of maxima Analysis for one Derivative Noise 11/28/2018 Correlations

Raw material selected all possible finished products 0.7 0.7 0.7 small subset of finished products guaranteed to contain maximum Noise 11/28/2018 Correlations

Risk analysis Life cycle of a derivative (pred,p) Two uncertainties offer risk high if I can find rm and fp with q(fp) > p buy risk high if I can find rm and fp with q(fp) < p raw material (rm) finished product (fp ,quality q(fp)) Two uncertainties raw material is not the worst (uncertainty_rm) finished product is not the best (uncertainty_fp) 11/28/2018 Correlations

To play SDG perfectly eliminate risk buy break-even price sell produce efficiently find worst-case example process efficiently achieve break-even quality 11/28/2018 Correlations

Goal: never lose with offer/buy Choose algorithms RM and FP 11/28/2018 Correlations

More Concerns for SDG(Max) Robot communication exchange language for derivatives, raw material, finished product centralized game with administrator managing store of derivatives decentralized game Winning in the market clever algorithm design 11/28/2018 Correlations

Analysis of SDG(Max) tpred = lim n -> ∞ min all raw materials rm of size n satisfying predicate pred max all finished products fp produced for rm q(fp) 11/28/2018 Correlations

Spec for RM and FP tpred = lim n -> ∞ min all raw materials rm of size n satisfying predicate pred and having property WORST(rm) max small subset of all finished products fp produced for rm q(fp) 11/28/2018 Correlations

Analysis of SDG(Max) tpred = lim n -> ∞ min all raw materials rm of size n produced by RM satisfying predicate pred max all finished products fp produced for rm by FP q(fp) 11/28/2018 Correlations

Hope Max is NP-hard SDG(Max) simplifies Max if our goal is to never lose. 11/28/2018 Correlations

SDG(MAX-SAT) Predicates using clause types. Example predicate S3-SAT = All CNFs with clauses of length at most 3 but clauses of length 1 must contain one positive literal. What is the right price p for derivative (S3-SAT, p, Specker) 11/28/2018 Correlations

SDG(MAX-SAT) Predicate space: any subset of clause types of S3-SAT: t S3-SAT = (√5 -1)/2 t {(2,0),(1,1)} = (√5 -1)/2 t {(100,50), (3,2), (2,0),(1,1)} = (√5 -1)/2 Noise for the purpose of constructing raw material. 11/28/2018 Correlations

SDG(MAX-SAT) t {(2,0),(1,1)} = t SYM{(2,0),(1,1)} = (√5 -1)/2 SYM stands for Symmetrization: Idea: if you give me a CNF with a satisfaction ratio f, I give you a symmetric CNF with a satisfaction ratio <= f. For a CNF in SYM{(2,0),(1,1)}, the MAX-SAT problem reduces to maximizing a polynomial. 11/28/2018 Correlations

Reductions Noise Elimination Abstraction: What is important to play the game well. Derivative reduction: to play well with derivative d, it is sufficient to play the game well with a simpler derivative d1 (pred(d1) => pred(d)). Raw Material Construction Reduction: to construct raw material for a derivative d, it is sufficient to construct only symmetric formulas. 11/28/2018 Correlations

Complexity theory connection Break-even prices are not only interesting for the SDG game. They also have complexity-theoretic significance: they are phase transition points separating P from NP (for “most” predicates). 11/28/2018 Correlations

General Dichotomy Theorem MAX-CSP(G,f): For each finite set G of relations there exists an algebraic number tG For f ≤ tG: MAX-CSP(G,f) has polynomial solution For f ≥ tG+ e: MAX-CSP(G,f) is NP-complete, e>0. 1 hard (solid) NP-complete polynomial solution: Use optimally biased coin. Derandomize. P-Optimal. tG = phase transition point easy (fluid) Polynomial due to Lieberherr/Specker (1979, 1982) 11/28/2018 Correlations

Other break-even prices (Lieberherr/Specker (1982)) G = {R0,R1,R2,R3}; Rj : rank 3, exactly j of 3 variables are true. tG= ¼ 11/28/2018 Correlations

Other break-even prices (Lieberherr/Specker (1982)) G(p,q) = {Rp,q = disjunctions containing at least p positive or q negative literals (p,q≥1)} Let a be the solution of (1-x)p=xq in (0,1). tG(p,q)=1-aq 11/28/2018 Correlations

Lessons learned from SDG / correlation perspective Noise elimination is essential to separate the concerns. Noise elimination leads to tractable minimization and maximization problems. Noise elimination is key to come up with needed correlations: Combination Generator, Randomized Algorithm and its Derandomized version, etc. 11/28/2018 Correlations

A model for correlations? The Socrates programming model A program in the Socrates core language consists of messages, representing abstract operations, branches, representing concrete units of behavior that implement parts of one or more operations, and fields, representing associations between data values. During the execution of a Socrates program, decision points, objects, and contexts are created and manipulated. These six entities are all first-class values. Socrates = Predicate Dispatch + … 11/28/2018 Correlations

Socrates Reference http://socrates-lang.sourceforge.net/ @INPROCEEDINGS{doug:aosd-2002, AUTHOR = "Doug Orleans", TITLE = "Incremental Programming with Extensible Decisions", BOOKTITLE = "First International Conference on Aspect-Oriented Software Developm ent", YEAR = "2002", ADDRESS = "Enschede, The Netherlands", PAGES = "", EDITOR = "Gregor Kiczales", PUBLISHER = "ACM Press”} 11/28/2018 Correlations

crosscutting concerns public class Shape implements ShapeI { protected AdjustableLocation loc; protected AdjustableDimension dim; public Shape() { loc = new AdjustableLocation(0, 0); dim = new AdjustableDimension(0, 0); } double get_x() throws RemoteException { return loc.x(); } void set_x(int x) throws RemoteException { loc.set_x(); } double get_y() throws RemoteException { return loc.y(); } void set_y(int y) throws RemoteException { loc.set_y(); } double get_width() throws RemoteException { return dim.width(); } void set_width(int w) throws RemoteException { dim.set_w(); } double get_height() throws RemoteException { return dim.height(); } void set_height(int h) throws RemoteException { dim.set_h(); } void adjustLocation() throws RemoteException { loc.adjust(); void adjustDimensions() throws RemoteException { dim.adjust(); class AdjustableLocation { protected double x_, y_; public AdjustableLocation(double x, double y) { x_ = x; y_ = y; synchronized double get_x() { return x_; } synchronized void set_x(int x) {x_ = x;} synchronized double get_y() { return y_; } synchronized void set_y(int y) {y_ = y;} synchronized void adjust() { x_ = longCalculation1(); y_ = longCalculation2(); class AdjustableDimension { protected double width_=0.0, height_=0.0; public AdjustableDimension(double h, double w) { height_ = h; width_ = w; synchronized double get_width() { return width_; } synchronized void set_w(int w) {width_ = w;} synchronized double get_height() { return height_; } synchronized void set_h(int h) {height_ = h;} width_ = longCalculation3(); height_ = longCalculation4(); interface ShapeI extends Remote { double get_x() throws RemoteException ; void set_x(int x) throws RemoteException ; double get_y() throws RemoteException ; void set_y(int y) throws RemoteException ; double get_width() throws RemoteException ; void set_width(int w) throws RemoteException ; double get_height() throws RemoteException ; void set_height(int h) throws RemoteException ; void adjustLocation() throws RemoteException ; void adjustDimensions() throws RemoteException ; Modularization of crosscutting concerns public class Shape { protected double x_= 0.0, y_= 0.0; protected double width_=0.0, height_=0.0; double get_x() { return x_(); } void set_x(int x) { x_ = x; } double get_y() { return y_(); } void set_y(int y) { y_ = y; } double get_width(){ return width_(); } void set_width(int w) { width_ = w; } double get_height(){ return height_(); } void set_height(int h) { height_ = h; } void adjustLocation() { x_ = longCalculation1(); y_ = longCalculation2(); } void adjustDimensions() { width_ = longCalculation3(); height_ = longCalculation4(); coordinator Shape { selfex adjustLocation, adjustDimensions; mutex {adjustLocation, get_x, set_x, get_y, set_y}; mutex {adjustDimensions, get_width, get_height, set_width, set_height}; portal Shape { double get_x() {} ; void set_x(int x) {}; double get_y() {}; void set_y(int y) {}; double get_width() {}; void set_width(int w) {}; double get_height() {}; void set_height(int h) {}; void adjustLocation() {}; void adjustDimensions() {}; Write this Instead of writing this © Crista Lopes 11/28/2018 Correlations

A Safe Form of Aspects: Adaptive Programming (AP) General Aspects (96) Base: any program join points pointcuts advice AP (92) Base: any traversal before / after events method signatures method bodies Connection to Theo: DemeterF was developed in the context of language interpreters and compilers. See DemeterF home page. AP is a special case of AOP. AP-F is a special case of AP. DemeterF is an implementation of AP-F (Java, C#) 11/28/2018 Correlations

Theme: noise elimination leads to better separation of concerns and better correlations. requirements: problem reductions eliminate noise irrelevant information is eliminated. Correlations: minimizing and maximizing continuous functions. programming: traversal abstractions eliminate noise irrelevant classes are not mentioned. Correlations: WhereToGo, WhatToDo. 11/28/2018 Correlations

Noise Elimination SDG (specific to SDG Game) DemeterF (very general) Only a subset of the formulas of the MAX-CSP problem is relevant to find a good raw material. Only a subset of the assignments is relevant to find a good finished product. DemeterF (very general) Only a subset of the classes is relevant to implement an object collaboration. Only a subset of the classes is relevant to specify WhereToGo WhatToDo 11/28/2018 Correlations

find all persons waiting at any bus stop on a bus route WhereToGo: Strategy first try: from BusRoute to Person busStops BusRoute BusStopList buses 0..* BusStop BusList waiting 0..* passengers Bus PersonList Person 0..* 11/28/2018 Correlations

find all persons waiting at any bus stop on a bus route WhereToGo: Strategy from BusRoute via BusStop to Person busStops BusRoute BusStopList buses 0..* BusStop BusList waiting 0..* passengers Bus PersonList Person 0..* 11/28/2018 Correlations

find all persons waiting at any bus stop on a bus route WhereToGo: Strategy Altern.: from BusRoute bypassing Bus to Person busStops BusRoute BusStopList buses 0..* BusStop BusList waiting 0..* passengers Bus PersonList Person 0..* 11/28/2018 Correlations

Robustness of Strategy find all persons waiting at any bus stop on a bus route Robustness of Strategy from BusRoute via BusStop to Person villages BusRoute BusStopList busses VillageList busStops 0..* 0..* BusStop BusList Village waiting 0..* passengers Bus PersonList Person 0..* 11/28/2018 Correlations

Filter out noise in class diagram only three out of seven classes are mentioned in traversal strategy! from BusRoute via BusStop to Person explain why strategy better replaces traversal methods for the classes BusRoute VillageList Village BusStopList BusStop PersonList Person 11/28/2018 Correlations

WhatToDo: Type-Unifying Return 1 for each Person-object. Return 0 for other leafs. Sum for lists: int up(ConsI c, int f, int r) {return f+r;} Pass for one part: int up(Object o, int p) {return p;} Processing works for many class graphs. Static type-checking protects against some bad changes to the class graph. 11/28/2018 Correlations

Genericity Works for a family of object graphs WhereToGo WhatToDo 11/28/2018 Correlations

Comparing Noise Elimination Noise elimination in SDG: have to prove theorems to show that nothing is lost and we get the correct result. Noise elimination in DemeterF: have to prove (or approximate it through testing) that the noise elimination leads to correct generalizations. 11/28/2018 Correlations

Other Benefit of AP-F If you want to take advantage of multi-core computers, it is beneficial to write programs in a style that is amenable to parallel execution. I will present such a style: AP-F and its implementation in Java and C#, called DemeterF. AP-F has other benefits: easier software variability. 11/28/2018 Correlations

Outline AP-F/DemeterF DemeterF: a traversal abstraction tool that supports structure-shy software Generalized data abstraction Intro to AP-F Intro to DemeterF 4 important classes (ID, Bc, EdgeControl, Traversal) 3 important methods (down, up, traverse) Examples Composition in DemeterF Conclusions 11/28/2018 Correlations

AP-F/DemeterF: a useful tool to process objects OOP and Functional Adaptive Programming Traversal abstraction library similar to a library supporting the visitor design pattern new: better traversal abstraction through multi-methods and traversal control 11/28/2018 Correlations

Builds on Adaptive Programming (AP): Graph-based traversal control based on expansions, separation of concerns into ClassGraph, WhereToGo, WhatToDo. Mostly the separation of functionality (WhatToDo) and traversal (WhereToGo). Functions don't need to traverse, and traversals don't mention any specific function. Functional Programming (FP): Side-effect-free programming, WhatToDo without local state. Programming with map and fold. Generalized fold. Generic functions (GF): dynamic dispatch for down and up methods. Predicate Dispatch. Socrates. PolyD. Scrap Your Boilerplate (SYB): separation into type preserving and type unifying computations. Datatype-Generic Programming (DGP): Polytypic Programming. 11/28/2018 Correlations

Recent related activity Attribute Grammars (AG): AOSD 2008 (Oxford paper). (Functional) Visitor Pattern (VP): OOPSLA 2008 (Oxford paper). Bruno Oliveira,  Jeremy Gibbons. 11/28/2018 Correlations

Important Software Topic To Which We Contribute with AP-F Generalizing the Data Abstraction Problem Through Better Abstractions for Traversals 11/28/2018 Correlations

Generalizing Data Abstraction well-accepted, since 70s Data Abstraction Principle: the implementation of objects can be changed without affecting clients provided the interface holds. Adaptive Programming (AP) Principle: the interface of objects can be changed without affecting clients provided the generalized interface holds. since 90s 11/28/2018 Correlations

Motto of AP-F Law of Demeter: Talk only to your friends. Law of AP-F: Listen only to your friends and selectively receive information from your superiors. You might not need all the information you get from your friends. You only take what you need and what the communication protocol requires. 11/28/2018 Correlations

What happens if you don’t use traversal abstraction? You write a lot of boiler plate code! Related work: Scrap Your Boilerplate (SYB) started by Ralf Laemmel and Simon Peyton-Jones. You tightly couple structure (changes frequently) and computation (more stable). That is against common sense. You always deal with the worst-case scenario of AP-F. 11/28/2018 Correlations

A Quick Intro to AP-F An AP-F program is defined by two sets of functions which adapt behavior of a predefined traversal with a multiple dispatch function. The two sets of functions: down up 11/28/2018 Correlations

crosscutting concerns public class Shape implements ShapeI { protected AdjustableLocation loc; protected AdjustableDimension dim; public Shape() { loc = new AdjustableLocation(0, 0); dim = new AdjustableDimension(0, 0); } double get_x() throws RemoteException { return loc.x(); } void set_x(int x) throws RemoteException { loc.set_x(); } double get_y() throws RemoteException { return loc.y(); } void set_y(int y) throws RemoteException { loc.set_y(); } double get_width() throws RemoteException { return dim.width(); } void set_width(int w) throws RemoteException { dim.set_w(); } double get_height() throws RemoteException { return dim.height(); } void set_height(int h) throws RemoteException { dim.set_h(); } void adjustLocation() throws RemoteException { loc.adjust(); void adjustDimensions() throws RemoteException { dim.adjust(); class AdjustableLocation { protected double x_, y_; public AdjustableLocation(double x, double y) { x_ = x; y_ = y; synchronized double get_x() { return x_; } synchronized void set_x(int x) {x_ = x;} synchronized double get_y() { return y_; } synchronized void set_y(int y) {y_ = y;} synchronized void adjust() { x_ = longCalculation1(); y_ = longCalculation2(); class AdjustableDimension { protected double width_=0.0, height_=0.0; public AdjustableDimension(double h, double w) { height_ = h; width_ = w; synchronized double get_width() { return width_; } synchronized void set_w(int w) {width_ = w;} synchronized double get_height() { return height_; } synchronized void set_h(int h) {height_ = h;} width_ = longCalculation3(); height_ = longCalculation4(); interface ShapeI extends Remote { double get_x() throws RemoteException ; void set_x(int x) throws RemoteException ; double get_y() throws RemoteException ; void set_y(int y) throws RemoteException ; double get_width() throws RemoteException ; void set_width(int w) throws RemoteException ; double get_height() throws RemoteException ; void set_height(int h) throws RemoteException ; void adjustLocation() throws RemoteException ; void adjustDimensions() throws RemoteException ; Modularization of crosscutting concerns In AP-F: concern 1: class graph concern 2: WhereToGo (down,up) concern 3: WhatToDo public class Shape { protected double x_= 0.0, y_= 0.0; protected double width_=0.0, height_=0.0; double get_x() { return x_(); } void set_x(int x) { x_ = x; } double get_y() { return y_(); } void set_y(int y) { y_ = y; } double get_width(){ return width_(); } void set_width(int w) { width_ = w; } double get_height(){ return height_(); } void set_height(int h) { height_ = h; } void adjustLocation() { x_ = longCalculation1(); y_ = longCalculation2(); } void adjustDimensions() { width_ = longCalculation3(); height_ = longCalculation4(); coordinator Shape { selfex adjustLocation, adjustDimensions; mutex {adjustLocation, get_x, set_x, get_y, set_y}; mutex {adjustDimensions, get_width, get_height, set_width, set_height}; portal Shape { double get_x() {} ; void set_x(int x) {}; double get_y() {}; void set_y(int y) {}; double get_width() {}; void set_width(int w) {}; double get_height() {}; void set_height(int h) {}; void adjustLocation() {}; void adjustDimensions() {}; Write this Instead of writing this 11/28/2018 Correlations

Motto of AP-F down A object graph Listen to C and D Receive from A B up Does NOT Receive from A C Receive from A D E Receive from A 11/28/2018 Correlations

Illustration of up 19a 19 :C (w1+w2+w3+w4,2) :Cons (w1+w2+w3+w4,1) :E 20 2 18a 3 18 3a 13a :Cons (w1+w2+w3,1) after blue arrow up is active (like after) 17a :C (w2+w3,1) 4 12a 17 :Cons (w1,0) 5 13 14 12 16a :Cons (w2+w3,0) 15a 6 :E (w1,0) :Empty (0,0) 11a 7a 15 16 11 :E (w3,0) :Cons (w2,0) 8 7 10a 9a :E (w2,0) :Empty (0,0) 9 11/28/2018 Correlations 10

DemeterF A significant class of applications can be coded modularly without tangled code and without having to write boiler plate code. DemeterF has an optional code generation tool to generate Java classes from grammars (data binding) and traversal strategies (navigation binding). 11/28/2018 Correlations

DemeterF Function Objects up methods several parameters (from your friends) default: reconstruct using (new) parameters down methods three parameters (where, which field/path, what came from above) default: identity (return third parameter) 11/28/2018 Correlations

What is new with DemeterF easy generalization of functionality thanks to multimethods: exploit similarities between up methods at different nodes. Type-checking captures missing methods. easy generalization of traversals easy parallelization 11/28/2018 Correlations

What Hinders Creation of Flexible Software Manually following rules like: Follow the structure, follow the grammar. Actively call traversal methods (explicit traversal problem). Also leads to manual passing of arguments (explicit argument problem). 11/28/2018 Correlations

Still follow the grammar? Might have to write customized methods for every node. Extreme case. Encode local information about structure in customized methods. 11/28/2018 Correlations

The 4 most important classes in DemeterF WhatToDo: ID: you take responsibility for writing sufficiently many up methods covering your objects. Type Unifying. Bc: you let the system write default up methods that rebuild the object and you override some of them. Type Preserving. WhereToGo: EdgeControl. For using ID, Bc and EdgeControl: Traversal 11/28/2018 Correlations

The 4 most important classes in DemeterF ID == id-for-trav-arguments and id-for-prims (int,String, etc.) ID inherits Bc Bc == ID + rebuild Use ID for type-unifying processing. Use Bc for type-preserving processing. Traversal(ID, EdgeControl) Traversal(WhatToDo, WhereToGo) EdgeControl which edges to avoid 11/28/2018 Correlations

The 3 most important methods in DemeterF for initiating a traversal: traverse for modifying a traversal (WhatToDo): down: method is activated when the down method of the parent is complete. up: method is activated when all subtraversals have returned. 11/28/2018 Correlations

Type-unifying: ID, Traversal up, traverse // Produces a string representation of the tree static class Str extends ID { String up(leaf l) { return ""; } String up(node n, int d, String l, String r) { return "("+d+" "+l+" "+r+")"; } public static String doStr(bst t){ return new Traversal(new Str()).<String>traverse(t); how generic? positions 2 and 3 are important 11/28/2018 Correlations

Example: Class Graph bst right left is-a is-a leaf node data int 11/28/2018 Correlations

Example: Class Graph bst middle right left is-a is-a y x leaf node is-a data node2 int data2 float 11/28/2018 Correlations

Type-preserving: Bc, Traversal up, traverse // Increments each int element and rebuilds the // resulting tree static class Incr extends Bc { int up(int i) { return i+1; } public static String doIncr(bst t){ return new Traversal(new Incr()). <bst>traverse(t); } how generic? very general 11/28/2018 Correlations

ID, Traversal, EdgeControl up, traverse // Find the minimum data element in the BST... keep going left static class Min extends ID { int up(node n, int d, leaf l) { return d; } int up(node n, int d, int l) { return l; } public static int min(bst t){ EdgeControl ec = EdgeControl.create( new Edge(node.class,"right")); ec.addBuiltIn(leaf.class); return new Traversal(new Min(), ec).<Integer>traverse(t); } 11/28/2018 Correlations

ID, Traversal down, up, traverse // Produces a list of strings representing the paths to each // leaf element in the tree static class Paths extends ID { String down(node n, node.left f, String path) { return path+".left"; } String down(node n, node.right f, String path) { return path+".right"; } List<String> up(leaf l) { return new Empty<String>(); } List<String> up(node n, int d, List<String> l, List<String> r, String p) { return l.append(r).push(" "+d+" : "+p); } public static String doPaths(bst t){ return new Traversal(new Paths()). < List<String>>traverse(t, “root”); } } 11/28/2018 Correlations

Type-unifying Tree Height, bottom up: Unifying type: (int height) fold: max + 1 class Height extends ID{ int up(node t, int d, int l, int r){ return Math.max(l,r)+1; } int up(leaf l){return 0;} } 11/28/2018 Correlations

Dispatch function of DemeterF chooses most specific method more information means more specific only list arguments up to last one that is needed 11/28/2018 Correlations

Traversal Constructors Traversal(ID ba)           Takes an ID function object Traversal(ID ba, EdgeControl eb)           In addition takes a traversal control object. Edge control may be specified using domain-specific language. 11/28/2018 Correlations

Unifying Type The unifying type which may contain many more elements than only what we need to compute. We want a type T = (U1, U2, ...) so that T carries just enough information to implement up: T up (X x, T t1, T t2, ...) at all nodes in the structure. 11/28/2018 Correlations

Type-unifying Examples Binary Search Tree Property: Given a binary tree of ints, are all nodes in the left tree smaller than the root and all nodes in the right subtree greater to or equal than the root. Consider the maximum max of the left subtree and the minimum min of the right subtree to do the max < d <= min check. Unifying type: (boolean BSTproperty, int min, int max) Base case: Empty tree: (true, +oo, -oo) fold: operation on two triples. 11/28/2018 Correlations

Check BST Property An integer tree T, with integer n at its root, is a binary search tree if (all must hold) the left and right tree of T are a BST the maximum of all nodes in the left tree of T is smaller than n the minimum of all nodes in the right tree of T is greater than or equal to n 11/28/2018 Correlations

Why do we need the last 2? 8 10 3 BST 1 9 BST not BST 11/28/2018 Correlations

Check for BST Property class Check extends ID{ Pack up(leaf l){ return new Pack(true); } Pack up(node t, int d, Pack lt, Pack rt){ return new Pack((lt.good && rt.good && (lt.max < d) && (d <= rt.min)), Math.min(lt.min,d), Math.max(rt.max,d)); } static boolean check(bst tree){ return new Traversal(new Check()) .<Pack>traverse(tree).good; }} 11/28/2018 Correlations

Pack Helper Class class Pack{ boolean good; int min,max; Pack(boolean g, int mn, int mx) { good = g; min = mn; max = mx; } Pack(boolean g){ this(g, Integer.MAX_VALUE, Integer.MIN_VALUE); } } 11/28/2018 Correlations

Example 5 (t,1,7) (t,1,3) 3 7 (t,7,7) E 1 E E (t,1,1) (t,+∞,-∞) (t/f,min,max) for entire tree. min and max are only valid if tree is BST. (t,+∞,-∞) (t,+∞,-∞) 11/28/2018 Correlations

Conclusions: A Technique for Finding Correlations Noise Elimination Requirements-level techniques to separate crosscutting concerns: reductions Specker Derivative Game (SDG) Programming-level techniques to separate crosscutting concerns: AP-F/ DemeterF Are the connections to Quantum Computing of any value? 11/28/2018 Correlations

11/28/2018 Correlations

Composition How to compose traversals? 11/28/2018 Correlations

Tree Diameter The diameter of a tree T is the largest of the following quantities: (1) diameter of T’s left subtree (2) diameter of T’s right subtree (3) the longest path between two leaves that goes through the root of T (computed from the heights of the subtrees of T) 11/28/2018 Correlations

Why do we need 1 and 2? 5 4 6 3 2 9 1 diameter 9, NOT through root 11/28/2018 Correlations

11/28/2018 Correlations

Correlations in the SDG World Consider organisms that need to survive in an artificial world through producing outsourced services, and consuming and processing outsourced services produced by other organisms. Consider simple trading robots that need to survive in an artificial derivative market through offering derivatives, and consuming and processing derivatives produced by other trading robots. 11/28/2018 Correlations

Services: Correlations in the SDG World Consider organisms that need to survive in an artificial world through offering services, and consuming and processing services produced by other organisms. Consider simple trading robots that need to survive in an artificial derivative market through offering derivatives, and consuming and processing derivatives produced by other trading robots. 11/28/2018 Correlations

(food,price,producer)) organisms need to decide whether to absorb a food based on partial information. Hopes to gain energy by absorption and processing of food. (food,price,producer)) energy gained where deducted? producing organism is nearby? seller and buyer of food meet. 11/28/2018 Correlations

correlations in SDG 11/28/2018 Correlations

Model SDG(Max) organisms: move around partial information = predicate food in organism: still connected to owner? 11/28/2018 Correlations

want to absorb a “kind” of food. inside: organism: food of the promised kind is delivered. food is digested with quality q. energy q belongs to consuming organism; deducted from providing organism. 11/28/2018 Correlations

correlations in biology: prices don’t have to come down. slightly different rules. organisms meet, explore each others food/service offers, choose and move on not a global store; distributed version in biology: “being” costs energy: need to consume and produce to generate energy; another way to make game interesting. 11/28/2018 Correlations

correlations in SDG(Max) think kind of: food, service, raw material what kind of skills make the organisms survive? buy, sell, produce, process what cuts across those 4? symmetrization, Polya’s inventor paradox problem solving techniques cut across 4 agents 11/28/2018 Correlations

buy sell produce process break-even price efficiently find worst case example process efficiently achieve break-even quality 11/28/2018 Correlations

Cutting across classes find polynomial for a maximization instance de-randomization 11/28/2018 Correlations

Cutting across problems symmetrization randomization de-randomization 11/28/2018 Correlations

A different form of crosscutting Problem solving techniques applicable in many domains. specific ways to solve problems are applicable in many sub problems of the same problem reductions and correlations 11/28/2018 Correlations

Demeter Interface Bus example: want no shortcuts, except list shortcuts. Or want one path with multiplicities allowed. 11/28/2018 Correlations

Variant SDG for CSP derivative: (predicate1 (on instances), predicate2 (on assignments and instances), price, creator) worst formula satisfying predicate1 and with assignment satisfying predicate2 we can satisfy fraction p. Example: independent set: predicate1 = relation q(x,y) = !(x = y). predicate2 = 2/3 of variables set to 1. Example: vertex cover: predicate1 = relation q(x,y) = {(1,0),(0,1),(1,1)} = x or y. predicate2 = 2/3 of variables set to 1. Example: maximum cut: predicate1 = relation q(x,y) = {(1,0).(0,1)} = x xor y. predicate2 = true. 11/28/2018 Correlations

11/28/2018 Correlations

Lefteris Rolling Stones: I can get no Satisfaction Beatles: yesterday, all you need is love. 11/28/2018 Correlations

Positive votes 15 Negative votes 7 Passes 2 Erik Ernst The notion of being an inherently cross-cutting concern does not exist. Posted by Erik Ernst, Friday, March 28, 2008 on aosd.net. It is a matter of (1) the power of the available abstraction mechanisms, and (2) the choice of organizational principles in the design of the software system under scrutiny whether or not the set of software elements associated with a given concern is cross-cutting or not. Such a set of software elements may be distributed in a cross-cutting manner in a concrete system written using one or more concrete languages. As soon as somebody manages to express the concern modularly by reshaping the architecture or creating and using new abstraction mechanisms, this concern becomes non-cross-cutting, so it was never *inherently* cross-cutting. The notion of being a *currently* cross-cutting concern does exist, though, and the really interesting issue is how we can move concrete concerns out of this sad state by means of improvements of the basic theories and technologies. ;-) 11/28/2018 Correlations

Theme: noise elimination leads to better separation of concerns and better correlations. requirements: problem reductions eliminate noise irrelevant information is eliminated. Correlations: minimizing and maximizing continuous functions. programming: traversal abstractions eliminate noise irrelevant classes are not mentioned. Correlations: WhereToGo, WhatToDo. 11/28/2018 Correlations

Example: class node // Functional Nodes public static class node extends bst { int data; bst left, right; public node(int d, bst l, bst r) { data = d; left = l; right = r; } public bst insert(int d) { if(d <= data) return new node(data, left.insert(d), right); return new node(data, left, right.insert(d)); } // Field classes... Must be public+static. public static class data { } public static class left { } public static class right { } 11/28/2018 Correlations

Difference: Coplien Abstraction (loses information) Generalization (Abstraction is here defined as the ability to perceive a concept irrespective of its concrete embodiment.) if appropriate conceptual abstraction (promoted by perceptual variability) and generalization (promoted by mathematical variability) are to occur. It seems reasonable to suggest that a child has truly learned a mathematical concept only when s(he) is able to abstract it from its physical context and generalize it to new situations. 11/28/2018 Correlations

the point of concern is that the psychological factors (abstraction and generalization) to which the perceptual and mathematical variability principles address themselves 11/28/2018 Correlations

A model for correlations? The Socrates programming model A program in the Socrates core language consists of messages, representing abstract operations, branches, representing concrete units of behavior that implement parts of one or more operations, and fields, representing associations between data values. During the execution of a Socrates program, decision points, objects, and contexts are created and manipulated. These six entities are all first-class values. Socrates = Predicate Dispatch + … 11/28/2018 Correlations

Socrates Reference http://socrates-lang.sourceforge.net/ @INPROCEEDINGS{doug:aosd-2002, AUTHOR = "Doug Orleans", TITLE = "Incremental Programming with Extensible Decisions", BOOKTITLE = "First International Conference on Aspect-Oriented Software Developm ent", YEAR = "2002", ADDRESS = "Enschede, The Netherlands", PAGES = "", EDITOR = "Gregor Kiczales", PUBLISHER = "ACM Press”} 11/28/2018 Correlations

Noise reduction: important topic To implement trading robots, we use a tool called DemeterF which is good at noise reduction during the programming process: focus on important classes and eliminate noise classes. 11/28/2018 Correlations

Positive votes 15 Negative votes 7 Passes 2 Erik Ernst The notion of being an inherently cross-cutting concern does not exist. Posted by Erik Ernst, Friday, March 28, 2008 on aosd.net. It is a matter of (1) the power of the available abstraction mechanisms, and (2) the choice of organizational principles in the design of the software system under scrutiny. 11/28/2018 Correlations

Positive votes 15 Negative votes 7 Passes 2 Erik Ernst The notion of being an inherently cross-cutting concern does not exist. Posted by Erik Ernst, Friday, March 28, 2008 on aosd.net. It is a matter of (1) the power of the available abstraction mechanisms, and (2) the choice of organizational principles in the design of the software system under scrutiny whether or not the set of software elements associated with a given concern is cross-cutting or not. Such a set of software elements may be distributed in a cross-cutting manner in a concrete system written using one or more concrete languages. As soon as somebody manages to express the concern modularly by reshaping the architecture or creating and using new abstraction mechanisms, this concern becomes non-cross-cutting, so it was never *inherently* cross-cutting. The notion of being a *currently* cross-cutting concern does exist, though, and the really interesting issue is how we can move concrete concerns out of this sad state by means of improvements of the basic theories and technologies. ;-) 11/28/2018 Correlations