1. A Programming Model and System Support for Disconnected-Aware Applications on Resource-Constrained Devices (FarGo-DA) MSc seminar by Weinsberg Yaron www.dsg.technion.ac.il/fargo-da January 6, 2002 www.dsg.technion.ac.il/fargo-da

2. Outline  Introduction  FarGo-DA’s Programming Model  A sample FarGo-DA application  Runtime infrastructure  Status and Future Work  Summary

3. Thesis Domain  Modern wide-area networks are characterized by: Dynamic changes of bandwidth, load, machine capacities and availability. Dynamic changes of bandwidth, load, machine capacities and availability.  Requires high level of adaptability from distributed software systems.  We follow the distributed object programming model and use Object mobility to: increase performance increase performance decrease cost decrease cost attain high availability attain high availability  The need: new models and techniques for designing wide-area sw systems.

4.  A generic framework for constructing disconnected- aware applications. Enable developers to encode connection semantics. Enable developers to encode connection semantics. Dynamically changing the application layout upon (re)disconnection. Dynamically changing the application layout upon (re)disconnection.  A new model and techniques for designing disconnected-aware applications. Separating the connection layout from the application logic. Separating the connection layout from the application logic.  A lightweight mechanism for management and of disconnection and reconnection Research Objectives

5.  Focus on Voluntary disconnection. User initiated User initiated enables the system to prepare for disconnection enables the system to prepare for disconnection One-to-many One-to-many  Uni-directional client-server applications. NLPCs act as active clients only, and are not servers of any interaction. NLPCs act as active clients only, and are not servers of any interaction. The disconnection request is always initiated by the NLPC (client), not by any server. The disconnection request is always initiated by the NLPC (client), not by any server. Assumptions

6.  Application objects interact across multiple hosts using remote method invocations (RMI)  FarGo adds a dynamic layout Add dynamic relocation while preserving the distributed object abstraction Add dynamic relocation while preserving the distributed object abstraction All references into and out of a migrating object must remain valid. All references into and out of a migrating object must remain valid. site B site A   FarGo Overview

7. site B site A  FarGo Overview  Application objects interact across multiple hosts using remote method invocations (RMI)  FarGo adds a dynamic layout Add dynamic relocation while preserving the distributed object abstraction Add dynamic relocation while preserving the distributed object abstraction All references into and out of a migrating object must remain valid. All references into and out of a migrating object must remain valid.

8.  A unique aspect of FarGo is the ability to associate movement semantics with component references.  The semantics can be changed dynamically. FarGo Overview  A component reference Source component Target component The semantic goes here…  FarGo support the following semantics: Link, Pull, Stamp, Duplicate. Link, Pull, Stamp, Duplicate.

9.  A Pull reference:  Pull  use r Link A relocator A Sample FarGo Reference  When  moves,  moves along.  Useful for frequent or data-intensive interaction. Core A Core B

10.  Pull   A Pull reference: Core A Core B use r  When  moves,  moves along.  Useful for frequent or data-intensive interaction. Link A Sample FarGo Reference

11.  A new programming model: A framework that enables developers to specify architecture-oriented (re)disconnection policies. A framework that enables developers to specify architecture-oriented (re)disconnection policies. A runtime for interpreting and carrying the policies upon disconnection,reconnection. A runtime for interpreting and carrying the policies upon disconnection,reconnection.  FarGo-DA Extends the FarGo implementation with: A whole new class of Disconnected-Aware (or DA) semantics. A whole new class of Disconnected-Aware (or DA) semantics. An extended FarGo invocation mechanism (and FarGo compiler) An extended FarGo invocation mechanism (and FarGo compiler) An extended FarGo serialization mechanism An extended FarGo serialization mechanism FarGo-DA’s Programming Model

12.  Complet: A mobile java component.  DA-Complet: A complet that is aware of the core’s connectivity Only DA-Complets are affected by (re)disconnection Only DA-Complets are affected by (re)disconnection  Core: A process that can execute, send, and receive complets.  Complet Reference: A reference that points to a complet ( “MetaRef”) Reifies (re)disconnection semantics. Reifies (re)disconnection semantics. Accessible via system methods: Accessible via system methods: FarGo-DA’s Programming Model MetaRef metaRef = Core.getMetaRef(target); metaRef.setDA(new Clone(),new Merge());

13.  Identify complets and their interactions. Complets are “semantically remote to each other” Complets are “semantically remote to each other”  object parameters are passed by value  complets are passed by reference. Physically, complets may be co-located or remote. Physically, complets may be co-located or remote. Facet 1 Application Logic Design

14.  Mark the DA components.  Non-DA components are not affected.  Define FarGo-DA’s connection semantics via reference types: Disconnection Semantics: Disconnection Semantics:  Clone, Replace, StoreAndForward, Depart  Define migration priorities. Reconnection Semantics: Reconnection Semantics:  Merge, Purge, Overwrite, Last Facet 2 DA Connection Layout

15.  A Clone reference. Disconnection Semantics Core A Core B  When core A disconnects,  is cloned to the locality of .  The default connection semantics.  Within each core, the system notifies all DA-complets about the coming disconnection. Enable to dynamically change the connection semantics Enable to dynamically change the connection semantics  Clone  I want to Disconnect!  Disconnection Event MetaRef metaRef = Core.getMetaRef(beta); metaRef.setDA(new Clone());

16.  A Clone reference. Core A Core B  Clone  I am the clone complet (replica) I am the original complet  The local reference is updated.  the clone complet has the same ID as the original complet. Disconnection Semantics

17.  A Replace reference. Core A Core B   locates a local instance of  ’s type and connects to it. If non-exist, a new one is created. If non-exist, a new one is created.  Useful when:  reduced functionality (smaller footprint) is sufficient.  the origin complet cannot be migrated.  Replace    lik e 1.same interface. 2. 2.same binding-name. 3. 3.equivalence method will figure it out… Disconnection Semantics

18.  A Replace reference. Core A Core B  Replace    lik e  Disconnection Event   locates a local instance of  ’s type and connects to it. If non-exist, a new one is created. If non-exist, a new one is created.  Useful when:  reduced functionality (smaller footprint) is sufficient.  the origin complet cannot be migrated. Disconnection Semantics

19.  A Replace reference. Core A Core B  Replace    lik e   locates a local instance of  ’s type and connects to it. If non-exist, a new one is created. If non-exist, a new one is created.  Useful when:  reduced functionality (smaller footprint) is sufficient.  the origin complet cannot be migrated. Disconnection Semantics

20.  A StoreAndForward reference. Core A Core B  StoreAndForward   Disconnection Event Disconnection Semantics  A special kind of reference: Store + Forward.  All invocations are one-way, in order to enable the source complet to continue its work.  All invocations are stored at the local NLPC, pending for reconnection.  The number of invocation stored is limited and can be set by the developer (with a runtime limit).

21.  A StoreAndForward reference. Core A Core B  StoreAndForward   A special kind of reference: Store + Forward.  All invocations are one-way, in order to enable the source complet to continue its work.  All invocations are stored at the local NLPC, pending for reconnection.  The number of invocation stored is limited and can be set by the developer (with a runtime limit). Local buffer Storing: - method number - parameters. Method invoked… Disconnected Operation Disconnection Semantics

22.  A StoreAndForward reference. Core A Core B  StoreAndForward   Upon Reconnection, the runtime forwards the pending invocations to the remote complet.  No return values.  An example use of this reference type is for outgoing email messages, CVS commits, instant messaging etc. Local buffer Upon Reconnection… method invoked! Disconnection Semantics

23.  A Depart reference. Core A Core B  Depart  Disconnection Semantics User Invoker return value method invoked!  All invocations are handed over to the remote core prior to disconnection, for remote execution  Upon reconnection the results, if any, are transferred to the source complet  Useful to spawn heavy remote computations without the need to be continuously connected  Partly implemented (based on Miki Abu’s work,ICDCS01) a remote thread created method invoked. return value is available: -  can pull it -can be notified return value is saved if  is currently disconnected.

24.  Complet Prioritization  NLPCs are resource constrained. not all clone jobs can be fulfilled. not all clone jobs can be fulfilled.  FarGo-DA provides a simple interface to setting priorities, by an additional parameter to the setDA method.  At runtime, the system carries out the disconnection semantics based on the priority order.  Invocations on references without target complets are gracefully handled via FarGo-DA exceptions MetaRef metaRef = Core.getMetaRef(target); metaRef.setDA(new Clone(),new Merge(),Priority.High); Disconnection Semantics

25.  A Merge reference. Core A Core B  When core A reconnects,  ’s clone is merged with the original complet (explained later).  The default reconnection semantics.  Merge  I want to Reconnect the network… Reconnection Event  Reconnection… Merging… Reconnection Semantics

26.  A Merge reference. Core A Core B  When core A reconnects,  ’s clone is merged with the original complet (explained later).  The default reconnection semantics.  Merge  merged!! ! Reconnected!!! Reconnection Semantics

27.  A Purge reference. Core A Core B  When core A reconnects,  ’s clone is purged.   is reconnected to .  Analogous to methods [in] parameters in RPC  Purge  I want to Reconnect the network… Reconnection Event  Reconnection… Reconnection Semantics

28.  A Purge reference. Core A Core B  Purge  Reconnected!!!  When core A reconnects,  ’s clone is purged.   is reconnected to .  Analogous to methods [in] parameters in RPC Reconnection Semantics

29.  An Overwrite reference. Core A Core B  The opposite of Purge.  The state of the cloned complet overwrites the state of the original complet.  Analogous to methods [in/out] parameters in RPC   is reconnected to .  Overwrite  I want to Reconnect the network… Reconnection Event  Reconnection… Overwrite Reconnection Semantics

30.  An Overwrite reference. Core A Core B  Overwrite  Reconnection Semantics  The opposite of Purge.  The state of the cloned complet overwrites the state of the original complet.  Analogous to methods [in/out] parameters in RPC   is reconnected to . Reconnected!!!

31.  A Last reference. Core A Core B  The state of the complet with the latest timestamp wins.  The clocks of the NLPC and the remote server should be synchronized.   is reconnected to .  Last  I want to Reconnect the network… Reconnection Event  Reconnection… Comparing timestamps… Reconnection Semantics

32.  A Last reference. Core A Core B  Last  Reconnected!!! I won!!!  The state of the complet with the latest timestamp wins.  The clocks of the NLPC and the remote server should be synchronized.   is reconnected to . Reconnection Semantics

33. Conflict Resolution  In order to resolve conflicting updates between replicas we must assume application specific knowledge.  FarGo-DA offers two mechanisms in order to resolve merging conflicts: Callback methods- The Merge callback method. Callback methods- The Merge callback method. void Merge(Complet archive,Complet replica, Log localLog,Log remoteLog) Log localLog,Log remoteLog){ // Application-specific merging code // Application-specific merging code} invoked on the original complet Core A Core B   rr original replica aa archive remote log local log

34.  Reconciliation operators encapsulation of primitive types + a state variable that represents a built-in merging method. encapsulation of primitive types + a state variable that represents a built-in merging method. public class BankAccount_ extends DisconnectedAwareComplet extends DisconnectedAwareComplet{ private discInt private discInt balance = new discInt(Operation.Add); balance = new discInt(Operation.Add);} Conflict Resolution  FarGo-DA provides the following types: discInt, discLong, discString discInt, discLong, discString  And the following operations: Add, Max, Min, Average, Random, Last, First. Add, Max, Min, Average, Random, Last, First.

35.  Each reconciliation operator must implement the discType interface.  User defined types are created by implementing the merge method declared at discType interface.  FarGo-DA’s Runtime supports the merging of reconciliation operators, contained in Java’s basic container classes (i.e., Vector, Array). Conflict Resolution

36. Core A Core B  Merge    How reconciliation operators work ? Upon Reconnection…   + Introspection -discInt balance; -discString account; -discInt balance’ ; -discString account ‘ ; -discInt balance;  merged!! ! -discString account;

37. FarGo-DA Example ManageAnywhere the current customer adds a new product to the order  ManageAnywhere is a simple tool for collaborative management to be used by remote sales forces.  Each sales agent is equipped with a NLPC.  The NLPC may or may not be connected to the central facility, depending on the cellular coverage in the visited area. get the available balance calculates the product price according to the quantity quantity

38. NLPC Server Logic Configurator Inventory Product Accounts CompanyAccount GUI BasicConfigurator 1 n 1 n Application Logic Design The agents user interface. Handles all Handles all service requests. service requests. - financial calc - financial calc - product’s price calc - product’s price calc - smaller footprint service provider service provider products inventory products inventory customers pre-paid accounts public class ComapnyAccount_ extends DisconnectedAwareComplet extends DisconnectedAwareComplet{ private Integer prePaidBalance= 60102; private discInt totalPurchase = new discInt(Operation.Add); }

39. Connection Layout Design public class Logic_extends DisconnectedAwareComplet { MetaRef mr = Core.getMetaRef(accounts); MetaRef mr = Core.getMetaRef(accounts); mr.setDA(new Clone(),new Merge(),Priority.LOW); mr.setDA(new Clone(),new Merge(),Priority.LOW); mr = Core.getMetaRef(configurator); mr = Core.getMetaRef(configurator); mr.setDA(new Replace(),new Purge(),Priority.High mr.setDA(new Replace(),new Purge(),Priority.High mr = Core.getMetaRef(inventory); mr = Core.getMetaRef(inventory); mr.setDA(new StoreAndForward(),null,Priority.LOW mr.setDA(new StoreAndForward(),null,Priority.LOW} A look at the Logic complet

40. Disconnection Exemplified

41. System Design Core Architecture DM - Manages the Reconnection and Disconnection procedures. and Disconnection procedures. - Realized by the DisconnectionManager (DM) DisconnectionManager (DM) The FarGo-DA’s Implementation enables disconnected operation. operation. FarGo-DA’s Implementation provide a general parameter passing scheme passing scheme

42. System Design Complet Reference Structure  The connection reference type is encapsulated in a meta-reference object. same interface as  holds the connection semantics. - points to  or to a remote tracker a remote tracker - RMI object

43. System Design FarGo-DA’s invocation model  In order to support disconnected operation, a new invocation scheme was implemented in FarGo-DA. FarGo assumed that the complet is “always- connected”: FarGo assumed that the complet is “always- connected”:  available on the local core  available on the remote core through a remote tracker. Upon invocation, FarGo always try to invoke the complet. In FarGo-DA this is a reference specific property (for example, StoreAndForward stops the invocation flow) Upon invocation, FarGo always try to invoke the complet. In FarGo-DA this is a reference specific property (for example, StoreAndForward stops the invocation flow) s Tc s T c Tremote

44. System Design FarGo-DA’s invocation model Invocation starts here… connector’s preInvoke method is activated: -continue invocation ? -store data ? -log ? etc… Invocation continues to complet! decision returned.

45. System Design The Disconnection Procedure Core A Clone    Replace  Disconnection Event User Asks to Disconnect… Core B

46. System Design The Disconnection Procedure Core A Clone    Replace DM is loaded…………. DMDM Core B

47. System Design The Disconnection Procedure Core A Clone    Replace Creating Internal Database… DMDM New Job: -  : should be cloned Core B

48. System Design The Disconnection Procedure Core A Clone    Replace Cloning… DMDM Core B  disconnection buffer  Is serialized…  type  Is serialized…  type Looking for  … Create a new one!!!  DM discovers that… -  : should be replaced

49. System Design The Disconnection Procedure Core A Clone    Replace Update References… DMDM Core B   Replace Disconnection Completed! ready to reconnect!

50. System Design The Reconnection Procedure Core A Merge    Purge DMDM Core B   Purge User Reconnects…. DM use the already created database.  +  merged =

51. System Design The Reconnection Procedure Core A Merge   Purge DMDM Core B   Purge User Reconnects….  merged

52. System Design The Reconnection Procedure Core A Merge   Purge DMDM Core B  merged Update References… Reconnection Completed!

53. Status and Future Work  Publications: A programming Model and System Support – to be published in ICSE’02 (May 2002) A programming Model and System Support – to be published in ICSE’02 (May 2002)  Simulation for NLPC using FarGo’s monitoring API (Hovav' s thesis, MA2000)  System is fully implemented (using JDK1.8) and will soon be available for download.  Ongoing work: DA-Application profiling. DA-Application profiling. Weakly-Connected Operation. Weakly-Connected Operation. More applications. More applications.

54. Summary  A new programming model: Design time (logical) structural rules. Design time (logical) structural rules. Loose coupling between application-logic and connection layout (re)design. Loose coupling between application-logic and connection layout (re)design. Keeping a uniform (and familiar) language, tools, and object model. Keeping a uniform (and familiar) language, tools, and object model.  System Design: Create a new invocation model to support disconnected operation. Create a new invocation model to support disconnected operation. A system component to handle the (re)disconnections. A system component to handle the (re)disconnections. Implementation of the disconnection semantics is built into a new serialization mechanism Implementation of the disconnection semantics is built into a new serialization mechanism Simple yet powerful conflict resolution mechanisms. Simple yet powerful conflict resolution mechanisms.  An implemented model and system.