Justine Rochas Thesis supervised by Ludovic Henrio (CNRS) Execution Support for Multi-threaded Active Objects: Design and Implementation Justine Rochas Thesis supervised by Ludovic Henrio (CNRS) Reviewers Einar Broch Johnsen University of Oslo Lionel Seinturier Université de Lille Examiners Florian Kammüller Middlesex University London Johan Montagnat CNRS Fabrice Huet Université de Nice Sophia Antipolis

Motivation Example: airline reservation system Challenges Simultaneous accesses (concurrency) From anywhere (distribution) Other examples World wide web Massively multiplayer online games Banking systems Peer-to-peer travel data booking

How do I build that? The toolbox Programming models Programming languages Middlewares APIs Technologies

Global objectives Help the programmer in developing safe distributed and concurrent systems, easily Our approach Programming model (design) Middleware (implementation) Goals to keep in mind High level of abstraction Correct behaviour

Outline Introduction Local scheduling for multiactive objects Encoding Support Conclusion Outline Introduction Actors Active objects Multiactive objects Local scheduling for multiactive objects Encoding of cooperative active objects Support for multiactive objects Conclusion & perspectives

The actor programming model (1985)

Active objects (1996) Object-oriented programming model for concurrency Asynchronous method call (request) Placeholder for the result (future) request queue active object request SAFE… NO DATA RACES thread method call

The ProActive middleware Java library for distributed active objects Active + passive objects (activity)  Java syntax & transparent futures Wait-by-necessity activity Bar bar = newActive(Bar.class, parameters, node); Foo foo = bar.compute(); active object foo.use(); local reference passive object

Limitations of active objects Inefficient local parallelism Single-threaded data manipulation Risk of deadlocks Circular dependency on future awaiting Deployment challenges Location transparency Exception handling Fault tolerance … SAFE… TOO STRICT? a b

Related works: cooperative active objects Explicit release points in requests (await) Enable request interleaving Can avoid some deadlocks Require more skills Creol (2006) JCoBox (2010) & ABS (2010) Sets of active objects sharing one active thread ABS Balancer smsb = new Balancer("sms", 15); Fut<Int> f = smsb!sendSms(); await f?; smsb two sets of active objects

Related works: Scala (2006) / Akka (2010) Single-threaded actors Still deadlock-prone (futures) Practical aspects fully addressed Akka class Master extends Actor { def receive = { case Calculate ⇒ println("Received") context.system.shutdown() }   def calculate(nbWorkers: Int) { val system = ActorSystem("PiSystem") val master = system.actorOf(Props[Master], name="m")   val future = master ? Calculate val result = Await.result(future, 3 seconds) .asInstanceOf[String] Low-level configuration mixed with the business logic

Multiactive objects (2013) Goal Local request parallelism (thread-based) Challenge Avoid data races between requests Principle Declare compatibility of requests multiple threads SAFE –> CONTROLLED

Multiactive objects in ProActive Class annotations Parallelisable requests add – log log – log Not parallelisable add – add @DefineGroups({ @Group(name="routing", selfCompatible=false), @Group(name="monitoring", selfCompatible=true) }) @DefineRules({ @Compatible({"routing","monitoring"}) }) class Peer { @MemberOf("routing") void add(Key k, Data d) { … } @MemberOf("monitoring") void log(String m) { … } }

Implementation and formal aspects of active object languages Results of the thesis Contributions to the multiactive object framework Enhanced local parallelism Execution support at the middleware level Contribution to the active object community Encoding of cooperative active objects Implementation and formal aspects of active object languages

Outline Introduction Local scheduling for multiactive objects Encoding Support Conclusion Outline Introduction Local scheduling for multiactive objects Default scheduling Thread limit Priority Encoding of cooperative active objects Support for multiactive objects Conclusion

Default scheduling of multiactive objects Maximisation of request parallelism Execute when compatible with Executing requests Or ahead in the queue Too many threads can be created

Controlling threads of multiactive objects Limit number Global thread limit Per multiactive object Limit type Hard thread limit (all threads) Soft thread limit (active threads) Can be changed programmatically @DefineThreadConfig(threadPoolSize=2, hardLimit=false) class Peer { … } threads in wait-by-necessity

Controlling threads of request groups … @Group(name="routing", selfCompatible=false, minThreads=2, maxThreads=5), @DefineThreadConfig(threadPoolSize=8, hardLimit=true) class Peer { } Threads never used by the routing group max min The programmer never  manipulates threads directly

Contention in multiactive object queue compatible compatible thread limit = 3 ready queue Request priority! [2] a [2] Declarative Scheduling for Active Objects, SAC 2014

Priority specification mechanism … @DefinePriorities({ @PriorityHierarchy({ @PrioritySet(groupNames={"G1"}), @PrioritySet(groupNames={"G2"}), @PrioritySet(groupNames={"G4","G5"}) }) @PrioritySet(groupNames={"G3"}), @PrioritySet(groupNames={"G4"}) class Peer { } G1 high priority G3 G2 G4 G5 low priority priority graph of request groups

Insertion time of requests with priority transitive graph Insertion time (ms) Same performance as integers, but more expressive Number of requests in the queue

Related works in active object scheduling Programmatic schedulers: ABS (2010), Parallel Actor Monitor (2014) Permissive Application-level scheduling: ProActive, Creol (2006) Safer Different priority locations

Local scheduling for multiactive objects Safe Expressive Easy to reason about Efficent Linked to request groups Conclusion A balanced approach

Outline Introduction Local scheduling for multiactive objects Encoding Support Conclusion Outline Introduction Local scheduling for multiactive objects Encoding of cooperative active objects ABS ProActive backend Fomalisation Support for multiactive objects Conclusion

DISTRIBUTED EXECUTION Motivation & results Translation of ABS programs into ProActive programs General comparison of active object languages MODELLING, VERIFICATION DEPLOYMENT, DISTRIBUTED EXECUTION ABS source code ProActive backend ProActive generated code

The ABS language Active object modelling language Toolset A a = new local A(); B b = new B(obj,…); b!foo(a); Active object modelling language Concurrent Object Groups (COG) Explicit syntax (! and Fut<T>) Cooperative scheduling (await) Toolset Verification & program analysis Execution through backends (Erlang, Haskell, Java…) COG foo(A a) { Fut<V> vFut = a!bar(); await vFut?; V v = vFut.get; v.doSmt(); } a o COG x y b

Challenges of the translation ABS ProActive active object model object groups active & passive objects asynchronous calls local distributed request scheduling cooperative multi-threaded

Challenges of the translation ABS ProActive active object model object groups active & passive objects asynchronous calls local distributed request scheduling cooperative multi-threaded registry COG 1 COG = 1 active object COG

Translation of a new statement ABS ProActive Server server = new Server() Server server = new Server() (1) COG cog = newActive(COG.class, {}, node2) (2) cog.registerObject(server) (3) node1 node2 (1) Create Java Object server (2) Create ProActive active object server (copy) (3) Make Java object available to ProActive active object cog (proxy) cog registry

Challenges of the translation ABS ProActive active object model object groups active & passive objects asynchronous calls local distributed request scheduling cooperative multi-threaded COG Two-level addressing system COG

Translation of an asynchronous call ABS ProActive server!start() server.getCog().execute("start", {}, server.getID()) node1 node2 server server (copy) PASSIVE OBJECTS BECOME REACHABLE getCog start registry cog (proxy) execute cog ID execute ref

Challenges of the translation ABS ProActive active object model object groups active & passive objects asynchronous calls local distributed request scheduling cooperative multi-threaded COG COG Soft thread limit

Translation of an await statement ABS ProActive await f? PAFuture.getFutureValue(f) @DefineGroups({ @Group(name="scheduling", selfCompatible=true) }) @DefineThreadConfig(threadPoolSize=1, hardLimit=false) public class COG { @MemberOf("scheduling") public ABSType execute(…) { … } (1) Make execute requests compatible (2) Limit the COG to 1 active thread Simulates the execution transfer of ABS

Formalisation: translation correctness [4] Theorem 1 from ABS to MultiASP Theorem 2 from MultiASP to ABS DISTRIBUTED ASPECTS programming model Multiactive Objects ABS MultiASP Async. call MultiASP ProActive ≈ Sync. call formalisation implementation Async. call silent actions Return sync. Assignment [4] From Modelling to Systematic Deployment of Distributed Active Objects, Coordination 2016

On representation of futures… Control flow (ABS) vs data flow (MultiASP/ProActive) No straight simulation, future indirection must be followed in the proof f1.get; { A a = new A(); Fut<Fut<Int>> f1 = a!foo(); Fut<Int> f2 = f1.get; return 0; } { A a = new A(); Fut<Fut<Int>> f1 = a!foo(); Int i = f1.get.get; return i; } class A { Fut<Int> foo() { B b = new B(); Fut<Int> f2 = b!bar(); return f2; } class B { Int bar() { while True { } ≠ future f1 future f2 empty future f1 empty in ABS… Only the progams with half-computed futures behave differently ✓ ❌ EQUIVALENCE LOST HERE

Restrictions of the translation The value of a future cannot be a future (from ABS to MultiASP) FIFO service / Causal ordering of requests (both directions) Distribution of future updates (from MultiASP to ABS) node1 node2 Restrictions due to distributed execution v v await f? f f node3 node4 v v await f? f f

Systematic deployment of cooperative active objects Cooperative active objects –> multiactive objects Faithful translation Practical result Implemented backend Formal result Proof of correctness of the simulation Conclusion Restrictions were defined thanks to formalisation

Outline Introduction Local scheduling for multiactive objects Encoding Support Conclusion Outline Introduction Local scheduling for multiactive objects Encoding of cooperative active objects Support for multiactive objects Debugging Fault tolerance Conclusion

Development and execution support For multiactive objects Help in finding bugs Help with unstable distributed environments DEBUGGING FAULT TOLERANCE

Debugger for multiactive objects Peer 2 Peer 4 Peer 1 Peer 3

Debugging: concurrent read & writes

Debugging: deadlock s

Fault tolerance

ProActive active object Recover upon faults request request ProActive active object

Checkpoint multiactive objects… thread 1 thread 2 ProActive multiactive object thread 3 thread 4 WHICH STATE SAVED??

Checkpoint with multiactive scheduling thread limit = 1 thread limit = 3 Checkpoint as-a-request, thread limit, and priority request checkpoint request request, n+1

Support for multiactive objects Request and communication debugger Several users Adapted fault tolerance protocol Extracted from the middleware as multiactive object annotations conclusion Implementation of non-functional aspects thanks to multiactive object scheduling

Outline Introduction Local scheduling for multiactive objects Encoding Support Conclusion Outline Introduction Local scheduling for multiactive objects Encoding of cooperative active objects Support for multiactive objects Conclusion Summary Perspectives

Recap(plication): peer-to-peer d d?

Recap(plication): peer-to-peer @DefineGroups({ @Group(name="dataManagement", selfCompatible=false), @Group(name="monitoring", selfCompatible=true), … }) @DefineRules({ @Compatible({"dataManagement", "monitoring"}), @DefinePriorities({ @PriorityHierarchy({ @PrioritySet({"dataManagement"}), @PrioritySet({"monitoring"}), @DefineThreadConfig(threadPoolSize=2, hardLimit=false); public class Peer { } a Multiactive object annotations Improved local scheduling Thread limit & priority

Recap(plication): peer-to-peer Efficient & robust implementation of peer-to-peer system INIT [1] An Optimal Broadcast Algorithm for Content-Addressable Networks, OPODIS 2013

Recap(plication): translation multiactive object COG Equivalent behaviour –> preservation of guarantees Translation of cooperative scheduling Proof of correctness Efficient ProActive backend for ABS

Conclusion multiactive object framework Compatibility Multi-threading Priority Thread limit Peer-to-peer Fault tolerance ProActive backend for ABS CHARACTERISTICS APPLICATIONS

Statically check absence of concurrent accesses Perspectives The multiactive object framework Mature fault tolerance Dynamic priorities Multiactive objects analysis Deadlock detection in MultiASP programs (on-going PhD) Specification and verification of multiactive components Static analysis of annotations @DefineGroups({ @Group(name="routing", …), @Group(name="monitoring", …) }) @DefineRules({ @Compatible({"routing","monitoring"}) }) class Peer { @MemberOf("routing") void add(Key k, Data d) { … } @MemberOf("monitoring") void log(String m) { … } } Statically check absence of concurrent accesses

Publications [1] Ludovic Henrio, Fabrice Huet and Justine Rochas. “An Optimal Broadcast Algorithm for Content- Addressable Networks’’. In: Proceedings of the 17th International Conference on Principles of Distributed Systems. OPODIS 2013. [2] Ludovic Henrio and Justine Rochas. “Declarative Scheduling for Active Objects”. In: Proceedings of the 29th Annual ACM Symposium on Applied Computing. SAC 2014. [3] Ge Song, Justine Rochas, Fabrice Huet and Frédéric Magoulès. “Solutions for Processing K Nearest Neighbor Joins for Massive Data on MapReduce”. In: 23rd Euromicro International Conference on Parallel, Distributed and Network-based Processing. PDP 2015. [4] Ludovic Henrio and Justine Rochas. “From Modelling to Systematic Deployment of Distributed Active Objects”. In: 18th International Conference, COORDINATION 2016, Held as Part of the 11th International Federated Conference on Distributed Computing Techniques. DisCoTec 2016. Selected for special issue of Logical Methods in Computer Science journal. [5] Ge Song, Justine Rochas, Léa El Beze, Fabrice Huet and Frédéric Magoulès. “K Nearest Neighbour Joins for Big Data on MapReduce: a Theoretical and Experimental Analysis”. In: IEEE Transactions on Knowledge and Data Engineering. 2016. Execution Support for Multi-threaded Active Objects: Design and Implementation – Justine Rochas