Implementing Chares in a High-Level Scripting Language LuaCharm: Implementing Chares in a High-Level Scripting Language Thiago Ponte, Noemi Rodriguez PUC-Rio – Brazil

If you optimize everything, you will always be unhappy. Don Knuth

Outline Introduction Objectives Lua LuaCharm Tests Summary Future Work

Introduction Scripting languages have been drawing much attention over the past years Dynamism, flexibility and simplicity are the main focus of scripting languages Parallel computing has not yet explored the potential of scripting

Objectives Integrate Charm++ and Lua in way that: Chares can be implemented in Lua Chares implemented in Lua are perceived as regular C++ chares Chares can have their implementation changed at runtime Allow flow of control to be writen in a high level language

Lua Developed at PUC-Rio Has become an important language

Lua

Lua fast, lightweight, powerful, embeddable scripting language dynamic typing; types are associated to values, not variables all values are first class single data-structuring facility tables implement associative arrays metadata mechanisms allow semantics to be extended

Lua Lua offers a object-oriented like syntax, in which objects are implemented as tables Previous experience in integrating Lua with other systems such as Corba and COM and languages, such as Java and .NET myObj = { … foo = function(self, …) -- lua code end, } myObj:foo(…) myObj = {} function myObj:foo(…) -- lua code end myObj:foo(…)

LuaCharm LuaCharm is an extension of Charm++ through changes in parser and code generator Developed after studying existing binding between Lua and Python LuaCharm offers a mechanism to implement chares in Lua The parser generates a façade C++ class responsible for the execution of the corresponding method in Lua

LuaCharm Changes in the interface file and code generator: Inclusion of the attribute lua to be used in chares, followed by the name of the file implementing the chare Besides the methods declared in the interface file, an updateChare method is created, and it may be used to execute a chunk of lua code inside the chare

LuaCharm Exports a library to Lua that allows instantiation of chares in a simples Lua-like manner Chares returned to Lua can be called with the “object-oriented” syntax of Lua

LuaCharm – Chare definition chare [lua "luaChare.lua"] luaChare{ entry luaChare(); entry someLuaMethod(); } luaChare = { ckNew = function() -- lua code end, someLuaMethod = function() end -- chare instantiation c = charm.luaChare() -- calling a remote method c.someLuaMethod()

Integrating with Load Balancing Chares implemented in Lua can be migrated by the load balancer. The Lua implementation must implement a pack and a unpack method to migrate values. pack must return a string, which unpack will receive as parameter

Tests Adaptive Quadrature of the ex function between 0 and 15 Implementations using bag of tasks and division in fixed intervals Four different approaches: Standard Charm++ implementation Controller in Lua and workers in C++ Both chares in Lua Both chares in Lua, but the workers using a library implemented in C for the mathematical calculations

Tests – Bag of Tasks Bag of tasks sends an interval to a worker N; Worker checks if interval should be broken down; If yes, sends back part of the interval; Calculates the result for its interval; Send back the result.

Tests First approach using bag of tasks in the mainchare and a chare group for workers. Tested in 8 processors 1 Impl 2 Impl 3 Impl 4 Impl Time(s) 29.75 29.85 168.77 30.59 Increase in relation to 1 0.35% 467.36% 2.82%

Tests Second approach, using an array of 500 chares, where each is responsible for a pre-determined part of the problem. Tested in 8 processors 1ª Impl 2ª Impl 3ª Impl 4ª Impl Time(s) 10.01 67.83 Increase in relation to 1 0% 577.66% 0.03%

Summary Even when the chares were all implemented in Lua, but with a C library for the mathematical work, the increase in processing time was acceptable Since Lua chares are regular chares, profiling results are easy to read

Future Work Optimizations in the generated code Experiments with runtime redefinitions application strategy load balancer Restrictions in the implementation: Read-only variables Array reductions Using Messages Synchronous non-blocking remote calls using coroutines

Thank You Questions? More details? Coming soon…  tponte@inf.puc-rio.br http://luaforge.net/luacharm ?