1. Dynamic Service Aggregation in Heterogeneous Grids Stavros Isaiadis and Vladimir Getov University of Westminster Heraklion, Crete, 13 th June 2007 http://www.coregrid.net s.isaiadis@wmin.ac.uks.isaiadis@wmin.ac.uk, v.s.getov@wmin.ac.ukv.s.getov@wmin.ac.uk

2. European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 2 Overview Hybrid Grid Systems Lightweight Server-side Container (LSC) Aggregator Framework Case Study Evaluation Conclusion Outline

3. European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 3 Overview Hybrid Grid Systems Lightweight Server-side Container (LSC) Aggregator Framework Case Study Evaluation Conclusion Outline

4. Overview/Objectives Integration of mobile and small-scale devices into Grid systems as service providers (and not just consumers!) Extend the Grid towards the less powerful but more flexible end of the spectrum Aggregation of small, mobile, and heterogeneous services Encapsulate the inherent dynamicity and provide high-level views and programming abstractions Ultimate vision: A truly pervasive hybrid Grid system European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 4

5. Why Mobile and Small Devices? Unique Functionality context and location awareness multimedia cameras intelligent wireless sensors Global Positioning Systems Flexibility Extend to new geographical areas Move business outside Interactive and Personalized Services With mobile phones, administrators are always near their services Attach status and context to services European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 5

6. 6 Overview Hybrid Grid Systems Lightweight Server-side Container (LSC) Aggregator Framework Case Study Evaluation Conclusion Outline

7. Classes of Hybrid Systems mobile interfaces to grid resources where mobile devices are merely interfaces to functionality available in the Grid system, and do not contribute any services. exploitation of raw resources focus is on distributing applications for parallel execution usually requiring the partitioning of the application into small independent tasks. exploitation of services and flexibility focus is on supporting mobile services in a SOA system, while also enabling small-scale devices to contribute services towards a fully integrated infrastructure. European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 7

8. Hybrid Systems European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 8 Application Client, User Tools, SOA MW Developer Mobile and Small-Scale Devices Fabric Layer Application/User Tools/ SOA Middleware Layer

9. Challenges Grid systems have typically evolved around HPC and e-Science Assumed a plethora of static resources Fast and reliable networking infrastructure Even with suitable solutions it is unreasonable to assume that everyone will adopt the same framework In dedicated and static environments certain guidelines and policies can be enforced but: in dynamic environments that aim at agility, opportunistic computing, flexibility and mobility, imposing such restrictions is unrealistic Providers should choose the framework that suits their devices (but results in heterogeneity WS for interoperability and easy integration RMI for a bit of performance and interoperability Proprietary performance-oriented methods? Something else??? Cannot consider each mobile device as a full single member! –Very unreliable –Limited contribution European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 9

10. Contribution A Lightweight Server-side Container suitable for limited devices Socket-based High performance No dependencies Easy deployment But: non-interoperable, low-level. What do we do? Related Work: –Mobile OGSI.NET Obsolete, restricted model, complex deployment –Microservices Very complex deployment, not very fast (WS-based) –Racoon Promising for small Web Services (still in progress), but not very fast –WSKPeer Promising hybrid Service/P2P approach, work in progress –no workable version European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 10

11. Contribution (contd.) An Aggregator framework to enable the efficient integration of mobile devices. An Aggregator provides: A proxy to a group of heterogeneous and mobile fabric services an abstraction layer that hides the dynamicity and heterogeneity of the mobile services a high-level view of a virtually stable and reliable entity, despite the unreliable nature of the inner fabric services Related Work: ? European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 11

12. Contribution (contd.) European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 12 Mobile and Small-Scale Devices Fabric Layer Application Client, User Tools, SOA MW Developer Application/User Tools/ SOA Middleware Layer Aggregators Development API Proxy Layer

13. European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 13 Overview Hybrid Grid Systems Lightweight Server-side Container (LSC) Aggregator Framework Case Study Evaluation Conclusion Outline

14. Lightweight Server-side Container (LSC) Small CPU and memory footprint Simpler services – not complex operations –no build in support for complex data types (currently) No software dependencies – only J2ME running! –for easy deployment Permanent and Stateful approach – To support context-awareness –To simplify lifecycle management –To reduce concurrent client connections and thus resource requirements Friendly GUI for easy service registration European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 14

15. LSC Stack European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 15 Service Registry Server Socket Listener Server Side Components Java Bean Java Class Other Java Selector Server Request Handler Service Container Socket Communicator Service Request Translator Service Interfaces TCP Socket ProxyMobile Device

16. European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 16 Overview Hybrid Grid Systems Lightweight Server-side Container (LSC) Aggregator Framework Case Study Evaluation Conclusion Outline

17. Aggregator Framework Integrated hybrid systems issues: –Encapsulate the big cardinality of the mobile domain –Provide reliability and availability –Encapsulate heterogeneity and dynamicity LSC-related issues: –Must enable a high-level view –Must provide interoperability European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 17 HTTP/SOAP TCP/LSC PROXY CLIENT VIRTUAL CLUSTER

18. Aggregator/Virtual Clusters Architecture European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 18 Application Client, User Tools, SOA MW Developer Aggregator + Platform Service Interfaces Development API Groups of Mobile and Small-Scale Devices Collective Layer Fabric Layer Aggregator + Platform Services VC Platform/ Proxy Layers Application/User Tools/ SOA Middleware Layer

19. Dynamic Service Aggregation European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 19

20. European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 20 Overview Hybrid Grid Systems Lightweight Server-side Container (LSC) Aggregator Framework Case Study Evaluation Conclusion Outline

21. Case Study Dynamic Content Aggregation from Multiple Heterogeneous Mobile Sources Phase 1: providers must register their services with an Aggregator Phase 2: registered services are automatically activated when they are within the “neighbourhood” of an Aggregator Phase 3: clients can now use that Aggregator (WS) to gather content without knowledge of the underlying fabric level details European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 21

22. Phases 1 and 2: Registration and Activation Providers specify the shared services Server-side Framework (WS, RMI, LSC, Other) Implementation class Interface location Human readable description They can then enable sharing on the press of a button Services are discovered by the relevant Aggregator and activated automatically Services are now shared! European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 22 VC Platform/ Proxy Layers

23. Phase 3: Invocation European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 23

24. European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 24 Overview Hybrid Grid Systems Lightweight Server-side Container (LSC) Aggregator Framework Case Study Evaluation Conclusion Outline

25. LSC Evaluation: Performance Testing TypeBase ModelMemoryOSServer-side framework Laptop A2.2GHz Intel1GBWin XPTomcat/Axis, RMI, LSC Laptop B1.6GHz AMD512MBWin XPTomcat/Axis, RMI, LSC SmartphoneSony P990i128MBWin MobileMicroservices, RMI-OP, LSC European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 25

26. LSC Evaluation: Resource Requirements ComponentSize vcluster-mobile.jar42 KB LSC.jar31 KB Service-registry.xml2 KB startup scripts1 KB Total76 KB European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 26

27. European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 27 Overview Hybrid Grid Systems Lightweight Server-side Container (LSC) Aggregator Framework Case Study Evaluation Conclusion Outline

28. Conclusions The LSC framework manages to marry the high performance of low-level TCP sockets with the interoperability of the standards-based Web Services paradigm Evaluation has shown that resource requirements are minimal, and there are no software dependencies or complex installation procedures The Aggregator framework: –enables high-level abstract views of the mobile devices –Provides a virtually stable and reliable virtual entity despite consisting of unreliable fabric services With the rapid evolution of mobile computing and related advances, the time is right to provide the foundations for truly pervasive systems European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 28

29. Related Resources Publications: http://www.isaiadis.com/publications/ International Science Grid This Week: http://www.isgtw.org/ (6 th June Edition) European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 29

30. Ευχαριστώ! Questions? The Virtual Clusters Project page: http://www.isaiadis.com/vc/ Please forward any suggestions, questions, comments to: s.isaiadis@wmin.ac.uk European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 30