1. XtreemOS IP project is funded by the European Commission under contract IST-FP6-033576 Overview of XtreemOS Christine Morin XtreemOS scientific coordinator xtreemos-projectleader@irisa.fr Phenix Workshop, Rennes December 07, 2006

2. Overview of XtreemOS - Phenix Workshop, December 7, 2006 2 Grid Environment & VO WAN VO2 VO1 Multiple users from different institutions Multiple geographically distributed resources in different administrative domains Large scale Uncountable number of resources Dynamicity VO, users, resources

3. Overview of XtreemOS - Phenix Workshop, December 7, 2006 3 State of the Art  Current OS are not Grid-aware & not VO-aware  A variety of Grid middleware & Toolkits for Grid Computing Resource management Changing interfaces Security pitfalls Complexity for users, programmers & administrators

4. Overview of XtreemOS - Phenix Workshop, December 7, 2006 4 XtreemOS Objectives  Design & implement a reference open source Grid operating system based on Linux –Native support for virtual organizations  Validate the XtreemOS Grid OS with a set of real use cases on a large Grid testbed  Promote XtreemOS software in the Linux community and create communities of users and developers

5. Overview of XtreemOS - Phenix Workshop, December 7, 2006 5 XtreemOS Research Challenges  Identify fundamental functionalities to be embedded in Linux for secure application execution in Grids  Build a set of scalable self-healing OS services for secure resource management in very large dynamic grids  Provide a simple Grid API compliant with Posix while adding new functionality and supporting Grid-aware applications  Aggregate cluster resources into powerful grid nodes by integrating single system image mechanisms in Linux  Build an XtreemOS flavour for mobile devices enabling ubiquitous access to grid resources

6. Overview of XtreemOS - Phenix Workshop, December 7, 2006 6 Linux Computer XtreemOS ApplicationMiddleware Appli XtreemOS Flavours  PC  Federation of PCs –Cluster  Mobile device –PDA –Mobile phone

7. Overview of XtreemOS - Phenix Workshop, December 7, 2006 7 XtreemOS Architecture Business Applications Scientific Applications XtreemOS API VO & Security Data Management Application Management Infrastructure for Highly Available and Scalable Services Linux-XOS: Grid-enabled Linux Operating System Linux-XOS for PC Linux-XOS for Cluster Linux-XOS for Mobile Devices

8. Overview of XtreemOS - Phenix Workshop, December 7, 2006 8 XtreemOS UseCases  14 applications –Simulation applications (aerospace, energy) –Business applications –Bioinformatics application –Virtual reality application –Finance application –Telecom application

9. Overview of XtreemOS - Phenix Workshop, December 7, 2006 9 XtreemOS & Linux  Acceptance in the Linux community is key for the success of the XtreemOS project –Packaging for multiple Linux distributions  Mandriva Linux  Red Flag Linux  Debian –Integration in OSCAR –Get XtreemOS patches accepted in Linux OS

10. Overview of XtreemOS - Phenix Workshop, December 7, 2006 10 XtreemOS Project Phases  Phase 1 (M1-M6) –Specification of XtreemOS  Phase 2 (M7-M18) –Design and implementation of XtreemOS basic version –Preliminary experiments with LinuxSSI  Phase 3 (M19-M24) –Integration of all XtreemOS components –Delivery of first XtreemOS prototype  Phase 4 (M25-M48) –Evaluation with real use cases –Design and implementation of advanced features of XtreemOS –Public releases

11. Overview of XtreemOS - Phenix Workshop, December 7, 2006 11 XtreemOS Sub-projects  SP1 - Project Management  SP2 - Linux for Virtual Organizations  SP3 - Grid Support for Linux  SP4 - Software integration, packaging, experimentation & validation  SP5 - Communication, dissemination, exploitation & training XtreemOS SP4 SP2SP3

12. Overview of XtreemOS - Phenix Workshop, December 7, 2006 12 VO and Security Management Business Applications Scientific Applications XtreemOS API VO & Security Data Management Application Management Infrastructure for Highly Available and Scalable Services Linux-XOS: Grid-enabled Linux Operating System Linux-XOS for PC Linux-XOS for Cluster Linux-XOS for Mobile Devices

13. Overview of XtreemOS - Phenix Workshop, December 7, 2006 13 VO & Security Management A VO can be seen as a temporary or permanent coalition of geographically dispersed entities (individuals, groups, organizational units or entire organizations) that pool resources, capabilities and information to achieve common objectives. –Legal or contractual arrangements between entities –Resources can be physical equipment or other capabilities such as knowledge, information or data

14. Overview of XtreemOS - Phenix Workshop, December 7, 2006 14 Some Lessons from the State of the Art  Open issues –Scalability of in-the-large VO management Short-lived VOs –Ease of management of VO and VO identities –Security and VO policy enforcement at the node and site level

15. Overview of XtreemOS - Phenix Workshop, December 7, 2006 15 VO & Security Management  Key components of VO –Owner/administrator of the VO –A set of participating users in different participating domains –A set of participating resources in different participating domains –A set of roles which users/resources can play in the VO –A set of rules/policies on resource availability and access control –An (renewable) expiry time of the VO

16. Overview of XtreemOS - Phenix Workshop, December 7, 2006 16 VO Lifecycle  VO identification –Identify and name VO candidates  VO formation –Creation and configuration of the VO according to the anticipated roles of members  VO operation –Members should be identified for effectively logging and auditing –The VO should be able to classify the resources to different access control level for effective management  VO evolution –Managing change in participating entities or in their condition of use –Members can be added and linked into a VO by authorization –Users can be classified at different levels with associated operation rights  VO dissolution –Non persistent information should be deleted, credentials reclaimed and user and resource providers notified –Should take place after all activities finished

17. Overview of XtreemOS - Phenix Workshop, December 7, 2006 17 VO Management  Two levels –VO level (administration) Performed by XtreemOS-G services  Distributed information management for membership tracking and accounting of users and resources –Node level Performed by XtreemOS-F Add mechanisms to Linux OS for recognizing, controlling, and enforcing usage of global Grid entities  Grid identity management  Resource access granting and accounting  VO policy checking, auditing and enforcing

18. Overview of XtreemOS - Phenix Workshop, December 7, 2006 18 Node Level VO Management  Minimal with respect to changes to the kernel code to reduce pressure to get VO related changes accepted in Linux community –Keep changes localized in dynamically loadable kernel modules  Features –PAM-plug-in based authentication –Static and dynamic identity mapping to local user/group ids –Kernel level key retention mechanisms –ACL mechanisms Flexible, secure, efficient and easily sustainable from the software engineering point of view VO model  Investigation of synergies with existing security enhancement for Linux –Linux Security Module (LSM) Refinement of access control and enforcement mechanisms

19. Overview of XtreemOS - Phenix Workshop, December 7, 2006 19 Infrastructure for Highly Available and Scalable Services Business Applications Scientific Applications XtreemOS API VO & Security Data Management Application Management Infrastructure for Highly Available and Scalable Services Linux-XOS: Grid-enabled Linux Operating System Linux-XOS for PC Linux-XOS for Cluster Linux-XOS for Mobile Devices

20. Overview of XtreemOS - Phenix Workshop, December 7, 2006 20 Infrastructure for Highly Available and Scalable Grid Services  Grid –Very large number of nodes that are distributed world- wide –Dynamicity: nodes join, leave, fail  Applications –Standalone (interact only with the user that launched them) –Services (present an interface to the outside world and can be invoked) System level functionalities Application-level functionalities  Targets of the infrastructure –XtreemOS-G services –Application-level services

21. Overview of XtreemOS - Phenix Workshop, December 7, 2006 21 Infrastructure for Highly Available and Scalable Grid Services  Management of collections of nodes

22. Overview of XtreemOS - Phenix Workshop, December 7, 2006 22 Infrastructure for Highly Available and Scalable Grid Services  Toolbox –Facilities to construct structured collections Application initialization DHT, N-dimensional matrix, ranked nodes –Distributed servers Present a single stable address to the external world hiding the internal organization of the service –Virtual nodes Fault tolerant groups of nodes capable of taking over each other’s tasks –Publish/Subscribe Useful for applications and also to build structured collections Fully decentralized implementation –Directory service Node monitoring and failure detection Adapt to the dynamicity of the monitored attributes

23. Overview of XtreemOS - Phenix Workshop, December 7, 2006 23 Application Management Business Applications Scientific Applications XtreemOS API VO & Security Data Management Application Management Infrastructure for Highly Available and Scalable Services Linux-XOS: Grid-enabled Linux Operating System Linux-XOS for PC Linux-XOS for Cluster Linux-XOS for Mobile Devices

24. Overview of XtreemOS - Phenix Workshop, December 7, 2006 24 Application Management  Entities taking part in job execution –Job One or more processes that collaborate to achieve a common goal Resource allocation unit –Resources Physical or virtual component of limited availability within a computer system  Have static and dynamic characteristics  Application execution management –Job submission and scheduling –Job and resource control –Job and resource monitoring

25. Overview of XtreemOS - Phenix Workshop, December 7, 2006 25 Application Life Cycle

26. Overview of XtreemOS - Phenix Workshop, December 7, 2006 26 Application Execution Management  AEM is generic and flexible as much as possible –Does not target specific users or types of jobs  AEM allows users to exploit advantages of executing a job in a Grid  AEM provides an easy to use job submission, control and monitoring interface –Unix-like submission (with default description of requirements) –Batch-like submission Requirements Hints (additional information optionally provided by users) –Adaptive and accurate monitoring  AEM deals with Grid dynamicity –Job migration and checkpointing –Hide failures and changes as much as possible to users

27. Overview of XtreemOS - Phenix Workshop, December 7, 2006 27 Application Execution Management  AEM has to guarantee access to authorized resources and their limited utilization –Jobs executed in the context of a grid user and a VO –Rely on VO and security management services (WP2.1, WP3.5)  Scalability and fault tolerance taken into account in the design of AEM –Most of AEM services are in the scope of a job which is suitable for scalability JExecMng and jMonitor could potentially have to manage hundreds of nodes –JobDirectory and jController need to be fault tolerant –WP3.2 services will be used as appropriate Resource discovery Distributed servers  Tight integration with the Linux OS –Enforcement in the usage of agreed resources (quota, access control) Job-id to be known by XtreemOS-F –Users will have more information and control on how their jobs are running Performance metrics, occurred errors, exit status, …  AEM provides a basic set of system-level functionalities –Users may rely on user-level services (eg. workflow manager, SAGA runtime)

28. Overview of XtreemOS - Phenix Workshop, December 7, 2006 28 Data management Business Applications Scientific Applications XtreemOS API VO & Security Data Management Application Management Infrastructure for Highly Available and Scalable Services Linux-XOS: Grid-enabled Linux Operating System Linux-XOS for PC Linux-XOS for Cluster Linux-XOS for Mobile Devices

29. Overview of XtreemOS - Phenix Workshop, December 7, 2006 29 Data Management  XtreemFS –Federated object-based file system for Grid environments Centralised metadata servers replaced by a federation of metadata servers  Independence of participating organizations while maintaining a global view of the system Designed with wide-area networks in mind  File replication  Location and access management based on an intelligent monitoring service oAccess pattern-aware replication Semantic naming and advanced query functions to allow users to find data in huge archives –Object Sharing Service (OSS) Inter-process communication via volatile memory, mapped files, dynamically allocated objects and grid pipes

30. Overview of XtreemOS - Phenix Workshop, December 7, 2006 30 XtreemFS Components  Object Storage Device (OSD) –Data access in the file system Read/write access, concurrency control –Object-based storage interface to hide complexity of underlying block-based storage mechanisms  Metadata and Replica Catalogue (MRC) –Maintenance of all file system metadata Posix metadata Extended (user defined) metadata Information on replica locations  Replica Management Service (RMS) –Decides when replicas have to be replicated and with what distribution among OSD –Replica removal  Client –Hosts running the access layer (file system adapter or XtreemFS library) Linux traditional file system interface for transparent access to MRC, OSD, RMS Native XtreemFS interface

31. Overview of XtreemOS - Phenix Workshop, December 7, 2006 31

32. Overview of XtreemOS - Phenix Workshop, December 7, 2006 32 Object Storage Device (OSD)  Container of objects –Reliably store and retrieve data from physical media –Security enforcement for access to stored objects Capabilities built by MRC and received with each request –Multi-object files Striping and/or replication Each file replica has its own striping policy –Transactional files Changes performed on a local copy (and not forwarded to other OSD) and committed or rolled back at some time

33. Overview of XtreemOS - Phenix Workshop, December 7, 2006 33 Replica Management Service (RMS)  Take care of autonomous creation and deletion of replicas  Replication policies –Must satisfy security needs and comply with local regulations Countries, real organization, VO, racks in a data centre  Replica creation –Gathering information from other services to decide when and where to create a replica Each time a file is open  RMS is contacted to see if a better replica should be created oDecision depends on the file size, OSD availability oA client may start accessing a “bad replica” during the creation of a new one MRC may keep track of opens to predict future access from the previous ones AEM can inform RMS that a job is about to start its execution  RMS can anticipate the creation of a new replica before the job execution  Removing “obsolete” replicas –Lack of free space, file or replica very seldom used, close replicas not anymore useful, … –A replica can be removed at any time even while being used

34. Overview of XtreemOS - Phenix Workshop, December 7, 2006 34 MetaData and Replica Catalogue (MRC)  MRC –Acts logically as one service but will be composed of replicated service instances to improve availability and performance –Access control management Support of a variety of policies Volume ACL  Data model –Hierarchical directory structure and/or extended metadata –Core abstraction for controlling access to file metadata and file data is the volume –Files can be copied between volumes and links to files in other volumes can be created  Internal architecture –Exactly one meta object per physical object on a storage device  To what extend it is possible to decouple system components while preserving a global view to the system

35. Overview of XtreemOS - Phenix Workshop, December 7, 2006 35 Object Sharing Service (OSS)  Inter-process communication via volatile memory, mapped files, dynamically allocated objects and grid pipes –All components designed to be scalable and fault tolerant to deal with the dynamic behaviour of the Grid  Features –Management of shared objects containing references –Object access detection Page based –Object access monitoring to control false sharing and object replicas –Object consistency management Strict, weak and transactional memory consistency models

36. Overview of XtreemOS - Phenix Workshop, December 7, 2006 36 LinuxSSI: Linux-XOS for Clusters Business Applications Scientific Applications XtreemOS API VO & Security Data Management Application Management Infrastructure for Highly Available and Scalable Services Linux-XOS: Grid-enabled Linux Operating System Linux-XOS for PC Linux-XOS for Cluster Linux-XOS for Mobile Devices

37. Overview of XtreemOS - Phenix Workshop, December 7, 2006 37 LinuxSSI: XtreemOS-F Cluster Flavour  LinuxSSI will leverage Kerrighed SSI OS for clusters  Four work directions for LinuxSSI –Scalability to hundreds of processors –LinuxSSI file system –Automatic reconfiguration of LinuxSSI –Checkpoint/restart mechanisms for parallel applications –Customizable scheduler

38. Overview of XtreemOS - Phenix Workshop, December 7, 2006 38 Scalability & Reconfiguration Management  Scalability to hundreds of processors –Removing hard limits on the amount of nodes –Evaluating the scalability of Kerrighed internal algorithms  Automatic reconfiguration of LinuxSSI –Node addition, eviction or failure management –Leverage the existing mechanisms provided by Kerrighed in the HotPlug module

39. Overview of XtreemOS - Phenix Workshop, December 7, 2006 39 LinuxSSI File System  LinuxSSI file system –Exploitation of the disks attached to cluster nodes Single name space (root file system) Policies for placing/replicating data on disk Efficient parallel accesses to large data volumes –Performance as a primary target in LinuxSSI basic version –LinuxSSI file system should not fail in the event of failures Better support to failures in the advanced version of LinuxSSI

40. Overview of XtreemOS - Phenix Workshop, December 7, 2006 40 Checkpoint/Restart in LinuxSSI  Checkpoint and restart of parallel application units in a cluster –Shared memory and message-passing programming models will be supported –Checkpointer multi-level architecture Kernel checkpointer  Process/thread checkpointing  Based on Kerrighed mechanisms  Transparent or application-aware checkpointing System checkpointer  Application unit checkpointing (inside a cluster)  Coordination of thread/process checkpoints for parallel applications  Configurable service Grid checkpointer  Application checkpointing (an application may span multiple Grid nodes)  Coordination of application unit checkpoints for an application comprising of multiple units

41. Overview of XtreemOS - Phenix Workshop, December 7, 2006 41 Customizable Scheduler  Customizable scheduler –Long-term scheduler Application admission in the cluster (job queuing system) –Load balancing scheduler Balance the current workload between cluster nodes  Long-term scheduler –DRMAA standard interface –Adapted to take advantage of the SSI “virtual multiprocessor” –Resource sharing (a CPU may not be dedicated to a single application) –Advanced monitoring capabilities  Load balancing scheduler –Policy customization Multilevel architecture (probes, analyzers, decision-making) –Self adaptation of policy based on the current state of the cluster –Advanced policies Shared memory, IPC  Interaction with the Grid level services when needed

42. Overview of XtreemOS - Phenix Workshop, December 7, 2006 42 From LinuxSSI to LinuxSSI-XOS  Virtual organization support –Support of the kernel key retention system Impact on the Ghost module –XtreemOS-G services will run as a single instance on a LinuxSSI cluster Example: daemons in charge of mapping global user, VO and group identities onto the Linux UID/GID

43. Overview of XtreemOS - Phenix Workshop, December 7, 2006 43 XtreemOS Consortium  19 partners –1 public financial institution as coordinator –9 research centers & universities –9 industrial partners 4 SME  8 countries –Europe France, Germany, Italy, Slovenia, Spain, The Netherlands, UK –China

44. Overview of XtreemOS - Phenix Workshop, December 7, 2006 44 XtreemOS Partners

45. Overview of XtreemOS - Phenix Workshop, December 7, 2006 45 Fact Sheet  Start date –June 1st, 2006  Duration –4 years  Budget –Approx. 30 Meuros –EC funding 14.2 Meuros  Website –http://www.xtreemos.euhttp://www.xtreemos.eu  Administrative and financial coordinator –CDC, Jean-Noël Forget  Scientific and technical staff –More than 100 persons