Agenda 1. Background: OASIS, ActiveEon 2. ProActive Overview 3. Programming (Components: GCM Standard) 4. Optimizing 5. Scheduling + Resourcing 6. SOA,

Slides:



Advertisements
Similar presentations
Elton Mathias and Jean Michael Legait 1 Elton Mathias, Jean Michael Legait, Denis Caromel, et al. OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis,
Advertisements

Agenda 1. Background: OASIS, ActiveEon 2. ProActive Overview : Programming, Scheduling, Resourcing 3. Use Case: Genomics 4. Cloud Seeding D. Caromel,
SLA-Oriented Resource Provisioning for Cloud Computing
European Commission Directorate-General Information Society Unit F2 – Grid Technologies INSERT PROJECT ACRONYM HERE BY EDITING THE MASTER SLIDE (VIEW.
Denis Caromel1 Joint work with Ludovic Henrio – Eric Madelaine et. OASIS members OASIS Team INRIA -- CNRS - I3S – Univ. of Nice Sophia-Antipolis, IUF.
1 Key Objectives  Parallel Programming Model and Tools  desesperatly needed  for the masses (New Scientist, New SME)  for new architectures (Multi-cores)
Eric MADELAINE1 E. Madelaine, Antonio Cansado, Emil Salageanu OASIS Team, INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis OSCAR meeting, Valparaiso,
Denis Caromel Denis Caromel 3 Clouds.
Architecture Grille Orientée Services ProActive Integration with HP Software in the AGOS project Towards the integration of business services within a.
Denis Caromel 1 Denis Caromel, et al. OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis, IUF September 4.
1 Ludovic Henrio Paris, An Open Source Middleware for the Grid Programming Wrapping Composing Deploying.
Denis Caromel1 Institut Universitaire de France (IUF) OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis ECCOMAS, July 2004 ProActive: Components.
Session 2: task 3.2 GCM, Kracow, June l Current status of GCM Denis Caromel (10 mn each talk) l Wrapping CCA Components as GCM Components Maciej.
Monitoring OSGi platforms with ProActive Virginie Legrand OASIS Team - INRIA Sophia Antipolis.
Denis Caromel1 Denis Caromel, et al. OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis, IUF 3 rd ProActive User Group, Nov Model.
Architecture overview 6/03/12 F. Desprez - ISC Cloud Context : Development of a toolbox for deploying application services providers with a hierarchical.
95-843: Service Oriented Architecture 1 Master of Information System Management Service Oriented Architecture Oracle SOA Suite 11g.
SOA – Development Organization Yogish Pai. 2 IT organization are structured to meet the business needs LOB-IT Aligned to a particular business unit for.
1 1 Hybrid Cloud Solutions (Private with Public Burst) Accelerate and Orchestrate Enterprise Applications.
1 1 Hybrid Cloud Solutions (Private with Public Burst) Accelerate and Orchestrate Enterprise Applications.
Fabien Viale 1 Matlab & Scilab Applications to Finance Fabien Viale, Denis Caromel, et al. OASIS Team INRIA -- CNRS - I3S.
Computer System Architectures Computer System Software
 Cloud computing  Workflow  Workflow lifecycle  Workflow design  Workflow tools : xcp, eucalyptus, open nebula.
KARMA with ProActive Parallel Suite 12/01/2009 Air France, Sophia Antipolis Solutions and Services for Accelerating your Applications.
Denis Caromel1 Institut universitaire de France (IUF) OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis REUNA, Santiago, May 2004 GRID.
@2011 Mihail L. Sichitiu1 Android Introduction Platform Overview.
Denis Caromel 1 A Strong Programming Model Bridging Distributed and Multi-Core Computing 1.Background: INRIA, Univ. Nice, OASIS.
The Grid Component Model: an Overview “Proposal for a Grid Component Model” DPM02 “Basic Features of the Grid Component Model (assessed)” -- DPM04 CoreGrid.
Oct Active objects: programming and composing safely large-scale distributed applications Ludovic Henrio SCALE team, CNRS – Sophia Antipolis July.
The Grid Component Model and its Implementation in ProActive CoreGrid Network of Excellence, Institute on Programming Models D.PM02 “Proposal for a Grid.
Ramiro Voicu December Design Considerations  Act as a true dynamic service and provide the necessary functionally to be used by any other services.
October 2008, Nice Using ProActive to bridge Grid and SOA The AGOS project Frédéric Linder Program Director, Service Oriented Architecture.
Taking the Complexity out of Cluster Computing Vendor Update HPC User Forum Arend Dittmer Director Product Management HPC April,
1 Update Strategies for First Class Futures Khan Muhammad, Ludovic Henrio INRIA, Univ. Nice Sophia Antipolis,CNRS.
Formalism and Platform for Autonomous Distributed Components Bio-inspired Networks and Services A Distributed Component Model Formalisation in Isabelle.
Denis Caromel 1 Denis Caromel, et al. OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis, IUF Open Source.
Heavy and lightweight dynamic network services: challenges and experiments for designing intelligent solutions in evolvable next generation networks Laurent.
OASIS OASIS Active Objects, Semantics, Internet, and Security Team: 30 persons Research Themes: Grid Computing Objects and Components Practice and Theory.
Eric MADELAINE1 T. Barros, L. Henrio, E. Madelaine OASIS Team, INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis (FACS’05), Fractal workshop, Grenoble.
Tool Integration with Data and Computation Grid GWE - “Grid Wizard Enterprise”
Issues Autonomic operation (fault tolerance) Minimize interference to applications Hardware support for new operating systems Resource management (global.
ServiceSs, a new programming model for the Cloud Daniele Lezzi, Rosa M. Badia, Jorge Ejarque, Raul Sirvent, Enric Tejedor Grid Computing and Clusters Group.
1 Romain Quilici ObjectWeb Architecture meeting July 2nd 2003 ProActive Architecture of an Open Middleware for the Grid.
Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components.
Grid programming with components: an advanced COMPonent platform for an effective invisible grid © 2006 GridCOMP Grids Programming with components. An.
1. 2 Objects to Distributed Components (1) Typed Group Java or Active Object ComponentIdentity Cpt = newActiveComponent (params); A a = Cpt ….getFcInterface.
Denis Caromel 1 Denis Caromel, et al. OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis, IUF Strong Programming.
A Component Platform for Experimenting with Autonomic Composition A component framework for supporting composition of autonomic services and bio-inspired.
Grid Computing Framework A Java framework for managed modular distributed parallel computing.
ProActive components and legacy code Matthieu MOREL.
A scalable and flexible platform to run various types of resource intensive applications on clouds ISWG June 2015 Budapest, Hungary Tamas Kiss,
A visualisation and debugging tool for multi-active objects Ludovic Henrio, Justine Rochas LAMHA, Nov 2015.
Denis Caromel1 Denis Caromel, et al. OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis, IUF October
Eric MADELAINE ---- OASIS1 E. Madelaine Oasis team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis RESECO ’08 Santiago – Nov. 24, 2008 Specification.
1 OASIS Team, INRIA Sophia-Antipolis/I3S CNRS, Univ. Nice Christian Delbé Data Grid Explorer 15/09/03 Large Scale Emulation Mobility in ProActive.
1 OASIS Active Objects, Semantics, Internet, and Security Large Scale, Parallel and Distributed Systems Middleware, Programming Models & Semantics, Verification.
VERIFYING THE CORRECT COMPOSITION OF DISTRIBUTED COMPONENTS: FORMALISATION AND TOOL Ludovic Henrio 1, Oleksandra Kulankhina 1,2, Dongqian Liu 3, Eric Madelaine.
Tool Integration with Data and Computation Grid “Grid Wizard 2”
1 ProActive GCM – CCA Interoperability Maciej Malawski, Ludovic Henrio, Matthieu Morel, Francoise Baude, Denis Caromel, Marian Bubak Institute of Computer.
Eric MADELAINE -- GridComp -- OASIS 1 E. Madelaine (A. Cansado) GridComp project OASIS team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis GridComp.
Eric MADELAINE1 T. Barros, L. Henrio, E. Madelaine OASIS Team, INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis DCC, University.
1 Romain Quilici OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis, IUF July 2nd 2003 ProActive Architecture.
GCM/ProActive: a distributed component model, its implementation, and its formalisation Ludovic Henrio OASIS Team INRIA – UNS – I3S – CNRS Sophia Antipolis.
2. CALCULUS: A S P. A Theory of Distributed Objects D. Caromel, L. Henrio, Springer 2005, Monograph A Calculus: ASP: Asynchronous Sequential Processes.
Mobile Analyzer A Distributed Computing Platform Juho Karppinen Helsinki Institute of Physics Technology Program May 23th, 2002 Mobile.
Agenda 1. Background: INRIA, ActiveEon 2. ProActive Open Source: Programming, Scheduling, Resourcing 3. Cloud Seeding with GPU 4. UC: Genomics, Finance,
Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components.
Consulting Services JobScheduler Architecture Decision Template
Consulting Services JobScheduler Architecture Decision Template
The Grid Component Model and its Implementation in ProActive
Presentation transcript:

Agenda 1. Background: OASIS, ActiveEon 2. ProActive Overview 3. Programming (Components: GCM Standard) 4. Optimizing 5. Scheduling + Resourcing 6. SOA, SLA and QoS D. Caromel, et al. ProActive Parallel Suite: Multi-Cores to Clouds to Autonomicity Parallelism+Distribution with Strong Model: Speed & Safety

Key Objectives  Parallel Programming Model and Tools  desesperatly needed  for the masses  for new architectures (Multi-cores)  As Effective as possible:  Efficient  However Programmer Productivity is first KSF  For both Multi-cores and Distributed  Actually the way around  Handling of ``Large-scale’’ (Grid, Clouds)

Background

4 4 OASIS Team & INRIA  A joint team, Now about 35 persons  2004: First ProActive User Group  2009, April: ProActive 4.1, Distributed & Parallel: From Multi-cores to Enterprise GRIDs

5 5 OASIS Team Composition (35)  Researchers (5):  D. Caromel (UNSA, Det. INRIA)  E. Madelaine (INRIA)  F. Baude (UNSA)  F. Huet (UNSA)  L. Henrio (CNRS)  PhDs (11):  Antonio Cansado (INRIA, Conicyt)  Brian Amedro (SCS-Agos)  Cristian Ruz (INRIA, Conicyt)  Elton Mathias (INRIA-Cordi)  Imen Filali (SCS-Agos / FP7 SOA4All)  Marcela Rivera (INRIA, Conicyt)  Muhammad Khan (STIC-Asia)  Paul Naoumenko (INRIA/Région PACA)  Viet Dung Doan (FP6 Bionets)  Virginie Contes (SOA4ALL)  Guilherme Pezzi (AGOS, CIFRE SCP)  + Visitors + Interns  PostDoc (1):  Regis Gascon (INRIA)  Engineers (10):  Elaine Isnard (AGOS)  Fabien Viale (ANR OMD2, Renault )  Franca Perrina (AGOS)  Germain Sigety (INRIA)  Yu Feng (ETSI, FP6 EchoGrid)  Bastien Sauvan (ADT Galaxy)  Florin-Alexandru.Bratu (INRIA CPER)  Igor Smirnov (Microsoft)  Fabrice Fontenoy (AGOS)  Open position (Thales)  Trainee (2):  Etienne Vallette d’Osia (Master 2 ISI)  Laurent Vanni (Master 2 ISI)  Assistants (2):  Patricia Maleyran (INRIA)  Sandra Devauchelle (I3S) Located in Sophia Antipolis, between Nice and Cannes, Visitors and Students Welcome!

6 6  Co-developing, Support for ProActive Parallel SuiteProActive Parallel Suite  Worldwide Customers: Fr, UK, Boston USA Startup Company Born of INRIA

Multi-Cores 7

8 8 Symetrical Multi-Core: 8-ways Niagara II  8 cores  4 Native threads per core  Linux see 32 cores!

9 9 Sun 16-core Rock: Fall 2009  16 cores  4 native threads per core    64 “Cores” or “Native Threads” at OS level

10 Intel 8-cores, 16-thread Nehalem-based Xeon processor confirmed (Feb. 2009)  Highly NUMA  Not an SMP: L1, L2, then L3 attached to a given core

Multi-Cores A Few Key Points  Not Shared Memory (NUMA)  Moore’s Law rephrased: Nb. of Cores double every 18 to 24 months  Key expected Milestones: Cores per Chips (OTS)  2010: 32 to 64  2012: 64 to 128  2014: 128 to Million Cores Parallel Machines in M cores coming in 2020  Multi-Cores are NUMA, and turning Heterogeneous (GPU) They are turning into SoC with NoC: NOT SMP!

12 2. Overview ProActive Parallel Suite

13 2. Programming Optimizing Parallel Acceleration Toolkit in Java: - Java Parallelism + Legacy-Code Wrapping and Control - Scheduling and Resource Manager Multi-Core + Distributed Open Source Used in production by industry

14 OW2: Object Web + Orient Ware

15

16 ProActive Contributors

17 3. ProActive Programming: Active Objects

18

19

20 A ProActive : Active objects Proxy Java Object A ag = newActive (“A”, […], VirtualNode) V v1 = ag.foo (param); V v2 = ag.bar (param);... v1.bar(); //Wait-By-Necessity V Wait-By-Necessity is a Dataflow Synchronization JVM A Active Object Future Object Request Req. Queue Thread v1 v2 ag WBN!

21 Standard system at Runtime: No Sharing NoC: Network On Chip Proofs of Determinism

Key Point: Software Evolution  Distributed To Multicores  Multi-Cores: 32 (2010) to 64 to 128 to 256 (2014) Shift the execution from several multi-cores executing the same application simultaneously to a single, larger multi-core chip. An application requiring 128 cores to correctly execute, can be executed in 2012 on four 32 cores, and seamlessly executed in 2016 on a single 128-core chips  Smooth evolutivity of applications: Distributed and Multi-core Platforms

23 Standard system at Runtime: No Sharing NoC: Network On Chip Proofs of Determinism

Key Point: Locality will more than ever be Fundamental  Let the programmer control it  No global shared memory

25 TYPED ASYNCHRONOUS GROUPS

26 A Creating AO and Groups Typed Group Java or Active Object A ag = newActiveGroup (“A”, […], VirtualNode) V v = ag.foo(param);... v.bar(); //Wait-by-necessity V Group, Type, and Asynchrony are crucial for Composition JVM

27 Broadcast and Scatter JVM ag cg ag.bar(cg); // broadcast cg ProActive.setScatterGroup(cg) ; ag.bar(cg); // scatter cg c1 c2 c3 c1 c2 c3 c1 c2 c3 c1 c2 c3 c1 c2 c3 c1 c2 c3 s c1 c2 c3 s Broadcast is the default behavior Use a group as parameter, Scattered depends on rankings

28 Dynamic Dispatch Group JVM ag cg c1 c2 c3 c4 c5 c6 c7 c8c0 c9c1 c2 c3 c4 c5 c6 c7 c8c0 c9 c1 c2 c3 c4 c5 c6 c7 c8c0 c9 Slowest Fastest ag.bar(cg);

Abstractions for Parallelism The right Tool to do the Task right

30 ProActive Parallel Suite  Workflows in Java  Master/Workers  SPMD  Components  …

31 Components: GCM Standard 31

32 GridCOMP Partners

33 GCM Standardization Grid Component Model Overall, the standardization is supported by industrials: BT, FT-Orange, Nokia-Siemens, NEC, Telefonica, Alcatel-Lucent, Huawei …

34 Objects to Distributed Components Typed Group Java or Active Object V A Example of component instance JVM Truly Distributed Components IoC: Inversion Of Control (set in XML)

35 GCM Components Scopes and Objectives: Grid Codes that Compose and Deploy No programming, No Scripting, … No Pain Innovation: Abstract Deployment Composite Components Multicast and GatherCast MultiCast GatherCast

36 Denis Caromel 36 GCM Standardization Fractal Based Grid Component Model 4 Standards: 1. GCM Interoperability Deployment 2. GCM Application Description 3. GCM Fractal ADL 4. GCM Management API

Key Points about Parallel Components  Parallelism is captured at the Module interface Identical to Typing for functional aspects  Composition, parallel word, becomes possible  Configuration of the Parallel aspects

38

39 4. Optimizing

40

41 IC2D

42 IC2D

43 ChartIt

44 Pies for Analysis and Optimization

45 Video 1: IC2D Optimizing Monitoring, Debugging, Optimizing

46 5. Scheduling & Resourcing

47

48 ProActive Scheduling Big Picture RESOURCES  Multi-platform Graphical Client (RCP)  File-based or LDAP authentication  Static Workflow Job Scheduling, Native and Java tasks, Retry on Error, Priority Policy, Configuration Scripts,…  Dynamic and Static node sources, Resource Selection by script, Monitoring and Control GUI,…  ProActive Deployment capabilities: Desktops, Clusters, Clouds,… ProActive Scheduler ProActive Scheduler ProActive Resource Manager ProActive Resource Manager

49 Scheduler: User Interface

50 Job Another Example : Picture Denoising Split Denoise Merge with selection on native executable availability (ImageMagik, GREYstoration) Multi-platform selection and command generation with file transfer in pre/post scripts

51 Video 2: Scheduler, Resource Manager

52 Clusters to Grids to Clouds: e.g. on Amazon EC2

Node source Usecase : Configuration for external cloud with EC2 ProActive Scheduler ProActive Scheduler ProActive Resource Manager ProActive Resource Manager Dedicated resources LSF Static Policy Amazon EC2 EC2 Dynamic Workload Policy Desktops SSH Timing Policy 12/24

54 Video 3: Provisioning Resources from Clouds

55 6. SOA, SLA and QoS

56 AGOS: Grid Architecture for SOA  AGOS Solutions Building a Platform for Agile SOA with Grid In Open Source with Professional Support

57 Denis Caromel 57 AGOS Generic Architecture for Autonomic SOA with GRIDs & Clouds OS, HW OS VirtualizationGrid Utility interface ESB Enterprise Service Bus SCA Service Component Architecture Resource Manager Task & Services Scheduling Parallel Programming SPMD, workflow Agent, Master/Worker Fork and Join In memory db cache (JSR / JPI / javaspaces) SOA BPEL Exec Repository, Registry, Orchestration SOA Monitoring Reporting, Notifications, alarms Business Intelligence BI Monitoring SLM Service Level Management

58 Conclusion 58

59 Conclusion: An Acceleration Toolkit : Concurrency+Parallelism Multi-Core+Distributed