1. Polish Infrastructure for Supporting Computational Science in the European Research Space EUROPEAN UNION PL-Grid: National Grid Initiative in Poland Structure and Status Jacek Kitowski Institute of Computer Science AGH-UST ACK CYFRONET AGH in collaboration with PL-Grid Representatives HP-CAST Madrid, May 10-13, 2009 BigView

2.  PL-Grid Consortium  Systematic approach establishing the Consortium and NGI  Motivation  Rationales  Foundations  International Collaboration  PL-Grid Project in a nutshell  Status  PL-Grid Project  Infrastructure  Workpackages  SummaryOutline

3. Motivation  E-Science approach to research  Integration activities ongoing in the world 3

4. Need for e-Science Approach  E-Science: collaborative research supported by advanced distributed computations  Multi-disciplinary, Multi-Site and Multi-National  Building with and demanding advances in Computing/Computer Sciences  Goal: to enable better research in all disciplines  System-level Science: beyond individual phenomena, components interact and interrelate  to generate, interpret and analyse rich data resources From experiments, observations and simulations Quality management, preservation and reliable evidence  to develop and explore models and simulations Computation and data at all scales Trustworthy, economic, timely and relevant results  to enable dynamic distributed collaboration Facilitating collaboration with information and resource sharing Security, trust, reliability, accountability, manageability and agility M. Atkinson, e-Science (...), Grid2006 & 2-nd Int.Conf.e-Social Science 2006, National e-Science Centre UK I. Foster, System Level Science and System Level Models, Snowmass, August 1-2, 2007

5.  Thanks to CYFRONET activity five High Performance Computing Polish Centres analysed according to:  Participation in International and National Projects and Collaboration  Need by Polish Scientific Communities  Computational resources to date  European/Worldwide integration Activities  National Network Infrastructure ready (thanks to Pionier National Project) Rationales behind PL-Grid Consortium GEANT2

6.  ~35 international projects FP5, FP6, FP7 on Grids (50% common)  Grid Projects: Crossgrid, Gridstart, Unicore, EGEE I,II,III, K-WfGrid, Coregrid, Virolab, Gredia, int.eu.grid, Eurogrid, Grip, Unigrids, Balticgrid I,II, Gridlab, Intelligrid, Ringrid, Brein, Beingrid, Qoscosgrid, Chemomentum, EUChinagrid, PRACE,…  ~10 Polish projects (50% common)  Pionier, Progress, SGIgrid, Clusterix, NDS, Platon, IT-SOA, ….  Close collaboration with Institute of Computer Science AGH-UST Participation in Projects and Collaboration

7. Need by Polish Scientific Communities Scientific Community / Representative% Polish publications Warsaw / ICM29,0 % Cracow / Cyfronet16,4 % Wrocław / WCSS11,1 % Poznań / PCSS10,1 % Gdańsk / TASK6,8 % SUM 73,4 % Poland100 % Distribution of Polish publications in period 01.2004 – 04.2008 according to Science Citation Index Expanded + Social Science Citation Index + Arts & Humanities Citation Index

8. Partners’ Resources to Date TOP500 – Nov.2008 Polish Sites

9. National and International Integration Initiatives Ongoing European and Worldwide activities and consolidation  EGEE  EGI_DS, EGI  DEISA  PRACE ...... European e- Infrastructur e 2000 Testbeds 2010 Utility Service Routine Usage National International SGI Grid, Progress (Clusterix, National Data Store...) Chemomomentum, Virolab, CoreGrid, Gredia, int.eu.grid, Baltic Grid, GridLab, Porta Optica, RINGRid, Phosphorus, QoSCoSGrid, Intelligrid, K-WfGrid, Unicore...

10. Other Activities – Collaboration and Dissemination  EGI_DS – CYFRONET activity  Policy Board (deputy: M. Turala),  NGI Observers -- „Polish Experts” (WP3, WP5) participation in Workshops, resulted in EGI_DS draft deliverables, use-cases...  EGEE  From Cyfronet: chairing Resource Allocation Group in EGEE-III – trying to influence EGI Group has a mandate to manage current process of resource allocation in EGEE and propose better model for future Operate CE ROC More: https://twiki.cern.ch/twiki/bin/view/EGEE/RAG  E-IRG – CYFRONET activity  Workshops  Discussions on EGEE – EGI – NGI future  EU Unit F3 „Research Infrastructure” Experts in Program Committee, nominated by the Polish Ministry of Science and High Education (since 2007) from CYFRONET  ENPG seminar in Cracow (2008)  Others  CGW 2007 and 2008  Cyfronet Users’ Annual Conference (2008 and 2009)  Cyfronet Open Day (2007, 2008)  ICCS 2008 Conference

11. Other Activities  Computational Chemistry (VO) in EGEE  Third CPU power consumer Nearly 3 million of jobs executed during 2007  Lead by Cyfronet since 2006 Recently management of Computational Chemistry Cluster of Excellence  In parallel current effort include: Grid ports of commercial and non commercial software packages with particular focus on their parallel version Development of “experiment centric” grid web portal for chemists  EGEE III / CE ROC

12. EGEE-III / CE ROC EGEE-III / CE ROC CE ROC – coordinator: ACK CYFRONET-AGH Consortium consists of institutes from 7 countries: Austria (2), Croatia (3), Czech (1), Hungary (5), Poland (4), Slovakia (1), Slovenia (1) Operate, maintain and support EGEE Grid Infrastructure in CE region Virtual Organizations supported in Central European region: HEP, computational chemistry, biomedicine, pharmacology, astrophysics, earth science and regional users (VOCE VO) Poznan-PSNC Cracow-CYFRONET Warsaw-ICM cores storage Published by sites

13. PL-Grid Foundations – Summary  Motivation  E-Science approach to research  EGI initiative ongoing in collaboration with NGIs  Creation of Polish Grid (PL-Grid) Consortium: http://plgrid.pl  Consortium Agreement signed in January 2007  PL-Grid Project (2009-2012)  Application in Operational Programme Innovative Economy, Activity 2.3 (in Sept. 2008)  Get funded March 2, 2009 (via European Structural Funds)  Consortium made up of five largest Polish supercomputing and networking centres (founders)  ACK CYFRONET AGH (Cracow) – Coordinator Polish Infrastructure for Supporting Computational Science in the European Research Space Response to the needs of Polish scientists and ongoing Grid activities in Poland, other European countries and all over the world

14. PL-Grid Consortium Founders Academic Computer Center Cyfronet AGH (ACK CYFRONET AGH) Coordinator Poznań Supercomputing and Networking Center (PCSS) Wrocław Centre for Networking and Supercomputing (WCSS) Academic Computer Center in Gdańsk (TASK) Interdisciplinary Center for Math. and Computat. Modelling, Warsaw University (ICM) GEANT2

15. Grid infrastructure (Grid services) PL-Grid Application Clusters High Performance ComputersData repositories National Computer Network PIONIER Domain Grid Advanced Service Platforms Domain Grid  Assumptions  Polish Grid is going to have a common base infrastructure – similar to solutions adopted in other countries.  Specialized, domain Grid systems – including services and tools focused on specific types of applications – will be built upon this infrastructure.  These domain Grid systems can be further developed and maintained in the framework of separate projects.  Such an approach should enable efficient use of available financial resources.  Creation of a Grid infrastructure fully compatible and interoperable with European and World Grids thanks to cooperation with teams involved in the development of European Grid systems (EGEE, DEISA, OMII, C-OMEGA, ESFRI).  Plans for HPC and Scalability Computing enabled. PL-Grid Infrastructure Users National computer network Grid Application Programming Interface Virtual organizations and security systems Basic Grid services Grid services LCG/gLite (EGEE) UNICORE (DEISA) Other Grids systems Grid resource s Distributed computationa l resources Grid portals, development tools Distributed data repositorie s

16. The PL-Grid Project is split into several workpackages Planned Realization of Aims PLANNING AND DEVELOPMENT OF INFRASTRUCTURE P2 Coordination Structure Operation Rules Dissemination PROJECT MANAGEMENT P1 SECURITY CENTER P6 Training SUPPORT FOR VARIOUS DOMAIN GRIDS P5 P4 GRID SOFTWARE AND USERS TOOLS DEVELOPMENT EGEEDEISA …. OPERATIONS CENTER P3 Main Project Indicators: Peak Perf.: 215 Tflops Disk Storage: 2500 TB

17. Challenges  Short term  To start – establishing PL-Grid VO using Partners’ local and EGEE resources  To provide resources for covering operational costs  To select computational/storage resources for PL-Grid infrastructure  Long term – continously  To provide the necessary infrastructure (!!) Computer rooms, electrical power, many organizational issues  Be prepared / work on approaching paradigms and integration development HPC – Scalable Computing Clouds (internal-external, computing clouds, data clouds) SOA paradigm, knowledge usage …

18. Thank you

19. Polish Infrastructure for Supporting Computational Science in the European Research Space EUROPEAN UNION PL-Grid: National Grid Initiative in Poland Structure and Status Jacek Kitowski Institute of Computer Science AGH-UST ACK CYFRONET AGH in collaboration with PL-Grid Representatives HP-CAST Madrid, May 10-13, 2009 Technical Issues

20. PL-Grid Foundations – Summary  Motivation  E-Science approach to research  EGI initiative ongoing in collaboration with NGIs  Creation of Polish Grid (PL-Grid) Consortium: http://plgrid.pl  Consortium Agreement signed in January 2007  PL-Grid Project (2009-2012)  Application in Operational Programme Innovative Economy, Activity 2.3 (in Sept. 2008)  Get funded March 2, 2009 (via European Structural Funds)  Consortium made up of five largest Polish supercomputing and networking centres (founders)  ACK CYFRONET AGH (Cracow) – Coordinator Polish Infrastructure for Supporting Computational Science in the European Research Space Response to the needs of Polish scientists and ongoing Grid activities in Poland, other European countries and all over the world

21. Grid infrastructure (Grid services) PL-Grid Application Clusters High Performance ComputersData repositories National Computer Network PIONIER Domain Grid Advanced Service Platforms Domain Grid  Assumptions  Polish Grid is going to have a common base infrastructure – similar to solutions adopted in other countries.  Specialized, domain Grid systems – including services and tools focused on specific types of applications – will be built upon this infrastructure.  These domain Grid systems can be further developed and maintained in the framework of separate projects.  Such an approach should enable efficient use of available financial resources.  Creation of a Grid infrastructure fully compatible and interoperable with European and World Grids thanks to cooperation with teams involved in the development of European Grid systems (EGEE, DEISA, OMII, C-OMEGA, ESFRI).  Plans for HPC and Scalability Computing enabled. PL-Grid Infrastructure

22. Elements and Functionality  PL-Grid software will comprise:  user tools (portals, systems for applications management and monitoring, result visualization and other purposes, compatible with the lower-layer software used in PL-Grid);  software libraries;  virtual organization systems: certificates, accounting, security, dynamic ;  data management systems: metadata catalogues, replica management, file transfer;  resource management systems: job management, applications, grid services and infrastructure monitoring, license management, local resource management, monitoring. Users National computer network Grid Application Programming Interface Virtual organizations and security systems Basic Grid services Grid services LCG/gLite (EGEE) UNICORE (DEISA) Other Grids systems Grid resources Distributed computational resources Grid portals, development tools Distributed data repositories  Three Grid structures will be maintained:  production,  reseach,  testing.

23. The PL-Grid Project is split into several workpackages Planned Realization of Aims PLANNING AND DEVELOPMENT OF INFRASTRUCTURE P2 Coordination Structure Operation Rules Dissemination PROJECT MANAGEMENT P1 SECURITY CENTER P6 Training SUPPORT FOR VARIOUS DOMAIN GRIDS P5 P4 GRID SOFTWARE AND USERS TOOLS DEVELOPMENT EGEEDEISA …. OPERATIONS CENTER P3 Main Project Indicators: Peak Perf.: 215 Tflops Disk Storage: 2500 TB

24. Management and Organization

25. WP2: Planning of Infrastructure Development  Analysis of users’ requirement  Analysis of worldwide activity in infrastructure development  Expecting results: Current activity: organization of pilot PL-Grid VO using EGEE infrastructure From analysis: clusters with x86 processors

26. WP3: Operations Center’s tasks EGI Group EGI Testing Middleware EGI Production Middleware  Coordination of Operation  Management and accounting  EGI and DEISA collaboration (HPC and Scalability Computing)  Users’ requirements  More spectacular (using researchand test infrastructures)  Computational Cloud  Data Cloud  Internal and External Clouds

27. WP4: Grid Software and Users Tools  Workpackage on Development/Research Analysis of users’ requirements Software repository (different tools and components developed by Partners) Reengineering of tools and applications (legacy codes, APIs, SOA applications) Virtual Laboratory (component applications, workflows, applications monitoring) – from EU FP6 Virolab Project High-level virtual organizations using knowledge (semi-automatic creation using contract, semantic description of resources, monitoring of SLA due to QoS), data access – FiVO from EU FP6 Gredia Project Tools for management, proactive monitoring and security (Bazaar, Permis)

28. WP5: Support and Training  Users’ support  Making commercial software available for the users (license activity)  Training, education

29. WP6: Security  Users’ support Authorization, authentication,... Certificates...

30. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 t.szepieniec@cyfronet.pl Gaussian VO created supported by several Partners (EGEE activity) Accepted by Vendor Registration: https://voms.cyf-kr.edu.pl:8443/vo/gaussian/vomrs GAUSSIAN in Grid VO Manager: Mariusz Sterzel (CYFRONET), EGEE II Comput. Chemistry coordinator m.sterzel@cyfronet.pl ACK: Tomasz Szepieniec

31. Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 t.szepieniec@cyfronet.pl Never Born Protein Folding ACK: Irena Roterman, Jagiellonian University, Tomasz Szepieniec

32. Activities  Short term  To start – establishing PL-Grid VO using Partners’ local and EGEE resources  To provide resources for covering operational costs  To select computational/storage resources for PL-Grid infrastructure  Long term – continously  To provide the necessary infrastructure (!!) Computer rooms, electrical power, many organizational issues Towards Cloud Computing (internal, external) Towards HPCaaS and Scalable computing