1. 1 CHEP 2000, 10.02.2000Roberto Barbera Roberto Barbera (*) GENIUS: a Web Portal for DataGRID ACAT 2002, Moscow, 24-28.06.2002 (*) work in collaboration with A. Andronico, A. Falzone and A. Rodolico

2. 2 Roberto Barbera Outline The Grid Vision: the concept of Collaboratory Grid activities in Europe: the DataGrid Project The main “actors” of the DataGrid Project “Easy” ed “ubiquitous” access to the Grid: the GENIUS web portal Conclusions and perspectives Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration ACAT 2002, Moscow, 24-28.06.2002

3. 3 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration ACAT 2002, Moscow, 24-28.06.2002

4. 4 GRID Projects – EU IST (~37 M€) An integrated approach Science Industry / business Applications Middleware & Tools Underlying Infrastructures GRIDSTART CROSSGRID DATAGRID DATATAG GRIDLAB EGSO GRIA GRIP EUROGRID DAMIEN iVDGL ACAT 2002, Moscow, 24-28.06.2002

5. 5 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration CERN – International (Switzerland/France) CNRS - France ESA/ESRIN – International (Italy) INFN - Italy NIKHEF – The Netherlands PPARC - UK DataGRID main contractors ACAT 2002, Moscow, 24-28.06.2002

6. 6 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Research and Academic Institutes CESNET (Czech Republic) Commissariat à l'énergie atomique (CEA) – France Computer and Automation Research Institute, Hungarian Academy of Sciences (MTA SZTAKI) Consiglio Nazionale delle Ricerche (Italy) Helsinki Institute of Physics – Finland Institut de Fisica d'Altes Energies (IFAE) - Spain Istituto Trentino di Cultura (IRST) – Italy Konrad-Zuse-Zentrum für Informationstechnik Berlin - Germany Royal Netherlands Meteorological Institute (KNMI) Ruprecht-Karls-Universität Heidelberg - Germany Stichting Academisch Rekencentrum Amsterdam (SARA) – Netherlands Swedish Research Council - Sweden Industrial Partners Datamat (Italy) IBM-UK (UK) CS-SI (France) Other DataGRID partners ACAT 2002, Moscow, 24-28.06.2002

7. 7 EDG overview : work packages EDG Project is structured in 12 Work Packages: WP1: Work Load Management System WP2: Data Management WP3: Grid Monitoring / Grid Information Systems WP4: Fabric Management WP5: Mass Storage Management WP6: Testbed and demonstrators WP7: Network Monitoring WP8: High Energy Physics Applications WP9: Earth Observation WP10: Biology WP11: Dissemination WP12: Management Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration } } Middleware } } Applications Roberto Barbera ACAT 2002, Moscow, 24-28.06.2002

8. 8 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Computational biology ACAT 2002, Moscow, 24-28.06.2002

9. 9 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Medical Diagnostic Imaging ACAT 2002, Moscow, 24-28.06.2002

10. Earth Observation Community GRID interactive scenario Common access to EO missions catalogues Acquisition plan, order, delivery Parametric data fusion and models integration Collaborative publishing of results On demand high level products generation Roberto Barbera ACAT 2002, Moscow, 24-28.06.2002

11. 11 ENVISAT (launched 01.03.2002 !) 3500 MEuro programme cost 10 instruments on board 10 instruments on board 200 Mbps data rate to ground 200 Mbps data rate to ground 400 Tbytes data archived/year 400 Tbytes data archived/year ~100 “standard” products ~100 “standard” products 10+ dedicated facilities in Europe 10+ dedicated facilities in Europe ~700 approved science user projects ~700 approved science user projects 3500 MEuro programme cost 10 instruments on board 10 instruments on board 200 Mbps data rate to ground 200 Mbps data rate to ground 400 Tbytes data archived/year 400 Tbytes data archived/year ~100 “standard” products ~100 “standard” products 10+ dedicated facilities in Europe 10+ dedicated facilities in Europe ~700 approved science user projects ~700 approved science user projects Earth Observation Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Roberto Barbera

12. 12 http://www.cern.ch ~9 km LHC SPS CERN Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration High Energy Physics ACAT 2002, Moscow, 24-28.06.2002

13. 13 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration High Energy Physics ACAT 2002, Moscow, 24-28.06.2002

14. 14 ATLASCMS LHCb Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration ~6-8 PetaBytes / year ~O(10 8 ) events/year ~O(10 3 ) batch and interactive users High Energy Physics ACAT 2002, Moscow, 24-28.06.2002

15. 15 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration 1/100 of a Pb+Pb @ LHC ! Simulation and reconstruction of a “full” (central) Pb+Pb collision at LHC (about 84000 primary tracks!) takes about 24 hours of a top-PC and produces an output bigger than 2 GB. ACAT 2002, Moscow, 24-28.06.2002

16. 16 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration 5 times the Eiffel Tower ˜1500 m 6-8 Petabytes ˜10.000.000 CD-ROM ACAT 2002, Moscow, 24-28.06.2002

17. 17 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Yerevan CERN Saclay Lyon Dubna Capetown, ZA Birmingham Cagliari NIKHEF GSI Catania Bologna Torino Padova IRB Kolkata, India OSU/OSC LBL/NERSC Merida Bari Nantes ALICE “grid” sites ACAT 2002, Moscow, 24-28.06.2002

18. 18 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Europe: 267 institutes, 4603 users Elsewhere: 208 institutes, 1632 users The LHC “web” in the world ACAT 2002, Moscow, 24-28.06.2002

19. 19 Roberto Barbera ACAT 2002, Moscow, 24-28.06.2002

20. Job Submission work-flow UI JDL Logging & Book-keeping ResourceBroker Job Submission ServiceStorageElementComputeElement InformationService Job Status ReplicaCatalogue DataSets info Author. &Authen. Job Submit Event Job Query Job Status Input “sandbox” Input “sandbox” + Broker Info Globus RSL Output “sandbox” Job Status Publish grid-proxy-init Expanded JDL SE & CE info Roberto Barbera ACAT 2002, Moscow, 24-28.06.2002

21. 21 Roberto Barbera EDG m/w has been released but… EDG software (Globus, UI, JDL, WP2, WP3, etc.) contains tens of commands/switches which also have their own logical sequences (“B” after “A”, “C” before “D” and so on). Browsing Grid VO “directories” (users, RC’s, DB’s, etc.) requires LDAP “speaking” and tomorrow could require SQL “speaking”. “User gridification” is a tough task for a “rookie”  this does not fit with the claim that “grids” are for everybody and that grid computing will be as easy as surfing the Internet ? Furthermore, all this holds for DataGrid. What will happen when other grids’ software (especially UI’s) will come up (PPDG, iVDGL, etc.) ? Will users have to learn tens of “grid dialects” ? Today “grid computing” is a rather complicated experience only possible at selected machines (UI’s)  this does not fit with the claim that one could do “grid computing” even from a PDA ? Is there any way to set-up a “user-friendly” grid ? Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration ACAT 2002, Moscow, 24-28.06.2002

22. 22 Roberto Barbera A web portal: why and how ? It can be accessed from everywhere and by “everything” (desktop, laptop, PDA, WAP phone). It can keep the same user interface to several back-ends (grid “dialects”  command-line UI’s). It must be redundantly “secure” at all levels: 1) secure for web transactions, 2) secure for user credentials, 3) secure for user authentication, 4) secure at VO level. All available grid services must be incorporated in a logic way, just “one mouse click away”. Its layout must be easily understandable and user friendly. Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration ACAT 2002, Moscow, 24-28.06.2002

23. 23 Roberto Barbera GENIUS ® (Grid Enabled web eNvironment for site Independent User job Submission) [https://genius.ct.infn.it]https://genius.ct.infn.it Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration OS & Net services Basic Services High level GRID middleware ALICEATLASCMSLHCb Applications’ specific layer Other apps GLOBU S toolkit DataGRID architecture GENIUS web portal ACAT 2002, Moscow, 24-28.06.2002

24. 24 Roberto Barbera GENIUS : how it works Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Apache EnginFrame GENIUS https+java/xml+rfb WEB Browser ED G UI Loc al WS the Grid EDG+GSI 3-tier model

25. 25 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: the main page

26. 26 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: the authentication

27. 27 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: file services

28. 28 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: the authorization

29. 29 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: security services

30. 30 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: job submission

31. 31 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: job submission

32. 32 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: job queue

33. 33 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: job output

34. 34 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: personal spooler

35. 35 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: job data

36. 36 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: interactive analysis

37. 37 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: the VO services

38. 38 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: the VO services

39. 39 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: monitoring services

40. 40 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: monitoring services

41. 41 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: monitoring services

42. 42 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration GENIUS show: monitoring services

43. 43 Roberto Barbera Present status and perspectives Current implementation of GENIUS already includes: secure web transactions, user authentication and authorization; remote interaction with the user’s file system; interfaces for job submission/control, to VO servers (users’ and catalogues), and to monitoring systems; persistent (user’s) book-keeping and spooler system; interactive analysis ! rpm available ! Todo: multi-jobs (parallel and sequential); interface to data management and other grid services; more application-specific customizations; web-guided creation of a work flow system. Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration

44. 44 Roberto Barbera GENIUS vs. other grid portals It is not a toolkit. It is a complete production-ready environment which combine the concepts of “user portal” and “science portal”. No client software needs to be installed apart from the web browser. GENIUS can be accessed from everywhere. No security delegation (“à la MyProxy”) is needed. Access passwords are securely “streamed” only when needed. Interactive analysis (via VNC) and web access to personal spooling areas are possible. User file system is not limited to input and output files. EnginFrame modularity makes different customizations easy to implement. Already available for EDG m/w, GLOBUS, and LSF. Under definition for CONDOR (hungarian grids). It is compatible with the Tomcat open source java servlet container available from SUN. Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration

45. 45 Roberto Barbera Conclusions Computational grids could represent the “natural” environment for next generation high energy physics experiments, computational bio-medecine, Earth observation and many other inter-disciplinary applications. “Grid” could be the Internet “new age” where users can seamlessly and ubiquitously access not only information but also huge computing resources and mass storage systems distributed worldwide with their own applications. However, in order to turn dreams into reality, grid access must be easy and intuitive especially for the vast majority of non-expert users and this is the mandate of the GENIUS team. Live demos of GENIUS at work could be made for interested people during breaks in the e-mail reading area. Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration ACAT 2002, Moscow, 24-28.06.2002