5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Development of GRID environment for interactive applications J.Marco (CSIC) DATAGRID WP6 MEETING (PARIS) 5 - III-2002

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) The CROSSGRID project EU V th Framework Programme (IST) –3 year project, starting date 1st March 2002 –4.86 M € contract signed last week! Objectives: –Final user interactive applications –Extend GRID into 11 EU countries in close collaboration with DATAGRID 21 Partners: –COORD: CYFRONET (PL, coordinator M.Turala) –CR: UvA (N), PSNC(PL), FZK (D),CSIC (E) –Companies: Algosystems(GR) + DATAMAT (I) –ICM,INP,IPJ (PL) + IISAS (SK) + USTUT+TUM (D) + ULINZ (A) + TCD(Ir) UAB+USC (E) + LIP (P) + Demo+AuTH (Gr) + UCY (Cy)

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) CROSSGRID Collaboration Poland: Cyfronet & INP Cracow PSNC Poznan ICM & IPJ Warsaw Portugal: LIP Lisbon Spain: CSIC Santander Valencia & RedIris UAB Barcelona USC Santiago & CESGA Ireland: TCD Dublin Italy: DATAMAT Netherlands: UvA Amsterdam Germany: FZK Karlsruhe TUM Munich USTU Stuttgart Slovakia: II SAS Bratislava Greece: Algosystems Demo Athens AuTh Thessaloniki Cyprus: UCY Nikosia Austria: U.Linz

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Project Structure Workpackages: –Applications (P.Sloot, UvA) –Programming Environment (H.Marten, FZK) –New Services & tools (N.Meyer, PSNC) –Testbed (J.Marco, CSIC) –Management (M.Turala) Architecture Team –Lead by M.Bubak, Cyfronet

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Applications Final user interactive applications: –Response loop in the range of seconds to minutes Computing & Data intensive Architecture basis: GLOBUS + DATAGRID Multidisciplinary: –Simulation & visualisation for surgical procedures –Air pollution combined with weather forecasting –Distributed Data Analysis in HEP –Flooding crisis team decision support system

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Tools & Services GRID Application Programming Environment –Verification of MPI use –Performance prediction –Monitoring Services & Tools –User friendly portals –Roaming access –Efficient distributed data access –Specific resource management Open Architecture approach

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Testbed Testing and validation for –Applications –Programming environment –New services & tools Emphasis on collaboration with DATAGRID + extension to DATATAG Extension of GRID across Europe

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Extension of GRID

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Application development Common requirements: –Near real-time performance across the network –Efficient access to large distributed databases –Large scale parallel simulations –Data discovery –Data mining –Visualisation Deployment on the CROSSGRID testbed will test the applications in the final user environment and provide feedback WP2 WP3 MPI Adressed in collaboration between WP1 tasks

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Distributed Physics Analysis in HEP Final user applications for interactive physics analysis in a GRID aware environment for LHC experiments Acess to large O/R DBMS through middleware server vs access to catalog + distributed root data files Distributed data-mining techniques (mainly Neural Networks, also self organizing maps,...) Integration of user-friendly interactive acess via portals CSIC + UAB + FZK + INP + ICM + IPJ

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) HEP Interactive Physics Analysis MPI Master Scalability: OK in local cluster…

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) WP4: International Testbed Organisation Objective: –Provide a distributed resource facility –Support testbed sites across Europe, integrating national facilities –Assure interoperability with DATAGRID testbed 15 testbed sites across 9 countries ~16 FTE working on them Start points: –Join DataGRID testbed gradually –Use GEANT for network connectivity –Extend testbed with incremental releases as applications and middleware demand and provide feedback Effort coordinated by CSIC, will benefit from experience in DATAGRID testbed also in TCD, UAB, LIP

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Testbed MAP Auth Thessaloniki UvA Amsterdam FZK Karlsruhe TCD Dublin UAB Barcelona LIP Lisbon CSIC IFIC Valencia CSIC Madrid USC Santiago CSIC IFCA Santander DEMO Athens UCY Nikosia CIFRONET Cracow II SAS Bratislava PSNC Poznan ICM & IPJ Warsaw

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Network (Geant) setup CrossGrid WP4 - International Testbed Organisation UCY NikosiaDEMO Athens Auth Thessaloniki CYFRONET Cracow ICM & IPJ Warsaw PSNC Poznan II SAS Bratislava FZK Karlsruhe UvA Amsterdam CSIC Valencia UAB Barcelona CSIC Santander CSIC Madrid LIP Lisbon USC Santiago TCD Dublin

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Tasks in WP4 4.0 Coordination and management (task leader: J.Marco, CSIC, Santander) –Coordination with WP1,2,3 –Collaborative tools (web+videoconf+repository) –Integration Team 4.1 Testbed setup & incremental evolution (task leader:R.Marco, CSIC, Santander) –Define installation –Deploy testbed releases –Trace security issues CrossGrid WP4 - International Testbed Organisation Testbed site responsibles: –CYFRONET (Krakow) A.Ozieblo –ICM(Warsaw) W.Wislicki –IPJ (Warsaw) K.Nawrocki –UvA (Amsterdam) D.van Albada –FZK (Karlsruhe) M.Kunze –IISAS (Bratislava) J.Astalos –PSNC(Poznan) P.Wolniewicz –UCY (Cyprus) M.Dikaiakos –TCD (Dublin) B.Coghlan –CSIC (Santander/Valencia) S.Gonzalez –UAB (Barcelona) G.Merino –USC (Santiago) A.Gomez –Demo (Athenas) C.Markou –AuTh (Thessaloniki) D.Sampsonidis –LIP (Lisbon) J.Martins

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Tasks in WP4 4.2 Integration with DATAGRID (task leader: M.Kunze, FZK) –Coordination of testbed setup –Exchange knowledge –Participate in WP meetings 4.3 Infrastructure Support (task leader: J.Salt, CSIC, Valencia) –Fabric management –HelpDesk –Provide Installation Kit –Network support 4.4 Verification & quality control (task leader: J.Gomes, LIP) –Feedback –Improve stability of the testbed CrossGrid WP4 - International Testbed Organisation

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Workflow and interfaces CrossGrid WP4 - International Testbed Organisation Setup report on requirements & planning 4.0 Coordination and management (PM1-36) Plan Evolution: Prototype 0FINAL WP 1,2,3 WP 1 requirements from 1st prototype releaset final testbed D4.5D4.5 D4.1 M D4.7D4.7 testbed setup on selected sites 6 D4.2 D4.4D4.4 Feedback from Feedback mainly from PU M4.3M4.3 internal progress report CO D4.6D4.6 internal progress report CO M4.4 final demo + report PU First testbedSupport Prototype 1 Integration Team 10 internal progress report DATAGRID testbed Feedback from D4.3D4.3 CO M4.2M4.2

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Questions… Interactive use of resources: –Priorities/Allocation/Booking policies How to allocate 500 machines for 5 minutes of interactive use? –“Connection”: Concept of Interactive Session? Use of Permanent and Volatile storage –Scheduler? Strongly data-location aware –Dynamic versioning/installation? Software configuration for different applications

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Testbed Organization: Use national certificates as in DataGrid: –Spain, Portugal, Ireland, Germany, Poland –Greece, Cyprus, Slovakia… V.O.: –Define CROSSGRID testbed V.O. –Join DataGrid HEP V.O. (Alice,Atlas,CMS,LHCb) –Define new V.O. for other applications: Med V.O. Flood Prevention V.O. Meteo V.O.

5 March 2002 DG PARIS Jesus Marco CSIC IFCA(Santander) Testbed Organization: Kick-off meeting in Krakow (17 th March) First three months: –Review and setup local infraestructure –Gather requirements from applications/middleware –Setup HelpDesk, define Integration Team –Reinforce connection with Datagrid: Extend testbed (Germany,Poland,Portugal,Spain) Find topics for contribution