SEE-GRID-2 The SEE-GRID-2 initiative is co-funded by the European Commission under the FP6 Research Infrastructures contract no WP5 Dragan Okiljević Belgrade University Computing Center (RCUB) Aplikacije

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, SEE-GRID-1 project Objectives  Establish at least 1 fully operational grid site in all participating SEE countries  Each contractor to set up at least one site; third-parties to set up additional sites  Deploy at least 2 Grid applications developed by EGEE (one from the HEP, one from Biomed) in the regional infrastructure and demonstrate their usage over the regional infrastructure  Deploy at least two Grid applications developed by SEE-GRID partners in the regional infrastructure.  Create a Human Network in eInfrastructures and raise awareness in the wide R&E community per country

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, SEE-GRID-1 Applications Regionals applications  VIVE – Volumetric Image Visualisation Environment  SE4SEE – Search Engine For SEE

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, VIVE - Introduction Objective Complete interactive analysis of 3D datasets from medical imaging devices (CT, MR, PET, SPECT…) Easy remote access over the Internet Generalized 3D tools for diagnosis, surgical planning, and therapy evaluation Virtual distant examination based on 3D datasets Basic idea Split the system into a client and server part Dedicate the server part to access and processing of volumetric data Produce a lightweight client for user interaction and 3D rendering Exploit grid resources and paradigms to overcome implementation issues

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro,

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, Screnshots - From Atoms to Galaxies

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, What Can We Gain From the Grid? Distribution of processing and data over several nodes  Grid submission used to allocate server jobs  Allocating several machines for texture and surface mesh creation  Field stripping is not necessary for the regular medical images  Grid reduces overall demand for CPUs even with one server job per session  Could build distributed virtual data centers, with data abstracted within electronic health record Data transfer and replication  Replication of data to locations close to computation elements  Remote file access Security – high sensitivity of medical data  Data encryption in transfer (GridFTP, dCache, RFIO and gsidcap)  Anonymisation - removal of personal details to prevent intrusion at machine level, solved by gLite Hydra Application-specific monitoring  Could select a site from performance indicators (responsiveness, bandwidth, round trip time, processing speed) provided by the grid  Currently we just collect and display monitoring data using in our implementation guidelines and ideas from various grid/network monitoring projects  Fault tolerance

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, SE4SEE - Introduction Grid-enabled crawling of all Web pages in SEE countries Grid-enabled categorization of these Web pages A navigational search facility over the classified content To sum up, we offer : A grid-enabled, directory-based search engine that will enhance the cultural integration in the region

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, SE4SEE - Grid-Enabled Web Crawling The size of the Web is enormous; Web crawling requires:  high download rates over the network  excessive storage (both memory and disk)  vast amount of processing power It is hard to crawl the Web by a centralized system Solution: Grid-enabled Web crawling since  geographically distributed grid nodes allow fast download rates and dispersion of the network load  excessive disk and memory capacities together with the huge CPU power of GRID provides the necessary medium for storing and processing the vast Web content

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, (Backup) SE4SEE Web Portal

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, (Backup) SE4SEE Web Portal

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, (Backup) SE4SEE Web Portal

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, (Backup) SE4SEE Web Portal Grid-Enabled Web Crawling

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, SEE-GRID-2 Application selection Large number of potential applications For the reason of scalability, it was decided that only a subset of the applications will be supported Candidate application developers filled online questionnaire submitting data on their applications Application ranking criteria developed jointly trough e- mail discussion within the consortium WP4 partners from all countries. 32 applications in total were submitted initially. 23 were assessed with the questionnaire.

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, SEE-GRID 2 Application statistics (1/2)

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, SEE-GRID 2 Application statistics (2/2)

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, Application selection for 1st round One application selected by country – “flagship applications” Chosen applications covers a broad range of applications fields and scientific communities

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, Selected applications for 1st round (1/2)

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, Selected applications for 1st round (2/2)

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, SEE-GRID-2 Application support Application support group ( ASG ) – experienced developers & admins, mostly participants of SG1 project  National level application support  SG - global level application support Work in close collaboration with WP5 (training) and WP3 (software requirements, maintenance of performance)

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, Application gridification guide Relevant topics for application developers identified trough online questionnaire system Some investiogation areas identified as well – candidates for future GG topics Gridification guide will provide information on these topics G G collaboration medium – Wiki

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, Application Lifecycle

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, Developer Resources Grid environment is constantly evolving, but  Useful features persist  New are constantly being added  Bugs are being fixed  Gained knowledge remains relevant, must be updated  Applications can be easily migrated to new/updated APIs gLite User Guide  SEEGRID Wiki 

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, Software/Job Management Software installation management using ESM tools Experiment Software Installation in LCG-2  SEEGRID Wiki “SEE-GRID ESM Software Installation Guide”   Site configuration  Using ESM tools  Example JDL and shell scripts for installation, validation, and removal Job management EDG WMS API (LCG-2) – C++ & Java  index.html Exec/spawn of shell scripts that create and submit JDL – simple, but no direct feedback on jobs gLite 

SEEGRID PSC 03 Meeting – Podgorica, Monte Negro, Data Management LCG-2 (LCG-2 User Guide, “man” pages)  LCG-UTILS API – C/C++  LFC API – C/C++, Python  GFAL API – C/C++, Python  SEEGRID Wiki “SG Using file replicas and RFIO: UI configuration, rfiod, usage in apps, limitations and workarounds”  see.org/index.php/SG_Using_file_replicas_and_RFIO:_UI_configuration%2C_rfiod% 2C_usage_in_apps%2C_limitations_and_workarounds  Configuring UI, SE, RB  Site testing of RFIO/GFAL  Typical problems and solutions  Java access to LFC and LCG-UTILS SEE-GRID File Management Java API  Customizable LFC web front end (upload, list, replicate, delete) 