DORII review Remote instrumentation communities and application support NA3 Roberto Pugliese Sincrotrone Trieste SCpA July 3, 2009, Brussels

 Workpackage introduction  Workpackage objectives  Workpackage deliverables  Workpackage overall schedule  Workpackage achievements  Workpackage future plans  Workpackage Summary Outline

 Remote instrumentation communities and application support  Give support for researchers for adaptation and integration of their applications in the extended e- Infrastructure NA3 Objectives

Connection with other WPs

Tasks and subtasks

Deliverables

Schedule

NA3.1NA3.2NA3.3NA3.4NA PMS110 PMs26 PMs12 PMs16 PMs Effort distribution

 Earthquake community  Network-centric seismic simulations  Environment community  Oceanographic and coastal observation and modelling Mediterranean Ocean Observing Network  Oceanographic and coastal observation and modelling using imaging  Experimental science community  On-line data analysis in experimental science Communities and pilot apps

 Earthquake community  Earthquake early warning system design and simulation  Environment community  Monitoring inland waters and reservoirs  Experimental science community  On-line data analysis in experimental science (other use cases) Communities and not pilots

 Described to the communities the existing e- Infrastructure  Requirement analysis for pilot and not pilot applications and evaluation metrics definition  Match the applications in the Instrument Infrastructure Architecture (VCR, CE, SE, IE, … )  For each application (focus on pilots)  defined a todo list (wbs, … )  defined a plan  constantly monitor application status and progress Achievements

Architecture

 DORII applications description  Requirements  General, instrumentation,  Data, storage, computing,  Visualisation, interactivity,  Networking, collective  Metrics definition  production, usability  Perceived gain (performance, reach, … )  Requirements template  Specific requirements Requirements

 Status of the VCR  Status of the VCR personalisation  Status of the IE  Status of IE personalisation (IM implementation)  Status of CE (porting of the application code)  Status of the SE (connection with the IE)  Status of the collective behaviour (workflow and connection of components) Application status

 We meet regularly every week via EVO  Meetings are focused on application short term and long term needs  Meetings try to give answers to specific application problems  Meetings have an agenda  Meetings have minutes with specific action items  We have installed an online survey tool to collect feedback and perform evaluations Teamwork achievements

Online survey

 Involves instruments computing, storage not yet collectively used, it is using he application manager and mpi  Next developments glider integration, separated VCR for personalisation and better performance, visualisation of NetCDF files OGS Pilot Application

 Involves instruments, computing, storage, not yet collectively used, application manager, example of system level science similar to the NEESGrid  Next developments other instruments integration, separated VCR for personalisation and better performance EUCENTRE Pilot Application

 Involves instruments, computing, storage not yet collectively used, it is using the application manager  Next developments more imaging algorithms, separated VCR for personalisation and better performance UC Pilot Application

(interactive sci.app: fit2D) (BL instrument) (VCR) (user in ELETTRA) Online/GRID: CE,SE,etc ELETTRA SAXS Application

 Earthquake early warning system design and simulation  Instrument integration in progress  Monitoring inland waters and reservoirs  Instrument integration in progress and modelling almost done  Experimental science community  Syrmep application under development; impressive tomography reconstruction  XRD yet to start, requirement for taverna integration Status of not pilot apps

 It has been developed to demonstrate and DORII (ICT2008, INGRID2009) and for training purposes actually involves only instruments, if we include the remote camera than we can demonstrate also instrument, storage integration  Next developments computing integration and permanent setup Robot Demo Application

 Experimental procedure  Focus on end users  Access to the DORII infrastructure via VCR  For each pilot application we have defined  list of key users (not involved in DORII)  a set of scenarios  a task list for each scenario  Each key user will first receive a basic training on DORII and the use of the VCR  After the training each key user will execute the scenario  After executing the scenario each user will fill an online questionnaire Early deployment evaluation

 Laboratory studies  Subjective assessment  General assessment  Specific feature assessment  Specific task assessment  Regults  Overall rating good  Security perceived too complex  VCR perceived as slow  Improvement required for reliability, application manager, visualisation capabilities, automatisation, metadata support, help and training  Evaluation process ok Evaluation results

 Need for a new VCR  Need for a better application manager  Need for a Common Library  Need for a workflow management system  Need to have a customized VCR at least for each community  Need to simplify security (robot certificates)  Need for better training material  Need for a summer school Feedback to JRA and SAs

 When the first release of the middleware will be available we will adapt pilot applications and port the other applications to production and operational use  We will provide further feedback and finalize the e-Infrastructure Future plans

 Papers, presentations and demonstrations have been included in the dissemination workpackage  VCR for all SRF users (SaaS)  MADBAG meteo and air quality data acquisition, nowcasting and forecasting using modelling, geographical visualisation of results  … Other archievements

 NA3 workpackage is about supporting instrumentation communities and helping them porting their applications to the e-Infrastructure  The involved effort is huge and has to deal not only with technical aspects  You have to build trust which is not easy  During this first half of the project we manage to establish a common language, establish a way to work together, collect requirements, deploy pilots, setup an evaluation procedure, provide feedback to the JRA and Sas, complete deliverables  Moreover we prepared good quality demo and involve new communities  We will now focus on pilot application improvements, not pilot application deployment, evaluation and feedback to JRA and SAs to obtain a production quality middleware and infrastructure. Summary

 All the pilot applications are available for demonstration on request  We will demonstrate the DORII e-Infrastructure staus with the SAXS application which is complete, shows integration of instrument, storage, computing resources,applications which can be interactive, online (new to the Grid world) and offline (traditional Grid use), and moreover is in production status and presents a collective use of the resources via application manager and workflow management system Introducing the demo