1. http://europlanet-ri.eu Summary of distributed tools of potential use for JRA3 Dugan Witherick HPC Programmer for the Miracle Consortium University College London JRA3 Workshop @ UCL: 22-24 April 2010

2. Talk Outline 2JRA3 Workshop @ UCL: 22-24/04/2010

3.  Summarise the Goals of JRA3 Tasks 3 and 4 Task 3: Distributed Numerical Modelling Laboratory (DNML) Task 4: Distributed Data Analysis Laboratory (DDAL)  What is IDIS?  What are other people doing (two examples)?  A possible solution from the Virtual Observatory Who’s using Virtual Observatory Technology The Universal Worker Service 3JRA3 Workshop @ UCL: 22-24/04/2010

4. Summary of Tasks 3 and 4 Goals (a reminder) 4JRA3 Workshop @ UCL: 22-24/04/2010

5.  Task 3: The main goal of this task is to bring together the numerical simulation efforts currently underway in Europe and to create a coordinated European multi- disciplinary modelling facility for Planetary Science: a Distributed Numerical Modelling Laboratory (DNML). ….feeding into the catalogue part of SA IDIS.  Task 4: The main goal of this task is to bring together expert teams in data analysis with their present research infrastructures and create a coordinated European interdisciplinary data analysis facility for Planetary Science: a Distributed Data Analysis Laboratory (DDAL). ….feeding into the catalogue part of SA IDIS.  The main technical requirements for these tasks could be satisfied with the same solution 5JRA3 Workshop @ UCL: 22-24/04/2010

6. Integrated and Distributed Information Service (IDIS) 6JRA3 Workshop @ UCL: 22-24/04/2010

7.  What is the purpose of IDIS? The task of IDIS is to provide an easy-to-use web-based platform to give access to available data, initiate new research activities needed to interpret observations or to solve open questions, locate teams and laboratories with special knowledge needed, and exploit synergies between space-based missions and capabilities of ground-based observatories. A set of tools for describing, accessing and combining information and data from different sources will be developed, with the objective to offer a Virtual Observatory like access to many data essential for planetary research from European and non-European sources.  Any solution we come up with must fit in with the existing plans of IDIS 7JRA3 Workshop @ UCL: 22-24/04/2010

8. What are other people doing? (two examples) 8JRA3 Workshop @ UCL: 22-24/04/2010

9.  Automated Multi Dataset Analysis (AMDA) A generic Webtool for Space Physics data o automated event search and characterisation o catalogue generation and exploitation o automated database conditional extraction o access to remote Data Centers  Web portal located at http://cdpp-amda.cesr.fr/  The service can be reached and used from other sites according to the Virtual Observatory paradigm.  Not so much a distributed data analysis tool but a single tool linked into the Virtual Observatory 9JRA3 Workshop @ UCL: 22-24/04/2010

10. 10JRA3 Workshop @ UCL: 22-24/04/2010

11.  Community Coordinated Modelling Centre CCMC http://ccmc.gsfc.nasa.gov/ Focus on Space Weather  Models are run through an automated request system: Instant Run – results are provided immediately or Run on Request – user is informed by email when run has completed o These longer running models are added to a queue to run on their Beowulf Cluster  Capable of running more than one model/analysis tool  Not really distributed Appears that all work is being completed at one site 11JRA3 Workshop @ UCL: 22-24/04/2010

12. 12JRA3 Workshop @ UCL: 22-24/04/2010

13. 13JRA3 Workshop @ UCL: 22-24/04/2010

14. Remember IDIS (Distributed is a keyword) 14JRA3 Workshop @ UCL: 22-24/04/2010

15. 15JRA3 Workshop @ UCL: 22-24/04/2010

16. A Possible Solution from the Virtual Observatory 16JRA3 Workshop @ UCL: 22-24/04/2010

17.  Common Execution Controller (CEC) Web service for running various asynchronous applications. Part of the Common Execution Architecture (CEA) suite of software from Astrogrid. Designed to wrap existing applications so that they are presented as IVOA standard web services. o Commandline applications o Existing web form based HTTP applications o Custom written Java applications Suitable for applications that: o have no graphical interface o need no user interaction once launch Supports the following standards: UWS, VOSI, General Security, Registry, Delegation, VOSpace JRA3 Workshop @ UCL: 22-24/04/201017

18.  The Universal Worker Service (UWS) An International Virtual Observatory Specification http://www.ivoa.net/Documents/UWS/ The Universal Worker Service (UWS) pattern defines how to manage asynchronous execution of jobs on a service Has features over the previous standards of o Control for job lifetime o Access to user sandbox (in order to inspect intermediate files There are UWS deployments that do not use the CEA o Not necessary to use CEA 18JRA3 Workshop @ UCL: 22-24/04/2010

19.  Virtual Observatory Registry Essentially the “Yellow Pages” or Telephone Directory for services User client (web portal or client tool) is used to search the registry o Information about the resource is returned  Brief summary  Contact Name  Inputs for the model/tool  Location for the model/tool Two types of registry o Local registry which lists the services installed locally o Full registry harvests the local registries to find out what they have installed o Users search the full registry JRA3 Workshop @ UCL: 22-24/04/201019

20. JRA3 Workshop @ UCL: 22-24/04/201020 EMDAF Full Registry UCL Publishing Registry CEA Service Running H3PCOOL CEA Service Running VOISE CEA Service Running Other Codes Graz Publishing Registry Codes Running at Graz Other Publishing Registries Codes running at Other Local Registries

21. JRA3 Workshop @ UCL: 22-24/04/201021 Virtual Observatory Full Registry IDIS Full Registry EMDAF Full Registry NVO (STSci) Publishing Registry Vizier Publishing Registry VAMDC Publishing Registry

22.  Advantages to this plan Allows us to take advantage of the extensive effort made in developing standards o Saves us a lot of effort We can use/modify existing VO compliant tools o Tool for OPerations on Catalogues And Tables o AladinVO – VO compliant image analysis tool o VOSpec – spectral analysis tool Possible access to workflow tools Provides us with VO access to other data/tools Some IDIS data sources are already published to the VO  Disadvantages to this plan Unclear whether this would be supported by IDIS The VO has had its own problems which we may end up getting caught in This may not be appropriate for fulfilling our deliverables JRA3 Workshop @ UCL: 22-24/04/201022