SCI-BUS is supported by the FP7 Capacities Programme under contract RI-283481 ER-FLOW is supported by the FP7 Infrastructures under contract RI-312579.

Slides:



Advertisements
Similar presentations
The map and reduce functions in MapReduce are easy to test in isolation, which is a consequence of their functional style. For known inputs, they produce.
Advertisements

1 Multi-Grid and Multi-VO Job Submission based on a Unified Computational Model Krakow Grid Workshop, November 22nd 2005 Trinity College Dublin Mr. Oliver.
Dynamically Discovering Likely Program Invariants to Support Program Evolution Michael Ernst, Jake Cockrell, William Griswold, David Notkin Presented by.
Module 12 Computation and Configurations Formal Definition Examples.
Personal Data Management Why is this such an issue? Data Provenance Representing links v Representing data Identifying resources: Life Science Identifiers.
EXTENDING SCIENTIFIC WORKFLOW SYSTEMS TO SUPPORT MAPREDUCE BASED APPLICATIONS IN THE CLOUD Shashank Gugnani Tamas Kiss.
Workflows Information Flows Prof. Silvia Olabarriaga Dr. Gabriele Pierantoni.
Content TCD’s past experience with portals Plans for WS-Pgrade and SCI-BUS – Integration with HELIO to build a portal for the Heliophysics community –
Hadoop Team: Role of Hadoop in the IDEAL Project ●Jose Cadena ●Chengyuan Wen ●Mengsu Chen CS5604 Spring 2015 Instructor: Dr. Edward Fox.
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI WS-PGRADE/gUSE Supporting e-Science communities in Europe Zoltan Farkas.
An Introduction to Designing and Executing Workflows with Taverna Aleksandra Pawlik University of Manchester materials by Dr Katy Wolstencroft and Dr Aleksandra.
Christopher Jeffers August 2012
Time Table exchange QSAS / CL / CAA / AMDA CESR, 25/26 feb
The GAVO Cross-Matcher Application Hans-Martin Adorf, Gerard Lemson, Wolfgang Voges GAVO, Max-Planck-Institut für extraterrestrische Physik, Garching b.
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI CloudBroker Platform integration into WS-PGRADE/gUSE Zoltán Farkas MTA.
An Introduction to Designing and Executing Workflows with Taverna Katy Wolstencroft University of Manchester.
1 Developing domain specific gateways based on the WS- PGRADE/gUSE framework Peter Kacsuk MTA SZTAKI Start date: Duration:
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI Creating the Autodock gateway from WS-PGRADE/gUSE and making it cloud-enabled.
A Framework for Elastic Execution of Existing MPI Programs Aarthi Raveendran Graduate Student Department Of CSE 1.
Sharing Workflows through Coarse-Grained Workflow Interoperability : Sharing Workflows through Coarse-Grained Workflow Interoperability G. Terstyanszky,
STAR net, Resources and VOs C. Vuerli, A. Costa, U. Becciani, P. Massimino, G. Castelli.
Bookkeeping Tutorial. Bookkeeping & Monitoring Tutorial2 Bookkeeping content  Contains records of all “jobs” and all “files” that are created by production.
Application portlets within the PROGRESS HPC Portal Michał Kosiedowski
An Introduction to Designing and Executing Workflows with Taverna Aleksandra Pawlik materials by: Katy Wolstencroft University of Manchester.
WS-Freefluo-VLAM Tutorial Spiros Koulouzis. Outline Get the WS-Freefluo-VLAM (GUI) Start the WS-Freefluo-VLAM (GUI) Generate modules from WSDL Compose.
Convert generic gUSE Portal into a science gateway Akos Balasko 02/07/
AgINFRA science gateway for workflows and integrated services 07/02/2012 Robert Lovas MTA SZTAKI.
Data Grid Research Group Dept. of Computer Science and Engineering The Ohio State University Columbus, Ohio 43210, USA David Chiu & Gagan Agrawal Enabling.
DataONE: Preserving Data and Enabling Data-Intensive Biological and Environmental Research Bob Cook Environmental Sciences Division Oak Ridge National.
An Introduction to Designing, Executing and Sharing Workflows with Taverna Katy Wolstencroft myGrid University of Manchester IMPACT/Taverna Hackathon 2011.
SolarFlows Dr. Gabriele Pierantoni (TCD). Contents What is Heliophysics ? How could workflows help ? Some examples What we are doing...
UCL DEPARTMENT OF SPACE AND CLIMATE PHYSICS MULLARD SPACE SCIENCE LABORATORY Taverna Plugin VAMDC and HELIO (part of the ‘taverna-astronomy’ edition) Kevin.
The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures contract no Workflow repository, user.
Bookkeeping Tutorial. 2 Bookkeeping content  Contains records of all “jobs” and all “files” that are produced by production jobs  Job:  In fact technically.
Event and Feature Catalogs in the Virtual Solar Observatory Joseph A. Hourclé and the VSO Team SP54A-07 : 2008 May 30.
Convert generic gUSE Portal into a science gateway Akos Balasko.
SHIWA and Coarse-grained Workflow Interoperability Gabor Terstyanszky, University of Westminster Summer School Budapest July 2012 SHIWA is supported.
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI Accessing Cloud Systems from WS-PGRADE/gUSE Zoltán Farkas MTA SZTAKI LPDS.
Building an European Research Community through Interoperable Workflows and Data ER-flow project Gabor Terstyanszky, University of Westminster, UK EGI.
Higher Computing Science 2016 Prelim Revision. Topics to revise Computational Constructs parameter passing (value and reference, formal and actual) sub-programs/routines,
SHIWA: Is the Workflow Interoperability a Myth or Reality PUCOWO, June 2011, London Gabor Terstyanszky, Tamas Kiss, Tamas Kukla University of Westminster.
Application Specific Module Tutorial Zoltán Farkas, Ákos Balaskó 03/27/
1 SCI-BUS: building e-Science gateways in Europe: building e-Science gateways in Europe Peter Kacsuk and Zoltan Farkas MTA SZTAKI.
Data Grid Research Group Dept. of Computer Science and Engineering The Ohio State University Columbus, Ohio 43210, USA David Chiu and Gagan Agrawal Enabling.
Introduction to the program of the summer school Peter Kacsuk MTA SZTAKI SCI-BUS is supported by the FP7 Capacities Programme under contract.
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI MTA SZTAKI background for the DARIAH CC Zoltan Farkas MTA SZTAKI LPDS,
Aggregator Stage : Definition : Aggregator classifies data rows from a single input link into groups and calculates totals or other aggregate functions.
Download Manager software Training Workshop Ostend, Belgium, 20 th May 2014 D.M.A. Schaap - Technical Coordinator.
Porting workflows for the Heliophysics Community Dr. Gabriele Pierantoni Trinity College Dublin.
SHIWA Simulation Platform (SSP) Gabor Terstyanszky, University of Westminster EGI Community Forum Munnich March 2012 SHIWA is supported by the FP7.
Usage of WS-PGRADE and gUSE in European and national projects Peter Kacsuk 03/27/
Taverna, myExperiment and HELIO services Anja Le Blanc Stian Soiland-Reyes Alan Willams University of Manchester.
RI EGI-InSPIRE RI Astronomy and Astrophysics Dr. Giuliano Taffoni Dr. Claudio Vuerli.
SCI-BUS is supported by the FP7 Capacities Programme under contract RI ER-FLOW is supported by the FP7 Infrastructures under contract RI
HELIO: Discovery and Analysis of Data in Heliophysics Robert Bentley, John Brooke, André Csillaghy, Donal Fellows, Anja Le Blanc, Mauro Messerotti, David.
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI CloudBroker usage Zoltán Farkas MTA SZTAKI LPDS
Convert generic gUSE Portal into a science gateway Akos Balasko.
SCI-BS is supported by the FP7 Capacities Programme under contract nr RI Quality assurance in SCI-BUS project by applying agile testing practices.
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI Improving the Swiss Grid Proteomics Portal Peter Kunszt, Lorenz Blum,
Exposing WS-PGRADE/gUSE for large user communities Peter Kacsuk, Zoltan Farkas, Krisztian Karoczkai, Istvan Marton, Akos Hajnal,
Building an European Research Community through Interoperable Workflows and Data ER-flow Prof. Gabor Terstyanszky, University of Westminster, UK Heildelberg,
HELIOGATE – A USER EXPERIENCE PERSPECTIVE G.Pierantoni, E. Carley, B. Bentley S. Olabarriaga E. Sciacca, G. Taffoni, G. Castelli, U. Becciani.
SHIWA SIMULATION PLATFORM = SSP Gabor Terstyanszky, University of Westminster e-Science Workflows Workshop Budapest 09 nd February 2012 SHIWA is supported.
WS-PGRADE for Molecular Sciences and XSEDE
CICC Project Meeting Developed two web services (VotableToTabbedFileService – convert votable to tabbed file) and (ExtractionFromSingleVotableService –
Versatile workflow management Tool
Introduction to the SHIWA Simulation Platform EGI User Forum,
Scientific Workflows Lecture 15
Login Main Functions Via SAS Information Delivery Portal
Presentation transcript:

SCI-BUS is supported by the FP7 Capacities Programme under contract RI ER-FLOW is supported by the FP7 Infrastructures under contract RI Workflows for HELIOPhysics Dr. Gabriele Pierantoni Dr. David Perez Suarez

Workflows for Heliophysics What advantages ? – Formal representation of a process – Allows to handle very complex programs – Allows sharing of knowledge through repositories.

Workflows for Heliophysics What are they for ? – Propagation Models – Search metadata – Extract metadata

Workflows for Heliophysics Where do they come from ? – Propagation Models WS-PGRADE (TCD-SCSS & Others) – Search metadata TAVERNA (HELIO) – Extract metadata WS-PGRADE (TCD-PHY & TCD-SCSS)

Propagation Models A propagation model is used to help determine which observations, where and when may be of interest in order to satisfy the user's search criteria.

Advanced Propagation Model Origin t Target t Period of interest E E E E ETA Range ETA Range P = P min P = P max E E P = P OK

Propagation Models

Download Event List from catalogues StepWorkflowMeaning SourceEventFinder Given a list of catalogues an extraction criteria returns a VOtable with the list of all relevant events ParameterExtractor Givent a list of events, it extracts the parameter values or ranges that are relevant for the execution of the propagation model. TimeExtractor Given a list of events, it extracts the time values or ranges that are relevant for the execution of the propagation model. Propagator It invokes the SHEBA propagation model with the parameters and times produced by ParameterExtractor and TimeExtractor RankerIt ranks the results of the propagation model.

Extract Parameters StepWorkflowMeaning SourceEventFinder Given a list of catalogues an extraction criteria returns a VOtable with the list of all relevant events ParameterExtractor Givent a list of events, it extracts the parameter values or ranges that are relevant for the execution of the propagation model. TimeExtractor Given a list of events, it extracts the time values or ranges that are relevant for the execution of the propagation model. Propagator It invokes the SHEBA propagation model with the parameters and times produced by ParameterExtractor and TimeExtractor RankerIt ranks the results of the propagation model.

Extract Time Ranges StepWorkflowMeaning SourceEventFinder Given a list of catalogues an extraction criteria returns a VOtable with the list of all relevant events ParameterExtractor Givent a list of events, it extracts the parameter values or ranges that are relevant for the execution of the propagation model. TimeExtractor Given a list of events, it extracts the time values or ranges that are relevant for the execution of the propagation model. Propagator It invokes the SHEBA propagation model with the parameters and times produced by ParameterExtractor and TimeExtractor RankerIt ranks the results of the propagation model.

Execute the model StepWorkflowMeaning SourceEventFinder Given a list of catalogues an extraction criteria returns a VOtable with the list of all relevant events ParameterExtractor Givent a list of events, it extracts the parameter values or ranges that are relevant for the execution of the propagation model. TimeExtractor Given a list of events, it extracts the time values or ranges that are relevant for the execution of the propagation model. Propagator It invokes the SHEBA propagation model with the parameters and times produced by ParameterExtractor and TimeExtractor RankerIt ranks the results of the propagation model.

Rank the results

LO-FAR LOFAR is a real-time multiple sensor array. Very different sensors can be placed along a common infrastructure and make use of it the same time. Analysis of raw frequency data requires significant storage and computation resources. We are developing a standardizes processing and storage workflow to the community to be easily configured

Processing Nodes Data Staging PreProcessing Processing PostProcessing DataPublications

Java Standard Node PORTLET / PORTAL WORFLOWS NODES PROGRAMS

Java Standard Node Standard Java Node Standard Java Node Input Data Input Status Input Parameter Output Data Output Status

Java Standard Node Input Data Load Files Input Status Input Parameter Check Status Failed Processing Completed Output Data Output Status

Java Standard Node AbstractGenericNode AbstractStringNode ConcreteStringNode Defines the sequence of actions Defines how to read and write files Defines the processing

Metadata Extraction (LO-FAR)

ER-FLOW WS-Pgrade Workflows Development of HELIO portal Access to DCIs Web Services TAVERNA workflows

Events Count

Counts the number of events for a period.

Fastest CMEs Propagation This workflow is used to investigate one of the most relevant events in HELIOphysics; Coronal Mass Ejections. The propagation of these events is studied throughout the Solar System by the means of the execution of the SHEBA propagation model on the HELIO Processing Service.

Fastest CMEs Propagation Find parameters for model Execute the model Validate the results

High Energy Flares Given a time range, this workflow looks for flares within the specified energy range and provides the observations for such time range for the list of instruments asked. It also provides the table of flares with its properties.

High Energy Flares