An Introduction to Designing, Executing and Sharing Workflows with Taverna Katy Wolstencroft myGrid University of Manchester IMPACT/Taverna Hackathon 2011.

Slides:



Advertisements
Similar presentations
KompoZer. This is what KompoZer will look like with a blank document open. As you can see, there are a lot of icons for beginning users. But don't be.
Advertisements

Customizing the MOSS 2007 Search Results November 2007 Rafael Perez.
An Introduction to Designing and Executing Workflows with Taverna Aleksandra Pawlik University of Manchester.
Logging In Go to web site:
1 of 5 This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT. © 2007 Microsoft Corporation.
1 of 6 This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT. © 2007 Microsoft Corporation.
1 of 6 Parts of Your Notebook Below is a graphic overview of the different parts of a OneNote 2007 notebook. Microsoft ® OneNote ® 2007 notebooks are digital.
1 of 6 This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT. © 2007 Microsoft Corporation.
Downloading and Installing AutoCAD Architecture 2015 This is a 4 step process 1.Register with the Autodesk Student Community 2.Downloading the software.
Chapter 1 Getting Started With Dreamweaver. Explore the Dreamweaver Workspace The Dreamweaver workspace is where you can find all the tools to create.
Installing geant4 v9.5 using Windows Daniel Brandt, 06 April 2012 Installing Geant4 v9.5 for Windows A step-by-step guide for Windows XP/Vista/7 using.
Quick Start Guide. This 22 page introduction to the Financial Assessment Subsystem provides the user with a visual overview of the components of the system.
Working with SharePoint Document Libraries. What are document libraries? Document libraries are collections of files that you can share with team members.
Google Earth How to create a Google Earth Tour and place it in your Wiki.
Creating a Web Page HTML, FrontPage, Word, Composer.
WorkPad 4 Quick Start WorkPad 4 Quick Start  Business Optix brings the rigor and discipline of business modelling and design into.
One to One instructions Installing and configuring samba on Ubuntu Linux to enable Linux to share files and documents with Windows XP.
An Introduction to Designing and Executing Workflows with Taverna Aleksandra Pawlik University of Manchester materials by Dr Katy Wolstencroft and Dr Aleksandra.
Classroom User Training June 29, 2005 Presented by:
® IBM Software Group © 2009 IBM Corporation Rational Publishing Engine RQM Multi Level Report Tutorial David Rennie, IBM Rational Services A/NZ
SharePoint You must use Internet Explorer Single click only on links and buttons There are two handouts 1.Importing a Web Part into SharePoint 2.Adding.
An Introduction to Designing, Executing and Sharing Workflows with Taverna Nowgen, Next Gen Workshop 17/01/2012.
Step 1: Import Pictures Step 2: Add Titles Step 3: Narrate Story & Customize Motion.
Creating a Web Site to Gather Data and Conduct Research.
To download PhotoStory: Go to On the left side under Product Resources, click on Downloads.
What is Museum Box? A Museum box is a way of presenting information that allows you to create a cube project that can be shared with others. You can use.
CIS 205—Web Design & Development Dreamweaver Chapter 1.
An Introduction to Designing and Executing Workflows with Taverna Katy Wolstencroft University of Manchester.
Introduction to Taverna, an environment For designing and executing workflows Franck Tanoh University of Manchester.
1 ITI 1120 Lab # 1 An Introduction to the Lab Environment Contributors: G. Arbez, M. Eid, D. Inkpen, A. Williams, D. Amyot.
Downloading and Installing Autodesk Revit 2016
An Introduction to Designing and Executing Workflows with Taverna Aleksandra Pawlik materials by: Katy Wolstencroft University of Manchester.
Sports Website Creation. In this project you will design and produce your own website.
Microsoft Access 2010 Chapter 10 Administering a Database System.
Downloading and Installing Autodesk Inventor Professional 2015 This is a 4 step process 1.Register with the Autodesk Student Community 2.Downloading the.
Tour Overview Introduction Collage Basics Collage Basics (Templates and Tools) Computer Configuration Bookmark Collage Getting Started Tour Collage Terminology.
THE 3M CLOUD LIBRARY AND YOUR KINDLE FIRE How to download 3M books to your Kindle Fire/HD Made possible by a grant from the Samuel P. Hunt foundation.
XP New Perspectives on Microsoft Office FrontPage 2003 Tutorial 7 1 Microsoft Office FrontPage 2003 Tutorial 8 – Integrating a Database with a FrontPage.
Introduction to Taverna Online and Interaction service Aleksandra Pawlik University of Manchester.
January 2006Colby College ITS Setting Up Course Pages.
External Study Credit Granting with SurveyGizmo Shepherd University Department of Psychology.
Creating and Editing a Web Page
Create new project or open existing project (here, we will create a new project)
Adding, editing, and deleting items using CONTENTdm Administration.
Access Queries and Forms. Adding a New Field  To insert a field after you have saved your table, open Access, and open the table  It is easier to add.
SNG via Webinar. Where’s Webinar??  Double click Aflac 2000 folder  Highlight “SNGWebCommunicator”  Right Click and “Send To - Desktop”
Invoices and Service Invoices Training Presentation for Raytheon Supply Chain Platform (RSCP) April 2016.
Fab25 User Training Cerium Labs LabCollector - LIMS Lynette Ballast.
Designing, Executing and Sharing Workflows with Taverna 2.2 Katy Wolstencroft myGrid University of Manchester.
TELEPORT PRO Website to Hard Drive Completely download a website, enabling you to “Browse Offline” at much greater speeds than if you were to browse the.
Exploring Taverna engine Aleksandra Pawlik materials by Katy Wolstencroft University of Manchester.
Advanced Taverna Aleksandra Pawlik University of Manchester materials by Katy Wolstencroft, Aleksandra Pawlik, Alan Williams
An Introduction to Running, Reusing and Sharing Workflows with Taverna – part 2 Aleksandra Pawlik materials by Katy Wolstencroft University of Manchester.
Taverna allows you to automatically iterate through large data sets. This section introduces you to some of the more advanced configuration options for.
The Next Step Hudson Fare Files 102 – Import & upload Rev. 10/14.
Exploring Taverna 2 Katy Wolstencroft myGrid University of Manchester.
An Introduction to Designing, Executing and Sharing Workflows with Taverna BioVel Workshop 2011.
Designing, Executing and Sharing Workflows with Taverna 2.4 Different Service Types Katy Wolstencroft Helen Hulme myGrid University of Manchester.
Journal of Mountain Science (JMS)
NOODLETOOLS SIGN-IN Student ID #
Weebly Elements, Continued
Designing and Sharing Taverna Workflows: Exploring Taverna 2.1 Beta
About SharePoint Server 2007 My Sites
An Introduction to Designing and Executing Workflows with Taverna
Module 6: Creating Web Pages and Working with Channels
An Introduction to Designing, Executing and Sharing Workflows with Taverna and myExperiment Katy Wolstencroft University of Manchester.
Shim (Helper) Services and Beanshell Services
Aleksandra Pawlik materials by Katy Wolstencroft
An Introduction to Designing and Executing Workflows with Taverna
Presentation transcript:

An Introduction to Designing, Executing and Sharing Workflows with Taverna Katy Wolstencroft myGrid University of Manchester IMPACT/Taverna Hackathon 2011

 Taverna can be downloaded from Go to the page and find the latest (2.3)  Download the correct version for your operating system  Follow the instructions in the Taverna installer The following page shows a screenshot of Taverna and the different panels that make up the workbench

Taverna Workbench Workflow Diagram Services Panel Workflow Explorer

The visual representation of workflow  Shows inputs/outputs, services and control flows  Allows editing of the workflow by dragging and dropping and connecting services together  Enables saving of workflow diagrams for publishing and sharing

 The Workflow Explorer shows the detailed view of your workflow. It shows default values and descriptions for service inputs and outputs and it shows where remote services are located. It also shows configuration details, such as iteration and looping (we will come back to these things later).  Workflow validation details can also be found here. Before a workflow is run, Taverna checks to see if it is connected correctly and if its services are available.

Lists services available by default in Taverna  Local java services  WSDL Web Service – secure and public  RESTful Services  R Processor services (for statistical analyses)  Beanshell scripts  Xpath scripts Allows the user to add new services or workflows from the web or from file systems – there are loads more available!

 In the Services panel, type ‘image’ into the search box.  Select ‘Get Image from URL’  This is a local service, but web services work the same way  Many historical documents are stored as images on the web. This is a simple, but useful service to help gather data Drag this service across to the workflow diagram panel

 In a blank space in the workflow diagram, right-click and select “Add Workflow Input Port”  Type a name (e.g. URL) for this input in the pop-up window and click “ok”  Do the same to create a new workflow output. Call this output “image”

 You now have 3 boxes in the diagram and we need to connect them up into a workflow  First, we need to find out how many inputs and outputs the ‘get image from URL’ service has  At the top of the workflow diagram, select the ‘show ports’ icon Show Ports

 Click on the workflow input box and drag the linking arrow across to the URL input of the ‘get_image_from_URL’ service. Link the image output of ‘get_image_from_url’ to the workflow output port

 You have now built your first workflow! It should look something like this.  In many cases, you have to supply input data for EVERY service input port. In this case, however, the ‘base’ input is optional, so we will leave it.  Save the workflow by going to file -> save workflow

 Run the workflow by selecting “file -> run workflow”, or by clicking on the play button at the top of the workbench

An input window will appear. As you can see, we have not yet added a description of the workflow or of the input Click on ‘New Value’ in the input window and add the url arta/images/magna_carta.jpg where it says “some input data goes here”

 Click “run workflow”  In the bottom left of the results window, click on the results. You will now see an image from the specified web page  Workflow results can be saved here if required by clicking on ‘save all values’

 Right-click on a blank part of the workflow diagram and select “show details”  In the workflow explorer panel, the details page will open up. Add some details about the workflow (e.g. who is the author, what does the workflow do).  You can also add examples and descriptions for the workflow inputs by selecting them in the explorer panel and selecting “details”  Adding this metadata makes the workflow much more reusable  Save the workflow by going to “File -> save workflow”

New services can be gathered from anywhere on the web We will find a new service and add it to the workbench  IMPACT and SACPE have a whole suite of services. We will add one (you will be using it later on today)  Go to Here you will find a list of IMPACT serviceshttps://fue.onb.ac.at/synapse  Click on IMPACTTesseractV3Proxy and copy the link you are directed to.  This is the WSDL address and is what Taverna needs to run the service Exercise 3: Adding New Services

 Go to the services panel in Taverna and click “import new services”. For each type of service, you are given the option to add a new service  Select ‘WSDL service…’ A window will pop-up asking for a web address

 Enter the service address you just copied  Scroll down the Services list, you will see your new service there

 Go to  myExperiment is a social networking site for sharing workflows and workflow expertise and experiences  Browse around the site and see what it contains  Find everything that has been tagged with ‘text mining’, for example  Look at the text mining workflows. You will see some that are specific to biology, some that are generally applicable, and some that are specific to other scientific disciplines

 IMPACT have many workflows on myExperiment, but they are not public. You must join an IMPACT group before you can see them and use them.  Create yourself an account and join the group called ‘IMPACT-myGrid-Hackathon’ (NOTE: you need to join this group to access content for future exercises)  Explore the shared items in this group. These are examples of the types of tasks IMPACT workflows can perform

 You can download and run the workflows from the myExperiment website, or you can use myExperiment directly from Taverna  To use workflows from the website, you can either download them, or copy the workflow file location into the ‘open workflows from the web’ option in Taverna’s file menu.

 Go back to Taverna and click on the myExperiment icon at the top of the workbench  Go to ‘my stuff’ and log in (using the same credentials as the web page)  Find the IMPACT-myGrid-Hackathon group by using the ‘search’ option.  Look at the shared items and find the workflow called ‘Text to List’  Click on ‘open’ and this workflow will be automatically imported into your Taverna design window

 Taverna checks to see that everything is connected properly and that all the required services are available  Go to the workflow explorer and click on ‘validation report’  See if Taverna has found any problems with the workflow. Errors will be displayed in red, warnings in yellow. Workflows with warnings often still run.  If there are problems, follow the instructions to resolve them by clicking on the ‘Solution’ tab  If not, run the workflow

 Use the default input suggested to run the workflow. The workflow will collect and list some example data stored at the given URL  It returns a list of image files  We can now combine this workflow with the one we made earlier to return the actual images.  In Taverna, you can add workflows as if they were any other kind of service – these are called ‘Nested Workflows’

 From the current workflow design window, go to ‘Insert -> Nested workflow  Import the workflow you made earlier, by selecting ‘import from file’  You can see a small version of the workflows, so you can check you are importing the correct workflow

 We now need to connect the two workflows together  Connect the Text2List service to the input of the nested workflow by dragging an arrow across.  Make a new workflow output port (by right-clicking and adding workflow output port)  Connect the output of the nested workflow to the new workflow output port

 Your new workflow should look something like this  Save and run the workflow  This time, as it runs, you will see Taverna automatically iterates over the list of data produced by Text2List  NOTE: some of the iterations will fail. See if you can tell which  Look at one of the resulting images

 You can track intermediate workflow values through the results view. This is very useful for working out where unexpected results came from.  On the diagram, click the Text2List service and look at its inputs and outputs in the results.  You can save the workflow in myExperiment if you wish, but make sure you give credit to the nested workflow author and make sure you ONLY share it with the IMPACT-myGrid-Hackathon group

As you have already seen, Taverna can automatically iterate over sets of data. When 2 sets of iterated data are combined, however, Taverna needs extra information about how they should be combined. You can have: A cross product – combining every item from list 1 with every item from list 2 - all against all A dot product – only combining item 1 from list 1 with item 1 from list 2, and so on – line against line

Find and load the workflow ‘Demonstration of configurable iteration’ from myExperiment  Read the workflow metadata to find out what the workflow does (by looking at the ‘Details’)  Select the ‘ColourAnimals’ service and select the ‘Details’ in the workflow explorer and ‘configure list handling’  Click on ‘dot product’ in the pop-up window. This allows you to switch to cross product

 Run the workflow twice – once with ‘dot product’ and once with ‘cross product’.  Save the first results so you can compare them – what is the difference? What does it mean to specify dot or cross product?

 Web services can sometimes fail due to network connectivity  If you are iterating over lots of data items, you can guard against these temporary interruptions by adding retries to your workflow  Upload the ‘Retry-Example’ workflow from the IMPACT- myGrid-Hackathon group. This workflow is designed to fail sometimes.  Run the workflow as it is and count the number of failed iterations

 Now, select the ‘sometimes_fails’ service and select the ‘details’ tab in the workflow explorer panel  Click on ‘advanced’ and ‘configure’ for retries  In the pop-up box, change it so that it retries each service iteration 2 times  Run the workflow again – how many failures do you get this time?  Change the workflow to retry 5 times – does it work every time now?

 From myExperiment, download and open the workflow “dummy_example_of_looping”  This workflow is asynchronous. This means that when you submit data (by running the workflows), it will return a jobID and place your job in a queue. This is very useful if your job will take a long time!  The ‘CheckStatus’ service will query your job ID to find out if it is complete

The default behaviour in a workflow is to call each service only once for each item of data – so what if your job has not finished when ‘Status’ workflow asks?  Run the workflow  Almost every time, the workflow will ‘fail’ (in this case, that means it will return 0) because the results have not been returned before the workflow reaches the ‘getResults’ service

This is where looping is useful. Taverna can keep running the ‘status’ service until it reports that the job is done.  Select the ‘CheckStatus’ service and click on the ‘details’ tab in the workflow explorer  Select ‘advanced’ and click on ‘add looping’  Use the drop-down boxes in the looping window to set ‘state’ ‘is_not_equal_to’ RUNNING

 Save the workflow and run it again  This time, the workflow will run until the ‘CheckStatus’ service reports that it is either COMPLETE, or it has an ERROR.  You will see results for ‘GetResults’, but you will still get an error for ‘GetResults2’. This is because there is one more configuration to change – we also need ‘Control Links’

 A control link specifies that there is a dependency of one service on another even though there is no data flowing between them.  A control link is a line with a white circle at the end that connects two services (see the link between ‘CheckStatus’ and ‘getResults’

 We will add control link to getResults2  Right-click on getResult2 and select ‘Run after’ from the drop down menu.  Set it to ‘Run after’ -> ‘CheckStatus’  Save and run the workflow  Now you will see both results returned