MSSL Astrogrid Workshop Kevin Benson Elizabeth Auden George Millward Silvia Dalla Kevin Benson

Agenda Time What Who Reference 10:30 Welcome Kevin 10:45 Learning Basics 11:30 Querying and Workflow Silvia 13:00 Lunch All Normal Sandwhiches  14:00 Solar Apps Elizabeth 15:15 Break Tea/Coffee 15:30 CTip George 16:30 Scripting 16:45 Solar and STP with Astrogrid 17:00 Closing Kevin Benson

But first... ...What is a Virtual Observatory? Dynamic collection of hardware, data and software working in harmony to solve arbitrarily large and complex astronomical problems. Kevin Benson

Bridging communities Semantic Desktop astronomy web/grid Virtual observatory (VO) Web archives in astronomy HPC Kevin Benson

AstroGrid’s position within VO Interactive exploration/ Experiment design Bulk processing/ Experiment execution AstroGrid’s emphasis Kevin Benson

AstroGrid-2 Layer Diagram Kevin Benson

Bulk processing  workflow Hands-off operation Reproducible experiments Common, reusable procedures Everything is recorded Integrated with interactive system “Virtual grad student” Kevin Benson

Components within AstroGrid Data grid Compute grid Web portal Workflow System Resource Registry Kevin Benson

Web portal Basic controls in web browser… …linked to traditional displays on desktop. Kevin Benson

Dashboard Client-side UI Kevin Benson

Resource registry Description of all accessible resources: Services Applications Archive data-sets Follows IVOA standards Standard interface for queries Standard XML schemata for resource entries Standard “harvesting” between registries Metadata for: Humans Software agents Kevin Benson

Portal and Workflow Supports workflow paradigm (from the simple to the complex) Queries and workflows are designed using the portal Work is run remotely and asynchronously Archives searched and results manipulated Results are stored in a virtual file system Queries and workflows can be re-used and shared Kevin Benson

Data grid: DSA and MySpace Desktop Data-selection Service Processing 1 Processing 2 Archive User-writeable storage Kevin Benson

Compute grid: Common Execution Architecture Web services Legacy/situated s/w Common Execution Connector Application as .exe file (local) JES Common Execution Connector Application as .exe file (grid) Common Execution Connector Application as HTTP service Registry Common Execution Connector Application as Java class Kevin Benson

Common Execution Architecture Uniform interface for running applications in VObs Set of WSDL and schema Define the interactions in language independent way Adapters (ExecutionControllers) implemented Using Java Web services To run legacy command-line applications To run legacy HTTP GET/POST apps within the framework ExecutionControllers Allow asynchronous operation of the applications Callback and polling mechanisms for delivery of results Tightly integrated with JES (Job Execution System) Allows programmable workflows Kevin Benson

Current Status Successfully used and demo'd at: V1.0 released April NAM ESO Cambridge Workshops V1.0 released April V1.1 Release August Cycle-2, Iteration-1 completed Cycle-2, Iteration-2 on track Kevin Benson

Future Complete auth/auth research and apply to all components Continue to improve usability Workflow libraries Assisted query builders Continue to work with partners Aladin, TopCat, NVO, ESO, etc Enable more science Harness Grid aspects Build in more “intelligence” Data routing Execution service selection Kevin Benson

Terms (Important to Know) Myspace/Filemanager – Your space. Holds files in some location. FileStore – Myspace is actually the metadata about your files sucha s names, size, type. But the actual file is sent to a Filestore service which may be on a different server than Myspace. Workflow – Allows a user to visually construct a workflow document which contains scripts and steps. Also has looping and parallel ability for calling applications. Steps – are always some application. Where by filling out the input and output of a application. Inputs or Outputs can be direct or indirect. Portal – HTML/Web Based display allowing you to run workflows, look in myspace, query and essentially run Astrogrid as a User. WorkBench – WebStart Application that runs on your machine. More flexiblity than Portal and better performance. Ivorn/Identifier –Users will mostly use this as references to files in myspace ex: ivo://mssl.ucl.ac.uk/kmb#workflow/solardemo.wf Kevin Benson

How does MSSL serve Astrogrid http://www.mssl.ucl.ac.uk/twiki/bin/view/MSSL/CurrentGridSetup 2 – Production Grid Machines Hosting Datasets of EIT, CDS, Yohkoh (SXT) Various Applications (CTipMovie, SolarMovie, SolarOverlay, GenericMovie, (Ace, GnuPlot, WDCQuery)) 1 – Development Grid Machine 1 – Production Hybrid with EGSO and contains SEC database. Kevin Benson

How To Login Username – {same as e-mail name} ex: kmb Password – same as above. Community – MSSL Staff – “mssl.ucl.ac.uk” UCL Staff – “ucl.ac.uk” Kevin Benson