ESDO Algorithms, Visualization and Data Access Elizabeth Auden 21 September 2006 AHM 2006 Nottingham.

Slides:

Advertisements

Similar presentations

September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.

Advertisements

A PPARC funded project The Grid Data Warehouse Description of prototype work in progress by AstroGrid. Access-Grid lecture to Universities of Leeds and.

C. Grimme, A. Papaspyrou Scheduling in C3-Grid AstroGrid-D Workshop Project: C3-Grid Collaborative Climate Community Data and Processing Grid Scheduling.

ESDO Algorithms, Visualization and Data Access Elizabeth Auden 27 September 2006 HELAS Workshop Nice.

19-20 March 2003 IVOA Registry Workgroup LeSc Astrogrid Registry: Early Designs Elizabeth Auden Astrogrid Registry Workgroup Leader IVOA Registry Workgroup.

A PPARC funded project Tony Linde Programme Manager eScience meets eFrameworks 28 th April 2006 NeSC, Edinburgh.

MySpace: Personalized Work Space in AstroGrid Catherine Ling Qin 26 th Nov 2003.

Solar and STP Physics with AstroGrid 1. Mullard Space Science Laboratory, University College London. 2. School of Physics and Astronomy, University of.

Page 1JSOC Peer Review – AIA Science Center – March 17, 2005 AIA Science Data Processing Infrastructure Neal Hurlburt AIA Data Scientist

Page 1HMI/AIA Joint Sciemce Meeting – AIA Science Center – Feb 2006 AIA Visualization Center N. Hurlburt.

JSOC Overview - 1HMI/AIA Team Meeting – Feb 2006 JSOC Summary- Phil Scherrer Pipeline Processing - Rasmus Larsen Data Access - Rick Bogart Data Visualization.

HMI/AIA Science Team Meeting, HMI Science Goals Alexander Kosovichev & HMI Team.

HMI Data Access Jeneen Sommers September 8, /23/2015HMI Team Meeting, Stanford CA.

[session no.]1 IV: Science investigation Key Observables Flux emergence and transport (both HMI and AIA) Magnetic connections among different areas (coronal.

ADASS, London Sep 23-26, 2007 VOExplorer : Visualising Data Discovery in the Virtual Observatory Jonathan Tedds University of Leicester/AstroGrid/ EuroVO.

25 September 2007eSDO and the VO, ADASS 2007Elizabeth Auden Accessing eSDO Solar Image Processing and Visualisation through AstroGrid Elizabeth Auden ADASS.

Advanced Technology Center 1 HMI Rasmus Larsen / Processing Modules Stanford University HMI Team Meeting – May 2003 Processing Module Development Rasmus.

EU 2nd Year Review – Jan – WP9 WP9 Earth Observation Applications Demonstration Pedro Goncalves :

QCDgrid Technology James Perry, George Beckett, Lorna Smith EPCC, The University Of Edinburgh.

Good practice in Research Data Management Module 6: Tools, training and support.

Data Processing and Display Challenges for Solar Dynamics Observatory Using the Heliophysics Event Knowledgebase Ralph Seguin Ankur Somani Lockheed Martin.

F. I. Suárez-Sol á 1, E. González-Suárez 1, I. González-Hernández 1, A.R. Davey 2,J. Hourcl é 3, VSO Team 1 National Solar Observatory, Tucson AZ – 2 Harvard-Smithsonian.

A long tradition. e-science, Data Centres, and the Virtual Observatory why is e-science important ? what is the structure of the VO ? what then must we.

Solar-B Global DataGrid 21 st September 2006 UK e-Science All Hands Meeting 2006 RALMSSL Tim Folkes Elizabeth Auden Jens Jensen Paul Lamb Matthew WildMatthew.

A PPARC funded project AstroGrid: new technology for the virtual observatory SC2004 Pittsburgh, PA November 2004 Guy Rixon AstroGrid Technical Architect.

MASSACHUSETTS INSTITUTE OF TECHNOLOGY NASA GODDARD SPACE FLIGHT CENTER ORBITAL SCIENCES CORPORATION NASA AMES RESEARCH CENTER SPACE TELESCOPE SCIENCE INSTITUTE.

Functions and Demo of Astrogrid 1.1 China-VO Haijun Tian.

Module 10: Monitoring ISA Server Overview Monitoring Overview Configuring Alerts Configuring Session Monitoring Configuring Logging Configuring.

QCDGrid Progress James Perry, Andrew Jackson, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.

Summary of distributed tools of potential use for JRA3 Dugan Witherick HPC Programmer for the Miracle Consortium University College.

Astrogrid Workshop – Group C (solar) Anne-Marie Broomhall, Bill Chaplin, Yvonne Elsworth, Steve Hale, Andrew Inglis, Brek Miller, David Pascoe, Graham.

GONG data and pipelines: Present & future. Present data products 800x800 full-disk images, one per minute, continuous (0.87 average duty cycle) Observables:

ILDG Middleware Status Chip Watson ILDG-6 Workshop May 12, 2005.

European Grid of Solar Observations EGSO European Grid of Solar Observations Simon Martin Rutherford Appleton Laboratory CDS Users Meeting, NAM 2003.

Solar-B Global DataGrid Update for AstroGrid SAG 10 RALMSSL Tim Folkes Elizabeth Auden Jens Jensen Paul Lamb Matthew WildMatthew Whillock Len Culhane Elizabeth.

AstroGrid: The UK’s Virtual Observatory Dr Dugan Witherick – Astrophysics Group, UCL Wednesday 5 th December 2007 The University of Warwick.

Page 1LWS Teams Day JSOC Overview HMI-AIA Joint Science Operations Center Science Data Processing a.k.a. JSOC-SDP Overview.

18 November 2004AstroGrid Solar Movie Maker VOTechKick-off VOTech Kick-off Meeting AstroGrid Solar Movie Maker Deploying SolarSoft IDL tools with CEA Elizabeth.

NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.

Solar and STP science with AstroGrid Silvia Dalla School of Physics & Astronomy, University of Manchester A PPARC funded project.

Solar and space physics datasets within a Virtual Observatory: the AstroGrid experience Silvia Dalla * and Nicholas A Walton  * School of Physics & Astronomy,

AstroGrid Solar/STP planning meeting Agenda: Helioscope Preparing for Solar-B Time-series viewing application IVOA and time series A PPARC funded project.

A PPARC funded project Tony Linde Programme Manager AG-SAG FM5: AG2/VOTech Scope & Targets Glasgow, 7 Feb 2005 VOTech Project.

Page 1JSOC Overview August 2007 HMI Status HMI is virtually done. –Virtually  similar to but not in fact –Front window issue resolved, flight window now.

SolarFlows Dr. Gabriele Pierantoni (TCD). Contents What is Heliophysics ? How could workflows help ? Some examples What we are doing...

1 e-Science AHM st Aug – 3 rd Sept 2004 Nottingham Distributed Storage management using SRB on UK National Grid Service Manandhar A, Haines K,

UCL DEPARTMENT OF SPACE AND CLIMATE PHYSICS MULLARD SPACE SCIENCE LABORATORY Taverna Plugin VAMDC and HELIO (part of the ‘taverna-astronomy’ edition) Kevin.

AstroGrid NAM 2001 Andy Lawrence Cambridge NAM 2001 Andy Lawrence Cambridge Belfast Cambridge Edinburgh Jodrell Leicester MSSL.

Algorithm Preparation and Data Availability 1. Mullard Space Science Laboratory, University College London. 2. Physics and Astronomy Department, University.

A Technical Overview Bill Branan DuraCloud Technical Lead.

ADNET Systems, Inc. Jack Ireland & Helioviewer Team ADNET Systems, Inc. Helioviewer Discovery for Everyone Everywhere.

Introduction to the VO ESAVO ESA/ESAC – Madrid, Spain.

AHM04: Sep 2004 Nottingham CCLRC e-Science Centre eMinerals: Environment from the Molecular Level Managing simulation data Lisa Blanshard e- Science Data.

Solar and STP with AstroGrid Silvia Dalla School of Physics & Astronomy, University of Manchester A PPARC funded project.

ESDO: UK Access to the Solar Dynamics Observatory 1. Mullard Space Science Laboratory, University College London. Holmbury St. Mary, Dorking, Surrey SM2.

Introduction: AstroGrid increases scientific research possibilities by enabling access to distributed astronomical data and information resources. AstroGrid.

DataGrid France 12 Feb – WP9 – n° 1 WP9 Earth Observation Applications.

AstroGrid and Virtual Observatories for Radio Interferometry arrays/ proposals Anita Richards Paul Harrison Noel Winstanley (Jodrell Bank Centre for Astrophysics,

STCE annual meeting 7 June, 2011

eSDO Consortium Meeting

MSSL Astrogrid Workshop

JSOC Status at SDO PSR-1 January 21-22, 2009 GSFC

E.C. Auden1, J.L. Culhane1, Y. P. Elsworth2, A. Fludra3, M. Thompson4

Flanders Marine Institute (VLIZ)

Grid Portal Services IeSE (the Integrated e-Science Environment)

Status and Overview of HMI–AIA Joint Science Operations Center (JSOC) Science Data Processing (SDP) April 22, 2008 P. Scherrer.

Intermountain West Data Warehouse

Data search & distribution

Gordon Erlebacher Florida State University

Presentation transcript:

eSDO Algorithms, Visualization and Data Access Elizabeth Auden 21 September 2006 AHM 2006 Nottingham

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden What is eSDO? eSDO is an e-science project funded by PPARC to make SDO data and algorithms available to the UK solar community using the virtual observatory. Algorithms Code: feature recognition, local helioseismology, global helioseismology Grid integration: AstroGrid and JSOC Date centres AstroGrid interface with US JSOC data centre UK data centre Visualization tools Catalogues SDO Streaming tool On-the-fly image galleries and movies

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden eSDO Schedule RESEARCH - Phase A: 1 October 2004 – 30 September 2005 IMPLEMENTATION - Phase B: 1 October 2005 – 30 September 2007 Overview of Phase A activities: Phase A Summary Report available on eSDO project wiki: Detailed report including 6 deliverable documents: Phase A Report available on eSDO project wiki:

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden eSDO Project Team MSSL –Elizabeth Auden, project manager –Len Culhane, PI –Lidia van Driel-Gesztelyi –Mike Smith RAL –Richard Harrison –Andrzej Fludra –Vito Graffagnino Sheffield –Mike Thompson –Sergei Zharkov Birmingham –Yvonne Elsworth –Bill Chaplin –Thierry Toutain US Links: JSOC, Joint Operations Centre Stanford University Lockheed Martin

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden Algorithms Code Feature Recognition / Event Analysis Coronal loop detection, MSSL Non-linear magnetic field extrapolation, MSSL Helicity computation, MSSL Small event detection, RAL CME dimming region recognition, RAL Local Helioseismology Perturbation map generation, Sheffield Subsurface flow analysis, Sheffield Local helioseismology inversions, Sheffield Global Helioseismology Mode parameters analysis, Birmingham Mode asymmetry analysis, Birmingham

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden Algorithms Grid Integration JSOC SDO pipeline at Stanford University / Lockheed Martin eSDO algorithms deployed as pipeline modules (automated or user-run) Data will be obtained locally from JSOC archive AstroGrid CEA UK hosted algorithms hosted as web services Accessible through AstroGrid workflows Data transferred from MySpace, remote data centre or user’s local storage SolarSoft Algorithm C modules wrapped in IDL Distribution through MSSL SolarSoft gateway Data transferred to algorithm from user’s local storage

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden Algorithms AstroGrid Deployment 9 algorithms have been deployed as CEA applications

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden Data Centres US Data Centre Primary US Data Store Files stored in SUMS (Storage Unit Management System) at Stanford University with metadata in distributed DRMS (Data Resource Management System) DSA / DRMS integration for grid data access investigations VSO (Virtual Solar Observatory) primary US access AstroGrid / VSO Interface VSO queries in AstroGrid workflows through STAP (Simple Time Access Protocol) web service US / UK Network Latency Tests Test expediency of file transfer mechanisms Test between Stanford, MSSL, UCL, RAL

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden Data Centres Possible UK Data Centre Storage 30 TB disc cache 60 day rolling cache HMI/AIA True cache for HMI / AIA DRMS / SUMS Transfer Export formats from JSOC Data search – DSA Data transfer - CEA Test installation MSSL Investigation of DRMS / SUMS

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden Data Centres Possible European Data Centre HELAS: European Helio and AstroSeismology Network EU Framework 6 support for European helioseismology data and algorithms Possible full mirror of SDO data in Germany Discussion of possible collaboration between eSDO and HELAS in late September 2006 at HELAS workshop in Nice Use UK-held DSA / DRMS interface for AstroGrid access to SDO data in Germany instead of UK?

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden Note: AstroGrid components (JES, DSA, MySpace, Workbenchl) may be located anywhere. Data Centres eSDO / Solar-B Prototype: system perspective UKSSDC ADS file system

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden Data Centres User access through AstroGrid HelioScope: solar data results from Simple Time Access Protocol services. AstroGrid workbench: STAP service access through HelioScope, DSA access through the Task Launcher. Task Launcher: locate DataSet Access components, submit ADQL queries STAP service will point to URLs in UK, US or Germany. DSA will integrated with DRMS (Data Resource Management System) in UK or Germany.

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden Visualization Tools Catalogues Small Event / CME Dimming catalogue Generated from continuously running of small event detection / CME dimming region recognition algorithms Tables of event / CME statistical information Helioseismology Mode Parameters catalogue GONG month of data generated from mode parameters analysis algorithm Generated in US Thumbnail catalogue Basic metadata for generated thumbnail images Catalogues can be accessed through ADQL query to DSA.

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden Visualization Tools SDO streaming tool User Facilities Pan in space  Zoom in space  Pan in time Zoom in time Data Products LoS Magnetogram HMI continuum map AIA images Future Work FFT / wavelet analysis for coronal wave investigations

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden Visualization Tools On-the-fly Image Galleries and Movies Thumbnail Extraction Extract thumbnails in US from HMI and AIA science products Image Gallery Generation User specifies start time, stop time, cadence, data product Image gallery generated on the fly in web browser Movie Generation User specifies start time, stop time, cadence, data product MPEG movie generated on the fly Find data with AstroGrid HelioScope. Use PLASTIC to load data into visualization tools.

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden Future Work Phase B implementation development now in progress: Summer / Autumn 2006: –Unit and science testing for algorithms –Add STAP service client to visualization tools –Add movie functionality to streaming tool –Investigate DRMS integration with DSA 2007: –completed algorithms –algorithm integration with AstroGrid, JSOC and SolarSoft –Coronal wave analysis in SDO streaming tool –Work with HELAS to explore European SDO data centre –Catalogues