eSDO Consortium Meeting Elizabeth Auden 24 October 2006 2nd year internal review RAL 24 October 2006 eSDO Consortium, RAL Elizabeth Auden

Agenda <12:00 Room booked all day all 12:00 Welcome and review of recent meetings EA, LC 12:30 Review of interactions with JSOC and AstroGrid EA 13:00 Progress report: data centre and visualization tools EA 13:30 Lunch all 14:00 Progress reports / demos: algorithms DB, TT, MS, SZ, VG 15:30 Technical update: unit tests, science tests, documentation EA 16:15 Discussion: increasing community awareness all 16:30 AOB all 17:00 finish    

Deliverables By end of December 2006 Completed unit tests 3 – 5 total science tests per algorithm defined Streaming tool movie maker functionality Remote DRMS at MSSL So far (October 2004 – October 2006) 9 solar algorithms 7 deployed through AstroGrid 1 submitted to JSOC pipeline 3 Visualization tool prototypes Streaming tool plus feedback report Data centre Network latency test report ADS / AstroGrid DSA prototype being revamped to work with DRMS Proposed UK data centre architecture evolving

Meetings AIA / HMI workshop, Monterey; Feb 2006 UK Solar Missions Forum, UCL; Feb 2006 MIST / UKSP, Aberystwyth; April 2006 Coronal Wave Meeting, UCL; July 2006 Solar Community Meeting, Manchester; July 2006 UK e-Science All Hands, Nottingham; Sept 2006 HELAS Roadmap workshop, Nice; Sept 2006

Update: Data Centre and Visualization Tools

Data Centre JSOC DRMS now available for Postgres Postgres 8.1.4 at MSSL; awaiting tables AstroGrid DSA 3.0.4 DRMS currently trialled at Stanford, remote installation being configured at MSSL Experimentation with remote DRMS / SUMS and AstroGrid underway now between MSSL and Stanford; next round of experimentation will include Lindau  

Data Centres Possible European Data Centre TLAs: DRMS: Data Resource Management System (JSOC) SUMS: Storage Unit Management System (JSOC) DSA: DataSet Access (AstroGrid) EU Framework 6 support for European helioseismology data and algorithms Possible full mirror of SDO data in Europe? DSA / DRMS interface for AstroGrid access to full mirror of SDO data in Europe? DRMS DRMS DRMS SUMS SUMS US Europe DSA UK UK datasets

Visualization Tools Working Prototypes SDO Streaming Tool prototype Image Gallery Thumbnail Maker (demos) Future Work  Movie Maker Serverside Streaming Tool, Image Gallery and Movie Maker combined

Visualization Tools SDO streaming tool User Facilities Pan in space P Zoom in space P Pan in time Zoom in time Data Products LoS Magnetogram HMI continuum map AIA images Future Work FFT / wavelet analysis for coronal wave investigations

Visualization Tools On-the-fly Image Galleries and Movies Thumbnail Extraction P Extract thumbnails in US from HMI and AIA science products Image Gallery Generation P User specifies start time, stop time, cadence, data product Image gallery generated on the fly in web browser Movie Generation MPEG movie generated on the fly Find data with AstroGrid HelioScope. Use PLASTIC to load data into visualization tools.

SDO Streaming Tool Feedback Feedback from formal and informal demonstrations at SOHO 18, HELAS workshop and All Hands Mtg. Change navigation from arrows to “google grab” Ensure that webstart tool works on Macs (tested on Linux and Windows; works on some Macs) Move to serverside hosting Add instrument specific processing Enable movie functionality Add coronal wave power analysis; add relevant eSDO algorithms

Update: Algorithms

Algorithm Talks and Demonstrations RAL: Small Event Detection, Coronal Dimming MSSL: Coronal Loop Detection, Magnetic Extrapolation, Helicity Computation Sheffield: Local Helioseismology Inversion, Perturbation Map Generation, Subsurface Flow Analysis  Birmingham: Mode Parameters

Technical Updates

Science Integration Tests Developers and scientists: Algorithm Deployment pages fulfil first half of science integration tests: compilation, execution, 1 test case. Choose 3 – 5 total test cases for each algorithm. Human-run end-to-end tests that showcase limits of algorithms. Document and attach test input files on wiki deployment pages. 

CuTests By 20 September 2006, nearly all algorithms had at least 1 CuTest per .C program. By end of December 2006, all algorithms should have thorough CuTest unit testing. Where possible, test: Varied numerical input: all zeros, negative numbers, numbers outside valid high and low ranges, expected input Empty files, invalid files, files with incorrect dimensions, valid files Varied string inputs: valid flags, incorrect flags, empty flags

Path to Files in Algorithms Any text or FITS file distributed with an algorithm must be called with a full path, not a relative path. Use makefiles to configure full paths; see Coronal Loop makefile in CVS for an example. Files with relative paths: Dataio.c optimal_constants.h Initialization.c Inversion.c Bfield.c inout.h Relax1.c Loop.c  

Makefiles Makefiles have been created for all algorithms at MSSL with configurable paths to external libraries. Makefiles, test inputs, README files and individual *.h libraries will be bundled into tarballs with C code for distribution. External libraries, such as cfitsio, FFT3W and GSL will be linked in deployment pages and makefiles, but not included in distribution.  

Documentation Algorithm Deployment pages look great. Please make sure that in-code documentation is good: Follow documentation patterns in eSDO code standard Variable and method names should be as self-describing as possible without becoming unwieldy Flag beginning and end of long for / if / while loops  

Community Awareness Try Our Software Instructions and links: http://www.mssl.ucl.ac.uk/twiki/bin/view/SDO/TrySoftware Instructions and links: Algorithms with AstroGrid workbench Visualization tools with HelioScope Data Centre prototype (under construction)