Australian Virtual Observatory Pacific Rim Applications and Grid Middleware Assembly The 4th Workshop 5th-6th June 2003 Monash University David Barnes School of Physics, The University of Melbourne

What is a Virtual Observatory? A Virtual Observatory (VO) is a distributed, uniform interface to the data archives of the worlds major astronomical observatories. A VO is explored with advanced data mining and visualisation tools which exploit the unified interface to enable cross-correlation and combined processing of distributed and diverse datasets. VOs will rely on, and provide motivation for, the development of national and international computational and data grids.

Scientific motivation Understanding of astrophysical processes depends on multi-wavelength observations and input from theoretical models. As telescopes and instruments grow in complexity, surveys generate massive databases which require increasing expertise to comprehend. Theoretical modeling codes are growing in sophistication to consume available compute time. Major advances in astrophysics will be enabled by transparently cross-matching, cross-correlating and inter-processing otherwise disparate data.

Sample multi-wavelength data for the galaxy IC5332 (Ryan-Weber) Visible blue light - young hot stars Infrared light - old cooler stars H-alpha spectral line - star forming sites HI spectral line - gas to form stars HI velocity field - kinematics HI velocity dispersion - gas stability, parameters Integrated HI spectrum - total neutral gas mass, distance from redshift And this is just the data on one object from three Australian telescopes!

Fundamental VO Challenges Data description: multi-wavelength, multi- resolution, multi-dimensional, multi-domain (optical, radio, X-ray, …), world coordinate systems, limited period ownership, … Data provision: distributed mass storage, high-bandwidth networks, registries, … Data processing: high performance clusters as grid nodes, data to code versus code to data, mountains of legacy software!, … Interface: portals, visual data flow control, analysis tools, display tools, …

Aus-VO structure 2003 Phase A funded AUD 260K by a 2003 ARC grant: –The University of Melbourne –The University of Sydney –CSIRO Australia Telescope National Facility –Anglo-Australian Observatory Additional institutes participating w/o direct funding from the ARC grant: –ANU, Mount Stromlo Observatory & APAC –CSIRO Mathematical and Information Sciences –University of Queensland –VPAC & GridBus (Melb) Lead investigator Rachel Webster (Melb) Project scientist David Barnes (Melb)

Aus-VO projects 2003 Common format on-line archive projects: –HIPASS catalog: HI Parkes All Sky Survey: neutral Hydrogen spectral line survey, ~4,300 sources with 138 parameters and 1024-channel spectra –SUMSS catalog: Sydney University Molonglo Sky Survey: radio continuum survey at 843 MHz, >100,000 sources –2dFGRS QSO catalog: 2-degree Field Galaxy Redshift Survey: optical spectra of >20,000 southern quasi-stellar objects –ATCA archive: Australia Telescope Compact Array archive: all observations since 1988, circa 1.5 TB of more than 1,000 separate observing projects! Massive exercise in describing data with metadata. –MACHO archive: Massive Compact Halo Objects archive: 8yr lightcurves for >18M stars

Aus-VO projects 2003 Server-based visualisation tools: –client Java canvas for legacy software package AIPS++ to draw on from a remote server (ATNF) –grid-service implementation of distributed volume rendering - remote data transferred to remote cluster, with display canvas applet supplied by coordinating portal Pipelines to enable on-line reprocessing of archived raw or pre-processed telescope data: –Molonglo Observatory Synthesis Telescope Interfaces: beta testers for the AstroGrid consortium software

VO Interface & Portal Agreement with AstroGrid (UK e-Science project) to be testers for their data publication and portal creation code. Collecting the necessary resources and intend to have an AstroGrid-based portal serving HIPASS catalog data for demonstration at IAU General Assembly in July Separately testing IBM Lotus Notes and Domino Server for publication of astronomical catalogs.

Grid-based Visualisation ATNF will build a Java PixelCanvas so that AIPS++ visualisation applications can be deployed as Web- Service and Grid- Service Java Applets AIPS++ is modern, OpenSource software for reducing (radio) astronomy data, 1.6M lines of code.

Grid-based Volume Rendering [see my talk at GridBus this Saturday]Agreement between Melbourne and AstroGrid to develop our existing distributed-data volume rendering code into a fully- fledged Grid-Service. [see my talk at GridBus this Saturday] Challenge is to interactively render a multi-GB cube at the IAU GA 2003, using GridFTP to transfer the data volume from a remote data warehouse to a remote rendering cluster and display and control the rendering from an applet.

The near future: data grids for Aus-VO Australian archives range from ~10 GB to ~10 TB in processed (reduced) size. providing just the processed images and spectra on-line requires a distributed, high- bandwidth network of data servers – that is, a data grid. users may want some simple operations such as smoothing or filtering, applied at the data server. This is a virtual data grid.

The near future: compute grids for Aus-VO More complex operations may be applied requiring significant processing: –source detection and parameterisation –reprocessing of raw or intermediate data products with new calibration algorithms –combined processing of raw, intermediate or "final product" data from different archives These operations require a distributed, high- bandwidth network of computational nodes – that is, a compute grid.

2004 ARC LIEF grant 10 partners! more data archives on-line more tools developed with special focus on server-based visualisation Australian Astronomy Grid…construction of the Australian Astronomy Grid…

The Australian Astronomy Grid 2004