Virtual Geophysics Laboratory (VGL) VGL v1.2 NeCTAR Project Close R.Fraser, T.Rankine, J.Vote, L.Wyborn, B.Evans, R.Woodcock, C.Kemp July 2013 CSIRO | MINERALS DOWN UNDER FLAGSHIP

Gather data, process it, publish results Simple, isn’t it? bedrock surficial mineral geochemical geochronologic hyrdrogeological Geo-information geophysical knowledge data Virtual Geophysics Laboratory v1.2 | Ryan Fraser 2 |

Hardcopy of data SSH Client Maths Package Proprietary Package Let’s map it out for Geophysics … Transform to a regular grid Crop data to area of interest Reformat data for processing Upload data to HPC resource Configure job and start processing Download results Get handed field data Visualise data Virtual Geophysics Laboratory v1.2 | Ryan Fraser 3 |

The Past Compile raw data using proprietary FORTRAN Also use other proprietary software Transform to a regular grid using more software MATLAB, Intrepid, ER Mapper, ESRI ArcGIS, QGIS, etc Crop data spatially eg: everything in Victoria Transform data into a file format that can be read by proprietary scientific code This is usually done with some handwritten python or c There is no version control, code is often rewritten / redone Upload data to HPC Manually enter input parameters/start job Where did my results go, how do I visualise? Virtual Geophysics Laboratory v1.2 | Ryan Fraser 4 |

There seems to be a problem… Reproducibility – there is none What was the input of your model? What transformations occurred? It’s a manual process Time consuming Error prone Expensive Licensing costs Effort/time Virtual Geophysics Laboratory v1.2 | Ryan Fraser 5 |

Scientific workflow – Virtual Geophysics Laboratory (VGL) Automated - Simplify access and integration Reproducible process Open access (less expense) Allow scientists to focus on the science and not the tech Exploiting the Cloud and HPC Operational or education tool Automates and massively expands geophysics computational capacity via the Cloud Virtual Geophysics Laboratory v1.2 | Ryan Fraser 6 |

VGL VGL - Virtual Geophysics Laboratory NeCTAR funded activity Collaboration with multiple partners (CSIRO, NCI, GA, UQ, Monash, ANU) Supporting multiple workflows New data types and collections New Use – Not just exploration. Done Virtual Geophysics Laboratory v1.2 | Ryan Fraser 7 |

Hardcopy of data SSH Client Maths Package Proprietary Package Let’s map it out… Transform to a regular grid Crop data to area of interest Reformat data for processing Upload data to HPC resource Configure job and start processing Download results Get handed field data Visualise data Virtual Geophysics Laboratory v1.2 | Ryan Fraser 8 |

…to this Virtual Geophysics Laboratory Build “science” from existing libraries Run job Collect and publish results Discover raw data Select spatial bounds Virtual Geophysics Laboratory v1.2 | Ryan Fraser 9 |

Virtual Geophysics Laboratory v1.2 | Ryan Fraser 10 | Introducing The Virtual Geophysics Laboratory

Data discovery 11 | Layers discovered via remote registries Layers consist of numerous remote data services Virtual Geophysics Laboratory v1.2 | Ryan Fraser

Data discovery 12 | Some data services support subsetting Some data services support reformatting e.g. CSV, NetCDF, GeoTIFF Virtual Geophysics Laboratory v1.2 | Ryan Fraser

Data discovery 13 | Virtual Geophysics Laboratory v1.2 | Ryan Fraser

Data processing 14 | A variety of different scientific codes are already available in the form of “Toolboxes” Flexibility in what computing resources to utilise Virtual Geophysics Laboratory v1.2 | Ryan Fraser

Data processing 15 | The steps so far have been building an environment to run a processing script...or build from existing templates Either write your own... Virtual Geophysics Laboratory v1.2 | Ryan Fraser

Managing results - provenance Presentation title | Presenter name 16 | All of a job’s outputs are also accessible Each job has a lifecycle that can be managed 16 | A job’s console log can be inspected Virtual Geophysics Laboratory v1.2 | Ryan Fraser

Managing results - provenance 17 | Successful jobs can have their entire process captured in a ISO ‘provenance record’ Each provenance record tracks all inputs, outputs, processing scripts and other metadata.... Spatial bounds... Input/output data... Virtual Geophysics Laboratory v1.2 | Ryan Fraser

Powered by the Spatial Information Services Stack Common Platform MarineEnvironment, Water Groundwater Geology Geophysics Virtual Geophysics Laboratory v1.2 | Ryan Fraser 18 |

VGL: Opportunities Free for research use Supporting multiple workflows Model Registry (3D) – researcher contributed models New Scientific Codes – Underworld, eScript, UBC, Airborne EM inversion codes + others (relatively “easy” to add) Data sets from GA: National Airborne Geophysical DB including –Gravity, Radiometric, AEM, Magnetics –Data sets from State Surveys Broad application and generic implementation Scalable Infrastructure (Cloud) Virtual Geophysics Laboratory v1.2 | Ryan Fraser 19 |

What’s next? VGL sustainability strategy kick-ins More Data and More Codes Extensions for new domains beyond the geosciences Reuse of infrastructure by additional clients Commercial connections Roadshow for state government geologic surveys (funded by AuScope) Provenance Service – take learnings of VGL and research to create a generic prov service 20 | Virtual Geophysics Laboratory v1.2 | Ryan Fraser

Thank you Ryan Fraser Research Manager t wsiss.auscope.org VGL is Power by: VGL acknowledges funding from the NeCTAR project NeCTAR is an Australian Government project conducted as part of the Super Science initiative and financed by the Education Investment Fund. VGL is a collaboration between: VGL is supported by: University of Queensland, Monash University and the Australian National University