1. www.csiro.au Towards Service Oriented Geoscience SEE Grid and APAC Grid Dr Robert Woodcock Executive Manager, e-Science

2. 2 Outline Industry drivers Inefficiencies in “geoscience” modelling workflow The Solid Earth and Environment Grid The APAC (Geoscience) Grid Putting it all together: pmd*CRC Modelling Workflow for Industry problems Results and what might the future hold?

3. 3 Australian National Research Priorities Frontier Technologies for Building and Transforming Australian Industries: Stimulating the growth of world-class Australian industries using innovative technologies developed from cutting-edge research Priority Goal 4: Smart information use Improved data management for existing and new business applications and creative applications for digital technologies  ICT applications are providing huge opportunities to deliver new systems, products, business solutions, and to make more efficient use of infrastructure  The ability of organisations to operate virtually and collaborate across huge distances in Australia and internationally hinges on our capabilities in this area

4. 4 Key points from case studies and support letters Show the diversity of use cases for the same data type throughout the mining value chain Show a strong business case for interoperability for management of your data in the external world Show an even stronger business case for interoperability for internal data management Show why standards need to be developed by groups working together as part of a community Highlight the emerging issue that responsibility of data quality becoming a legislative issue

5. 5 Key Driver: Input to the Minerals Exploration Action Agenda – July 2003 Industry input highlighted  problems in gaining access to pre-competitive geoscience information  described existing information as commonly incomplete and fragmented across eight government agencies, each with its own information management systems and structures  noted that the disparate systems lead to inefficiencies causing higher costs, reduced effectiveness and increased risk incurred by the industry and its service providers Source: http://www.industry.gov.au/assets/documents/itrinternet/minerals_aa_finalreport_July2003.pdfhttp://www.industry.gov.au/assets/documents/itrinternet/minerals_aa_finalreport_July2003.pdf

6. 6 What is the role of: Competency contrasts? Permeability? Pore fluid pressure & flow fields? Modelling Workflow Define the geological problem Build the model Run the model View and Interpret Results Iterate to achieve Understanding Report and feed into knowledge base …Must be repeatable, robust and timely very weak strong weak strong Tensile failure mod. strong Block model of dilation: showing impact of Fault set “A” Dip variation

7. 7 Inefficiencies in the Workflow Information is scattered across:  Organisations – company, geological survey, etc  Resources – different hardware and software platforms  Geography – geological surveys in each state and territory (region) in Australia  Cost of data integration is high, in some situations exceeding all other costs Computational resources:  Different architectures suit different numerical codes better  Are often available but outside your organisations direct control  Are setup in different ways  Cost of adapting an investigators specific toolkit to use multiple sites is often prohibitive Can these issues be removed?

8. www.csiro.au The Solid Earth and Environment Grid Obtaining information…

9. 9 The SEE Grid Community Working together (loosely) to develop a toolkit for interoperability for the Solid Earth and Environmental Sciences  Together… because our information and services need to be shared more easily to achieve our goals  Loosely… because ultimately we are separated by political and economic boundaries  Toolkit… because our World is dynamic and we need tools that can be reconfigured and chained together quickly to answer our questions …in this context we must reduce the barriers to becoming a part of the community

10. 10 Data Structures Proprietary Software Versions of Software Client Pre-competitive geoscience data - The trouble is… Slide courtesy of Stuart Girvan

11. 11 XML GML/XMML Client Our aim… Slide courtesy of Stuart Girvan

12. 12 PIRSA Web Feature Service (WFS) Common Interface Binding – GML/XMML GA Geochemistry Feature Data Source DOIR Geochemistry Feature Data Source DOIR Web Feature Service (WFS) GA Web Feature Service (WFS) Geoserver (Open Source) PostGIS (Open Source) Oracle PostGIS (Open Source) CLIENT APPLICATIONS DATA ACCESS SERVICES DATA SOURCES WebMap Composer GA Reports Application PIRSA Geochemistry Feature Data Source Little or no change required here Translation to standards here

13. 13 Common Interface Binding – GML/XMML WebMap Composer GA Reports Application PIRSA WFS DOIR WFS GA WFS NTGS WFS MRT WFS NRM WFS NSWDPI WFS VICDPI WFS ? FracSIS pmd*CRC Model Tools CLIENTS DATA SOURCES DATA SERVICES

14. www.csiro.au The Solid Earth and Environment Grid Information - Implementation and Examples

15. 15 Common Interface Binding - Details Two parts 1.Service interface standard – how you communicate with the service, sending requests and receiving results 2.Information standards – how information is encoded in a community agreed form We use and develop Open Geospatial Consortium and the Exploration and Mining Mark-up Language and its successor, GeosciML

16. 16 Open Geospatial Consortium Web Feature Service (WFS) Web Feature Service Get Capabilities Request Get Capabilities Response Describe Feature Type Request Describe Feature Type Response Get Feature Request Get Feature Response http protocol XML/ KVP XML XML/ KVP GML Schema XML/ KVP GML Data Source Config Files Application (web based or desktop) Response in Geography Mark-up Language (GML) - Or more usefully, a GML Application Schema

17. 17 Features – Geoscience Community (XMML & GeoSciML) Borehole  collar location  shape  collar diameter  length  operator  logs  related observations  … Fault  shape  surface trace  displacement  age  … Ore-body  commodity  deposit type  host formation  shape  resource estimate  … Observation  location  subject/specimen/station  property/theme  method  operator  date/time  result (+ type/reference system/scale/classification)  … Basin?  formations  shape – time dependent  resource estimate  …

18. 18 Data source to community schemas Community schemas provide the common or shared model All data providers have their own local data model  All data providers must map data from local source (database) to community schema, irrespective of technology implementation

19. 19

20. 20 Why XML?  Extensibility  Self describing  Ability to be (remotely) validated against schema  XML Schema provides “loose tolerances”  All software languages have tools to deal with XML But… Problematic for large data sets… though nobody said you can’t use binary as well (even over WFS)  Community agreement is what matters

21. 21 A user makes a request and gets back GML based data which can be …. Rendered into a map layer AND queried by a user or…. … formatted into a report or …. … read and used by any enabled application Slides courtesy Stuart Girvan – Geoscience Australia How would you use an interoperable service?

22. 22 Web Map Interface (courtesy of Social Change Online) Bounding Box Known Layers

23. 23 Tabular Reports by Source (courtesy of Geoscience Australia)

24. 24 Desktop Visualisation (courtesy of Fractal Technologies)

25. www.csiro.au High Performance Computing in Exploration and Mining

26. 26 Why use simulation and modelling? Mineral exploration has considerable uncertainty We use simulation and modelling to analyse an ensemble of possible geological structures and histories that could have produced the observations seen today The result is reduced uncertainty and some quantification of risk This same approach applies to many fields – hazards, environment, … which is why we formed SEE Grid community

27. 27 Our toolkit… Our toolkit contains a variety of codes (usually more than one each type) for  Mechanics  Chemistry  Transport  Thermal  Fluid flow Some of these can be coupled together:  Reactive Transport – Chemistry+Transport+Thermal+Fluid flow Some scenarios only require a subset… It becomes very computationally intensive when using many… AND we run many scenarios Grid Computing provides a solution Darcy flow and Streamlines

28. 28 Community Agreed Service Interfaces and Information Models Industry Data and Knowledg e Grid APAC Data and Compute Grid APAC Web Feature Service (WFS) Industry Web Feature Service (WFS) Client Applications Gateway Services Facilities Drill Core Analysis Workflow Government Geological Surveys Data and Knowledge Grid Mantle Convection Modelling Workflow Tsunami Workflow Reactive Transport Workflow Geological Survey Web Feature Service (WFS)

29. 29 Grid Technology Layers APAC Grid pmd*CRC SEE Grid

30. 30 Client The Grid Application… Service Interactions Resource Registry Data Management Service HPC Repository Login Job Monitor Run Simulation Edit Problem Description Local Repository Archive Search Geology S.A Geology W.A Geochem N.S.W Geochem W.A Information Authentication Job Management Service Escript Service FastfloRT Service User Workflow... Computation Community Infrastructure Physical Resource

31. 31 Traditional Mechanical Modelling Workflow “Powerful” desktop computes several models at a time Limitations are in the order of ~2 models per week Models (mesh + data files) are individually and laboriously constructed The manual process is error prone Results are manually visualised one at a time Screenshots are manually taken and made into “movies” Very little, if any, standardised data archiving is done. This results in potential confusion or loss of the originating conditions of the experiments, making it unrepeatable in the long term Slide courtesy of Robert Cheung and Warren Potma

32. 32 New Refined Workflow Parameterised template or wizard driven model geometry/mesh creation Boundary condition & model properties parameter sweep utilities automatically creates a “family” of model, data files based on varying a set of parameters Inversion algorithms determine input parameters of future iterations automatically based on the user ranking of previous results Automated generation of visualisations Automated movie generation Automated archiving 3D Time varying volume visualisation Parameterised Geometry Creation Multi-site data storage via Storage Resource Broker Slide courtesy of Robert Cheung and Warren Potma

33. 33 Results to Date For one Investigator, on one investigation: 500 Models in 4 months (100x more!) Inversion/parameter sweep algorithms – semi-automated model creation; faster, less errors Automated post-processing/visualisation – all views X all timescale X all models await the investigator automatically Automated archiving – metadata searchable, more accurate store of experimental conditions, delivered to your store!

34. 34 Results Major inefficiencies have been removed by: Integrating the pmd*CRC geoscience modelling workflow with the: Solid Earth and Environment Grid, and APAC (Geoscience) Grid Industry response to approach is supportive as evidenced by SEE Grid Roadshow survey results and pmd*CRC applications

35. www.csiro.au Thank You NameDr Robert Woodcock TitleExecutive Manager, e-Science Phone+61 8 6436 8780 EmailRobert.Woodcock@csiro.au Webwww.csiro.auwww.csiro.au www.seegrid.csiro.au Contact CSIRO Phone1300 363 400 +61 3 9545 2176 Emailenquiries@csiro.au Webwww.csiro.au