Using the Assay Data Exchange standard with WFS to build a complete minerals exploration data-transfer chain Simon CoxA.Dent, S.Girvan, R.Atkinson CSIRO Exploration and MiningI. Whitehouse, C. Legg 5 September 2006R.M.Woodcock

ADX – IAMG 2006 Outline ADX history OGC “Observations and Measurements” ADX3 issues Interoperability

ADX – IAMG 2006 History ADX 1  Aug-Sept 2002  Sponsors: Metech, Anglo-American, Newmont, Placer-Dome, WMC  Contractor: Pindan Software LIMS-focussed XML document matched workflow, not information model Iffy use of XML and XML Schema

ADX – IAMG 2006 History (2) ADX 2  May-Oct 2003 (report: Feb 2004)  Sponsors: Metech, Anglo-American, Newmont, Placer-Dome, WMC, BHP-Billiton, ALS Chemex  Contractor: CSIRO Introduced normalised Procedure/Specimen/Analysis model Inherited some limitations from ADX1

ADX – IAMG 2006 ADX3 Goals and experiments 1.Harmonize ADX with generic standards (O&M) + GeoSciML 2.Refine Specimen + Procedure model tracking specimen processing history is key to QA/QC procedures tracking chain-of-custody is key to audit trail 3.Testing, feedback, documentation, examples, training  Collaborators: Metech, Newmont, ALS Chemex

ADX – IAMG 2006 Standard transfer format allows multiple data sources WFS Client WFS Server WFS Server B WFS Server C

ADX – IAMG 2006 User GML for assay data: XMML and OGC O&M XMML Demonstrated: Same format and standards

ADX – IAMG 2006 Completing the loop? “From field to desktop” Regulator Assay Lab ADX Assay-WFS Assay-WFS/T ADX-request?

ADX – IAMG 2006 Design methodology: model in UML, serialize in XML Specimen test instance Specimen test 1 … chips 0.35 … GML Application Schema

ADX – IAMG 2006 Generic “Observation” model An Observation is an Event whose result is an estimate of the value of some Property of the Feature-of-interest, obtained using a specified Procedure - “metadata for a property estimate”

ADX – IAMG 2006 Assay measurements – specialized observation model “Rock specimen” “Concentration[analyte]” “Instrument/technique” Domain-specialization of observed property, feature of interest, procedure

ADX – IAMG 2006 Assay data issues Many measurements per (delivered) specimen: different analytes same analyte QA/QC - “checks”, “splits”, “duplicates”, “replicates” 1.different instruments/labs 2.same instrument, different time 3.different specimen preparation (“splits”)

ADX – IAMG 2006 How to model measurements on split specimens? customer viewpoint: one specimen, complex procedure  compound “procedure” composed of  preparation phase (as above)  followed by analytical phase lab viewpoint: many specimens, standard analytical procedure  derived from incoming specimen, with different preparation histories  multi-step preparation procedure  splitting may occur at various points  “biased” vs “unbiased” splits  need to record exact splitting point …  pulp may then go into several analytical systems

ADX – IAMG 2006 Generic “Specimen” model

ADX – IAMG 2006 Procedure systems, events, histories

ADX – IAMG 2006 Standard procedure? preparation and analytical phases  based on a standard design sequence (“path”),  but a specific instance (“history”) 1.has parameters recorded for each step  % passing a given sieve, weight-in/weight-out, instrument settings, … etc 2.may vary from the prototype How much information can be “inherited”, how much should be explicit …? N.B. This information does matter:  analysis of results  billing!

ADX – IAMG 2006 Raw XMLZipped XML Explicit1470 kB39 kB Hand-optimised650 kB36 kB Different encoding strategies 1.fully explicit “UML→GML” – direct from UML model to an XML document apparently verbose lines of XML per split much duplication between splits 2.manual optimization – inherit parameters from standard procedures less duplication indirect mapping to information model– XML must be maintained separately from model, processing requires more conditions and loops Concerned about file-size? 650 kBHand-optimised 1470 kBExplicit Raw XML

ADX – IAMG 2006 Summary ADX can potentially harmonize with GeoSciML, O&M challenge in modeling prep/analytical procedures let Shannon take care of document size issue standards support interoperability, lead to round-tripping

Thank You CSIRO Exploration and Mining NameSimon Cox TitleResearch Scientist Phone Webwww.seegrid.csiro.au Contact CSIRO Phone Webwww.csiro.au

ADX – IAMG 2006 Conceptual object model: features Digital objects correspond with identifiable, typed, objects in the real world  mountain, road, specimen, event, tract, catchment, wetland, farm, bore, reach, property, license-area, station Feature-type is characterised by a specific set of properties Specimen  ID (name)  description  mass  processing details  sampling location  sampling time  related observation  material ……

ADX – IAMG 2006 Why is a standard methdology useful? GML-based encoding is supported by standard web data access protocols from Open Geospatial Consortium

ADX – IAMG 2006 Is any of this for real? SEEGrid interoperability demonstrator

ADX – IAMG 2006 PIRSA Web Feature Service (WFS) Common Interface Binding – GML/XMML GA Geochemistry Feature Data Source DOIR Geochemistry Feature Data Source DOIR Web Feature Service (WFS) GA Web Feature Service (WFS) Geoserver (Open Source) PostGIS (Open Source) Oracle PostGIS (Open Source) CLIENT APPLICATIONS DATA ACCESS SERVICES DATA SOURCES Mapping Reports PIRSA Geochemistry Feature Data Source Little or no change required here Translation to standards here

ADX – IAMG 2006 Completing the loop “From field to desktop” Regulator Assay Lab ADX Assay-WFS Assay-WFS/T ADX-request?

ADX – IAMG 2006 Assignment of property values Each property value is either i.asserted  name, owner, price, boundary (cadastral feature types) ii.estimated  colour, mass, shape (natural feature types)  i.e. error in the value is of interest

ADX – IAMG 2006 Observations and Features An estimated value is determined through observation i.e. by application of an observation procedure

ADX – IAMG 2006 Feature of interest may be any feature type from any domain-model … observations provide values for properties whose values are not asserted i.e. the application-domain supplies the feature types

ADX – IAMG 2006 Some feature types only exist to support observations

ADX – IAMG 2006 Observation model Generic Observation has dynamically typed result

ADX – IAMG 2006 Observation specializations Override result type

ADX – IAMG 2006 Observations support property assignment These must match if the observation is coherent with the feature property N.B. Each “Phenomenon” definition reifies a (Feature) property definition

ADX – IAMG 2006 Observations support property assignment

ADX – IAMG 2006 premises: O&M is the high-level information model SOS is the primary information-access interface SOS can serve: an Observation (Feature)  getObservation == “getFeature” (WFS/Obs) operation a feature of interest (Feature)  getFeatureOfInterest == getFeature (WFS) operation or Observation/result (often a time-series == discrete Coverage)  getResult == “getCoverage” (WCS) operation or Sensor == Observation/procedure (SensorML document)  describeSensor == “getFeature” (WFS) or “getRecord” (CSW) operation Sensor service optional – probably required for dynamic sensor use-cases

ADX – IAMG 2006 SOS vs WFS, WCS, CS/W? WFS/ Obs getFeature, type=Observation WCS getCoverage getCoverage (result) Sensor Registry getRecord SOS getObservation getResult describeSensor getFeatureOfInterest WFS getFeature SOS interface is effectively a composition of (specialised) WFS+WCS+CS/W operations e.g. SOS::getResult == “convenience” interface for WCS

ADX – IAMG 2006 Non-map use of the service

ADX – IAMG 2006 Tabular Reports by Source Three servers, one report

ADX – IAMG 2006 GML-based data can be …. Rendered into a queryable map … … formatted into a report or …. … read and used by any WFS/GML enabled application Condition: community-standard data model/encoding One service – many uses

ADX – IAMG 2006 PIRSA DOIR GA Three servers, one map Mapping client

ADX – IAMG 2006 Bounding Box Layers Initial Map Interface – HTML & SVG Click a Map Feature

ADX – IAMG 2006 …and see all Analytes on that sample

ADX – IAMG 2006 Desktop Visualisation (Fractal Technologies)

ADX – IAMG 2006 Data Structures Proprietary Software Versions of Software Client Now – Different formats and standards