1. Using ISO/IEC 11179 to Help with Metadata Management Problems Graeme Oakley Australian Bureau of Statistics

2. Content of Presentation  ABS Strategy for Management of Metadata  Examples of Problems to Address  How are we using ISO 11179  Further work

3. Strategy for Management of ABS metadata  Background - senior management project  Problems to address  Principles  Metadata Model  Strategies

4. "Metadata Management refers to the content, structure, and designs necessary to manage the vocabulary and other metadata that describes statistical data, designs and processes.... includes the development of metadata models..., building metadata registries to organise the metadata..., developing statistical terminologies which define and organise terms..." (Bargmeyer and Gillman, METIS 2000) Metadata Management

5. Examples of ABS metadata  Data Item definitions, eg person, sales  Classification description, category values and hierarchy eg Industry, Household type  Collection - who, how, when, where  Quality eg response rate, survey error measures  Provider load - how long to complete forms  Collection form development  Process - edits, imputation  Glossary, Themes, Products

6. Problems (not an exhaustive list)  SMA's have to re-enter metadata  Metadata is not reused  What is 'standard'?  Terminology  Existing business processes not widely used  Gaps in Infrastructure

7. Principles for Metadata Management  Metadata should be managed as part of a 'life- cycle'  One, authoritative source for each metadata type  Registration process for each type of metadata  New metadata not created until designer has determined that no useable metadata already exists  Business processes have a metadata flow described

8. Principles (ctd)  Metadata gathering activities should be minimised so that no metadata be entered more than once, and derivable metadata be automatically generated  Cost/benefit mechanism to ensure benefits to users justify costs to producers of metadata  Single approved standard description for each metadata type. Variants may be permitted but must always be linked to the 'standard'

10. Strategies  Communication and education  Allocation of 'registration' tasks to appropriate organisational units  Enhance metadata model to cover quality and process metadata  Development projects to fill infrastructure gaps  Use opportunities in other development projects to promote metadata mgt

11. Examples of Problems to address  What is the IDW and its relationship to metadata?  Definition of large number of data elements  Linking CMR and Input Data Warehouse  "Useful Metadata Structures" to consider  concordances  definition of legal and illegal combinations  ETL descriptions

12. Input Data Warehouse  Underpins Business Statistics Infrastructure Project  Repository for all business statistics unit record data  Star schema architecture  Use of shared definitional metadata from CMR  Process and quality metadata to be accessed and/or created  "Gatekeeper role" by Standards group

13. Data source Change reason Business location Organisation type Data item Processing stage Industry Reference period Fact Provider Change module IDW v1.0 dimensional model

14. Definition of data elements  Need for framework of concepts, properties, classifications  Potentially thousands of data elements to define and approve  Concepts and terminology of ISO 11179  Links to question wording and question modules needed

17. Linking IDW and CMR  'Star schema' elements should be based on metadata in CMR  data sources (surveys and their cycles)  key classifications eg industry  'data items'  'Snapshot' rather than dynamic access  Only use 'approved' metadata  More services in 2004 eg form information, better searching, transfer from IDW to CMR

18. Useful Metadata Structures  Definition of legal, illegal and questionable combinations  Concordances between an IDW value domain and underlying 'standard' classification  Specialised transformations - related to ETL processes

19. How are we using ISO 11179?  11179 provides advice about registration  Concepts of properties, object classes, qualifiers and value domains (classifications)  11179 is a general standard, so wanted framework to link to existing statistical standards and to link to metadata objects beyond boundaries of 11179

21. Further Work  ABS senior management endorsed metadata mgt strategy  Now to look at implementation  new metadata stores  enhance existing metadata stores & services  deployment of 'registration authority' concept along with education & communication to staff

22. Investigations related to 11179  Definition of table or matrix, ie the result of the 'aggregation' process  When to use qualifiers  Expression of 11179 model in an XML schema  Defining value domains and rules to drive 'editing engines', ETL process

23. QUESTIONS AND DISCUSSION