1. Metadata – A Bedrock for Official Statistics Dr. S. M. Tam, Chief Methodologist, ABS

2. Outline What questions am I addressing? –What metadata to use to support dissemination? –How much metadata to use? -The answer Is ……….. for the paper world; and …….for the WWW. Metadata –A “boring” and “confusing” subject? –“Data about data”? –Metadata for Fitness of purpose (Reference metadata) information exchange – humans and WWW (Structural metadata) Improving efficiency of processes (Process metadata) – will not cover in this talk

3. Reference metadata “Data on fitness for purpose” –An example from ABS 2006 CensusABS 2006 Census It exemplifies –“good practice” to hyper-link up information –Conceptual metadata –Production metadata (eg Quality declarations)

7. 3,38067.1255,05271.415,017,84769.8 1553.12,9560.8185,0390.9 1382.713,0503.7911,5934.2 1272.56,5921.8318,9691.5 1002.05,8861.6295,3621.4 791.62,42420.7171,2340.8 What do these numbers signify?

8. Country of BirthFlorey%Australian Capital Territory %Australia% 3,38067.1255,05271.415,017,84769.8 Vietnam1553.12,9560.8185,0390.9 England1382.713,0503.7911,5934.2 China (excludes SARs and Taiwan)1272.56,5921.8318,9691.5 India1002.05,8861.6295,3621.4 Philippines791.62,42420.7171,2340.8 In Florey (State Suburbs), 67.1% of people were born in Australia. The most common countries of birth were Vietnam 3.1%, England 2.7%, China (excludes SARs and Taiwan) 2.5%, India 2.0% and Philippines 1.6%. Numbers will only have meaning if they have context

9. Structural metadata “Data about content and container” –provides the context for human consumption for machine-to-machine communication Structural metadata can be described by –Container :Dimensions (variables) –Content: Attributes (observations) –Content: Measures (units of measurement)

10. Data Set Structure: Concept Usage Unit Multiplier Unit Topic Time/Frequency Country Stock/Flow Observation (Dimension) (Attribute) (Dimension) (Attribute) (Measure)

11. Structural metadata used to support Discovery of official statistics –Search engines “Linked Data” –Semantic Web/Web 3.0 Data visualisation Machine to machine communication Technical standards for structural metadata –Statistical Data and Metadata Exchange (SDMX) –Data Documentation Initiative (DDI) –Data Cube Vocabulary (DCV – W3C)

12. Structural metadata to discover statistics Discovery metadata –Variable names (Container Structural Metadata) Other means (specially created) to aid WWW search –Key words –Catalogues etc. Google search –“Page rank” to rank matches based on Frequency of keywords on webpage Age of webpage No. of other sites linking to the webpage –SEO is a big industry

13. Search engines do not always provide meaningful answers How many Web 3.0 companies are there in Abu Dhabi? –143 million hits from Google search –Yet there are only …. companies from SCAD A “deficiency” of Web 2.0 –Content of the structural metadata is NOT the problem –Need relationship between “objects” recorded on the web, and query technologies So a new approach Is needed –Web 3.0 or web of linked data Brain child of Sir Tim Berners-Lee

14. Structural metadata to support “linked data” What is the difference between Web 2.0 and Web 3.0? What is linked data?

15. Linked data – Tim Berners-Lee

16. 5 star Open Data Format

17. In a nutshell –Structure the “Structural metadata” using Resources Description Framework (RDF) Identity statistical concepts in Universal Resources Identifiers (URIs) –GSIM uniquely identifies metadata objects Linked data is an emerging but an increasingly important field for official statistics –Help us “ingest” data better –Help other better “digest” our data –USBC, CSO Ireland, Statistics Switzerland etc. have trialled linked data SemStat 2013 to be held Sydney, Australia

18. Structural metadata to support …….. Data visualisation (DV) –Structural metadata harvested for the visualisation application SDMX converter for Google Public Data Explorer –DV applications built on/support SDMX Flex – CB NCOMVA’s Statistics eXplorer Exchange of statistics from one computer to another – Web Services Structural metadata Technical standards to describe structural metadata such as SDMX Web Services protocols or standards –WSDL, SOAP and XML

19. To summarise What metadata to use to support dissemination? How much metadata to use? –Dissemination goals Assist users to determine fitness for purpose “Consume” the data – increasingly through linked data Data visualisation Machine to machine communication -The answer Is Reference Metadata the paper world; and -Reference Metadata and Structural Metadata (+ Suitable Technical Standards) for the WWW.

20. Useful references for linked data cubes Still a new an emerging field for statistical data –General introduction of Data Cube Vocabulary http://www.slideshare.net/der42/linked-data-hypercubes http://www.slideshare.net/der42/linked-data-hypercubes –General introduction to Linked Statistical Data Statistical Linked Dataspaces –A simple description of how linked data works http://data.gov.uk/blog/what-is-linked-data –TED talks by Sir Tim Berners-Lee Tim Berners-Lee on the next Web | Video on TED.com http://blog.ted.com/2009/03/13/tim_berners_lee_web/

21. Questions? Siu-Ming.Tam@abs.gov.au