Metadata-driven Business Process in the Australian Bureau of Statistics Aurito Rivera, Simon Wall, Michael Glasson – 8 May 2013

Content 1.Why have it? 2.What is meant by metadata-driven business process? 3.What is the ABS doing? Two case studies 4.Implications for metadata management 5.Key messages

Why have metadata driven business process ABS perspective - the business benefits include Reduced time and cost of statistics production Improved quality of statistical products Increased agility in meeting new demands for statistical products and services. Increased agility in harnessing new sources of statistical data. Industry perspective - directly supports the HLG’s approach to industrialising and standardising statistics production.

What is metadata-driven business process In the ABS, it is the systematic and consistent use of metadata to determine the inputs, outputs and behaviour of a statistical business process.

What is metadata-driven business process Key characteristics 1.Metadata is used systematically Metadata is used in a planned and managed way across the organisation. 2.Metadata is used consistently Authoritative ‘single source of truth’ metadata is used throughout the end- to-end lifecycle of an activity and/or across activities. 3.Metadata is used actively Metadata is used to automate the definition and execution of statistical processes Metadata is structured so as to be machine-consumable.

What is metadata driven business process Necessary components for ABS implementation: 1.GSIM/DDI/SDMX-based metadata that serves as or identifies inputs to and outputs from processes standardised and discoverable via a metadata registry analogous to ‘blood that flows through vital organs’. 2.Libraries of business processes, methods and IT components that can be assembled in a ‘plug and play’ way that are atomic or are complex processes that are assembled from atomic process components 3.Effective governance framework to ensure the appropriate specification and use of contents that dictate what and how metadata is used in what processes.

What the ABS is doing in this area Examples of earlier work: 2002 – Business Statistics Innovation Program (BSIP) 2003 – development of ABS strategy for e-2-e management of metadata 2009 – formal adoption of SDMX and DDI Standards 2010 – Information Management Transformation Program (IMTP). Examples of more recent work (under the ABS 2017 Program): Metadata Registry and Repository (MRR), Statistical Workflow Management systems (SWM) Online Forms Prototype Project (eForms) Enterprise Data Warehouse (EDW).

Case study 1 – MRR/SWM Two Core applications that will form the ‘central nervous system’ of future ABS metadata management infrastructure. MRR/SWM are intended to guide and quicken the work of Statistics Producers as well improve metadata management. MRR/SWM in themselves are not sufficient to realise the business benefits. Also need Business Process re-engineering, instilling Information Management governance, etc.

Case study 1 – MRR/SWM Components MRR is a store of statistical metadata that is authoritative, centralised, and standards based Repository – logically centralised store Registry – metadata catalogue and data directory. SWM helps to assemble business processes from a library of re-useable building blocks SWM helps to automate and manage the execution of these business processes. SWM in concert with MRR will: Guide and quicken the work of Statistic Producers Greatly improve the way we manage metadata.

Case study 1 – MRR/SWM Registry Repository Register key elements of metadata package E.g. Register metadata Complete metadata package resides in Repository

Case study 1 – MRR/SWM Repository Search for metadata here E.g. Metadata search Returns a list of matches based on search criteria Registry ? 1.Match1 2.Match2 3.Match3

Case study 1 – MRR/SWM Registry Repository E.g. Retrieve metadata based on search results ]

Case study 1 – MRR/SWM Proof of Concept undertaken in 2011/2012 Lessons learnt from POC Concepts and technologies employed are sound Automation of business processes feasible. First production version due December 2013

Case study 2 – eForms Prototype web-based collection tool Forms the basis for ABS capability for building and deploying web-based collection tool. Test the feasibility of using DDI 3.1 Instrument specification, Strategic ABS technologies, especially SWM/MRR GSBPM-aligned processes.

Case study 2 – eForms Components Web Instrument Production Application MRR SWM ABS Collections Electronic System (ACES) DDI to ACES Transformer Service.

Brief case study 2 – eForms Lessons learnt from POC Automatic generation of web-base collection tool was possible Need an ABS (or a wider-NSI) DDI profile Need for a Metadata Creation and Management Toolbox Centrally registered, accessible and governed metadata.

Conclusions from the two case studies Metadata-driven business process is feasible There are technical challenges but the more significant ones are non-technical. There are implications for metadata management Move from siloed systems to integrated metadata infrastructure Conceptualise and implement metadata consistently Govern metadata definition, use, etc. effectively.

Likely implications for metadata management – based on ABS experience ‘Templates’ depicting what GSIM object are required as input/output to GSBPM sub-processes are needed Informal Task Force on Metadata Flows across the GSBPM – a good start. Versioning strategy for metadata is required ‘Eco system’ of governance arrangements – e.g. IT governance, Information Management governance, etc. should all cohere

Likely implications for metadata management – based on ABS experience (cont.) Statistical Metadata Systems should be made to be shareable Broadening and deepening role of metadata practitioners Metadata practitioners must be up-to-date with frameworks and Standards.

Key Messages Metadata driven business process is feasible Metadata driven business process credibly presents tangible business benefits Metadata driven business process is consistent with HLG Vision Metadata driven business process will require potentially significant changes to the way metadata is managed in an NSI.