SISAI STATISTICAL INFORMATION SYSTEMS ARCHITECTURE AND INTEGRATION WORKING GROUP 3rdMEETING 13-14 MAY 2013 ITEM 1.5.1-3 Modernising Statistical Production and Services

Modernising Statistical Production and Services Thérèse Lalor UNECE

Introducing UNECE Statistics

UNECE Statistics: Priorities Population censuses, migration, Millennium Development Goals Globalisation, National Accounts, employment, business registers Sustainable development, environmental accounts, climate change Modernisation

Why is modernisation important? http://www.isleofideas.com/2011/06/28/information-overload/

In the last 2 years more information was created than in the whole of the rest of human history!

The Challenges Riding the big data wave Increasing cost & difficulty of acquiring data New competitors & changing expectations ABS, like other National Statistical Institutions, faces shared constraints and challenges. External Challenges rapidly changing external environment More sophisticated users Demand for timeliness and responsiveness increasing demand for more accessible and ‘joined up’ data to solve complex policy questions Constraints Reduced funding and volatility in funding Our costs are increasing significantly – unable to contact many households, response rates dropping, difficult to recruit and retain interviewers skills shortages – competing for statistical and ICT skills across government complex work programs siloed processes and aging infrastructure Competition for skilled resources Reducing budget Rapid changes in the environment

These challenges are too big for statistical organisations to tackle on their own We need to work together

Statistics fights back! High Level Group for the Modernisation of Statistical Production and Services Created by the Conference of European Statisticians in 2010 10 heads of national and international statistical organisations

A strategy for modernisation Transform vision to reality New sources and products Streamlined processes Managing organisational change to support modernisation and collaboration

A strategy for modernisation Transform vision to reality New sources and products Streamlined processes Managing organisational change to support modernisation and collaboration Conference of European Statisticians, June 2012 Endorsed

Using common standards, statistics can be produced more efficiently No domain is special! Do new methods and tools support this vision, or do they reinforce a stove-pipe mentality?

Introducing the GSBPM

Why do we need the GSBPM? To define and describe statistical processes in a coherent way To compare and benchmark processes within and between organisations To make better decisions on production systems and organisation of resources 17

The GSBPM is used by more than 50 statistical organizations worldwide

Beyond statistics: Data archives Generic Longitudinal Business Process Model 26 May 2019

Information objects Things that flow between GSBPM sub-processes Things that drive and integrate sub-processes

GSIM and GSBPM GSIM describes the information objects and flows within the statistical business process.

GSIM and GSBPM-processes GSIM object structures (formats) GSBPM -process GSIM object instances http://www.illeccio.com/page.php?lang=en&product_id=37

So what is GSIM? GSIM gives us standard terminology A reference framework of information objects: Definitions Attributes Relationships GSIM aligns with relevant standards such as DDI and SDMX GSIM gives us standard terminology

Production Business Concepts Structures Statistical Program Production Activity Process Input Process Method Statistical Program Design Process Step Production Dissemination Activity Acquisition Activity Rule Process Output Business Concepts Data Set Data Point Variable Population Data Structure Structures Concept Data Resource Product Unit Classification

It is a new way of thinking for statistical organizations 2727 The conceptual model is explicitly chosen to be independent of design or implementation concerns, The aim of a conceptual model is to express the meaning of terms and concepts used by domain experts to discuss the problem, and to find the correct relationships between different concepts. The conceptual model attempts to clarify the meaning of various, usually ambiguous terms, and ensure that problems with different interpretations of the terms and concepts cannot occur. Such differing interpretations could easily cause confusion amongst stakeholders, especially those responsible for designing and implementing a solution, where the conceptual model provides a key artifact of business understanding and clarity. Once the domain concepts have been modeled, the model becomes a stable basis for subsequent development of applications in the domain. The concepts of the conceptual model can be mapped into physical design or implementation constructs It is a new way of thinking for statistical organizations

GSIM documentation There are many layers to the GSIM documentation!

GSIM: The “sprint’ approach Locked people in rooms 3030 GSIM: The “sprint’ approach The HLG decided to accelerate the development of the GSIM using an Agile approach. Sprints Collaboration of multi-disciplinary experts A “time-boxed” period of work, and A closely defined and agreed output

GSIM: The “sprint’ approach 3131 Locked people in rooms GSIM: The “sprint’ approach Sprint 1 – Slovenia, February 2012 Sprint 2 – Republic of Korea, April 2012 Integration Workshop, Netherlands, September 2012

Developing GSIM 113 person weeks 3 “Sprint” sessions

What next? 2 big projects for 2013

Implementing GSIM http://www1.unece.org/stat/platform/display/metis/GSIM+v1.0+discussion+forum

GSIM Map GSIM to SDMX and DDI Conceptual model DDI SDMX Implementation Other relevant standards DDI SDMX GSIM Conceptual model Implementation standards

Reviewing / revising GSBPM and GSIM http://www1.unece.org/stat/platform/display/metis/GSBPM+discussion+forum

Geospatial Standards Task team to investigate role geospatial standards play in the modernisation of official statistics

Get involved! Anyone is welcome to contribute to this work. Contact: Thérèse Lalor Therese.lalor@unece.org There are lots of resources on the wiki for you to use: http://www1.unece.org/stat/platform/pages/viewpage.action?pageId=59703371

The problem 2 big barriers that hinder modernisation of statistical organisations are: Rigid processes and methods; and Inflexible and ageing technology environment.

NSI 1 Collect Process Analyse Disseminate Survey A Survey B Historically statistical organisations have produced specialised business processes and IT systems

Now we have an agreed conceptual basis to move forward on, we can focus on “How To”

Desired Project Outcomes Increased: interoperability in Official Statistics through the sharing of processes and components ability to find real/genuine collaboration opportunities ability to make international decisions and investments sharing of architectural/design knowledge and practices

How does Architecture help? Many statistical organisations are modernising and transforming using Enterprise Architecture Enterprise architecture shows what the business needs are, where the organisation wants to be and aligns the IT strategy to this. It can help to remove silos and improve collaboration across an organisation.

EA helps you get to this NSI 1 Survey A Survey B Survey C Collect Process Analyse Survey A Survey B Survey C Disseminate

…but if each statistical organisation works by themselves…..

…we get this…. Canada Collect Process Analyse Disseminate Sweden

This makes it hard to share and reuse! Canada Collect Process Analyse Disseminate Sweden ?

…but if statistical organisations works together?

Common Statistical Production Architecture An industry architecture will make it easier for each organisation to standardise and combine the components of statistical production, regardless of where the statistical services are built

An industry architecture is: “set of agreed common principles and standards designed to promote interoperability in an industry” “an architecture template for statistical production” “common vocabulary to discuss implementations” “enables the vision and strategy of an industry, showing a clear picture of how to get there”

It has 4 layers Business Architecture Information Architecture Application Architecture Technology Architecture

Based on Service Oriented Architecture A Statistical Service: is a representation of a business activity with a specified outcome is self contained performs a task in a business process can be reused in a number of business processes

Based on Service Oriented Architecture Project is sometimes called “plug and play” Statistical service is the pluggable bit The idea is that installing a new statistical service should be easy

This makes it easier to share and reuse! Collect Process Analyse Disseminate ? Sweden Canada

Wrapped legacy/existing Lego pieces could be: Wrapped legacy/existing Brand new

Architecture Sprint!

Open for public review for 3 weeks CSPA v0.1 released 22 April Open for public review for 3 weeks

Design Artefacts

Quality Attributes GSIM – Compliance GSBPM – Compliance Statistical Service Architectural Pattern

GSIM and GSBPM-processes GSIM object structures (formats) GSBPM -process GSIM object instances http://www.illeccio.com/page.php?lang=en&product_id=37

Quality Attributes Statistical Service

Statistical Service Architectural Pattern

User Stories Architecture gives an industry strategy Collaborating Gives common To Be state for statistical organisations Helps statistical organisations to transition Aligns roadmaps to achieve future industry vision International investment Catalogues show what statistical services exists Genuine collaboration opportunities Collaborating Adapting an existing statistical service or designing new ones Hosting/Deploying services available to all organisations Influencing vendors to align to industry requirements

Proof of Concept Demonstrate the process of working together Advantages in cooperation Demonstrate business viability to senior management There are benefits to pursuing this Show it is feasible Give energy to idea Prove the value of the Architecture Here is something that we could not do before

NSI-Specific communication platform E-form Service ? Coding Service E-form Service Editing Service? E-form Service Coding Service Editing Service? ? Repository NSI-Specific communication platform

Project Timeline to end June Strand 1: Architecture Pre Sprint Architecture discussions Revisions to Architecture based on feedback Collect feedback of Sprint Output Advise on PoC work 18 – 24 March 25 – 31 March 1 – 7 April 8 – 14 April SPRINT 15 – 21 April 22 –28 April 29 April – 4 May 5 – 11 May 12– 18 May 19 – 25 May 26 May – 1 June 2 – 8 June 9 – 15 June SPRINT 16 – 22 June 23 – 30 June Teams for Business and IT requirements for PoC Sprint Virtual PoC work Negotiating for licences for PoC Strand 2: Proof of Concept

Get involved! Anyone is welcome to contribute to this work. Contact: Thérèse Lalor Therese.lalor@unece.org CSPA v0.1 is out for public review until 17 May: http://www1.unece.org/stat/platform/display/msis/CSPA+v0.1