COmmon REference Environment - CORE: Architecture and Information Model Monica Scannapieco Italian National Institute of Statistics - Istat SISAI Meeting - Luxembourg June 7-8 2011 1

Outline Introduction and history CORE objectives CORE: where we are Architecture implementation Information model SISAI Meeting - Luxembourg June 7-8 2011

COmmon Reference Architecture (CORA) Need for a common reference architecture NSIs share similar goals and surveys Different choices wrt business and technical architectures Benefits of a common reference architecture Sharing of solutions among NSIs Quality improvement SISAI Meeting - Luxembourg June 7-8 2011

CORA ESSnet Financed by Eurostat under 2009 Statistical Workprogramme Countries involved: it (coordinator), ch, dk, lv, nl, no, se Duration: October 2009 - October 2010 SISAI Meeting - Luxembourg June 7-8 2011

CORA Workplan Technical architecture Requirements and state of the art analysis Organizational architecture SISAI Meeting - Luxembourg June 7-8 2011

WP3: Technical Architecture CORA Model: two dimensions Functional dimension Construction dimension Adoption of GSBPM 1.2 9 subprocesses of level 2 1 Specify Needs 2 Design 3 Build 4 Collect 5 Process 6 Analyse 7 Disseminate 8 Archive 9 Evaluate SISAI Meeting - Luxembourg June 7-8 2011

Construction Dimension: Layers Figures Time Series Statistic Population Unit Variable Value The end result of the process Integration over time One measurement aggregate Statistical population observed The units that make up the population The variables that make up the unit The data values SISAI Meeting - Luxembourg June 7-8 2011

CORA Model Grid SISAI Meeting - Luxembourg June 7-8 2011

Statistical Design vs. Technological Specification (1) Statistical processes compliant to the CORA model are intended to be designed by statisticians Services composing processes are indeed statistical services and are technological independent SISAI Meeting - Luxembourg June 7-8 2011

Statistical Design vs. Technological Specification (2) Given the specification of a CORA compliant process, technological services must be designed and implemented SISAI Meeting - Luxembourg June 7-8 2011

After CORA…CORE! COmmon Reference Environment (CORE), financed by Eurostat under 2010 Statistical Workprogramme Countries involved: it (coordinator), fr, nl, no, pt, se Duration: December 2010 - January 2012 SISAI Meeting - Luxembourg June 7-8 2011

CORE Principal Outcomes Environment for the definition and execution of statistical processes Definition of a process in terms of services selected by an available repository Execution of the composed workflow SISAI Meeting - Luxembourg June 7-8 2011

Principal Outcomes: Design CORA information model Design of CORE services and processes SISAI Meeting - Luxembourg June 7-8 2011

Principal Outcomes: Implementation Selection of an available middleware solution for process execution Realization of an environment able to permit the execution of processes Interfaces (GUIs) for defining CORE processes for statistical users Integration APIs Repository of integration layers SISAI Meeting - Luxembourg June 7-8 2011

Principal Outcomes: Testing Realization of processes starting from services implementing some GSBPM blocks Evaluation of costs related to integration Prototype implementation (to be engineered) SISAI Meeting - Luxembourg June 7-8 2011

CORE Architecture (1) GUIs to support modelling CORA processes according to CORA grid Modeling & control flow constructs Drag & drop facilities for process design Global schema Implementation: we are evaluating the usage of an open process editor tool (Oryx - http://bpt.hpi.uni- potsdam.de/Oryx/WebHome ) SISAI Meeting - Luxembourg June 7-8 2011 16 16

CORE Architecture (2) Process Runtime Controlled execution of services Implementation: integration of existing WF solution, currently in evaluation phase Service Runtime Integration APIs (in-out data transformation) Service execution Implementation: CSV and SQL data transformations are currently being implemented Service Repository Deployment of services SISAI Meeting - Luxembourg June 7-8 2011 17

CORE Information Model (1) First draft of CORE information model To be released, currently in discussion phase SISAI Meeting - Luxembourg June 7-8 2011

CORE Information Model (2) Design Principles: Rectangular data sets Strong typing Kinds Free-style arguments Other SISAI Meeting - Luxembourg June 7-8 2011

CORE Information Model (3) Service info package Name CORA vertical axis CORA horizontal axis Version Documentation SISAI Meeting - Luxembourg June 7-8 2011

SISAI Meeting - Luxembourg June 7-8 2011

CORE Information Model (3) Rectangular data sets A data set consists of rows and columns of data Same proposal independently made by WP2 leader (CBS) and WP3 leader (Istat) SISAI Meeting - Luxembourg June 7-8 2011

CORE Information Model (4) Strong Typing Each column contains data of one value type only Data sets are strongly typed Rules are typed. This means that a rule can be applied to (a row of) data only if the types of the rule and the (row of) data match Parameters are typed. Actual parameter values are limited to values of a given predefined type only Proposed by WP2, hard to get in WP3 opinion SISAI Meeting - Luxembourg June 7-8 2011

CORE Information Model (5) Kinds: The information model leaves open the actual form or structure of data sets, but provides a mechanism for specifying such a form or structure instead, through data set kinds Examples: micro datasets consisting of variables, aggregated data set consisting of dimensions and measures etc. SISAI Meeting - Luxembourg June 7-8 2011

CORE Information Model (6) Free-style arguments: Rules and parameters for which it is not feasible to express them in the format offered by the information model Examples: scripts that are expressed in a tool-dependent format SISAI Meeting - Luxembourg June 7-8 2011

CORE Information Model (7) CORE Services Arguments Data sets, rectangular data sets (with possible specification of data kinds) Expressions, e.g. if-then-else(<(x,5),x,5) Parameters, supply a single item of information, e.g. number of loops Rules, expressions applied to data set definition, e.g. editing rule of the form if- then-else(<(x,y),x,y) SISAI Meeting - Luxembourg June 7-8 2011