CORE Final Meeting – 11 January CORE Demo Scenario Diego Zardetto, Istat & CBS CORE team

CORE Final Meeting – 11 January Demo Scenario Involves 3 typical processing steps performed by NSIs for sample surveys: Sample Allocation Sample Selection Estimation It has been used as empirical test-bed during the whole implementation cycle of the CORE environment

CORE Final Meeting – 11 January Rationale for the Scenario Minimality: very easy workflow (no conditionals, nor cycles), can be run without a Workflow Engine Appropriateness: addresses heterogeneity issues heterogeneity is precisely what CORE must be able to get rid of

CORE Final Meeting – 11 January Spreading Heterogeneity over the Scenario The Scenario incorporates both: Data Heterogeneity: Via data exchanged by CORE services belonging to the scenario process Technological Heterogeneity: Via IT tools implementing scenario services – A batch job based on a SAS script – Two full-fledged R-based systems

CORE Final Meeting – 11 January The Scenario at a glance START MAUSS-R ALLOCATION SAS SCRIPT SELECTION STOP ReGenesees System ESTIMATION

CORE Final Meeting – 11 January Sample Allocation Service Overall Goal: determine the minimum number of units to be sampled inside each stratum, when lower bounds are imposed on the expected level of precision of the estimates the survey has to deliver IT tool: Istat MAUSS-R system implemented in R and Java CORA tag: “Statistics” START MAUSS-R ALLOCATION

CORE Final Meeting – 11 January Sample Selection Service Goal: draw a stratified random sample of units from the sampling frame, according to the previously computed optimal allocation IT tool: a simple SAS script to be executed in batch mode CORA tag: “Population” SAS SCRIPT SELECTION

CORE Final Meeting – 11 January Goal: compute the estimates the survey has to provide (typically for different subpopulations of interest) along with the corresponding confidence intervals IT tool: Istat ReGenesees System R-based CORA tag: “Statistics” Estimates and Errors Service STOP ReGenesees System ESTIMATION

From the Scenario to the Demo CORE Final Meeting – 11 January

Allocation (MAUSS-R) Selection (SAS Script) Estimation (ReGenesees) bethel_out stratiferrors xml bethel_outsample xml estimates frame Runtime Process Engine CORE transformations CORE transformations

Allocation (MAUSS-R) Selection (SAS Script) Estimation (ReGenesees) bethel_out stratiferrors xml bethel_outsample xml estimates frame Java/Webserver CORE transformations CORE transformations ISTAT

Allocation (MAUSS-R) Selection (SAS Script) Estimation (ReGenesees) bethel_out stratiferrors xml bethel_outsample xml estimates frame Bonita/Windows CORE transformations CORE transformations CBS ISTAT

What we are going to see: A set of GUIs for process, services and data design A set of GUIs for process execution CORE Final Meeting – 11 January Demo Details: Istat

Istat Demo back-end What lies “behind” the GUIs Integration API for CSV-CORE transformations Core Repository Data Flow Control System CORE Final Meeting – 11 January

A process run executed via Bonita workflow engine CORE Final Meeting – 11 January Demo Details: CBS