YTY − an integrated production system for business statistics COC on S-DWH workshop, Helsinki 24.9.2014

Topics Introduction YTY and layer architecture Process Tools: Tibco, ProceX, Variable Editor Case studies: Reception of administrative data: Customer Register of National Board of Taxes The dataflow of annual data Antti Santaharju 24/09/2014

Topics (2) YTY-application (Juha) Treatment of the single observations YTYWiki (Juha) Data Analysis (Mark) Data cubes The Publishing process (Mark) Antti Santaharju 24/09/2014

What is YTY? Statistics Finland’s integrated production system for business statistics Integrates the production processes of the business register and 13 different business statistics Introduced in November 2013 The first production cycle for reference year 2013 comes to an end in December 2014 Antti Santaharju 24/09/2014

Antti Santaharju Information Services 27.11.2018 24/09/2014 5

YTY integrates production processes of Business register and 13 different statistics Structural business and financial statement statistics (preliminary t+9, final T+12) Regional statistics on entrepreneurial activity (preliminary t+9, final t+12) Industrial output (t+11) Enterprise openings and closures (Q1:t-5, Q2:t-2, Q3:t+1, Q4:t+4,) Index of turnover in industry (t+75 days) Index of turnover of construction (t+75 days) Turnover of trade (preliminary t+22 and t+45 days, final t+75 days) Turnover of service industries (t+75 days) Wage and salary indices (t+45 days) International trade in services (preliminary t+5, final t+12) Foreign affiliates in Finland (FATS) (t+12) Finnish affiliates abroad (FATS) (t+16) Business Services Statistics (t+10) Antti Santaharju 24/09/2014

Users of the YTY S-DWH YTY S-DWH has about 300 users 150 users edit the database 150 users have read-only rights to data warehouse Main users: Data collection unit Business statistics unit National account Antti Santaharju 24/09/2014

------------------Introduction-------------------- BR into the core of data on enterprises – timetable for 2009-2014 ------------------Introduction-------------------- Antti Santaharju 24/09/2014

YTY and layer architecture Antti Santaharju 24/09/2014

Meta Data Data Acces Layer Interpretation and data analyse Layer GSBPM Phase 7 Disseminate Macro Data Macro Data Interpretation and data analyse Layer GSBPM Phase 6 Analysis Integration Layer GSBPM Phase 5 Process Sources Layer GSBPM Phase 4 Collect Antti Santaharju 24/09/2014

Antti Santaharu 24/09/2014

Process Antti Santaharju 24/09/2014

Antti Santaharju 24/09/2014

Source layer − Reception of input data Administrative data Transfer of data to internal network Data sources (and every variable) are described in the Statistics Finland’s metadata database Technical checking (based on metadata) Data are as ordered Data from direct collections Direct surveys are carried out by web application Conversion of data to right format Standardised procedures (e.g data descriptions) and tools for data reception

Source layer − What are raw/interim warehouses of data collections? Original data from both administrative files and direct data collections are in safe keeping and usable No corrections are made to data contents Validity of data has been established (no major deficiencies) Source data have been described into metadata system

Integration layer − Data update Data are transferred from interim warehouses to production database for processing Automatic processing and correcting

Antti Santaharju 24/09/2014

Antti Santaharju 24/09/2014

Antti Santaharju 24/09/2014

Antti Santaharju 24/09/2014

Integration layer − Production database Place for processing unit-specific data Unfinished data (controlled with quality variables) Data update launches modular data integration, coding, validating, editing and imputing procedures Erroneous observations are flagged Unit-level data are analyzed/edited with "Unit data processing application“ Conversion of data collection units into statistical units

Antti Santaharju 24/09/2014

Antti Santaharju 24/09/2014

Interpretation and data analysis layer − Analysis in production database Tools for data analysis in production database Data cubes Aggregate reports

Interpretation and data analysis layer ETL to dissemination database Data are transferred from production database to YTY S-DWH for validated micro data Automatic or user’s choice (process management application) Updating once a day, timed overnight

Interpretation and data analysis layer/access layer − Enterprise data warehouse? Data in consistent place and format (classifications and unit structures) Contains different versions: Latest and Released unit level data Data cubes for analysis Passive, i.e. changes are made via production database Data selections and inquiries All data of business statistics in the long run Different application alternatives for browsing data

Data access layer Unit data extractions File extractions Compilation processes of statistics Information and researcher services National Accounts system extraction Archiving In future: Updates of external information services (internet)

Metadata "Technical and file metadata", e.g.: Names and properties of variables Classifications Managed with variable and classification editors “System specific metadata“ Control variables Control tables Editing rules "Process metadata" Information from process management application, e.g.

Process metadata − Progress of the process Managed with process management application We know where we are Tasks are performed in right order (e.g. data selections after certain checks) Activation of automatic and manual work phases New features will be published at the end of the September Tasks have contact/responsible persons Documentation on sub-tasks easily accessible Planned dates of completion and tracking of adherence to them (→ automatic email messages)

Process management ProceX An in-house solution based on BPMN / XPDL standard Requires a diagram that defines the flow and the tasks to execute Tibco Possible to execute Microsoft .NET procedures, SAS procedures, SQL database procedures and SSIS procedure Tasks can also be manual expecting some user interaction (eg. approval) Antti Santaharju 24/09/2014

Case study 1 Reception of administrative data: Customer Register of National Board of Taxes Aloita ASITK demo Tibcolla 0_Tuotanto tasolta Antti Santaharju 24/09/2014

Case study 2 The dataflow of annual data Antti Santaharju 24/09/2014

Antti Santaharju 24/09/2014

Antti Santaharju 24/09/2014