1. S-DWH layered architecture – Statiscs Finland

2. NSI: Statistics Finland Name: Antti Santaharju
Date:
Institutional Output
Reporting
Analysis
Data mining
Meta Data
&
Process
Control
System
Macro Data
Data Acces Layer
GSBPM Phase 7 Disseminate
Micro Data
YTY S-DWH for validated micro data
Data
cubes
Future plan
Data
cubes
Interpretation and data analysis Layer
GSBPM Phase 6 Analysis
YTY operational
database
Reports
Integration Layer
GSBPM Phase 5 Process
Statistics Finland’s
other databases
Sources Layer
GSBPM Phase 4 Collect
11 Direct
surveys
16 Administrative
Data sources

3. YTY business statistics production system
”Dependent” statistics of the integrated YTY operational database
Business Register
Some SBS and other statistics: * Financial statement * Regional and industrial statistics * International trade in services * FATS statistics (inward and outward) * Industrial output /Commodity (PRODCOM)
STS statistics (partial integration)
Turnover indices
Wage and salary indices
Antti Santaharju

4. 1. YTY Source layer Direct surveys are carried out by web application
Administrative datasets (sequential text files) are transmitted from other administrative institutes to Statistics Finland as sequential textfiles
All data sources (and every variable) are described in the Statistics Finland’s metadata database
Metadata descriptions are accessible by Variable Editor application
Using the metadata received administrative datasets
are converted into SAS datasets
are technically validated
All data sources (external and internal) are stored into YTY Source layer as SAS datasets (raw data warehouse)
All the processing is controlled by tailor made process control system process engine X (procX)
Antti Santaharju

5. YTY Integration layer & YTY Interpretation and data analysis layer
YTY Integration layer & YTY Interpretation and data analysis layer work in a single database called YTY operational database
Microsoft SQL2012 database
All database tables are described in the Statistics Finland’s metadata database
All the processing is controlled by Statistics Finland’s tailor made process control system procX
Antti Santaharju

6. 2. YTY Integration layer
Data sources are uploaded to YTY integration layer
By SAS programs
Some information is extracted directly from Statistics Finland’s other databases to YTY operational database
Input for estimation
Data update launches modular data integration, coding, validating, editing and imputing processes
These process modules are SAS/.Net programs
Erroneous units are flagged
Flagged units are analyzed and edited manually by Statistics Finland's tailor made .Net application
Tietopalveluyksikkö/Viestintä

7. 3. Interpretation and data analysis layer
Data analysis and data mining is done by
SSAS Database cubes
MS Excel
SAS EG
Microsoft SQL2012 report builder reports
Data analysis and data mining is based on real time data in YTY production database
Tietopalveluyksikkö/Viestintä

8. 4. Access layer All validated micro data are loaded to access layer
Microsoft SQL2012 server database for validated micro data
Data analysis, data mining, dissemination and delivery is based on SSAS Database cubes
Daily updated MOLAP cubes
One data cube for each statistics
Publication process creates frozen micro data version into database
Publication process tools:
SAS, Tau-Argus, PC-Axis, PX Web..
In future Some validated information is loaded directly from other production databases to YTY S-DWH for validated micro data (YTY Access layer)
Tietopalveluyksikkö/Viestintä