1. Pekka Korhonen - Statistical services
The publishing system of statistics in Natural Resources Institute Finland (LUKE)
Pekka Korhonen - Statistical services

2. Natural Resources Institute Finland (LUKE)
Established
Fusion: MTT (Agrifood Research Finland), Metla (Finnish Forest Research Institute), RKTL (Finnish Game and Fisheries Research Institute) & MMM Tike (Information Centre of the Ministry of Agriculture and Forestry)
Statistical Services Unit
Official statistical authority - Official Statistics of Finland (OSF)
Statistics on agriculture, forestry and fisheries
62 statistics ( )
Agriculture: 27 statistics
Forestry: 25 statistics
Fisheries and game: 10 statistics
Indicators
Agrikaattori: The EU Common Agricultural Policy (CAP): 38 indicators
Forest Indicators (coming)
Total 422 statistical tables ( )
Pekka Korhonen - Statistical services

3. The structure of the statistics publishing system of LUKE
2 separate parts those are linked to each other
Statistics Portal (html site):
basic maintenance, development and location purchased from outside
normal html pages (drupal), tailored to TiPa
TiPa has its own maintenance interface
Statistics Database (PX-Web):
Virtual Machines - maintained by Government ICT Centre (VALTORI)
PX-Web application, own installation and maintenance
linked to the Statistics Portal: database, ready-made tables, dynamic images
automatic maintenance (timing + scripts)
API Interface: Conductive Data for 3rd Parties (open data)
Three-language (Finnish, Swedish and English)
User Management (separate PX Web installation)
Pekka Korhonen - Statistical services

4. The publishing system of the statistical services -unit of LUKE
STATISTICAL BULLETINS
LUKE-
PORTAL
PX-WEB
AUTOMATIC
GRAPHICS
HOMEPAGE, INFO, QUESTIONNAIRES
etc.
STATISTICAL
PAGES
0 min.
valmis-taulut
portaaliin
STATISTICS
DATABASE
STATISTICS
PORTAL
LINKS:
DATABASE
READYMADE
TABLES
GRAPHICS
Pekka Korhonen - Statistical services

5. PX-Web specs in LUKE Statistics
2 virtual servers:
Test / Production sites (2 virtual machines) (maintained by VALTORI)
Windows Server 2012 R2 & IIS 8.5
remote access (RDC), disk allocation, admin rights, automation
TEST:
test PX-Web (Luke intra)
maintenance of automation (scripts + data storage + backups + logs)
PRODUCTION:
production PX-Web (public)
graphical automation (basic javascript + js- & html-files of graphics)
Common network disk space
PX-Web folder structure
update and timing files (in the right folders)
access by statisticians (statistics take care themselves)
Pekka Korhonen - Statistical services

6. Structure & operation of the PX-Web database
INTRA WEB 7 9 8 6
TEST SERVER
PRODUCTION SERVER 5 3 2
TEST
PX-Web
(check)
PRODUCTION
PX-Web
(public)
STATISTICS
Release
calender:
update +
timing
0 min.
readymade tables
to portal 5
COMMON DISKSPACE
Update
files (csv,px,txt)
UPDATE
AUTOMATION
Scripts,
Data storage,
Logs, Backups
GRAPHICS
AUTOMATION
Basic code,
Page codes
(js,html)
0 min.
API
9:00 if update
15 min.
0 min.
graphs
to portal
15 min.
Pekka Korhonen - Statistical services

7. The automatic maintenance of PX-Web
Update + timing files on a shared network disk
update: px- or csv-file, timing: txt-file (eg )
A 15-minute script
retrieves updates from the network disk to the test server and combines them with the original tables and moves the tables to the test-PX-Web and updates the menu
-> Updates the Test-PX-Web every 15 mins (checking)
Every morning script (mon-fri 9:00)
combines updates with original production tables if the timing file is ok (same day or past) and moves tables to the production-PX-Web and updates the menu
-> Updates the Production-PX-Web every working day at 9:00 am according to the release calendar (release)
Scripts own production + PX-Job & Menubuilder (Statistics Finland)
Timing with Task Scheduler (Windows)
Pekka Korhonen - Statistical services

8. Automatic graphics
All the LUKE Statistics Portal graphics come through API-interface
basic patterns + maps (over 250 graphs total)
dynamic links, downloaded to iFrame, updated from the database table
linking with URLs to any website (including customers)
languages parameterized (?lang = fi / sv / en)
PX-Web API-interface (open data)
API-address (url)
JSON query (TiPa: json-stat - also xlsx, json, csv, sdmx, px)
TiPa's own graphics application (charts)
JavaScript: basic code + picture-specific js- & html-files
libraries: JSON-stat Javascript Toolkit, Idescat Visual (flot)
+ own code (development & production in Git)
Ready made tables (and pictures)
with PX-Web Saved Search (statistical updaters / customers)
Pekka Korhonen - Statistical services

9. The last page
Tools:
PX-Edit: designing, creating and maintaining tables
PX-Job: automatic updates (part of the script)
PX-Win: classification (vs- & agg-files)
Demo:
Indexed images
Nominal / real values
Data from many sources, through two APIs
LUKE / Statistic Finland
Calculation on the fly
Pekka Korhonen - Statistical services