1. National Institute of Statistics – Italy
MICROCOE: A multilevel data warehouse to support the production and expand the analysis and the dissemination of foreign trade and FATS statistics
Marco Giorcelli
National Institute of Statistics – Italy
ESSnet on micro data linking and data warehousing
in statistical production.
Rome, April 6,

2. Issues covered by this presentation
History and key characteristics of the current data warehouse for the production and dissemination of foreign trade statistics
The drivers underpinning a radical change in IT support to the production and dissemination in the domain of Globalisation statistics
Key characteristics of MICROCOE, the new micro level datawarehouse for foreign trade and foreign affiliates (FATS) statistics
Conclusions

3. Key achievements in the last 15 years:
History and key characteristics of the current data warehouse for the production and dissemination of foreign trade statistics
Key achievements in the last 15 years:
Successful migration from Adabas/Natural and SAS to Oracle RDBMS in the late ’90s
Set up and maintenance of the first and still largest on line ISTAT database for the dissemination:
Strong interaction between statisticians and IT people
IT support to the link between foreign trade operators and the BR, and to changes in statistical production related to alteration of fiscal inputs or induced by New Foreign Trade Regulations
Key characteristics:
Huge amount of data uploaded every month
The production data warehouse stocks on line foreign trade micro-transactions since 1991
Full reversibility between operators and product transactions, partial with respect to variables linked to BR 3

4. THE STRUCTURE OF THE CURRENT DATA WAREHOUSE FOR STATISTICAL PRODUCTION
History and key characteristics of the current data warehouse for the production and dissemination of foreign trade statistics
THE STRUCTURE OF THE CURRENT DATA WAREHOUSE FOR STATISTICAL PRODUCTION 4

5. History and key characteristics of the current data warehouse for the production and dissemination of foreign trade statistics
IT TOOLS FOR IN DEPTH ANALYSIS OF TRADE FLOWS MICRO DATA INCLUDED IN THE PRODUCTION DATA WAREHOUSE 5

6. History and key characteristics of the current data warehouse for the production and dissemination of foreign trade statistics
STRUCTURE OF THE CURRENT DISSEMINATION DATA WAREHOUSE 6

7. The drivers underpinning a radical change in IT support to
production and dissemination of Globalisation statistics
It is hard to reconsider an IT database from the scratch, given its strong reputation and reliability, but it is time for a radical change determined by
External drivers
New mission: Inward and Outward FATS surveys and trade in services intra EU have entered the Division following a reorganisation finalised to cover all statistics related to the Globalisation domain
The production of statistics on exporting and importing companies is now under a EU Regulation
Strong demand for calculation of indicators and figures based on micro-level trade transactions, including accessibility to micro data
Convergence toward a better integration between SBS and trade statistics in the framework of MEETS
Internal drivers
Potential conflict between statistical rules and IT procedures given 15 years of cumulative but not always optimal IT development
Strong reduction in human resources and need to support a deeper engagement in rebalancing automatic checks and manual revisions 7

8. MICROCOE is finalised to meet the following goals:
Key characteristics of MICROCOE, the new datawarehouse on foreign trade and foreign affiliates (FATS) statistics
MICROCOE is finalised to meet the following goals:
To fully integrate at the micro level all Globalisation related surveys. These include trade in goods Intra and extra EU, trade in services intra EU, Inward and Outward FATS data
To be open and technologically compatible to external sources, including single surveys or other firm level data warehousing
To be internally consistent with respect to three different but fully hierarchical level of analysis:
Level one: enterprise group level – FATS statistics
Level two: enterprise level – FATS data, SBS and balance sheet administrative data, trade in goods and services reclassified by operators
Level three: trade flows individual transactions – trade in goods and trade in services data potentially linked to FATS data (intra-firm trade by products!!!)
To separate statistical rules from IT procedures thus improving transparency and efficiency in statistical production and IT management 8

9. An embryonic scheme of MICROCOE database
Key characteristics of MICROCOE, the new datawarehouse on foreign trade and foreign affiliates (FATS) statistics
An embryonic scheme of MICROCOE database
National and European EGR
Consolidated balance sheet data
Parent company of MNE
Level 1
1-N
Italian affiliates abroad
Foreign affiliates in italy
Other resident enterprises
BR, SBS data, unconsolidated balance sheet data
Level 2
1-1
Firms engaged in trade in goods and sevices
Firms not engaged in trade in goods and sevices
1-M
Micro trade flows by product, country and resident / non resident operator
Level 3
INTERNAL FRAME
EXTERNAL SOURCES 9

10. A possible view of MICROCOE user interface
Key characteristics of MICROCOE, the new datawarehouse on foreign trade and foreign affiliates (FATS) statistics
A possible view of MICROCOE user interface
Parent level data
Company name
Country of
Residence
Consolidated turnover
Exports of goods
Sum of unconsolidated
turnover abroad
Number of foreign affiliates
MARIO SPA
Italy
55.000
30.000 1
Company (Affiliate) level data
Company name
Country of residence
Unconsolidated turnover
Exports of goods
Imports of goods
MARIO 1 SPA IT
70.000
10.000
30.000
MARIO 2 SPA
50.000
20.000
25.000
MARIO GERMANY DE
missing
Trade flows level microdata (partial)
Company name
Direction
of flows
Type of goods (Combined nomenclature)
Partner Country
Monetary value
Volume
Trade partner operator
MARIO 1 SPA
Exports US
3.000 3 DE
7.000 2
MARIO GERM
Imports CN
5.000 5
25.000 10 10

11. Full integration at the micro-level of all globalisation surveys
Conclusions
Data warehousing in foreign trade statistics has an history of success in ISTAT both for micro level production and aggregated data dissemination sides, so it has being hard for us to make a decision to change.
MICROCOE benefits from what exists, but further expands along several directions:
Full integration at the micro-level of all globalisation surveys
Multilevel complex structure of the database
Open to be integrated with other ISTAT data warehouses
Complete user interface with all relevant information
Key advantages: What we have is not just the “SUM” of already existing information, new information can be created (SBS figures by exporting companies and Intra-firm trade by product!!) plus more comprehensive analysis of trade flows based on new info on trade operators
Threats: Entropy. This is a complex system with potentially huge cumulative knowledge processes, but it needs to be jointly managed by IT experts, statisticians and business analysts 11