1. Composite Indicators for the Measurement of Economic Performance P. Roberti F. Oropallo ISTAT Productivity, Competitiveness and the New Information Economy Business, Systemic and Measurement Issues NESIS FP5 ISTAT – Rome June 26, 2003

2. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 2 We live with too many Indicators LISBON OBJECTIVES “to become the most competitive and dynamic knowledge-based economy in the world, capable of sustainable economic growth...and social cohesion” A host of indicators that have been proposed to monitor this goal. They can be grouped in five areas: General economic background Employment Economic Reform Social Cohesion Environment

3. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 3 Indicators can be one-dimensional or multi-dimensional Composite indicators have received increasing attention in recent years. Various methodologies have been developed to handle aggregation and related problems: Aggregation systems Deciding on the phenomenon to be measured Selection of sub-indicators Assessing the quality of the data Assessing the relationships between the sub-indicators Testing for Robustness and Sensitivity

4. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 4 State of the Art – Methodological Issues A number of methodologies can be applied for the development of composite indicators. They include: Multiple linear regression models Principal components analysis and factor analysis Cronbach alpha Neutralization of correlation effect Efficiency frontier Distance to targets Experts opinion (budget allocation) Public opinion Analytic Hierarchy Process JRC – EC (2002) - State-of-the-art Report on Current Methodologies and Practices for Composite Indicator Development – Joint research Centre – European Commission - Institute for the Protection and Security of the Citizen Technological and Economic Risk Management I-21020 Ispra (VA) Italy - Prepared by the Applied Statistics Group June 2002

5. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 5 Is there a clear Framework yet? How can a framework be developed? By defining an analytical framework with precise properties!

6. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 6 Property of Indicators Micro founded Scope fulfilling Purpose oriented Well- behaved Consistent Decomposable Multidimensional - Composite

7. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 7 MICRO FOUNDED that is, based on a comprehensive database that embraces all aspects of enterprise features (ad hoc surveys can cover some aspects) Integration of different data sources of micro data Quality test of the integration process (matching procedures / estimation) In the second part of this presentation examples of the opportunities opened up by an Integrated Database are shown (Diecofis Project www.istat.it/diecofis Year of reference is 1998-2000)www.istat.it/diecofis Sources are: (1) Structural Business Statistics (2) Administrative data (Foreign Trade, Commercial account, Fiscal and Social security data)

8. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 8 SCOPE FULFILLING Indicators can measure size, change and dispersion Changes in an Indicator’s value can have different causes and lead to different conclusions depending on underlying combinations Socio-economic phenomena have different and complex dimensions. Different indicators can serve different purposes, i.e. measure: - heterogeneity/dispersion -performance (moving toward the mean) - overall systemic performance ß-performance (generalised move upward/downward) overall comparative performance (e.g. catching up/lagging behind) stratification (to evaluate differences in systemic structures and whether they represent a “stratum” - as with Yitzhaki’s decomposition)

9. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 9 PURPOSE ORIENTED Socio-economic phenomena may have many facets and change can result from a combination of different patterns Appropriate indicators may be needed in different circumstances

10. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 10 WELL-BEHAVED AND CONSISTENT Indicators inconsistency may arise for different reasons Reference to condition of - Lorenz dominance (focusing on relative differences) - Pareto superiority (focusing on levels) - stochastic dominance (focusing on both dimensions)

11. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 11 DECOMPOSABLE & MULTIDIMENSIONAL To be able to study patterns take into consideration more than one dimension/aspects

12. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 12 Developments in poverty analysis are a good example of the possible problems and avenues to solve them Number of people  Income gaps  Welfare dimensions  Multidimensional aspects Headcounts  Gaps  FGT indices (squared gaps, etc.)  Multidimensional indices

13. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 13 Developing a similar framework is important for the analysis of systemic performance and the benchmarking of “economic textures” Since many factors and forces are at work to determine, condition and produce different outcomes. The quote that follows can serve to grasp the problem and possible approach to address it

14. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 14 “The fact that many of the smaller EU economies do either better or worse than the larger ones is partly due to larger EU economies contributing more to the overall EU mean than smaller economies, which means that they are less able to diverge from the mean. A second explanation is due to structural conditions. The industrial distribution of small economies is often concentrated in a few sectors, while larger economies are more diverse. This can shift the scores towards the mean for many indicators in large economies, while small economies can exhibit either a high or low innovative capacity, depending on the sectors that dominate the economy. Of course, this shift towards high or low technology sectors is not accidental, but reflects both public and private institutions seeking out areas of comparative advantage and high profitability”. (EU Commission)

15. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 15 The “analytical framework” and indicators that can serve to study and benchmark different levels/dimensions of “systemic features”  Systemic maps (whole jigsaw)  The overall aggregate picture: EU level  The national, regional, local picture  The sectional picture  The occupational picture  Systemic strength and weakness, at a point in time (cross-section analysis) and overtime (longitudinal analysis)  Map transitions: features, patterns and evolution (“New” vs. “Old”)  Systemic change and its features, at the aggregate/disaggregate levels

16. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 16 Decomposable indicator how The GINI index measures the concentration of a particular phenomenon (0=no concentration, 100=maximum concentration). It can be divided into three elements:

17. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 17 Distributions and Overlap When overlap in the decomposition is high, it is very hard to judge which group is the best/worst, because distributions cross Mean (A) ≈ Mean (B) ≈ Mean (C)performance N

18. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 18 Distributions and Overlap When overlap in the decomposition is low it is easy to determine which group is the best/worst performer Mean (A)Mean (B)Mean (C)performance N

19. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 19 Mono-dimensional Analysis of performance (Overall) (GINI index calculated on exports)

20. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 20 Mono-dimensional analysis of performance (Exports) (By NACE Sectors)

21. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 21 Multi-dimensional analysis of performance Decomposition for each dimension Decomposition of the between component for each dimension across K classes

22. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 22 To make composite indicators easy to use and interpret Requires Normalization Requires Aggregation

23. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 23 Benchmarking performance levels or gaps? How far away from best performers enterprises are?

24. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 24 Weighted Composite Gap How far away from best performers enterprises are?

25. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 25 Composite Indicator (Breakdown by NACE sector) Three dimensions  One Dimension

26. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 26 NACE sectors The Sectors in italic are defined “ICT sectors” by OECD

27. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 27 Composite Indicator (Breakdown by NACE sector) Three dimensions of enterprises’ performance: (1)Value Added (2)Employment (3)Exports a) one dimension-three areas

28. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 28 Composite Indicator (Breakdown by NACE sector) b) 3 dimensions into one indicator Short fall from top or average

29. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 29 Composite Indicator (Breakdown by regions) Three dimensions of enterprises’ performance: (1)Value Added (2)Employment (3)Exports a) 3 mono-dimensional indicators

30. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 30 Composite Indicator (Breakdown by regions) b) from 3 to one 3-dimensional indicator Short fall

31. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 31 Composite Indicator (by size of firm) Three dimensions of enterprises’ performance: (1)Value Added (2)Employment (3)Exports Short fall

32. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 32 Conclusions The analysis which has been presented draws from research results under two related FP5 projects DIECOFIS and NESIS that deal with different mapping and benchmarking aspects. A strong investment in the design and development of a complex and wide ranging system of enterprise micro data which have been integrated and systematised into one single “hub”. The analysis is founded on micro-data drawn from the integrated and systematised enterprise SIS, which gives high flexibility and allows to aggregate and disaggregate indicators a la carte.

33. June, 26 2003Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo 33 Thank You