1. HANDBOOK ON ECONOMIC TENDENCY SURVEYS Presentation by Luciana Crosilla and Mauro Politi (ISTAT – National Statistical Institute, Italy) Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014

2. Outline 1.Motivation (introductory notes; why an ETS handbook?) 2.The process (preparation step; milestones; method of work; arranging the drafting) 3.The handbook (purposes; intended readership; about the handbook; an overview) 4.The gobal consultation: preliminary results (summary of comments) 5.Concluding remarks Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 1

3. Introductory notes  UN Statistical Commission considered an international programme of work on short-term economic statistics as part of a coordinated statistical response to the economic and financial crisis ;  ETS are a theme comprised in the international programme because: o ETS data have been recognized as tools to improve the estimation of other economic indicators and as components of composite indicators; o Confidence indicators are useful in providing an early warning of changes in economic activity (UN, EUROSTAT); Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 Motivation 2

4. Why an ETS handbook?  UN encourages countries to conduct ETS (if not doing so already) in respect the role of the surveys data in the economic research;  UN has identified a handbook as a tool to assist member States in compiling and reporting internationally comparable tendency survey data;  the shortcoming of a unique guide for “economic” tendency surveys (the current handbooks deal with the consumer surveys or the business “traditional” surveys, excluding e.g. agriculture sector) comprising: best practices harmonized principles Guidance; Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 Motivation 3

5. Preparation step: the team International working group  The Italian National Statistical Institute (ISTAT) – leader – Italy  Statistics Netherlands (CBS) – Netherlands  The Swiss Economic Institute (KOF – ETH) – Switzerland  Philippine Statistical Authority (PSA former NSCB) – Philippines  The Organisation for Economic Co-operation and Development (OECD) Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The process 4

6. Preparation step: the Team External revisers  The European Commission (DG Ecfin)  French National Institute of Statistics and Economic Studies (INSEE) Contributors  Instituto Brasileiro de Economia, Fundação Getulio Vargas, IBRE- FGV Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The process 5

7. Milestones  February 2012: decision of the Statistical Commission of UN to produce a handbook;  October 2012: constitution of the Working Group;  Time span for the realization of a first draft of the handbook: January 2013 - October 2013;  November 2013: the first draft was submitted to the revisers;  Realization of a second draft (including further comments of the revisers): January-may 2014;  UN Global consultation: 28 July – 29 August 2014;  The finalization of the handbook is expected by beginning of 2015. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The process 6

8. Method of work No meetings (pros and cons); No multi-conference call (time gap between countries and language difficulties); Bilateral conference call; Contacts only by e-mail; Standardization of the template to write the chapters; always in touch with UNSD; Hard centralized work of harmonization and revision (different experiences, languages…) to avoid repetitions, non homogeneous Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The process 7

9. Arranging the drafting The drafting of the handbook was arranged for chapters: each chapter deals with a (main) topic.  For each chapter:  three/four working group members worked  One role was assigned to each member, among: Leader (responsible of the chapter: it coordinated the drafting of the chapter and, in some cases, it has contributed in the drafting) Contributor Reviser  For some chapters (it depends on the topic), an external reviser (EC, INSEE) was assigned. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The Process 8

10. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The process Arranging the drafting  The deadline for the finalization of the first draft and of the revised draft was assigned;  To each contributor was assigned the drafting of one/two sub- paragraphs expected in the chapter.  EXAMPLE : Chapter 4 4.1 Business surveys (assigned to CBS) 4.2 The consumers survey sample (assigned to ISTAT) 4.3 Frequency and timeline (assigned to NSCB) 9

11. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook Purposes  to constitute internationally accepted set of guidelines applicable to all countries and multiple economic activities;  to provide guidance on the process of setting up tendency survey operations, data processing and analysis;  to make users aware of the statistical methods and techniques that are employed in tendency surveys (associated with composite tendency indicators). 10

12. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook Purposes More specifically, the handbook should provide best practices and harmonized principles on:  tendency survey sample selection  questionnaire design  survey questions  survey execution  data processing  dissemination of results  use of composite tendency indicators 11

13. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook Intended readership  National statistical organizations that already carry out or intend to carry out tendency surveys in future;  Users of the survey data as researchers, analysts, compilers of composite indicators etc.;  Policymakers and anybody related to monitoring economic conditions and business cycles. 12

14. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook About the handbook  The handbook is about: methods that are currently used (or could be used) to produce tendency survey data; some related aspects (e.g., process design, quality)  The handbook is not about: laws, regulations, etc. official definitions of concepts 13

15. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook About the handbook  In preparing the handbook, the working group has taken into account the existing guidelines on the subject (European Commission and OECD);  The new handbook expands the scope of the existing guidelines to include more economic sectors ;  it focuses explicitly on the needs of developing countries in administering tendency surveys. 14

16. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook About the handbook The handbook is made up of:  10 chapters  9 topics: each chapter deals with one main topic Examples of topic: survey frame and sample design the questionnaire design  List of abbreviations and acronyms 15

17. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview CONTENTS Chapter 1Introduction 2Constructing ETS: scope 3The questionnaire design 4Survey frame and sample design 5Estimation procedures and accuracy 6Data collection 7Managing sources of non-sampling errors 8Processing tendency survey data 9Data dissemination and publication 10Use of tendency survey results 16

18. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 1 - Introduction Define the scope of the handbook and introduces the main characteristics of the ETS.  Main goals of the handbook  Qualitative data: characteristics and place in the system of economic statistics  On the use of ETS  Survey process  Outline of the handbook 17

19. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 2 - Constructing ETS: scope Describes the goal of ETS and contains the main definitions.  The scope of BTS Sector coverage Regional Coverage Survey units Basic principles of questions Survey frequency  The scope of CTS Target population, sampling and reporting unit Principle of the questions Survey mode and frequency 18

20. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 3 – The questionnaire design Principles guiding the construction and selection of questions are discussed as well as the choice of response scales.  Introduction  General considerations Business and consumer tendency surveys: main differences Formulation of questions Choosing subjects and sectors Measurement scale Reference period Seasonality Structural Information on respondents Pre-testing of the questionnaire 19

21. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 3 – The questionnaire design (continued)  An example of harmonized questionnaire BTS at a monthly frequency BTS at a quarterly frequency BTS at a (semi-) annual frequency CTS at monthly and quarterly frequencies 20

22. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 4 – Survey frame and sample design It focuses on important steps in the entire process of ETS: from where and how to select a subset of units (i.e. the sample). Moreover, the selection methods of units from survey frame, the allocation criteria and the sample size are discussed.  Introduction  Survey frame Business tendency surveys Consumer tendency surveys 21

23. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 4 – Survey frame and sample design (continued)  Sample design Selecting units: probabilistic and deterministic samples Sample design features Panel sample BTS: specific features of sample design CTS: specific features of sample design 22

24. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 5 – Estimation procedure and accuracy In ETS estimation processes, weighting schemes are usually used: the goal of the weighting procedures and the choice of specific weights are described in detail. Sampling error and its calculation are illustrated.  Introduction  Target variables and size weights Why use size weights in BTS? Choice of size weights in BTS  Estimators and sample weights Horvitz-Thompson and ratio estimators Inclusion probabilities 23

25. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 5 – Estimation procedure and accuracy (continued)  Post-stratification  Accuracy and sample size General theory Example: Simple Random Sampling Design effect 24

26. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 6 – Data collection Survey modes, reminder policy and training of interviewers are topics discussed within this chapter.  Data collection: designing a strategy Primary data collection Data collection strategy: survey techniques and communication with respondents Determinants in designing the process of data collection  Designing the process of data collection Pre survey: preparations and communication Field work: collecting the data Response enhancing measures  Organizing the data collection Training of enumerators Technical facilities and documents 25

27. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 7 – Managing sources of non-sampling errors In chapter 7 non-sampling errors and the possible approaches to reduce errors are analysed.  Non-sampling errors Coverage errors Measurement and processing errors Non-responses  Treatment of non-responses Total non-responses Partial non-responses 26

28. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 8 – Processing tendency survey data Qualitative responses have to be quantified to make them easier to interpret and to obtain time series from different survey waves. In addition, various standard methods for seasonal de-composition of time series and widely adopted solutions for ETS time series are discussed here.  Conversion of multiple choice questions into a time series The balance statistic Diffusion indexes The probabilistic approach and other methods Latest proposals for the quantification of qualitative information The “disaggregate” approach 27

29. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 8 – Processing tendency survey data (continued)  The seasonal adjustment Seasonal patterns in opinion survey data General principles of seasonal adjustment of BCS data Widely adopted solutions for seasonal adjustment 28

30. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 9 – Data dissemination and publication Recommendations regarding metadata and output tables are made. Strategies of publication are also discussed.  Metadata Contact data The methodology Data and dissemination  Publication procedures Variety of users Quality criteria Release calendar Forms of dissemination Confidentiality Revisions 29

31. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The handbook An overview Chapter 10 – Use of tendency survey results The most common business cycle composite indicators are presented; examples of forecasting applications, turning point detection and micro-econometric approaches are given.  Users of survey data  The use of survey data Business cycle composite indicators Visualizing tools Forecasting using survey data Consumer confidence, macroeconomic analysis and business cycle Micro-econometric methods 30

32. Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 The Global consultation: preliminary results  A Global Consultation on the handbook was carried out during the period from 28 July 2014 to 29 August 2014 by UN. To facilitate the consultation a comment form was prepared.  A total of 62 responses were received from countries (57) and international/regional organizations (5).  Overall there was a strong support for the Handbook: the responses were extremely positive and a number of useful comments were received. 31

33. The Global consultation: preliminary results Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 YesNo Per centNumberPer centNumber Question 1: Do you consider that the Handbook provides useful recommendations on Economic Tendency Surveys? 1005000 Question 2: Do you agree with the recommendations provided in the handbook? 1005000 Question 3: Overall do you consider the structure of the handbook clear and coherent? 1005000 Question 6: Do you consider the Handbook ready for publication, subject to the incorporation of the comments made during this global review? 984821 Summary of the responses that were received to the questions (as of 19 september) 32

34. The Global consultation: preliminary results Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 Summary of comments  Positive comments. The respondents noted:  the comprehensive coverage of the topics (all the phases of the statistical production processes);  the usefulness of the recommendations provided in the handbook;  the current structure of the handbook: clear, coherent and easy to follow;  98 per cent of the respondents considered the Handbook ready for publication subject to the incorporation of the comments received through the global consultation. 33

35. The Global consultation: preliminary results Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 Summary of comments  Main suggestions of the respondents:  To include references or examples from non EU countries (not only experience from EU countries);  To elaborate the text more on issues relating to countries with poor statistical systems and poor data quality;  chapter-specific technical and editorial comments and/or proposals for revision;  relevant reference should be made to ISIC classification;  the inclusion of additional text on the calculation and interpretation of composite indicators. 34

36.  The finalization of the handbook is expected by 2015 after:  Incorporating all the comments from the Global Consultation and alignment the text with the UN style for publications: by end December 2014;  final review of the working group and steering group: by end January 2015;  submit final draft (subject to language editing) to the 46th session of the UN Statistical Commission.  Final assessment of the Working Group – UNSD on the work/experience Concluding remarks Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 35

37. THANK YOU FOR YOUR ATTENTION !!! lcrosilla@istat.it politi@istat.it Handbook on Economic Tendency Surveys - UN Workshop on short-term economic indicators, Hangzhou, 8 October 2014 36