Ifo Institute for Economic Research Effectiveness of the Visual Analog Scale for the Measurement of Business Expectations Anna Stangl Ifo Institute for Economic Research Brussels, 12. October 2009

Visual Analog Scale (VAS) in Web Surveys 3-Category Scale. Visual Analog Scale (VAS) Present business situation Business expectations Ifo Economic Climate Index

Motivation: Measurement of Expectations Qualitative measurement of economic expectations: category-rating scales limited, coarse data information loss in the neutral category no information on dispersion strong assumptions for modeling easy to apply reliable measurement Quantitative measurement of economic expectations: point forecasts costly and time-demanding prone to inaccuracies not available in business surveys interval-scale measurement information on dispersion and the shape of the distribution Probabilistic expectations theoretically appealing information on dispersion and the shape of the distribution information on uncertainty applicable in surveys of professionals general tendency of respondents to be optimistic / further drawbacks not available in business surveys

Categorical Scale vs. Visual Analog Scale Symmetric properties of the indifference interval are assumed Information loss due to the central tendency of responses Score range: 1 to 3 Economic confidence bad good satisfactory a b I. II. This country's overall economic situation at present? This country's overall economic situation at present? Economic confidence I. bad satisfactory good Score range: 1 to 100 Enables scores between categories Shows direction of change and magnitude Delivers dispersion and uncertainty measures

Categorical Scale vs. Visual Analog Scale 3-Category Scale Visual Analog Scale bad satisfactory good ¨ Uncertainty

Data Internet Business survey in manufacturing, Germany Number of responses

Observation period

Scale Reliability Tests Parallel-form reliability Test-retest reliability Internal consistency Inter-rater reliability VAS 3-Cat. bad satisfactory good 1. 2. 1. Step: Common factor of „econ. situation“ items extracted 2. Step: Correlation of the common factor with VAS / 3-Cat.

3-Category and VAS Business Expectations and Production Index 3-Category Business Climate VAS Business Climate

Measures of Uncertainty and Heterogeneity Direct survey-based measures: probability distribution of a point forecast uncertainty of the individual forecaster responses are comparable internal consistency and accuracy surveys of professionals not available in business & consumer surveys Time-series based measure of uncertainty: forecast errors accuracy not available in business and consumer surveys proxy for uncertainty readily available Dispersion as a proxy for uncertainty (Zarnowith and Labros, 1987) readily available proxy for uncertainty important in its own right as measure of heterogeneity quantitative point-forecasts not available in business and consumer surveys

Measuring Uncertainty and Heterogeneity with VAS “Epistemic uncertainty” Bruine de Bruin et al., 2000 Dispersion of business expectations (Heterogeneity) Zarnowith and Labros, 1987, Bomberger, 1996, Batchelor and Dua, 1996, Giordani and Soderlind, 2003, Rich and Tracy, 2006, Mitchel et al., 2005, Lahiri and Liu, 2006, Doepke and Fritsche, 2006, Boero et al., 2007… 3. Kurtosis Doepke and Fritsche, 2006: A significant kurtosis above 3 indicates that the forecasters are very close to each other. 4. Skewness Doepke and Fritsche, 2006: If the distribution of economic expectations is significantly skewed, consensus among forecasters is rejected

Growth fall / turning point Dispersion of VAS Business Expectations and the Production Index Growth fall / turning point Correlation: -0.84

Measuring Uncertainty with VAS Correlation: 0.80 Correlation: 0.72

Skewness of the VAS Present Business Situation

Skewness of the VAS Business Expectations

Skewness of the VAS Business Expectations and Production Index

Correlation with Production Index in Manufacturing Correlation of the VAS Indicators with the Production Index in the German Manufacturing Sector Lead in months 1 2 3 VAS Climate 0.94 0.95 0.93 0.89 3-Cat. Climate 0.92 0.95 0.95 0.93 VAS Expectations 0.90 0.94 0.95 0.94 3-Cat. Expectations 0.74 0.84 0.90 0.93 VAS Situation 0.94 0.92 0.88 0.83 3-Cat. Situation 0.93 0.91 0.87 0.81 Skewness of VAS situation 0.93 0.93 0.91 0.88 Skewness of VAS expectations 0.65 0.72 0.77 0.82 Epistemic uncertainty 0.56 0.62 0.66 0.61 Standard deviation of VAS expectations 0.84 0.82 0.78 0.76 Kurtosis of VAS expectations 0.90 0.89 0.88 0.86

Summary: Effectiveness of the VAS Dispersion and Kurtosis of business expectations - Contra-cyclical - Proxy for uncertainty and heterogeneity “Epistemic uncertainty” (Bruine et al. definition) - Increases around turning points - Proxy for uncertainty Skewness of the VAS distribution - Cyclical - More pronounced in VAS business situation

The Present Study Adds to the Literature: Dispersion as proxy of uncertainty was previously calculated only from point forecasts Presents a meaningful dispersion measure based on qualitative expectations There was no direct uncertainty measure derived from qualitative responses Presents a meaningful measure of “epistemic” uncertainty without eliciting probabilistic forecasts Business expectations were assumed to be normally distributed Demonstrates systematically variable skewness of business expectations over the business cycle

General Summary VAS is easy to apply and does not require any quantitative information VAS presents a direct, almost interval scale measure of economic expectations VAS delivers reliable and valid information VAS is highly efficient, delivering a variety of valid economic indicators - Dispersion measures - Uncertainty proxies - Distributional shape - Enables differentiation between uncertainty and heterogeneity Assumptions of the three-category expectations are systematically violated VAS is an effective instrument for the measurement of business expectations

Research outlook Gathering longer time-series Improving the business cycle forecasts with the information on uncertainty and heterogeneity of expectations (out-of-sample forecasts) VAS measurement of consumer expectations Application of VAS batteries in surveys, analysis of drop-outs

THANK YOU! Email: stangl@ifo.de