Importance–performance analysis Prof Ivan Lai

Importance-Performance Analysis Importance-performance analysis (IPA) has been applied to different areas in the services industries since it was introduced by Martilla and James (1977) in the 1970s. In IPA studies, respondents are asked to rate the importance and performance of selected attributes in relation to their experiences. The direct measurement results were plotted on four quadrants of importance–performance mapping (I–P mapping). The ‘data-centred quadrants approach’ has been widely used to display the empirical data as cross-points. Lai and Hitchcock (2015a)

I-P Mapping

Example of Direct IPA Approach Lai and Hitchcock (2015a)

Indirect IPA In the indirect IPA method, the relationship between the attribute’s importance and customer satisfaction is statistically inferred, commonly using two approaches. The first is the ‘regression coefficient approach’ that calculates the relative importance of an attribute by using a multiple regression with attribute performance (independent variables) and overall customer satisfaction (dependent variable). The second is the ‘partial correlation method’ that evaluates the relative importance of an attribute by analysing the correlation between attribute performance and overall satisfaction. Lai and Hitchcock (2015a)

Example of Indirect IPA Approach Lai and Hitchcock (2015a)

Different I-P Mapping Approaches Scale-centered quadrants approach (Martilla and James, 1977) Data-centered quadrants approach (Alberty and Mihalik, 1989) Diagonal line model (Hawes and Rao, 1985) Scale-centered diagonal line model (Abalo, Varela, and Rial, 2006) Means and diagonal line model (Rial, Rial, Varela, and Real, 2008) Three-factor theory model (Fallon and Schofield, 2006)

Different I-P Mapping Approaches Lai and Hitchcock (2015b)

Reliability and Validity Issues Due to the lack of comprehensive guidelines, researchers over the years have pursued IPA in various ways. Common questions: Is it necessary to build a new set of attributes? Which scale is the best one for IPA studies? What should be the appropriate sample size for IPA study? Which sampling method is preferable? And how should data be collected? In what situation(s) should factor analysis be performed? Should it be EFA or CFA? Is it necessary to calculate the difference between importance and performance with t-value? And if so, why? Is I-P mapping an effective tool for setting strategic actions and how may it be presented graphically? Lai and Hitchcock (2015b)

IPA Research Framework Lai and Hitchcock (2015b)

Research Direction In cases where importance and performance of attributes are designed to suit an individual firm (e.g., Holiday Inn), the aim of this approach is to provide guidance for a given company on how to improve its weaknesses. (XX) If the importance of the attributes is measured in an industrial setting (e.g., the hotel industry in Macau) and the performance of attributes is measured for an individual firm (e.g., Venetian Macao-Resort-Hotel), then the outcome of this setting is to show how individual firms may enhance competitiveness. (X) Using IPA to evaluate different segments' preferences could generate more meaningful results. (√) Cross-cultural comparisons. (√) Well-developed attributes deriving from previous studies. (X) Own unique set of attributes for the study. (√ √) Lai and Hitchcock (2015b)

Questionnaire Design Unique set of attributes Qualitative studies such as focus groups and/or unstructured personal interviews to generate their unique set of attributes. For performing indirect performance measures, measurable items for overall satisfaction should be included. Likert-type scale Using 5-point or 7-point Likert-type scales 7-point Likert-type scale shows more reliable results in measuring the gaps between importance and performance of the attributes Asking importance-performance sequences simultaneous sequence: a monotonous task which may lead respondents to produce stereotyped and unreliable information (X) separate sequence: respondents to initially rate the importance of all attributes and then rate the performance of all attributes back translation was used in order to ensure that there was no translation bias Lai and Hitchcock (2015b)

Data Collection Sample size Pilot studies Sampling methods Time frame should be conducted to test content validity minimum sample size is 30 Sample size the item-to-response ratios should be 1:20 for 15 measurable items, 300 samples Sampling methods Convenience, random, full Systematic random sampling (√) Time frame measuring importance occurs before the product or service is obtained ascertaining performance is after consumption Lai and Hitchcock (2015b)

Multivariate Normality measured by the use of Mardia's multivariate kurtosis cutoff value of 3 for skewness cutoff value of 10 for kurtosis the value of multivariate kurtosis should not be 30 or greater Lai and Hitchcock (2015b)

Data Analysis Process Step 1 – descriptive mean and standard deviation of attributes (not enough) skew and kurtosis distribution a multivariate normality test Eliminating these abnormal attributes in order to obtain a multivariate normal distribution detect influential multivariate outliers by using the Mahalanobis distance approach as a precursor for removing abnormal data keeping non-normal attributes and perform bootstrapping to overcome multivariate normality problems for indirect measures (details will be discussed later) Lai and Hitchcock (2015b)

Data Analysis Process Step 2 – reliability and validity Exploratory Factor Analysis (EFA) provides evidence of construct validity Convergent EFA is recommended for refining new attributes Confirmatory Factor Analysis (CFA) is conducted to assess the quality of the factor structure CFA provides further evidence of the construct validity of the new attributes Cronbach's alpha is commonly used to test internal consistency reliability The average variance extracted (AVE) and composite reliability (CR) are used for accessing convergent validity Lai and Hitchcock (2015b)

Exploratory Factor Analysis A Kaiser-Meyer-Olkin (KMO) test of sample adequacy (>0.70) EFA is based on the principal component analysis with varimax rotation and eigenvalue exceeding 1 with factor loadings exceeding 0.4 Lai and Hitchcock (2015b)

Confirmatory Factor Analysis The CFA fit indices include: Chi-square relative/normal Chi-square (<3) RMSEA (<0.08) CFI (>0.90) NFI (>0.90) Lai and Hitchcock (2015b)

Cronbach's alpha, AVE, CR, and Correlation Coefficients The square-root of the construct's AVE exceeds its correlations with other dimensions and thus the discriminant validity of the latent dimensions is also valid. Lai and Hitchcock (2015b)

Data Analysis Process Step 3 - measure the levels of importance it is assumed that the levels of importance of attributes for products or services are different repeated measures ANOVA will help to find the rating of importance among the attributes Post Hoc tests can be applied in case the researcher wants to further elucidate the differences between attributes Lai and Hitchcock (2015b)

Mauchly’s Test of Sphericity If the test of sphericity is significant (i.e., p-value < 0.05), then the assumption of null hypothesis is met. For the tests of Within-Subject Effects, the main effect is significant (p-value < 0.05). Lai and Hitchcock (2015b)

Data Analysis Process Step 4: measure the gaps between importance and performance individual paired-samples t-test should be used to confirm that there are significant differences among the levels of importance of the attributes and their respective performances gap analysis can also be viewed as an effective means of benchmarking against competitors Lai and Hitchcock (2015b)

Individual Paired-samples t-test the attributes with p-value higher than 0.05 should be eliminated and should not be plotted in IeP mapping for further interpretation Lai and Hitchcock (2015b)

Data Analysis Process Step 5: interpret the results of IPA The most common direct measurement method is the ‘data-centered quadrants approach’ in I-P mapping. ‘regression coefficient approach’ has a significant advantage because researchers can perform a multiple regression analysis by using standard statistical tools (such as SPSS, LISREL, AMOS, and PLS). bootstrapping can be performed to adjust for distributional problem in case a multivariate normality problem exists. The Partial Least Squares (PLS) approach is recommended because it is more robust with small sample sizes and non-normal data Lai and Hitchcock (2015b)

I-P Mapping for Direct Measures Lai and Hitchcock (2015b)

PLS Results Lai and Hitchcock (2015b)

Latent Variable Correlations Lai and Hitchcock (2015b)

I-P Mapping for Indirect Measures Lai and Hitchcock (2015b)

Dimensions Distribution researchers should provide a dimensions (factors/constructs) distribution in I-P mapping after showing the attributes‘ spread in cases where the attributes can be classified into different dimensions Lai and Hitchcock (2015b)

Data Analysis Process Step 6: practical recommendation The strategic actions should relate to the existing market position of the Lai and Hitchcock (2015b)

Flexible Data-Centered Diagonal Line Model Lai and Hitchcock (2015b)

Flexible Data-Centered Diagonal Line Model Lai and Hitchcock (2015b)

Q & A

References Abalo, J., Varela, J., & Rial, A. (2006). El Analisis de Importancia-Valoracion aplicado a la gestion de servicios. Psicothema, 18(4), 730-737. Alberty, S., & Mihalik, B. (1989). The use of importance-performance analysis as an evaluative technique in adult education. Evaluation Review, 13(1), 33-44. Fallon, P., & Schofield, P. (2006). The dynamics of destination attribute importance. Journal of Business Research, 59(6), 709-713. Han, S., Ham, S. S., Yang, I., & Baek, S. (2012). Passengers’ perceptions of airline lounges: Importance of attributes that determine usage and service quality measurement. Tourism Management, 33(5), 1103–1111. Hawes, J. M., & Rao, C. P. (1985). Using importance-performance analysis to develop health care marketing strategies. Journal of Health Care Marketing, 5(3), 19-25. Lai, I.K.W., & Hitchcock, M. (2015a). A Consideration of Normality in Importance-Performance Analysis, Current Issues in Tourism, Jan 2015 online publication. Lai, I.K.W., & Hitchcock, M. (2015b). Importance-Performance Analysis in Tourism: A Framework for Researchers, Tourism Management, 48, 242–267. Martilla, J. A., & James, J. C. (1977). Importance-performance analysis. Journal of Marketing, 41(1), 77-79. O’Leary, S., & Deegan, J. (2005). Ireland’s image as a tourism destination in France: Attribute importance and performance. Journal of Travel Research, 43(3), 247–256. Rial, A., Rial, J., Varela, J., & Real, E. (2008). An application of importance-performance analysis (IPA) to the management of sport centres. Managing Leisure, 13(3/4), 179-188.