1 Improving Data Quality through Two Dimensional Surveying: the Kano Method Stephen M. Bauer PhD, & Vathsala I. Stone PhD RERC on Technology Transfer, University at Buffalo Center for Assistive Technology

2 Abstract The Kano Method is a unique way of conceptualizing, measuring and understanding customer defined quality for designing and developing products and services. Uses a non-traditional, “two-dimensional” (2-D) survey method to capture in-depth data. Strong basis for decision making. Demonstrably superior to “one- dimensional” (1-D) survey methods. Applied especially to the design and improvement of products and services. Growing practice in private sector. Potential application to decision-making in unexplored areas (e.g. education, health and social service). Poster presents the Kano rationale, describes the Method, and illustrates applications within and beyond product evaluation.

3 Recycle Implement Setup Design Evaluate Context User Surveys User Surveys for Design Decisions Design  new or improved products, services, curriculum… Basis: CIPP Model (Stufflebeam, 1971) Are “standard” 1-D surveys the best tools for this purpose?

4 Decision versus Precision Kano 2-D Survey Yes / No Include / Exclude Pick This One / Pick That One Take Action / Don’t Take Action Change / Don’t Change Increase / Decrease OR Weigh Measure Find Range Find Percent Average Traditional 1-D Survey

5 Understanding Customer Satisfaction FeatureDiffusionPerceptionDiscussion Exciting Not familiar to most people Unknown… Radical, impractical, complex, strange or costly… May not be discussed w/o probing questions, models… (Too “weird” to discuss) RevealedFamiliar Need it… Should be better... Readily discussed… (Very “reasonable” to discuss) ExpectedPervasive Obvious… common sense… Should be that way… May not be discussed w/o probing questions… (Why discuss? Everyone knows!) Dr. Kano drew upon psychology research Three basic types of design “Features”

6 Kano Questions Standard Kano surveys use question pairs that either include or exclude some feature, function…(Y) from some product, service…(X). Question pairs have the form: Each question employs the same 5-point response scale of the form: How would you feel if “…X HAS Y…” How would you feel if “…X LACKS Y…” Like It Don’t Like It Little Interest How would you feel if “your program HAS a carbon copy function?” How would you feel if “your program LACKS a carbon copy function?” Example: Don’t Like It … …Like It

7 5x5 Response Frame (-) (-) Revealed Exciting Expected Little Interest Reverse Revealed Reverse Exciting Reverse Expected Weak Expected Weak Exciting Not Applicable X LACKS Y X HAS Y Standard method employs a 5x5 response frame and response pairs map to one of 25 cells in the frame. View “Reverse” as though the “X HAS Y” and “X LACKS Y” questions have been “flipped.” Not quite this simple though…

8 Judgment with 1-D & 2-D Surveys Question Form …X LACKS Y… is EXPECTED is REVEALED is EXCITING Feature Y Is Y Important? Yes No Error! 1D questions of form “X HAS Y” do not “catch” EXPECTED Features. 1D questions of form “X LACKS Y” do not “catch” EXCITING Features. 2D question pairs (Kano) “catch” both EXPECTED & EXCITING Features. (Hate It!) (Like It!) Question Form …X HAS Y… Is Y Important? No Yes Error! (Hate It!) (Like It!)

9 5x5 Response Frame (-) (-) Revealed Exciting Expected Little Interest Reverse Revealed Reverse Exciting Reverse Expected Weak Expected Weak Exciting Not Applicable X LACKS Y X HAS Y Unfortunately, at least 16 of the 25 cells in the 5x5 response frame are undefined, unnecessary or ambiguous.

10 3x3 Response Frame (-) X LACKS Y X HAS Y (-) X LACKS Y X HAS Y “Subtract” Unneeded and Ambiguous Cells For decision making, added precision that contributes to ambiguous interpretation should be eliminated.

11 Feature Importance Example response distributions for items having great importance Example response distributions for items having little importance ExCiting Revealed ExPected Little Interest … … … ExCiting Revealed ExPected Little Interest … … … (C + R + P) / N  Importance, where N = # of Responses Exciting, Revealed and Expected characteristics are different – but they are all important. This suggests the following importance estimate: Item

12 Estimating Importance E.g. Design of a mainstream consumer product. 21 study participants. Kano method with 3 point response scale. 35 design features considered. Follow up “importance” survey of the form: How important is it that X HAS Y ? …Not Important… …No Opinion……Important… Estimated item (feature) importance is highly correlated (r 2  0.948) with separate Importance Survey results. Note: study limited by small sample size and lack of “Reverse” (-) features.

13 Response Averaging? (-) 3 2 (-) 8 (-) X LACKS Y X HAS Y Averaging incorrectly suggests that Y is a Feature of Little Interest. Y is primarily an Expected Feature (3,2). In general, averaging (and related approaches) can cause incorrect interpretation of the data resulting in poor design decisions. Average  (1.96, 1.82)  (2,2)

14 Use of Categories 12 (-) 15 3 (-) 20 (-) X LACKS Y X HAS Y Primary category (1,2): Y is unexpected and disliked by most “customers.” Secondary category (2,3): Y is unexpected and liked by many “customers.” Simultaneously looking at two or more categories provides important guidance for design decisions. Consider a software application where some feature can be “switched” on / off by the user…

15 Graphical Representation 0.5  Revealed Increasingly Exciting Increasingly Expected E.g. Design of an product. 22 study participants. Kano method with 3 point response scale. 123 design features considered. Note: study limited by small sample size. Analysis for “Reverse” (-) features requires an extension to this approach. Feature tendency toward being exciting, revealed, or expected is ~ (1 + (C-P)) / 2 ≥ 0.8 important 0.8 to 0.5 moderately important 0.0 to 0.5 gen. not important

16 Further Explorations How would you feel if “X HAS MORE OF Y” ? How would you feel if “X HAS LESS OF Y” ? 1) In practice, design decisions are often made whether to increase or decrease a feature, function… (Y) for some product, service… (X). Kano question pairs of the following form can be used for this purpose: 2) Response distributions can be evaluated for diffusion of innovation (technology transfer) and related phenomena. The common lifecycle for innovation is: Exciting (introduction of Feature)  Revealed (period of refinement)  Expected (perfected and pervasive)  Reverse Categories (obsolescence). 3) Response distributions can be used to study market segmentation (business) and related phenomena. 4) No Sense responses can be used to flag survey response errors and flawed question construction. It is clear that respondents should not like (hate) both the presence and absence of any Feature.

17 Acknowledgement This is a presentation of the Rehabilitation Engineering Research Center on Technology Transfer, which is funded by the National Institute on Disability and Rehabilitation Research of the Department of Education under grant number H133E The opinions contained in this publication are those of the grantee and do not necessarily reflect those of the Department of Education.