1. a.k.a. Voice of the Customer
The 17th Annual International Deming Research Seminar
Test it in Service
a.k.a. Voice of the Customer

2. What We Plan to Cover (AIM)
Why Test in Service? Why Good Sample Design? Accuracy versus Precision Some Statistical issues Some Methodological Issues Some Data Issues Some Questionnaire Issues
23 March 2011
The 17th Annual International Deming Research Seminar

3. Deming Lecture Japan 1950 23 March 2011
The 17th Annual International Deming Research Seminar

4. Evolution of Shewhart Cycle
Production
Make
Specification
Design
Inspection
Sell
(a. Old Way)
4. Test in
Service
5. Redesign
Specification
1. Design
Production
3. Sell
2. Make
Inspection
(b. Original Shewhart 1939) (c. Deming 1950 Version)

5. Production Viewed as a System
Design
and
Redesign
Consumer
Research
Suppliers of
Materials and
Equipment
Consumer A
Receipt and test of materials
Consumer B
Production, assembly, inspection
Consumer
Consumer
Consumer C
Consumer D
Tests of Processes, machines, methods, costs
Consumer
23 March 2011
The 17th Annual International Deming Research Seminar

6. Process GAP Input Conversion Output Voice of the Customer
Voice of the Process
GAP
In order to be sure that the conversion is operating as intended, one samples the output. Selecting which output of which sub-process to measure is the subject of another chapter. In general, it depends on what is a matter of importance to the customer. A method with the acronym of “QMS” (Quality Measuring System) helps management to determine where to measure to get the most value from the measurement.
Any deviation that lessens the value of the item to the customer is subject to measurement. As Mr. William Scherkenbach pointed out, “There is powerful leverage in going upstream” (Scherkenbach, 1986). Upstream means the first input source than can be impacted. For example, if the material (or information) used is faulty, it is not likely to improve as it works its way through the network of processes. The process input is upstream while the process output is downstream.
To determine the effectiveness of the conversion one needs two measures called “Voice of the Customer” and “Voice of the Process”. The first measure gives us an understanding of the customer’s needs. The second tells us how well the conversion process is meeting those needs. For the voice of the customer one uses proper market research methods. A method that obtains a great deal of information is a quadrant analysis developed by Mr. David Saunders (1992).
To determine how well the conversion process meets the customers’ needs one gets the voice of the process. This text is concerned with how to get the appropriate measure from the process. One should keep in mind the customer may well be an internal one. For example, the accounting function has as its primary customer the Board of Directors and the officers of the organization.
Input
Conversion
Output
23 March 2011
The 17th Annual International Deming Research Seminar

7. How do we get the Voice of the Customer?
Test it in Service
Consumer Research
In order to be sure that the conversion is operating as intended, one samples the output. Selecting which output of which sub-process to measure is the subject of another chapter. In general, it depends on what is a matter of importance to the customer. A method with the acronym of “QMS” (Quality Measuring System) helps management to determine where to measure to get the most value from the measurement.
Any deviation that lessens the value of the item to the customer is subject to measurement. As Mr. William Scherkenbach pointed out, “There is powerful leverage in going upstream” (Scherkenbach, 1986). Upstream means the first input source than can be impacted. For example, if the material (or information) used is faulty, it is not likely to improve as it works its way through the network of processes. The process input is upstream while the process output is downstream.
To determine the effectiveness of the conversion one needs two measures called “Voice of the Customer” and “Voice of the Process”. The first measure gives us an understanding of the customer’s needs. The second tells us how well the conversion process is meeting those needs. For the voice of the customer one uses proper market research methods. A method that obtains a great deal of information is a quadrant analysis developed by Mr. David Saunders (1992).
To determine how well the conversion process meets the customers’ needs one gets the voice of the process. This text is concerned with how to get the appropriate measure from the process. One should keep in mind the customer may well be an internal one. For example, the accounting function has as its primary customer the Board of Directors and the officers of the organization.
Input
Conversion
Output
23 March 2011
The 17th Annual International Deming Research Seminar

8. Test it in Service Thought Data Knowledge 23 March 2011
The 17th Annual International Deming Research Seminar

9. Survey Impact on Management
Wrong Design
Wrong Information
Wrong Decision
23 March 2011
The 17th Annual International Deming Research Seminar

10. What is the aim of the survey?
Who determines the aim?
The Subject Matter Expert
Why is the aim important?
Without an aim there is no system
23 March 2011
The 17th Annual International Deming Research Seminar

11. Who is involved in Surveys
Subject Matter Expert Statistician Respondents
23 March 2011
The 17th Annual International Deming Research Seminar

12. The Subject Matter Expert
Commissions the survey
Determines the aim
Can act on the results
Has budget authority
23 March 2011
The 17th Annual International Deming Research Seminar

13. Aspects of Sample Design
Statistical
Sample size
Variance
Accuracy
Methodological
Sample selection
Computation
23 March 2011
The 17th Annual International Deming Research Seminar

14. What is a sample Complete coverage gives answer
A subset of the interest group
Draw Sample
23 March 2011
The 17th Annual International Deming Research Seminar

15. The Frame The Frame is a list of sample units
Frames are not always complete
The Gap
23 March 2011
The 17th Annual International Deming Research Seminar

16. Data vs. Information Data The sample yields data Analysis
Results
Data
Analysis
The Subject Matter Expert needs Information
Information
23 March 2011
The 17th Annual International Deming Research Seminar

17. Information contains Result Measure of uncertainty 23 March 2011
The 17th Annual International Deming Research Seminar

18. Which is more important
Sample size? or Sample method?
Sample size? or
Sample method?
Bias
Total Error
Sampling Error
23 March 2011
The 17th Annual International Deming Research Seminar

19. Accuracy and Precision
Sample Method
Not Accurate
Accurate
Not Precise
Sample Size
Precise
23 March 2011
The 17th Annual International Deming Research Seminar

20. Sample Size (n) Expected result percentage Desired precision
Where is the actual result
t the number of standard deviations under the Normal Curve
Then t
Area for Mean ±ts 1
68% 2
95% 3
99.73%
23 March 2011
The 17th Annual International Deming Research Seminar

21. Sample Size Example
Q. Was the Conference useful? Please check one: Yes □ or No □ If one wants 95% reliability that the actual responses fall within 10% of the response of 50% then .
23 March 2011
The 17th Annual International Deming Research Seminar

22. 23 March 2011
The 17th Annual International Deming Research Seminar

23. Because it is the maximum sample size
23 March 2011
The 17th Annual International Deming Research Seminar

24. Sample Method Clear Aim Random Sampling Unbiased Questions
23 March 2011
The 17th Annual International Deming Research Seminar

25. Clear Aim What does the owner wish to accomplish with the data?
How will the data be used in practice?
Is the data measureable
Cause
Effect
Is the data useable?
e.g. measure engagement with an Association
e.g. measure how often contacted by Association
23 March 2011
The 17th Annual International Deming Research Seminar

26. Random sample Equal probability of selection Random numbers
Random Start nth Sample
Replicated design with random start per zone
Deming's Replicated Design: Total sample cut into zones like sliced bread
One zone
Total sample size
23 March 2011
The 17th Annual International Deming Research Seminar

27. Reduce Bias in Questionnaire
If questionnaire…
…is for interviewing then train interviewer to
Avoid loaded questions
Avoid verbal bias
Avoid body language bias
…is for respondent's completion
Avoid question overload
23 March 2011
The 17th Annual International Deming Research Seminar

28. Open Ended Questions
An open-ended question is designed to encourage a full, meaningful answer using the subject's own knowledge and/or feelings.
Opposite of closed-ended question, which encourages a short or single-word answer.
Open-ended questions also tend to be more objective and less leading than closed-ended questions.
Courtesy Myriam Ochart, ASQ CSSBB, CMQ/OE

29. Open-Ended Question Examples
Closed-Ended Questions
Open-Ended Questions
Do you get on well with your boss?
Tell me about your relationship with your boss.
Who will you vote for this election?
What do you think about the two candidates in this election?
Is that a photograph of your children?
Tell me about the children in this photograph.
Courtesy Myriam Ochart, ASQ CSSBB, CMQ/OE

30. Prevent Response Bias
Avoid Ambiguity: Questions not focused enough to obtain the needed information. Poor question: How often do you buy fast food? Better question: The last 10 times you’ve eaten lunch out, how often have you purchased each of the following types of food?
Courtesy Myriam Ochart, ASQ CSSBB, CMQ/OE

31. Prevent Response Bias
Avoid questions that are too general. Poor question: Do you like orange juice? (Do you mean taste? texture? price?) Better question - more specificity: Do you like the taste of fresh-squeezed orange juice?
Courtesy Myriam Ochart, ASQ CSSBB, CMQ/OE

32. Prevent Response Bias
Avoid loaded questions: use of language likely to bias response: Do you actually support Proposition X?
Courtesy Myriam Ochart, ASQ CSSBB, CMQ/OE

33. Prevent Response Bias
Avoid leading questions: Questions that lead respondents toward a particular answer. Isn’t it true that women are more likely to talk on the phone while driving?
Courtesy Myriam Ochart, ASQ CSSBB, CMQ/OE

34. What We Covered
Why Test in Service? Why Good Sample Design? Accuracy versus Precision Some Statistical issues Some Methodological Issues Some Data Issues Some Questionnaire Issues
23 March 2011
The 17th Annual International Deming Research Seminar

35. Questions?
23 March 2011
The 17th Annual International Deming Research Seminar