Prepared by Stephen M. Thebaut, Ph.D. University of Florida G&W Chapter 21: Measuring Satisfaction Software Specification Lecture 28 Prepared by Stephen M. Thebaut, Ph.D. University of Florida

Part V: Greatly Improving the Odds of Success Ambiguity Metrics Technical Reviews Measuring Satisfaction Test Cases Studying Existing Products Making Agreements Ending

Rationale Perhaps 90% of product development efforts fail. About 30% fail to produce anything at all, but most failures produce a product that people simply don’t like. The easiest way to avoid this problem is to measure user satisfaction as the design takes form.

Addressed to a group of students at UF: Case Study: Development of a Satisfaction Test for Graduate Students at UF Addressed to a group of students at UF: We need half-a-dozen or so attributes by which, in the final analysis, your experience as a graduate student will be evaluated. Using a round-robin approach, please identify an attribute that is important to you.

DRAFT How do you rate your graduate student experience at this time? useless useful not fun fun not satisfied with performance satisfied with performance poor interaction with faculty good interaction with faculty poor job preparation good job preparation poor course selections good course selections Students: If you fill this out each semester, will it enable you to express what you like and don’t like about your graduate student experience?

How do you rate your graduate student experience at this time? Comments? Name ___________________

Using the Satisfaction Test Each completed test has “face validity” and is therefore a useful communication vehicle. Changes in average response can reflect major shifts in satisfaction. Follow-up with an interview. 1 2 3 4 Semester

Interpreting Comments When people take the time to write something free-hand, it’s probably important. Puzzling or strongly worded comments must be tracked down. (Hypothetical) Examples: “This program stinks! I can’t even take the courses I really want to.” “I think the faculty could do much more to help students!”

Attributes of G&W’s Test Allows frequent measurement so that changes in satisfaction can be detected quickly. Provides a periodic indication of divergence of opinion. (Can warn of unrecognized ambiguity or political problems.)

Attributes of G&W’s Test (cont’d) Allows designer to pinpoint specific areas of dissatisfaction, and to identify dissatisfaction within specific user constituencies. Usually yields a high response rate. Allows designers to determine statistical reliability of the sample.

Attributes of G&W’s Test (cont'd) Provides insight into how the completed design will be evaluated. (I.e., the relative importance of attributes.) Useful to those who take it (keeps users involved) and create it. Cheap, easy to use, non-intrusive, and educational to all.

Other Tests The existence of a regular survey should never be used as an excuse for not supplementing it with other, commonsense methods. Try the occasional “How do you feel about the system right now?” You can also observe how people use, avoid, or modify a product increment or prototype to gauge satisfaction.

Prepared by Stephen M. Thebaut, Ph.D. University of Florida G&W Chapter 21: Measuring Satisfaction Software Specification Lecture 28 Prepared by Stephen M. Thebaut, Ph.D. University of Florida