1. How to Get Insightful and Useful Audience Input for User Friendly Tools Maritza E. Matheus, Principal AHRQ Annual Conference Monday, September 14, 2009

2. Audience Exercise Need: paper and pen On middle of page, write “packing suitcase for business trip” and circle it Packing suitcase for business trip

3. What are we discussing today? When is it useful to obtain audience input? Why conduct formal testing? Approaches Testing mechanics Do’s and Don’ts Examples of eliciting information Useful tactical tips Audience exercise

4. When is it Useful to Obtain Audience Input? Developing a consumer product –Planning a product –Pre-testing of a prototype –Post-testing of a model Testing an existing product Testing messages Developing an advertising campaign

5. Qualitative Research To gather an in-depth understanding of human behavior; answers the “why” Provides insights, feedback and information not gleaned from quantitative approaches Key: use gleaned information into actionable items and strategies

6. Approaches Focus group Dyad/Triad One on One/In Depth Interviews Telephone Online Blogs

7. Approaches ApproachParticipantsDurationProsCons Focus Group8-101.5-2 hoursRich in insights, group dynamics $ Dyads/Triads2 or 31 -1.5 hoursProbe deepConduct numerous One on One Interviews / In Depth Interviews 1 person1 hour or lessFace to face, phone Delve deep No group dynamics Conduct numerous Telephone1 person1 hour or lessGood for difficult to schedule participants Miss face to face cues, interaction OnlineNumerous1 hour or lessGood for difficult to schedule participants Miss face to face cues, interaction, $

8. Testing Mechanics Clear Objectives Planning! Appropriate Approach or Combination of Approaches Budget Qualitative Researcher or In-house Venue - field facility or other Recruiting (Demographics; Recruiting Guide or Screener) Moderating (Topic Areas; Discussion Guide) Reporting

9. Do’s Do allow enough time for testing, including approvals Do be inclusive in selecting audience (users, non- users, clinicians, family, educators, etc) Use tools: flip chart, markers, actual products Do honor time limit set with participants Do provide food Do provide incentives - be creative Do audiotape and video tape Do set time limit on topic areas Do pre and post testing

10. Don’ts Don’t allow friends or family in same groups Don’t allow for “seasoned” participants Don’t accept all of the recruited participants (do over recruit) Don’t ask participants to share their job title or profession unless necessary Don’t ask “WHY” - use other ways to elicit information

11. Examples of Eliciting Information Flip Chart –Easel, markers Design Team Visualization Word Bubbles Picture Decks Mind Map

12. Word Bubbles Use: when learning about sensitive, conflicting or politically-charged attitudes Tools: handout with stick figures with bubbles

13. Picture Decks Use: need to learn about emotional associations about an experience or an issue Tools: deck of images

14. Mind Mapping Use: when learning more details or more indepth understanding of behavior, situation, or relationship Tools: paper and markers “Quit Day” Triggers Bad Moods Eating Weight Gain

15. Mind Map Example of Completed Mind Map Activities Disney World Family vacation Fun Memorable Costly No adult time ChildhoodCommercializationFamily Flying Summer Clothes Kids cranky Florida Mickey Mouse Long lines Save $ Exercise - toning

16. Tactical Tips Over-recruit Latinos and African-Americans Pay for transportation for indigent or elderly Mail publications in advance for participants to review before group testing Best not to mix some participants, such as physician specialties or physicians with other clinicians Consider on-site interviews when appropriate Ask colleagues and observers for their insights

17. Audience Exercise Need: paper and pen On middle of page, write “packing suitcase for business trip” and circle it Example Take 2 minutes to write thoughts that come to mind when packing a suitcase When done, circle the most significant branch Packing suitcase for business trip

18. Thank You! Questions? Contact: Maritza E. Matheus Matheus Marketing, LLC Oakton, VA 22124 Tel 703 758-7595 Email matheusmarket@aol.com