1. Listening to the Learners: A Case Study in Health Information Website Design Jennifer Turns Acting Assistant Professor Technical Communication Tracey Wagner Educational Technology Analyst Program for Educational Transformation Through Technology Acknowledgements: This work has been supported by the Program for Educational Transformation Through Technology (PETTT). Many people have contributed to this work including Scott Macklin, Aaron Louie, Kristen Shuyler, Brett Shelton, Kristina Liu, Alice Tanada, Jake Burghardt, Julianne Fondiller, Regina Yap, Ralph Warren, and Dr. Frederick Matsen.

2. Today’s Goal –Background –Case Study –Results –Implications –Conclusions Demonstrate how a characterization of users as learners can inform the design of an informational website. Background - Case Study - Results – Implications - Conclusion

3. Informational Website Design Examples: –legal information –cooking information –home repairs and renovations Issues –Type of content –Organization of content –Style of content –Access to content Background - Case Study - Results – Implications - Conclusion

4. Audience Analysis “Know thy user” Common dimensions include –Role – Dominant persona of users (job, affiliation) –Goals – Reason for the interaction –Knowledge – The extent and nature of prior relevant knowledge –Circumstances of Use – Setting, resources, strategy, timing –Culture – Group level beliefs, language, preferences –Ergonomics – Relevant perceptual & motor abilities, skills Background - Case Study - Results – Implications - Conclusion

5. When users are learners Learner centered design (Soloway et al, 1994) Key Challenges –Diversity: learners are diverse –Engagement: learners need support for engaging in learning –Growth: learning changes the learner Background - Case Study - Results – Implications - Conclusion

6. Design Relevance? What is the nature of diversity, engagement, and growth in a specific domain and with a specific user populations? How can we characterize users along these dimensions? How do we design informational web sites to address these dimensions? Background - Case Study - Results – Implications - Conclusion

7. Arthritis Source Origin: Developed in 1995 by Dr. Rick Matsen Goal: Help learners-at- large satisfy their knowledge needs with authorized information. Challenges: Expand scope and ensure effectiveness of Arthritis Source Background - Case Study - Results – Implications - Conclusion

8. Audience Analysis Methods Challenge –Getting to actual users Our methods include –Online Survey –Phone Interviews –Logfile Analysis Background - Case Study - Results – Implications - Conclusion

9. Diversity Users with many roles Many goals / questions –Condition information (54%) –Ways to minimize pain (7%) –Seek clarification or a second opinion on diagnoses (5%) –Preparing for surgery (5%) –Desire to better control arthritis condition (2.5%) –… n=462/472 Background - Case Study - Results – Implications - Conclusion

10. Diversity Educational spread Misconceptions –Bone spurs cause arthritic pain –Joint injections are helpful –Low bone density is associated with Osteoarthritis –There is little you can do –(and 3 others) n=399/472 Background - Case Study - Results – Implications - Conclusion

11. Diversity International Use - 128 countries in 2½ months Background - Case Study - Results – Implications - Conclusion

12. Engagement –Motivated learners: 48% from search engine –Personally engaged: 65% have arthritis –Heavily one-time visitors: 80% first visit –Moderate time on page: avg. visit ~ 7 mins. –Common misconceptions and questions suggest a need for longer engagement periods and deeper interaction with information Background - Case Study - Results – Implications - Conclusion

13. Growth –Windows into growth Many statements of information needs Users at different points in patient careers –Opportunities for growth Efforts to articulate questions –Ongoing interest in exploring Growth Background - Case Study - Results – Implications - Conclusion

14. Design Challenges Diversity –Support users with variety of questions/goals –Support users in different contexts –Support users with different knowledge Engagement –Help users find relevant information quickly –Find ways to engage users in knowledge that might be useful to them (e.g., correcting misconceptions) Growth –Support user concerns and knowledge over time Background - Case Study - Results – Implications - Conclusion

15. Design Direction – Content Content via Templates –Common organization –Linked to questions –Ensures coverage Links to –Diversity –Engagement –Growth Background - Case Study - Results – Implications - Conclusion

16. Design Direction – Search Question-based Search –Start with question –Choose from “similar” questions –Enter content at point where question is answered. Links to –Diversity –Engagement –Growth Background - Case Study - Results – Implications - Conclusion

17. What we are working on Building on these strategies –Expanding content –Improving search Evaluation Studies – Is it working? –User evaluation of content –“Longitudinal” Study - Impact of extended site interaction on knowledge and health outcomes Background - Case Study - Results – Implications - Conclusion

18. Moving in the Right Direction: Evidence of Growth? Sequences of searches showing growth? 8:26 am "my husbands leg bone below knee is splinering and knee has moved over it,is there a surgery to save his leg" At 9:22 am "aproximately 6inches below knee cap the long bone is spintering is there a surgery that can cover this and help support the leg bone“ 10:01am "can a femur tibia splintering of the bone be surgicially be repaired ?“ Background - Case Study - Results – Implications - Conclusion

19. Concluding Remarks Learner-Centered Design - Diversity, engagement, growth Learner-Centered Design and Audience Analysis –One strategy for audience analysis –Complements other audience analysis approaches (e.g., roles, tasks) Learner-Centered Design and Our Case Study –Variety of data can be organized via this framework –The dimensions are powerful organizer –Diversity is easy to capture –Engagement & growth are challenging to capture for web populations. –Using this framework informed our design Background - Case Study - Results – Implications - Conclusion

21. Method – Logfile Analysis Analyze record of users interactions with site Available data include –Entry/exit pages, pages viewed, time on page, country of origin, etc. Background - Case Study - Results – Implications - Conclusion

22. Method - Phone Interview About Visit Could you tell me about your visit or visits to the Arthritis Source? Could you tell me what you were trying to do when you visited the Arthritis Source? Did you benefit from your visit or visits to the Arthritis Source? What kind of information do you think other arthritis patients should know? About Knowledge of Condition Could you tell me what you think arthritis is in general? Could you tell me how RA/OA affects the body? Do you know what contributes to getting RA/OA? Do you know how RA/OA is diagnosed? If no, Do you remember what your doctor told you about your diagnosis? What is most difficult to understand about RA/OA? Background - Approach – Results - Implications

23. Method – Online Survey Questions: Adaptive, ~25 questions Participants –Duration: 9/1/2000 – 7/2/2001 (10 months) –472 respondents / 710 starts Analyses –Descriptive Statistics –Content Analysis –Qualitative Coding –Statistical Analysis Background - Case Study - Results – Implications - Conclusion

24. What we are working on Supporting goals (finding/using content) Support evolving content Evaluating effects –Learning –Health outcomes –Use patterns –Various changes over time—growth Background - Case Study - Results – Implications - Conclusion

25. Method – Online Survey Questions: Adaptive, ~25 questions –Demographics –Role –Goal –Successful during today’s visit Participants –Duration: 9/1/2000 – 7/2/2001 (10 months) –472 respondents / 710 starts Background - Case Study - Results – Implications - Conclusion

26. Method - Phone Interview Questions: –About your visit (5): Could you tell me what you were trying to do when you visited the Arthritis Source? –Knowledge of condition (5): Could you tell me how RA/OA affects the body? Participants – 20 users (10 OA, 10 RA) Analyses –Conceptions/misconceptions –Overarching Goals –Specific Information Needs Background - Case Study - Results – Implications - Conclusion

27. Method – Logfile Analysis Analyze record of users interactions with site Available data include –Entry/exit pages, pages viewed, time on page, country of origin, etc. Background - Case Study - Results – Implications - Conclusion Sample Logfile Data: 2 requests cpe-65-172-233-177.sanbrunocable.com [29/Apr/2002:00:01:07 -0700] "GET /shoulder_elbow/shoulder/roughness/management/exercise/02/exercise06 HTTP/1.1" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461)" cpe-65-172-233-177.sanbrunocable.com [29/Apr/2002:00:01:11 -0700] "GET /shoulder_elbow/shoulder/roughness/management/exercise/02/exercise05 HTTP/1.1" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461)"