Evaluating the Changes in Knowledge and Attitudes of Digital Library Users Gemma Madle, Patty Kostkova, Jane Mani-Saada, Julius R Weinberg City eHealth Research Centre Institute of Health Sciences, City University, London

Talk outline Introduction to the theme Health information on the Internet Usability testing of Digital Libraries Antimicrobial Resistance Website Evaluating User Knowledge & Attitude changes Summary of Results & Future Research

1a. Health information on the Internet Explosion of health information on the internet, e.g. NHS Direct, Netdoctor, Surgery Door, NeLH, Medline Plus The informed patient, e.g. Expert Patient Change in doctor-patient relationship, shift in the balance of power, shared decision-making “One has a greater sense of intellectual degradation after an interview with a doctor than from any human experience.” Alice James

1a. Health information on the Internet Medical digital libraries Life critical applications Empower health professionals & patients National electronic Library for Health Communicable Diseases specialist library Freely accessible to all via the Internet

1b.Usability testing of DLs Graphical interface, navigation, download time etc e.g. Nielsen Accessibility e.g. Bobby User information seeking behaviour User satisfaction

1b. Usability testing of DLs So we know that….. Does it have any impact on the quality of clinical care?

2. Antimicrobial Resistance Website Acts as an interface to a subsection of the NeLCD Aim of the site “to inform the public of current evidence-based guidelines on antimicrobial prescribing and the issues surrounding those guidelines in an effort to reduce patient pressure on doctors and subsequently reduce inappropriate prescribing” Useful testbed to evaluate the impact of the NeLCD on user knowledge & attitudes and subsequent behaviour

2. Antimicrobial Resistance Website Identification of content Ask the Doctor archives Health information websites Current news topics Organisation of content MeSH Reasons for visiting site

2. Antimicrobial Resistance Website Lotus Domino Database Left navigation menu Search facility Dynamically generated content: Frequently asked questions Related news articles Related resources

2. Antimicrobial Resistance Website Antimicrobial Resistance is a global problem Patient expectations influence prescribing by doctors Importance of educating the public highlighted by influential UK reports So can we change public (and professionals ) behaviour?

3. Evaluation Methods Pre and Post Use Questionnaires Used to evaluate differences in patient knowledge and attitudes before and after using the site Users were free to browse the site between questionnaires 227 Science Museum visitors took part as part of ‘live science’ of which 177 completed both questionnaires

3. Evaluation Methods - Knowledge Knowledge about something or someone can be gained through information or facts (Oxford English Dictionary) 7 true/false statements about antibiotic use

3. Evaluation Methods - Attitude “a psychological tendency that is expressed by evaluating a particular entity with some degree of favour or disfavour” Six Likert scale questions 1 = strongly disagree, 5 = strongly agree Four evaluated attitude to information on the site Two evaluated attitude to prescribing

3. Evaluation Methods Relationship between knowledge and attitude Changes in knowledge may or may not be reflected by changes in attitude It is important to change people’s attitudes and behaviour as well as improve their knowledge

3. Evaluation Results - Knowledge

3. Evaluation Results - Attitude

3. Evaluation Results 24% users increased knowledge score and decreased attitude score 7% users decreased knowledge score and decreased attitude score 5% users decreased knowledge score and increased attitude score 11% users increased knowledge score and increased attitude score

4. Summary of Results Significant improvements in user knowledge about antibiotics Significant changes in attitude towards antibiotic use in Acute Otitis Media Reduced expectation of receiving antibiotics for AOM Indication of relationship between knowledge and attitude changes

4. Future Research Methods of learning i.e. active or passive using logs and observation Long term impact on user knowledge/attitudes Mapping questions to site content Development of a methodology to evaluate the impact of digital libraries on user knowledge, attitudes and behaviour

Contact Gemma Madle g.c.madle@city.ac.uk Antimicrobial Resistance site and study results available at the National electronic Library for Communicable Disease website http://www.nelcd.co.uk