1. Evaluating the Changes in Knowledge and Attitudes of Digital Library Users
Gemma Madle, Patty Kostkova, Jane Mani-Saada, Julius R Weinberg
City eHealth Research Centre
Institute of Health Sciences, City University, London

2. Talk outline Introduction to the theme
Health information on the Internet
Usability testing of Digital Libraries
Antimicrobial Resistance Website
Evaluating User Knowledge & Attitude changes
Summary of Results & Future Research

3. 1a. Health information on the Internet
Explosion of health information on the internet, e.g. NHS Direct, Netdoctor, Surgery Door, NeLH, Medline Plus
The informed patient, e.g. Expert Patient
Change in doctor-patient relationship, shift in the balance of power, shared decision-making
“One has a greater sense of intellectual degradation after an interview with a doctor than from any human experience.”
Alice James

4. 1a. Health information on the Internet
Medical digital libraries
Life critical applications
Empower health professionals & patients
National electronic Library for Health
Communicable Diseases specialist library
Freely accessible to all via the Internet

5. 1b.Usability testing of DLs
Graphical interface, navigation, download time etc e.g. Nielsen
Accessibility e.g. Bobby
User information seeking behaviour
User satisfaction

6. 1b. Usability testing of DLs
So we know that…..
Does it have any impact on the quality of clinical care?

7. 2. Antimicrobial Resistance Website
Acts as an interface to a subsection of the NeLCD
Aim of the site “to inform the public of current evidence-based guidelines on antimicrobial prescribing and the issues surrounding those guidelines in an effort to reduce patient pressure on doctors and subsequently reduce inappropriate prescribing”
Useful testbed to evaluate the impact of the NeLCD on user knowledge & attitudes and subsequent behaviour

8. 2. Antimicrobial Resistance Website
Identification of content
Ask the Doctor archives
Health information websites
Current news topics
Organisation of content
MeSH
Reasons for visiting site

9. 2. Antimicrobial Resistance Website
Lotus Domino Database
Left navigation menu
Search facility
Dynamically generated content:
Frequently asked questions
Related news articles
Related resources

11. 2. Antimicrobial Resistance Website
Antimicrobial Resistance is a global problem
Patient expectations influence prescribing by doctors
Importance of educating the public highlighted by influential UK reports
So can we change public (and professionals ) behaviour?

12. 3. Evaluation Methods Pre and Post Use Questionnaires
Used to evaluate differences in patient knowledge and attitudes before and after using the site
Users were free to browse the site between questionnaires
227 Science Museum visitors took part as part of ‘live science’ of which 177 completed both questionnaires

13. 3. Evaluation Methods - Knowledge
Knowledge about something or someone can be gained through information or facts (Oxford English Dictionary)
7 true/false statements about antibiotic use

14. 3. Evaluation Methods - Attitude
“a psychological tendency that is expressed by evaluating a particular entity with some degree of favour or disfavour”
Six Likert scale questions
1 = strongly disagree, 5 = strongly agree
Four evaluated attitude to information on the site
Two evaluated attitude to prescribing

15. 3. Evaluation Methods Relationship between knowledge and attitude
Changes in knowledge may or may not be reflected by changes in attitude
It is important to change people’s attitudes and behaviour as well as improve their knowledge

16. 3. Evaluation Results - Knowledge

17. 3. Evaluation Results - Attitude

18. 3. Evaluation Results
24% users increased knowledge score and decreased attitude score
7% users decreased knowledge score and decreased attitude score
5% users decreased knowledge score and increased attitude score
11% users increased knowledge score and increased attitude score

19. 4. Summary of Results
Significant improvements in user knowledge about antibiotics
Significant changes in attitude towards antibiotic use in Acute Otitis Media
Reduced expectation of receiving antibiotics for AOM
Indication of relationship between knowledge and attitude changes

20. 4. Future Research
Methods of learning i.e. active or passive using logs and observation
Long term impact on user knowledge/attitudes
Mapping questions to site content
Development of a methodology to evaluate the impact of digital libraries on user knowledge, attitudes and behaviour

21. Contact Gemma Madle g.c.madle@city.ac.uk
Antimicrobial Resistance site and study results available at the National electronic Library for Communicable Disease website