1. Does numeracy affect the comprehension of medicines information in users of a patient information website? Peter Gardner Institute of Psychological Sciences collaborators: Peter Knapp, Theo Raynor, Liz Woolf, Brian McMillan

2. PLAN
Patient Information Leaflets Problems with risk communication Cancer Research UK Studies Effects of Numeracy

3. Patient Information Leaflets (PILs)

4. Which parts of the leaflet do people read?
Side effects %
How and when to take it 91%
What is your medicine for? 85%
Things to do before you take 66%
What is in your medicine? 53%
MORI Survey for Medicines Partnership, 2003, 2004
Ask about Medicines Week
Established in Q Spun out of Leeds University

5. Testing the EU terms EU People Very common > 10% 54%
Uncommon % 11%
Rare % 8%
Very rare < 0.01% 4%
Knapp, Raynor, Berry (2004) Quality & Safety in Health Care.
Berry, Knapp, Raynor (2002) Lancet.

6. Information on side effects
People need information on
their personal risk
potential effects on health
what they should do
Survey of 50 most-prescribed medicines in England:
most leaflets gave no frequency information
inconsistent use of words (‘common’, ‘rare’)
Carrigan, Raynor and Knapp (2008) Drug Safety, 31,

7. Problems with communicating risk
Gigerenzer and Edwards (2003)
Experienced doctors, asked to imagine a test for colorectal cancer:
Prevalence of Cancer is 0.3%
Sensitivity of the test is 50%
False positive rate is 3%
If someone tests positive what is the probability that they actually have colorectal cancer?
3 out of 1000 have cancer
1.5 of these will test positive as the test has 50% sensitivity
30 more will test positive, but don’t have cancer (because of the false positive rate)
So, 1.5 out of 31.5 who test positive will ACTUALLY have cancer = 4.76%
Answers ranged from 1% to 99% with half saying 50% (level of sensitivity) or 47% (sensitivity minus false positives)
The correct answer is 4.76%

8. Natural frequencies – a solution?
Provide a REFERENCE CLASS of events to provide context for the probability value
e.g., in the previous example, out of the same reference class of 1000 people:
3 people will have colorectal cancer
1.5 of these will be detected by the test
30 people will have a positive test but do not have cancer
So, if a person has a positive test there is a 1.5/31.5 chance that they have cancer, or 4.76%.

9. CR-UK studies Using a pop-up on a medicine page of Cancerhelp.org.uk
7 on-line studies since 2004
3 studies published in British Journal of Health Psychology, Drug Safety, Patient Education and Counseling.
Test alternative formats for presenting side effect risk
Controlled design with random allocation
Taxol, Ibuprofen, Tamoxifen
Approximately 15 participants per month
Participants more likely to have a personal interest

10. CR-UK 2
Knapp, Raynor, Woolf, Gardner, Carrigan, McMillan (2009). Drug Safety 32(10),
Following the MHRA recommendation to combine words with frequency bands
Information about tamoxifen
hot flushes (48%)
cataracts (3%)
DVT (0.2%)
pulmonary embolism (0.06%)
3 conditions – verbal, absolute frequencies and combined (verbal plus frequency bands)
187 completed the study

11. Subjective rating scales

12. Estimations of side effect risk
What do you think is the chance that you will have hot flushes from taking tamoxifen? Please state as a percentage in the box below. Also asked for cataracts, deep vein thrombosis and pulmonary embolism What do you think is the chance that you will have any side effect from taking tamoxifen? What do you think is the chance that the average person taking tamoxifen will have any side effects? (actual likelihood is 49.7%)

13. Results

14. Effects of numeracy
Growing evidence that level of numeracy has an effect on understanding risk information:
Schwartz et al (1997) – highly numerate women 13 times more likely to interpret risk estimates correctly
Keller and Siegrist (2009) – low numerate participants unable to distinguish between low and high risk scenarios in three presentation formats
Wright et al (2009) – highly numerate participants better at objective risk comprehension
Reyna et al (2009) – large review

15. Numeracy and the CR-UK studies
Numeracy was measured in the four ‘Tamoxifen’ studies
Data combined (N=591) and accuracy of risk estimate for each of four side effects was correlated with numeracy score and subjective ratings of the information
Gardner PH, McMillan B, Raynor DK, Woolf E, and Knapp P. (in press) Patient Education and Counseling 2011

16. Numeracy measure
Please answer some questions on how you think about certain kinds of numbers. Try to be as accurate as you can but don't worry if you find it hard, just give your best guess. Remember, your responses are anonymous 1) Imagine that we flip a fair coin 1,000 times. What is your best guess about how many times the coin would come up heads? 2) In an imaginary lottery, the chance of winning a prize is 1%. If 1,000 people each buy a single ticket, how many would win a prize? 3) In an imaginary sweepstake, the chance of winning a car is 1 in 1,000. What percent of tickets in the sweepstake wins a car? 4) What does 40 percent mean? a) One quarter b) 4 out of 10 c) every 40th person
Adapted from Lipkus, Samsa and Rimer (2001), and Gigerenzer (2002).

17. Characteristics of the sample
Gender
568 Female; 21 Male; 2 missing
Age
Mean 46.5 (SD 10.8). Range 15-66
Location
428 UK; 93 USA; 70 Other
English as first language
573 Yes; 17 No; 1 Missing
Reason for visiting the webpage
230 (38.9%) Currently taking Tamoxifen
50 (8.5%) Have cancer but not taking Tamoxifen
32 (5.4%) Have previously taken Tamoxifen
110 (18.6%) Were about to take Tamoxifen
97 (16.4%) Have a close relative or friend with cancer
31 (5.2%) Health professionals
41 (6.9%) None of the above, just looking

18. Numeracy and perception of the information
Correlations between numeracy and subjective ratings of the information
Whole sample
(n=591)
Those who have cancer
(n=461)
Those who have not had cancer
(n=130)
Those who have taken/are taking tamoxifen
(n=262)
Those who have not taken tamoxifen
(n=329)
Satisfaction with information about tamoxifen
-.10*
-.09
-.13
-.12*
-.07
How bad overall are side effects from taking tamoxifen
-.04
-.08
.13
-.01
Likelihood of side effects for taking tamoxifen
-.03
-.02
.04
General risk to health from taking tamoxifen
-.06
.03
-.13*
How much would this affect your decision to take tamoxifen
.10*
.08
.24**
.09
.13*
Likely benefit from taking tamoxifen
.07
.10
** Correlation is significant at the 0.01 level (2-tailed).
* Correlation is significant at the 0.05 level (2-tailed).

19. Numeracy and accuracy 1 -.09 -.15** -.08 -.25** -.03 -.24** -.27**
Correlations between numeracy and accuracy of risk estimate
(excluding participants with missing data)
Whole sample
(n=591)
Those who have cancer
(n=461)
Those who have not had cancer
(n=130)
Those who have taken/are taking tamoxifen
(n=262)
Those who have not taken tamoxifen
(n=329)
Hot Flushes †
-.09
-.15**
-.08
-.25**
-.03
Cataracts
-.24**
-.27**
-.17
-.31**
-.20**
Deep Vein Thrombosis
-.41**
-.44**
-.29**
-.39**
Pulmonary Embolism
-.48**
-.50**
-.38**
-.46**
Risk of self getting any side effect
-.02
-.04
.002
.04
Risk of average person getting any side effect
.03
-.003
.07
** Correlation is significant at the 0.01 level (2-tailed).
* Correlation is significant at the 0.05 level (2-tailed).

20. Numeracy and accuracy 2 Numeracy level (score on numeracy test) N
Mean accuracy (sd)
% correct estimates
(across all side effects)
Zero 12
36.7 (22.4) 1 53
23.9 (21.3)
1.6 2
133
23.3 (20.8)
6.9 3
169
14.9 (16.1)
10.4 4
224
10.7 (12.6)
20.2

21. Broad conclusions
Regardless of presentation format, those with higher levels of numeracy were more accurate in their understanding of side effect risk information This appears to be more so for those with personal interest in the information There is some evidence to suggest that numeracy may be related to the decision to take medicine and, in those with cancer, to the perception of the likely benefits

22. Future research
How can this make a difference to practice? Alternative measures of numeracy Different representations of risk information, particularly for small side effect risks Deeper understanding of link between risk information and behaviour Qualitative understanding of medicines risk information

23. References (if not provided in the slides)
Berry, D.C. (2004) Risk, communication and health psychology. Maidenhead:Open University Press. Berry, D.C., Knapp, P. and Raynor, D.K. (2002) Provision of information about drug side effects to patients. Lancet, 359, Gigerenzer G. (2002) Reckoning with Risk. London: Penguin. Gigerenzer, G. and Edwards, A. (2003) Simple tools for understanding risks: from innumeracy to insight. British Medical Journal, 327 (27 September), Knapp, P., Gardner P.H., Carrigan N., Raynor, D.K. and Woolf, E. (2009) Perceived risk of medicine side effects in users of a patient information website: a study of the use of verbal descriptors, percentages and natural frequencies. Brit J Health Psychol,14, Knapp, P., Gardner, P.H., Raynor, D.K., Woolf, E. and McMillan, B.R.W. (2010) Perceived risk of Tamoxifen side effects: a study of the use of absolute frequencies or frequency bands, with or without verbal descriptors. Patient Education and Counseling, 79(2), Knapp, P., Raynor, D.K. and Berry, D.C. (2004) Comparison of two methods of presenting risk information to patients about the side effects of medicines. Qual Saf Health Care,13, Knapp, P., Raynor, D.K., Woolf, E., Gardner, P.H., Carrigan, N. and McMillan, B.R.W. (2009) Communicating the risk of side effects to patients: an evaluation of UK regulatory recommendations. Drug Safety, 32(10), Lipkus, I.M., Samsa, G. and Rimer BK. (2001) General performance on a numeracy scale among highly educated samples. Med Decis Making, 21, Reyna, V.F., Nelson, W.L., Han, P.K. and Dieckmann, N.F. (2009) How numeracy influences risk comprehension and medical decision making. Psychological Bulletin, 135(6),