1. Evaluation of Written Prescription Information Provided in Community Pharmacies: A National Study Bonnie L. Svarstad, Ph.D. Jeanine K. Mount, Ph.D. and R.Ph. University of Wisconsin - Madison

2. Study done in cooperation with... FDA National Association of Boards of Pharmacy National expert panel

3. National Expert Panel 16 experts Nominated by seven pharmacy organizations Pharmacy practitioners and experts in pharmacotherapy and communications Faculty at 9 pharmacy colleges and universities

4. Panelists Mary Amato Heidi Anderson-Harper Robert Beardsley C. A. Bond Marie Gardner Betty Dong Carole Kimberlin Duane Kirking Sharlea Leatherwood Matt Osterhaus Anthony Provenzano Mary Pubentz Betsy Sleath Jenene Spencer Judith Sommers Hanson Gayle Dichter Bonnie Svarstad, Chair

5. Past Studies Distribution of patient information increased from 16% in 1982 to 74% in 1998 1999 study in 8 states found 87% given information but quality was variable [Svarstad & Bultman, 1999]

6. How does this study differ? Pharmacies sampled from national list Professional shoppers visited pharmacies Experts and consumers rated information Additional analyses performed

7. Primary Aims of Study What percent of patients are given written prescription information? How do experts rate this information? How do consumers rate this information? How well does information adhere to criteria for useful medication information?

8. Secondary Aims of Study How do expert/consumer ratings compare? Do ratings vary by consumer background? Do ratings vary by pharmacy type? Do ratings vary by leaflet characteristics?

9. Objectives for Today Review study design and procedures Review evaluation criteria and forms Present results: - Leaflet distribution and ratings - Factors that might influence ratings

10. Study Design Shoppers acting as patients presented four prescriptions at each pharmacy Patient information leaflets mailed to UW Expert panelists rated leaflets Consumer panelists rated leaflets Rating forms mailed to UW for analysis

11. Sampling of Pharmacies Excluded: hospital, clinic, LTC, mail order, IV infusion, dispensing physician, government settings, Alaska, Hawaii, Puerto Rico, states prohibiting filling of prescriptions for research Selected from 57,157 community pharmacies Simple random sample of 384 pharmacies 35% independent, 65% chain Data collected in 44 states

12. Observer Protocol Shoppers hired by professional shopper firm 72% of visits by females 66% of visits by persons 45 years or older Mean age = 50

13. Standard Scenario Presented new prescriptions for atenolol, glyburide, atorvastatin, nitroglycerin SL Did not ask questions or initiate talk If asked, patient-observer reported: - new diagnoses: heart disease, diabetes - no prior use of study drugs

14. Observer Protocol (cont.) Shopper mailed materials to shopper firm Firm removed pharmacy/pharmacist names All leaflets, brochures, other information items mailed to researchers All items referred to as “leaflets”

15. Expert Evaluation Forms Each form included 8 general criteria and 62-63 sub-criteria Eight criteria from 1996 Action Plan for “useful” information Sub-criteria based on approved labeling Forms revised until all panelists approved

16. Eight Criteria from 1996 Action Plan for Useful Information 1. Drug names and indications 2. Contraindications and what to do before using 3. Specific directions about how to use, monitor, get most benefit 4. Specific precautions and how to avoid harm

17. Eight Criteria from 1996 Action Plan for Useful Information 5. Serious and frequent adverse reactions, what to do 6. General information, encouraged to ask questions 7. Scientifically accurate, unbiased, up-to-date 8. Readily comprehensible and legible

18. Scoring Method for Expert Forms Each criterion rated by 4-10 sub-criteria Each sub-criterion rated by adherence level: full (2), partial (1), no adherence (0) Computer calculated % points obtained Scale ranged from 0 to 100% (best)

19. Scoring Categories Level of AdherencePoint Range % 580-100 460-79 340-59 220-39 10-19

20. Inter-rater Reliability Experts assigned to one of four drug groups Independently rated same sub-set of leaflets Good inter-rater reliability obtained Pearson r for overall adherence = 0.90-0.97

21. Expert Rating Process Each expert assigned to one of four drugs Each leaflet rated by one expert Experts rated 1,367 pharmacy-generated leaflets and 31 manufacturer-generated leaflets (mean = 87 per expert)

22. Consumer Evaluation Form 1-page form with 12 items Based on 1996 Action Plan and Krass et al Items scored 1 (poor) to 5 (good) Overall scale ranged from 0 - 100% (best)

23. Scoring Categories for Consumer Form Level of AdherencePoint Range % 580-100 460-79 340-59 220-39 10-19

24. Three Legibility Items Poor-good print size Poor-good print quality Poor-good spacing between lines

25. Six Comprehensibility Items Poorly-well organized Poor-good length Unclear-clear Unhelpful-helpful Incomplete-complete Hard-easy to find important information

26. Three Summary Items Hard-easy to read Hard-easy to understand Not useful-useful

27. Test-Retest Reliability Nine consumers independently rated 18 leaflets at two sessions Good test-retest reliability obtained Pearson r for overall score = 0.82

28. Consumer Rating Process Recruited 154 consumer raters in 11 states Recruited at senior centers, clinics, work and social organizations Facilitator met with 8-15 raters per session Each rater independently rated ~ 10 leaflets

29. Rater Characteristics Mean age = 61 yrs (20-89 yrs) 68% female, 89% white 77% used medication daily 8% not completed high school, 38% completed high school, 54% beyond h.s.

30. % Patient-Observers Given Any Written Information atenolol89.6 glyburide88.8 atorvastatin89.3 nitroglycerin88.3

31. Expert Ratings, All Criteria (n=1,367 leaflets)

32. Expert Ratings by Criterion Highest Ratings: #7: accuracy, unbiased Moderate Ratings: #1: names, indication #3: directions Low Ratings: #5: ADRs, what to do #6: general information Lowest Ratings: #2: contraindications #4: precautions #8: legibility/comprehens.

33. Highest Expert Ratings - Criterion 7 Scientifically Accurate, Unbiased, Up-to-date <1

34. Moderate Expert Ratings - Criteria 1 & 3

35. Low Expert Ratings - Criteria 5 & 6

36. Lowest Expert Ratings - Criteria 2 & 4

37. Lowest Expert Ratings - Criterion 8 Legibility/Comprehensibility

38. Consumer Ratings, All Items (%)

39. Consumer Scores by Item Scores varied by item Lowest scores: print size, print quality, spacing, overall readability Moderate-high scores: easy to understand, useful 36% of leaflets given low ratings on readability

40. Consumer Scores on Readability & Understandability Items

41. Consumer Scores on Usefulness Item

42. Expert Versus Consumer Ratings Low correlation between total scores Expert rating of usefulness was related to consumer usefulness item Expert rating of usefulness was unrelated to consumer readability item

43. Expert Rating of Usefulness by Consumer Rating of Usefulness Item (n=339 atenolol leaflets) Consumer Score Mean expert (%) 1 (poor)40.3 246.3 350.0 455.0 5 (good)55.0 (p<.001)

44. Conclusions -- Part I 89% given some information Ratings varied by criterion Experts most critical of information on contraindications and precautions and legibility/comprehensibility Consumers most critical of readability

45. Results -- Part II What factors might influence leaflet ratings? Consumer characteristics? Pharmacy type? Leaflet characteristics? Changes made by pharmacy organization or pharmacy integrator?

46. Leaflet Ratings by Consumer Characteristics Ratings unrelated to gender, age, education, current drug use White raters gave higher scores than non- white raters for nitroglycerin leaflets Race unrelated to other ratings

47. Leaflet Distribution and Ratings by Pharmacy Type P<.001 all tests

48. Expert Ratings by Leaflet Length P<.001 all tests

49. Consumer Ratings by Leaflet Length P<.001 all tests

50. Consumer Ratings by Font Size P<.001 all tests

51. Consumer Ratings of Readability by Font Size P<.001 all tests

52. Data Vendors Examined Vendor - not ascertainable13.5% (46) Vendor 1 - partial message 5.0% (17) Vendor 1 - full message 81.5% (277) Vendor 2* ( comparison leaflets,institutional )

53. Leaflet Length by Vendor Vendor% leaflets <5.6 inches Vendor - not ascertainable82.6% Vendor 1- partial message94.1% Vendor 1- full message27.1% Vendor 2* 0.0% P<.001

54. Results By Data Vendor Substantial variability by data vendor Highest: Vendor 1 and Vendor 2 Lowest: Partial messages and unidentified vendors

55. Expert Ratings by Vendor P<.001 all tests

56. Ratings of Atenolol Leaflets by Vendor. P<.001 all tests

57. Ratings of Atenolol Leaflets by Vendor P.05)

58. Consumer Ratings by Vendor P<.001 all tests

59. Consumer Ratings of Atenolol Leaflets by Vendor P<.001 all tests

60. Data Vendor by Pharmacy Type P<.001

61. Analysis of Vendor 1 Leaflets Analyzed full leaflets used by 16 organizations 5 organizations used Version 1 (33 leaflets) 5 organizations used Version 2 (50 leaflets) 6 organizations used Version 3 (72 leaflets) Compared leaflets with prototypes

62. Results Ratings varied by leaflet version No prototype fully met criteria Some organizations added and deleted information from prototype

63. Expert Ratings by Leaflet Version P<.001 all tests

64. Expert Ratings by Leaflet Version P<.001 for all criteria

65. Additions/Deletions from Prototypes V1: One organization deleted publisher and disclaimer information; two used leaflet+label V2: One deleted additional information and added label V3: Five deleted warning box and overdose sections; four deleted drug names and notes; five added disclaimer; 1 added publisher section

66. Ratings of Distributed Versus Prototype Leaflets -- Atenolol Version 1

67. Ratings of Distributed Versus Prototype Leaflets -- Atenolol Version 2

68. Ratings of Distributed Versus Prototype Leaflets -- Atenolol Version 3

69. Conclusions (1) Highest ratings for scientific accuracy and being nonpromotional Lowest ratings for contraindications and precautions

70. Conclusions (2) Lowest ratings for leaflets: - < 5.5 inches long or - font size < 10 point Lowest ratings for leaflets from: - independent pharmacies - unidentified vendors

71. Conclusions (3) No prototype fully met criteria –Experts, consumers both critical of legibility Pharmacy organizations can influence ratings by –selecting vendor and leaflet version –modifying leaflets themselves

72. Examples Leaflets with low versus high ratings Leaflets from different vendors Changes made by pharmacy organizations or integrators