1. How Can We Improve the Accuracy of Routine Pain Screening? L.R. Shugarman RAND Corporation VA HSR&D Center for the Study of Healthcare Provider Behavior, VA of Greater Los Angeles

2. Colleagues K.A. Lorenz, Study PI K.A. Lorenz, Study PI C.D. Sherbourne C.D. Sherbourne L.V. Rubenstein L.V. Rubenstein L. Wen L. Wen A. Cohen A. Cohen J. Goebel J. Goebel A. Lanto A. Lanto S. Asch, Study Co-PI S. Asch, Study Co-PI

3. Background Despite available, effective treatment, clinicians and patients often have trouble achieving adequate pain control Despite available, effective treatment, clinicians and patients often have trouble achieving adequate pain control Although there is widespread use of the `5th vital sign´ in the VA, its accuracy is unclear. Although there is widespread use of the `5th vital sign´ in the VA, its accuracy is unclear.

4. Objectives Compare routine pain rating during vital sign intake to the same pain scale applied under ideal research conditions and to a gold standard measure, the Brief Pain Inventory (BPI) Compare routine pain rating during vital sign intake to the same pain scale applied under ideal research conditions and to a gold standard measure, the Brief Pain Inventory (BPI) Distinguish variation associated with instrumentation from that associated with routine measurement processes Distinguish variation associated with instrumentation from that associated with routine measurement processes

5. Methods Randomly sampled (March 2006-April 2007) veterans in primary care, urgent care, women’s health, oncology and cardiology clinics Randomly sampled (March 2006-April 2007) veterans in primary care, urgent care, women’s health, oncology and cardiology clinics 19 clinics from 2 hospitals, 6 affiliated sites in 3 large urban counties (Los Angeles, Ventura, and Orange) 19 clinics from 2 hospitals, 6 affiliated sites in 3 large urban counties (Los Angeles, Ventura, and Orange) Surveyed patients, their nurses, & treating providers Surveyed patients, their nurses, & treating providers Patients surveyed immediately following clinic visit Patients surveyed immediately following clinic visit Additional measures derived from chart review Additional measures derived from chart review

6. Measures Numeric Rating Scale (NRS) – 0-10 pain rating Numeric Rating Scale (NRS) – 0-10 pain rating Most commonly used method for assessing pain Most commonly used method for assessing pain NRS of 5+ = moderate/severe pain NRS of 5+ = moderate/severe pain Brief Pain Inventory (BPI) – developed by the WHO Brief Pain Inventory (BPI) – developed by the WHO Measures pain intensity and pain interference with various activities Measures pain intensity and pain interference with various activities Demonstrated reliability across patient populations and settings Demonstrated reliability across patient populations and settings Study measures: Study measures: NRS gathered during vital sign intake (Nurse-NRS) – from chart review NRS gathered during vital sign intake (Nurse-NRS) – from chart review NRS gathered under research conditions (Research-NRS) – from patient survey NRS gathered under research conditions (Research-NRS) – from patient survey BPI-24 hours, BPI-one week, BPI-interference – from patient survey BPI-24 hours, BPI-one week, BPI-interference – from patient survey

7. Analysis Intraclass correlation used to assess agreement between Research-NRS and Nurse-NRS and Nurse-NRS and BPI Intraclass correlation used to assess agreement between Research-NRS and Nurse-NRS and Nurse-NRS and BPI Determined sensitivity/specificity of cutpoints on the Nurse-NRS to BPI (gold standard) by fitting ROC curves and calculated the area under the curve (AUC) Determined sensitivity/specificity of cutpoints on the Nurse-NRS to BPI (gold standard) by fitting ROC curves and calculated the area under the curve (AUC) Variation in agreement further evaluated: Variation in agreement further evaluated: Did RN ask patient to rate pain on 0-10 scale? Did RN ask patient to rate pain on 0-10 scale? Has patient’s pain changed since arrival at clinic? Has patient’s pain changed since arrival at clinic?

8. Results – Sample Characteristics (N=627) Mean Age (SD) 62.5 (12.9) % Male 94.7 % White 49.1 Mean SRHS (1=Poor, 5=Excellent) (SD) 3.4 (1.1) % Mental Health Problem 44 % Cancer 13 % Cardiovascular Disease 38 % Musculoskeletal Condition 45 Mean Research-NRS (SD) 3.1 (3.2) Mean Nurse-NRS (SD) 2.1 (3.2)

9. Direction of Difference in Ratings Between Nurse-NRS and Research-NRS Ratings Difference in Ratings N(%) Nurse > Research 3+ points Nurse > Research 3+ points 24 (3.8) Nurse > Research 2 points Nurse > Research 2 points 19 (3.0) Nurse > Research 1 point Nurse > Research 1 point 24 (3.8) Nurse = Research Nurse = Research 347 (55.3) Nurse < Research 1 point Nurse < Research 1 point 51 (8.1) Nurse < Research 2 points Nurse < Research 2 points 39 (6.2) Nurse < Research 3+ points Nurse < Research 3+ points 123 (19.6)

10. Intraclass Correlations Among Pain Rating Measures Intraclass Correlations Nurse- NRS Research- NRS BPI- Sev/24 hr BPI- Sev/last week BPI-Int Nurse-NRS- Research- NRS.627- BPI-Sev/24 hr.613.866- BPI-Sev/last week.588.845.947- BPI-Int.409.597.693.753-

11. Results AUC for Nurse-NRS compared to Research- NRS was 0.78 for a cutoff of 5 and 0.77 for a cutoff of 7 AUC for Nurse-NRS compared to Research- NRS was 0.78 for a cutoff of 5 and 0.77 for a cutoff of 7 Similar AUC results found for Nurse-NRS compared to BPI-24 hour and BPI-last week Similar AUC results found for Nurse-NRS compared to BPI-24 hour and BPI-last week Nurses more likely to underestimate pain if they did not use 0-10 scale Nurses more likely to underestimate pain if they did not use 0-10 scale Agreement attenuated when patient pain changed Agreement attenuated when patient pain changed

12. Limitations VA system institutionalized routine pain screening in the last decade; as such, findings may conservatively estimate the challenges of consistent implementation in more diverse, non- VA settings VA system institutionalized routine pain screening in the last decade; as such, findings may conservatively estimate the challenges of consistent implementation in more diverse, non- VA settings Study limited to outpatient evaluation of the 5th vital sign Study limited to outpatient evaluation of the 5th vital sign

13. Conclusions Accuracy of the 5th vital sign is moderate Accuracy of the 5th vital sign is moderate Nurses may not always use the 0-10 NRS to properly quantify pain levels Nurses may not always use the 0-10 NRS to properly quantify pain levels Nursing staff training in pain measurement may be warranted Nursing staff training in pain measurement may be warranted