1. Quantitative validation of central line-associated bloodstream infections (CLABSI) in Oregon intensive care units (ICU) 2009
Zintars Beldavs
Manager HAI Program, Acute and Communicable Disease Section, Oregon Public Health Division, Oregon Health Authority
June, 2012

2. Central Line-Associated Bloodstream Infection (CLABSI)
Deadly: 18% mortality
14,000 deaths/ year in ICU patients
Prolong hospitalization by mean of 7 days
Expensive: $3,700 - $29,000/episode
Preventable: hand hygiene, barrier precautions, skin antisepsis, catheter site selection
Pittet D, Tarara D, Wenzel RP. Nosocomial bloodstream infection in critically ill patients. Excess length of stay, extra costs, and attributable mortality. JAMA. 1994;271:
Soufir L et al. Infect Control Hosp Epidemiol 1999 Jun;20(6):

3. Mandated Reporting in Oregon
Reportable as of 1/1/2009
CLABSIs in ICU
SSI knee prostheses and coronary artery bypass grafts
More reportable surgical site infections as of 1/1/11
Colon surgery
Hip prosthesis
Laminectomy
Abdominal hysterectomy
Rationale behind CLABSI: high morbidity and mortality
Behnd CABG: cardiac prodedures also high-risk
Behind KPRO: consequences of an infection are severe...(i.e. Getting your knee opened up again!). Also, this allows for inclusion of a reasonable number of facilities; orthopedic procedures done at more facilities than cardiacs
SCIP-1: antibiotic received 1-hour prior to surgical incision
SCIP-2: prophylactic antibiotic selection
SCIP-3: prophylactic antibx discontinued < 24 hours after surgery end

4. 2009 Oregon CLABSI Pathogens
Charts indicate primary organism associated with for each (~4 true pos cases had secondary and tertiary orgs).
This is statewide aggregate data.
“other” = 1 Moraxella (gram neg diplococcus),1 Stenotrophomonas, 1 Veillonella.
All but 2 “other yeast” are non- albicans Candida spp.

5. CLABSI Rate

6. CLABSI Rate Change

7. CLABSI Count by Hospital Size

8. Validation for Accurate Data
Concern: surveillance definitions applied inconsistently by IPs
Poor inter-rater reliability: kappas .30 to .58
Previous validation studies: potentially > half of cases not reported (Connecticut)
Oregon hospitals:Lin MY, et al., SHEA % of 661 positive blood cultures considered CLABSIs:
Infection Preventionists: 21%
Standardized review: 35%
Electronic algorithm: 57
Kappa values 0.43, 0.30, 0.58
Mayer J, et al., SHEA 2010: 120 VAMC records, 18 IPs compared to electronic algorithm
Overall kappa 0.42, range 0.30–0.50
Individual IPs varied up to 2-fold in CLABSI assessments
Backman et al: 23 of 48 (48%) CLABSI reported

9. Objectives Evaluate quality of reported data
Assess under- and over-reporting
Gauge the reliability and consistency of surveillance case definitions
Provide feedback to facilities on surveillance case definitions and reporting methods

10. Methods Study period: 2009 Included: 44 acute care hospitals
28 with < 50 beds
10 with > 200 beds
Median central line days 210, range
OPHD validation team:
HAI Program Manager
Epidemiologist
EIS Officer/Physician
3 public health nurses
58 total hospitals. 14 excluded because they had no ICU or no more than 10 patients with CL annually.
Map: Oregon Association of Hospitals and Healthcare Systems, oahhs.org

11. Methods All blood culture (+) Drawn in ICU or up to 48 hours after and
Organism not isolated in 14 days before admit

12. Methods Unit of analysis: Single bacteremia episode
More than one CLABSI/patient possible

13. All (+) culture reviewed
Methods
37 Hospitals
7 Hospitals
All reported CLABSI and random sample 60 (+) culture not reported as CLABSI
All (+) culture reviewed
March April 2011: on-site hospital visit for chart review
Retrospective record review by 2-4 reviewers
At 37 hospitals: all ICU pts. blood culture (+)
At 7 largest hospitals: all reported CLABSI plus random sample of 60 patients with ICU blood-culture(+) not reported as CLABSI
Validators blinded as to whether cases reported as CLABSI
Reviewers blind as to reported CLABSI status

14. Methods
Adjudication Discussion
NHSN
Hospital Determination
Validator
Determination
After visit, all cases with discordant CLABSI determinations (suspected false positives or false negatives) adjudicated by phone with hospital staff
Participants
Hospital IP staff
Hospital physician
OPHD validators
OPHD physician
Review of all findings for final CLABSI determination
If no consensus reached, case referred to CDC staff

15. Methods 1926 (+) cultures received on line lists
1204 from 7 highest volume facilities
722 from 41 facilities < than 60 (+) cultures

16. Methods 1199 medical records reviewed
477 sampled at 7 highest-volume facilities
722 at small- and medium- volume facilities

17. 817 included in final analysis
Results
817 included in final analysis
Hospitals included cultures not requested:
(+) blood cultures obtained prior admit or
> 48 hours after discharge from ICU
382 of 1199 reviewed censored

18. Results Discordant Cases
35 records with disagreement
18 (51%) adjudicated as CLABSI
17 (49%) adjudicated as not CLABSI
Hospital correct in 13 (37%)
OPHD correct in 22 (63%)
4 required NHSN consult (2 CLABSI)

19. Reasons for discrepancies
Reason for discrepancy for under-reported CLABSI
No. episodes %
No clearly discernible reason determined 7 44
Misattributed CLABSI to other infection
Recognized CLABSI failed to attribute ICU 1 6
Misclassified CLABSI as present on admit
Total
100
Reason for discrepancy for over-reported CLABSI
No. episodes %
Infection attributable to other site 2 33
Infection not attributable to ICU
Single blood culture for probable contaminant 1
16.5
Unknown why reported CLABSI
Total 6
100
For CLABSI “just missed”: at some facilities, IP staff had changed since 2009 and current staff unaware of rationale for previous reporting decisions.

20. Results Validation outcome, unadjusted
CLABSIs among All ICU Patients with Positive Blood Cultures, by Initial Hospital Report — Oregon, 2009
CLABSI
Final determination
Present
Absent
Total
Hospital report
70 (TP)
6 (FP) 76
16 (FN)
725(TN)
741 86
731
817

21. Example Calculation to Adjust for Sampling Fraction
60 Records sampled at hospital
142 Total BSI
2 BSI reported as CLABSI
Sampling fraction: 60/140=.43
True positive and false positive remain (all reported CLABSI reviewed)
False negative and true negative results divided by sampling fraction
Final Determination
CLABSI Present
CLABSI Absent
Hospital 1
True Positive
False Positive
“Overreport” 2
(Estimate: 2/.43=4.7)
False Negative
“Underreport” 42
(Estimate: 42/.43=98)
True Negative

22. Results Estimated # of CLABSI adjusted for sampling fraction
Estimated CLABSIs among All ICU Patients with Positive Blood Cultures, by Initial Hospital Report — Oregon, 2009
CLABSI
Final determination
Present
Absent
Total
Hospital report
70 (TP)
6 (FP) 76
27a (FN)
1089a (TN)
1116 97
1095
1192
72% of true CLABSIs had been reported (Sensitivity = 0.72)
99% of true non-CLABSI were correctly not reported (Specificity = 0.99)
92% of CLABSIs reported were true CLABSI (Positive predictive value = 0.92)
98% of cases not reported CLABSI were not CLABSI (Negative predictive value = 0.98)
8% of positive cultures were CLABSIs (Prevalence = 0.08 )

23. Validation Impact on CLABSI Rate
Validation increased the statewide ICU CLABSI rate from 1.21 (95% CI: 0.95–1.51) to 1.54 (95% CI: 1.25–1.88) CLABSI per 1,000 central-line days
Change after validation in CLABSI rate
No. hospitals %
Rate decreased 0.70 1 2
No change
33a 75
0.01–0.50 higher 5
0.51–1.00 higher
>1.00 higher 6b 14
Total 44
100
1 hospital reported 2 CLABSIs and these were later deemed non CLABSI: rate change .7 to 0.
> 1 rate change: 4 to 146 bed. 4 of 6 with >1 smaller census hospitals reported 1 collectively but validation found Sampled hospitals reported 28 but were found to have estimated 40.
a 23/33 had no CLABSI identified either before or after the validation.
b 3/6 had no CLABSI before the validation.

24. Importance of Inter-Agency Follow-up Discussion
Of 27 unreported cases identified as CLABSI by OPHD, 16 (59%) true CLABSI
Sensitivity of reporting:
72% based on follow-up adjudication
vs. 60% based on OPHD review alone (P= 0.07), closer to some previous validation efforts

25. Limitations Denominator data not rigorously assessed
Did not adjudicate cases when reviews concordant with reported data
Unbiased 3rd party not involved in adjudication discussions

26. Conclusions
Validating hospital CLABSI reporting improves accuracy of hospital-based CLABSI surveillance
Discussing discordant findings improves the quality of validation

27. Future Work
Currently completing coronary artery bypass graft validation of all 14 hospitals
Similar adjudication procedure
Sampling higher duration procedures
Given funding: comprehensive baseline and yearly sampled validation other HAIs

28. Acknowledgments Questions? OPHD HAI program staff and others assisting
Paul Cieslak – Public Health Physician
Ann Thomas – Public Health Physician
Margaret Cunningham – HAI Epidemiologist
Diane Roy – HAI Administrative Assistant
John Oh – EIS Officer
Steve Moore – Public Health Nurse
Jennifer Tujo – Infection Preventionist
Valerie Ocampo – HAI Public Health Nurse
Oregon Patient Safety Commission
Office for Oregon Health Policy and Research
Association of Professionals in Infection Control, Oregon-SW
Washington Chapter
Questions?

29. Questions?