1. Scorecards that save lives: But which score is the winner?
A critical appraisal of Early Warning System scores Joshua Adams, BS, RN, CNML
The Pennsylvania State University
In adult inpatients in an acute care hospital setting, which early warning score system (EWSS) is
most effective in predicting adverse outcomes (e.g. cardiac/respiratory arrest or unplanned transfer to the intensive care unit)?
Background/Problem
Hospitalized patients who suffer adverse events may show subtle signs hours before they crash
Subtle signs can be viewed in aggregate in an EWS, for which a variety of systems for calculating exist
The IHI supports using an EWS system but does not recommend a particular one
This review evaluates evidence for which EWSS is most effective in predicting adverse events
Summary of Literature
Synthesis/Summary of Findings
Six JH Level III studies, one JH Level I systematic review, and one JH Level II quasi-experimental study
Variability among EWSS in predicting adverse events
It could not be definitively determined which EWSS is most effective due to measurement of different endpoints, though the RI showed most promise with AUC of 0.93
All systems incorporated use of EMR vs. manual scoring
Source
Design/Sample
Variables/
Intervention
Measurement
Findings
Quality and Strength
Albert et al., 2011
RCR; N=150 codes & RRTs; 550 bed tertiary AMC with Magnet
IV = MEWS
DV=RRT or codes
MEWS calculated from EMR at time of event, 4 & 8 hours prior
-Pre-event score >= 3, P=.002 for respiratory events
-Pre-event score >=3, P=.065 for cardiac
-S: simple tool in EMR
-L: small N, 1 hospital
-JH Level: III
-JH Qual: B
Churpek et al., 2013
RCR; N=59,643 admissions, 109 cardiac arrests, 291 deaths, 2,655 upgrades
IV=scores of various EWSS
DV=cardiac arrest, ICU transfer, mortality, composite
Vital signs from EMR
-CART had highest AUC for cardiac arrest (0.83), ICU upgrade (0.77) and composite (0.78)
-CART, ViEWS, SEWS had same AUC (0.88) for mortality
-S: compared 8 tools
-L: unclear whether EWSS improves outcomes
Finlay et al., 2014
RCR; N=32,472 pt visits to a 665 bed AMC in PA
IV=RI or MEWS
DV=death
RI (including nursing assessment parameters), and MEWS calculated from EMR
-RI had AUC of 0.93, MEWS 0.82
-S: RI incorporating nursing assessment
-L: retrospective design
Hollis et al., 2016
RCR; N=552 gen surg procedures involving GI and surg onc at 1 facility
IV=EWS
DV=likelihood of postop complication
EWS derived from algorithm against the rate of occurrence of serious complications
-EWS had AUC of 0.9 for grade IV/V complications
-S: understudied pop. of surgical pt
-L: retrospective, limited high severity comp
Kang et al., 2016
Prospective black box validation study; N=3,889 pts in a single center
IV=eCART score
DV=cardiac arrests and ICU transfers
eCART score at defined risk threshold against ICU transfer and cardiac arrest
-eCART had AUC of 0.8 for preducting ICU transfer, 0.88 for predicting cardiac arrest
-S: use of EMR
-L: unable to make generalizations d/t ICU transfer criteria being local
-JH Qual: C
Mapp et al., 2013
Systematic review; 9 studies from individual facilities
IV=MEWS, CART, SI, EWS adapted for facility
DV=RRT, codes, unplanned upgrades, death
Vitals, LOC, age, BMI, UOP, age
-Studies using AUC analysis ranged from 0.67 to 0.84, higher than competing systems
-S: comparison study
-L: single facilities with retrospective analysis
-JH Level: II
Nishijima et al., 2016
Quasi-experimental; single site acute care hospital in Japan
IV=MEWS
DV=IHCA
MEWS was calculated from EMR and incidence of IHCAs per 1000 admissions analyzed
-IHCA per 1000 admissions 5.21 before introducing, 2.39 after
-S: alert threshold for team to be activated
-L: single site study, unable to exclude other factors that could decrease IHCA
Smith et al., 2014
Systematic review; one controlled trial and 20 observation studies
IV=CART, MEWS, EWS, ViEWS, proprietary clinical marker
DV=multiple
HR, RR, SBP, Temp, UOP, sat, dyspnea, increased O2 need, LOC, subj. concern
-8 studies finding strong predictive value for death (AUC )
Methods
Keywords: scorecards, early warning score, EWS, modified early warning score, MEWS, Rothman Index, track and trigger
Inclusion: Adults, scholarly, peer-reviewed, published after Exclusion: Duplicates, studies of pediatric, obstetric, or behavioral health patients
Critical appraisal of 8 articles for quality & strength using Johns Hopkins (JH) Critical Appraisal Tool
Limitations
Low generalizability
Measurement of different endpoints
Implications for Practice
Based on current evidence, hospitals wishing to create or implement an existing EWSS should pilot concepts before implementing housewide
Once an EWSS is validated in a particular setting, it can then be used to trigger proactive interventions by the healthcare team preventing further decline
Total articles identified (n=26)
CINAHL (n=4)
PubMed MEDLINE (n=21)
Cochrane (n=1)
Duplicate articles (n=2)
Distinct articles after exclusion of 2 duplicates (n=24)
Eligible articles (n=24)
Articles excluded for content (16)
Articles identified after content (n=8)
More research examining how EWSS can be used to direct resources to a potentially deteriorating patient before an adverse event
…..Next Steps
Abbreviations: EMR – Electronic Medical Record, MEWS – Modified Early Warning Score, RI – Rothman Index (measure of X), EWS – Early Warning System, ICU – Intensive Care Unit, RRT – Rapid Response Team, RCR – retrospective chart review, RCC – retrospective case control, AMC – academic medical center, accred. – accreditation, AUC – area under curve, LOC – level of consciousness, UOP – urine output, BMI – body mass index, LOS – length of stay, ROC – receiver operating characteristic, eCART – electronic Cardiac Arrest Risk triage, IHCA – in-hospital cardiac arrest, tx
Johns Hopkins Level of Evidence (JH Level) where I = RCT or meta-analysis of RCTs, II = Quasi-experimental, III = non-experimental, qualitative, meta-synthesis; Quality Ratings (JH Qual) where A= High, B= Good, C= low/major flaws