1. WG1 Presenters Connie White Delaney, PhD, RN, FAAN, FACMI, Professor & Dean Delaney@umn.edu Tom Clancy, PhD, MBA, RN, FAAN, Clinical Professor Associate Dean for Faculty Practice, Partnership, Professional Development Clanc027@umn.edu Bonnie Westra, PhD, RN, FAAN, FACMI, Associate Professor Director, Center for Nursing Informatics Westr006@umn.edu Karen Monsen, PhD, RN, FAAN, Associate Professor Specialty Coordinator Doctorate of Nursing Practice in Nursing Informatics Mons0122@umn.edu Chih Lin Chi, PhD, MBA Assistant Professor cchi@umn.edu

2. Big Data Research Extended Clinical Data Bonnie L. Westra, PhD, RN, FAAN, FACMI Beverly Christie, DNP, RN; Connie W. Delaney, PhD, RN, FAAN, FACMI; Grace Gao, DNP, RN; Steven G. Johnson, MS; Anne LaFlamme, DNP, RN; Jung In Park, PhD-C, RN; Lisiane Pruinelli, PhD-C, RN; Suzan Sherman, PhD, RN; Piper Svensson-Ranallo PhD; Stuart Speedie, PhD

3. Acknowledgment This was supported by Grant Number 1UL1RR033183 from the National Center for Research Resources (NCRR) of the National Institutes of Health (NIH) to the University of Minnesota Clinical and Translational Science Institute (CTSI). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the CTSI or the NIH. The University of Minnesota CTSI is part of a national Clinical and Translational Science Award (CTSA) consortium created to accelerate laboratory discoveries into treatments for patients.“

4. Purpose The purpose of this study was to Demonstrate feasibility of creating a hierarchical flowsheet ontology in i2b2 using data-derived information models Determine the underlying informatics and technical issues Demonstrate the applicability of flowsheet data to nursing / interprofessional research

5. Requirements for Useful Data Common data models Standardized coding of data Standardized queries

6. http://www.pcornet.org/resource-center/pcornet-common-data-model/

7. 7 Vision – Inclusion of Nursing and Other Interprofessional Data Clinical Data NMDS Management Data NMMDS Other Data Sets Continuum of Care

8. Example Flowsheet

9. Flowsheet Data Challenges Volume of data There are multiple measures for the same concepts Different people building screens Software upgrades Discipline/ practice specific needs No information models exist Data driven information modeling required

10. Information Model Development Process Identify Clinical Data Model Topic Identify Concepts Map Flowsheets to Concepts PresentValidate

11. UMN – Academic Health Center CDR Flowsheets constitute 34% of all data 14,564 measure types 2,972 groups 562 templates 1.2 billion observations 2,000 measures cover 95% of observations

12. Sample Data Source - Clinical Data Models T 562 Groups 2,696 Flowsheet Measures 14,550 Data Points 153,049,704 Flowsheet Data from 10/20/2010 - 12/27/2013 66,660 patients 199,665 encounters

13. Development Process Details Identify clinical topic important to researchers/ operations Develop a list of concepts from research questions, clinical guidelines and literature Search for concepts in templates/groups/measures Search associated groups for additional concepts Add matched concepts to running list Categorize into assessment and interventions Organize into hierarchy Combine similar concepts that have similar value sets Validated by a second researcher

14. PainNeuromusculoskeletal System Falls/ Safety Respiratory system Peripheral Neurovascular (VTE) Vital Signs, Height & Weight Genitourinary System/ CAUTI Aggression and Interpersonal Violence Pressure UlcersPsychiatric Mental Status Exam Cardiovascular System Substance Abuse Gastrointestinal SystemSuicide and Self Harm Flowsheet Information Models

15. Example Information Model

16. What is i2b2? Informatics for Integrating Biology and the Bedside (i2b2) Framework for research cohort discovery

17. 14 information models - approximately 81 million new rows i2b2 OBSERVATION_FACT table I2b2 – every row has to be unique Create Flowsheet Ontology in i2b2

18. Informatics Issues Encountered Redundancy – flowsheet and value sets 7 blood pressure and 10 heart rate measures Mapped multiple flowsheet measures to same concept Variations in value sets Created a unique list of all for same concept Measures with similar names represented different concept – i.e. search “display name” – Urine Output R IP URINE FOLEY URINE OUTPUT URINE OUTPUT.MODIFIED ALDRETE R NEPROSTOMY URINE OUTPUT URINE OUTPUT (ML) 0-unable to void and uncomfortable 1-unable to void but comfortable 2-has voided, adequate urine output per device, or not applicable

19. Technical Issues Encountered Free text response Included name of measure, no data included in i2b2 Multi-response items Created a separate row OBSERVATION_FACT table Choice list - comment or “other” option Created a row for each type of comment Numeric response measures - units of measure not clearly identifiable Modified name to include unit of measure Mapping issues Changed names to exclude “* | / \ “ ? %” Constructed synthetic value item id’s Names must be unique within first 32 characters Changed from fully specified names to multiple levels

20. Falls/ Fall Risk Concepts You plan to use a clinical data repository to predict who is likely to fall and what interventions prevent falls. What are the concepts related to Risk assessment? What interventions prevent falls? Do all disciplines in all units/ settings use the same risk assessment instrument? How is the outcome of falls captured in the EHR?

21. Discussion/ Conclusion Flowsheet data represent the largest portion of CDR, rich source of nursing and interprofessional clinical data Created 14 information models, 81M observations Transformed models for flowsheet measures into i2b2 Identified a number of informatics and technical issues and developed processes for managing these issues Continue to clean up information models External validation initiated Flowsheet data can extend knowledge of interprofessional evidence-based practice to improve health outcomes

22. Next Steps External validation of information models with additional organizations http://www.fhims.org/press_ulcer.html Adding conceptual definitions Mapping to standardized terminology – LOINC/ SNOMED CT Demonstrate comparative effectiveness research across organizations Collaborate with other common data model efforts to expand CDMs to include assessments and additional interventions in IM’s derived from flowsheet data

23. A Data Mining Approach to Determine Sepsis Guideline Impact on Inpatient Mortality and Complications Michael Steinbach, PhD; Bonnie L. Westra, PhD, RN, FAAN, FACMI; György J. Simon, PhD Lisiane Pruinelli, MSN, RN, PhD-C; Pranjul Yadav, PhD-C; Andrew Hangsleben; Jakob Johnson; Sanjoy Dey, PhD; Maribet McCarty, PhD, RN; Vipin Kumar, PhD; Connie W. Delaney, PhD, RN, FAAN, FACMI

24. Acknowledgments Support for this study is provided by NSF grant IIS-1344135 National Center for Research Resources of the NIH 1UL1RR033183. Contents of this document are the sole responsibility of the authors and do not necessarily represent official views of the NSF, CTSI, or NIH

25. Introduction Sepsis or septicemia has doubled from 2000 to 2008 Hospitalizations increased 70% Severe sepsis and septic shock have higher mortality – 18%- 40% Patients are sicker, have longer length of stay, more expensive EBP guidelines (SSC) could lead to earlier diagnosis and treatment Guidelines are not fully implemented in clinical practice The effectiveness of these guidelines are unclear

26. Aim The overall aim is to evaluate and extend evidence-based guidelines for patients with health disparities for the prevention and management of sepsis complications 1.Map EHRs data to SSC guideline recommendations 2.Estimate the compliance with the SSC guideline recommendations; and 3.Estimate the effect of the SSC individual recommendations on the prevention of in-hospital mortality and sepsis-related complications

27. Data Source De-identified EHR data obtained, after Institutional Review Board approval Data obtained from a Midwest hospital All data from patients hospitalized between 1/1/09 - 12/31/11 (including all encounters through 12/31/13) Billing diagnosis code of sepsis (ICD-9: 785.5*, 038.*, 998.*, 599.*, 995.9*) 1,993 patients (1,270 with little missing data) 189 (177) Severe sepsis/ septic shock (995.92 and 785.5*) 1,804 (1,093) Other sepsis diagnoses Exclusion criteria: Patients with cardiogenic shock Patients with no antibiotic therapy

28. Study Sequence Baseline and Comorbidities Propensity Score Matching “TimeZero” Onset of Sepsis Start SSC Recommenda- tions Mortality and complications

29. Baseline Sociodemographics Age Gender Race/ ethnicity Payer (Medicaid for low income) Vital signs Heart rate (HR) Respiratory Rate (RR) Temperature (Temp) Mean arterial pressure (MAP) Laboratory results Lactate White Blood Cell Count (WBC)

30. Sepsis Time Zero At least 2 of the following criteria: MAP < 65 HR > 100 RR > 20 Temp 100.94 F WBC 12 Lactate > 2.0

31. Baseline/ Outcomes 5 outcome variables In-hospital mortality New complication (in hospital and up to 30 days after discharge) Cerebrovascular Respiratory Cardiovascular Kidney

32. SSC guideline - Interventions

33. Data Preparation Matching SSC guidelines to data elements Data quality assessment based on literature and domain knowledge Missing values (lactate – 7.7%, temp – 3%, WBC – 3%) Out of range values (CVP, > 50 for 133 patients, some negative values Excluded negative values and those > 30 For each data element, we evaluated range and created rules for suitable range Compared with other values i.e. MAP and SBP/ DBP Determine use of one or more flowsheet measures for vital signs

34. Baseline Characteristics Characteristics Patient Count n=177 Characteristics Patient Count n=177 Mean (IQR) Age (years)61 (51-71)Temperature98.4 (97.3-99.5) Gender (Male)102Heart rate101.3 (87.4-200.4) Race (Caucasian)97Respiratory rate20.6 (17.1-22.8) Ethnicity (Latino)11Cardiovascular100 Payer (Medicaid)102Cerebrovascular66 White blood cell15.8 (9.1-18.6))Respiratory69 Lactate2.8 (1.6-2.8)Kidney62 Mean blood pressure73.9 (40.7)

35. Compliance with SSC Guidelines Rules DescriptionPatient Count / % YN% ComplN/A 1. Was Blood Culture done? (BCulture)12651710 2. Was Antibiotic given after Blood Culture? (Antibiotic)99277951 3. Was Lactate checked? (Lactate)12750720 4. Was Fluid Resuscitation done if Lactate > 4? (LactateFluid)360100141 5. Was Blood Glucose checked? (BGlucose)13245750 6. Was Insulin given if two Blood Glucose measures were > 180? (GlucoseInsulin) 38883131 7. Was MAP checked? (MAP)17701000 8. Was Fluid Resuscitation give if MAP < 65? (MAPFluids)16069611 9. Was Vasopressor given if MAP < 65 after Fluid Resuscitation? (Vasopressor) 261401611 10. Was CVP checked? (CVP)12156680 11. Was Fluid Resuscitation done if CVP < 2? (CVPFluids)1516290 12. Was Albumin given if CVP < 2 after Fluid Resuscitation? (Albumin) 41127162 13. Was a Diuretic given if CVP above 12? (Diuretic)10711296 14. Was there Respiratory Distress*? (RespDistress)16710940 15. Was a ventilator given if there was Respiratory Distress? (Ventilator) 92755510

36. Results - Mortality CI= (0.03, 0.20 )

37. Results - Complications CardiovascularRespiratoryKidneyCerebrovascularDeath BCulture(-0.11, 0.15)(-0.16, 0.12)(-0.15, 0.11)(-0.09, 0.20)(-0.14, 0.09) Antibiotic(-0.16, 0.10)(-0.23, 0.13)(-0.08, 0.26)(-0.09, 0.28)(-0.21, 0.10) Lactose(-0.05, 0.19)(-0.20, 0.07)(-0.08, 0.18)(-0.04, 0.21)(-0.12, 0.10) BGlucose(-0.02, 0.25)(-0.02, 0.28)(-0.16, 0.14)(-0.06, 0.18)(-0.19, 0.09) Vasopressor(-0.11, 0.27)(0.04, 0.35)(-0.20, 0.17)(-0.32, -0.07)(-0.10, 0.21) CVP(-0.03, 0.16)(-0.06, 0.17)(-0.10, 0.14)(-0.08, 0.16)(-0.08, 0.13) RespDistress(-0.25, 0.36)(-0.36, 0.37)(-0.14, 0.40)(-0.30, 0.37)(-0.25, 0.14) Ventilator(0.04, 0.19)(0.08, 0.32)(-0.11, 0.09)(-0.08, 0.11)(0.03, 0.20) CI (0.04, 0.35)CI (0.04, 0.19) CI (0.08, 0.32) CI (-0.32, -0.07)

38. Limitations Small sample size No attempt to add guideline timing of 3 or 6 hours Guidelines as a whole may affect outcome vs single recommendations within guideline – no comparison group Timing of data used (2009 – 2013) – may not reflect current practice SSC Guidelines may not have been thoroughly implemented at health organization

39. Conclusions EHR data can be used to estimate compliance with individual guideline recommendations EHR can be used to estimate the effect of the guideline adherence on sepsis-related complications Some guideline recommendations are protective for patients for certain outcomes Other variables may be needed to control for variation in severity of illness or variation in practice

40. Overall Conclusion Flowsheet data represents a significant contribution to research Challenges in using flowsheet data without standardization What question would you want to investigate using flowsheet data? Who would be on your research team (type of expertise)?

41. Questions? Further Information Bonnie L. Westra, PhD, RN, FAAN, FACMI westr006@umn.edu westr006@umn.edu