0 The Facts Don’t Speak For Themselves: AHRQ 2007 HS Kaplan R Levitan B Rabin Fastman CUMC/NYPH Getting the Story from Aggregate Data AHRQ 2007

1 Event Reports as Rumble Strips Both can help increase safety by revealing danger Neither is reliably quantitative Both may create some unwanted noise in the environment

2 Counting: A Means to an End

3 Steinbeck on Counting

4 Focus of Today’s Presentation Discuss real-time database queries using built-in tools: RAI: Risk Assessment Index “Single Click” Standard reports QBF: Query by Field Data Mining: Clustering, CBR, and HAWK – Constraint - Access: Permissions and roles

5 Access: Permissions and Roles User-based –Role within organization –Role within MERS system Overarching access rule set –Location –Service line/Department –Employee events/Patient complaints –Type: falls, meds, transfusion, equip, etc. –Read-only HIPAA

6 Filter no-harm reports to improve signal-to- noise ratio (SNR) Risk Assessment Index

7 Two Filters to Enhance SNR Impact Frequency Low High Medium

8 “Single Click” Standard Reports Ad-hoc reporting in real-time MERS has a comprehensive list of reports

9 “Single Click” Standard Reports

10 Medication Events by Specific Type

11 Query By Field (QBF): Exact field matches QBF’s filtered results can be fed into any report, graph, or spreadsheet

12 Generation of Graphics on data subset using QBF filter

13 Medication Drill-Down: Categories of Ordering Errors

14 Benchmark Against Total and Other Reporting Sites/Hospitals

15 Real-Time, Formatted Spreadsheets

16 User-Customized Spreadsheets

17 Mining Association Rules Decision Trees ID Clustering Statistical Clustering Data Mining CBR Similarity Matching Textual Numeric Semantic Neural Networks © 2007 by The Trustees of Columbia University in the City of New York. CBR

18 Why Cluster? Clusters show us event reports that are similar across predefined dimensions They may represent: –frequency of a type of event –event trends in time –potential prevention, etc © 2007 by The Trustees of Columbia University in the City of New York.

19 Case-Based Reasoning (CBR) What is case-based reasoning? Case-based reasoning is another methodology for, among other things, identifying clusters of similar events in large databases © 2007 by The Trustees of Columbia University in the City of New York.

20 Clustering/CBR Clustering divides large data sets into coherent subsets that can be studied more easily Given an event report, CBR will –go through all event reports in database –compute similarity between them –find all reports within a certain distance or similarity (defined by the user) These reports form a cluster © 2007 by The Trustees of Columbia University in the City of New York.

21 CBR and Similarity Matching Using CBR, the computer system can establish the closest matches to any target event It can cluster based on similarity It can also identify unique events © 2007 by The Trustees of Columbia University in the City of New York. CBR and Similarity Matching

22 HAWK MERS’ similarity function, HAWK, uses a vector of pre-assigned weights that corresponds to the vector of variables in an event report record. HAWK provides information that can be used to evaluate trends

23 CBR: Another Use CBR can be extended to provide solutions to problems based on past experiences in the database –e.g., a help desk

24 The Facts Don’t Speak For Themselves: “Knowledge resides in the user and not in the collection.” C. West Churchman in The Design of Inquiring Systems