Bisma Sayed, M.S.W. University of Miami Department of Sociology John Dow, M.S.W. South Florida Behavioral Health Network

 Understand the value of utilizing data for decision-making  Determine what should be measured and what data elements should be used  Data analysis and interpretation  Recognize limitations  Validate findings using other data sources

 The recent recession coupled with health care reform has had cascading consequences on behavioral health care service delivery in Florida.  Current funding limitations and budget cuts have increased the urgency for cost-effective and efficient delivery of behavioral health care services.

 How can behavioral health care organizations lower cost, raise quality, and still offer accessible services to increasing numbers of consumers? ◦ Meet standards ◦ Coordinate ◦ Demonstrate outcomes ◦ Eliminate duplication ◦ Produce reportable, effective, and sustainable results

 Quality Improvement vs. Quality Assessment

 Quality Improvement Processes allow organizations to analyze current practices, identify strengths and weaknesses, set goals, and monitor progress  Quality in the behavioral health care setting may be defined as the ‘extent to which a health care service or product produces a desired outcome’

 Quality of care measures ◦ Effective ◦ Appropriate ◦ Safe ◦ Efficient ◦ Responsive ◦ Accessible ◦ Continuous ◦ Capable ◦ Sustainable

 Examine current organizational functioning  Identify target problems  Identify quality of care measures  Identify goals (short term or long term)  Measure baseline performance on quality measures

 Develop and conduct interventions designed to affect the targeted measures  Repeat measurement of performance based on quality indicator  Document and disseminate results.

 “If you do not measure it (or cannot measure it), it didn’t happen.”  How can we measure it?

 Data provides the foundation for quality improvement initiatives ◦ Timely ◦ Transparent ◦ Presented with humility ◦ Based on past lessons learned ◦ Accountability ◦ Presented with compassion and understanding

 The shift to evidence based care coupled with increased technological and statistical advances have resulted in an explosion of data...

... However, this knowledge remains to be harnessed

 The influx of data has led organizations to report data, rather than analyze data.  Data Reporting Data Analysis

Data Information Knowledge Decision Action Data Information Knowledge Decision Action

 Quality Improvement ◦ What is happening? ◦ What factors affect delivery ◦ How can we influence them  Reactive and Proactive  We need data to guide this.  “Data helps to push improvement (by identifying problems) and pull improvement (by identifying opportunities)”

 Facts and statistics collected together for reference or analysis Surveys Literature Reviews Key informants Surveillance data Focus Groups Surveys Literature Reviews Key informants Surveillance data Focus Groups

 Develop overall goal for improvement  Identify objectives using quality of care measures  Identify target populations  Identify data to be collected

 Determine data sources and/or collection method.  Determine data storage, management, and analysis techniques.  Analyze and Interpret Data  Utilize data for decision-making

 Plan ◦ Consider scope and purpose ◦ Target Audience  Learn ( Do not reinvent the wheel) ◦ Literature Reviews ◦ Other sources of data  Test ◦ Pilot-test on a smaller scale to identify challenges  Team work ◦ Involve and Integrate

 Internal Data  External Data  Administrative or Clinical  Regardless of source of data or type of data, it must be reliable and valid ◦ What is reliability and validity?

 Process mapping: (Who? How long? Steps? Costs?)  Brainstorm  Quantitative or Qualitative ◦ Nominal ◦ Ordinal ◦ Interval ◦ Ratio

 Surveys and questionnaires ◦ Ethical Standards ◦ Confidentiality and Anonymity ◦ Response Rates ◦ Existing Surveys ◦ Guidance ◦ Pilot test

 What is your target population? ◦ Consumers? Their families? Providers? Community?

 Clear and Understandable ◦ Specific ◦ Not loaded or leading ◦ No double barreled question ◦ No jargon or acronyms  Allow choice of only one option  Provide reasonable ranges of variation in the response options

 Social Desirability Bias  Target towards population ◦ Appropriate for age, culture and literacy  Include adequate demographic information

 Why do we sample?  Sampling must be representative of your population  Selection bias

 Important step that can cause significant error if not done properly  Identify inconsistencies ◦ For example, the mean age of adolescents sampled across the nation is The range is ◦ Why do we have a 56 year old adolescent?

 Spreadsheet programs ◦ Reporting, not analysis  Database programs ◦ Database changes – Store data with reports ◦ Reporting, not analysis  Statistical Programs ◦ Analysis

 Understand the variables ◦ Categorical and numerical variables  Frequency Distribution  Median and Percentile  Counts and Sums  Measures of central tendency  Measures of variability

 Measures of Central Tendency ◦ Mean ◦ Median ◦ Mode

 Range  Standard Deviation  What does this tell you about your population?

 What is the goal of data analysis in QI?  Descriptive Analyses and Measures of Variation are useful, but...  Inferential statistics can add to the power of your conclusions. ◦ Examine Relationship/Estimate size of difference ◦ Confidence Intervals ◦ Tests of statistical significance

 Correlation Analysis ◦ Correlation Coefficient: Pearson Product Moment Correlation Coefficient (r)  Scatter plots ◦ Linear Relationships ◦ Non-Linear Relationships  Correlation does not equal causation

 Nominal Level Data: Non-Parametric Tests ◦ Chi Square ◦ Cramer’s V/ Contingency Coefficient/Others  Numerical Data: Parametric Tests ◦ T-tests (independent or dependent) ◦ ANOVA ◦ Regressions  Confidence Intervals ◦ What are they? ◦ How can they be used? ◦ Sample size matters

 When you combine your sample value with the margin of error, you obtain a confidence interval.  The confidence interval is the level of confidence that the sample value represents the true value as seen in the overall population.

 For example, the waiting time for appointments for clients referred to your clinic might be expressed as a mean of 13.5 weeks with a 95% confidence interval of 11.6 to 15.3 weeks (95% CI ).  This means that you expect your population on average would wait between 11.6 and 15.3 weeks for an appointment.

 The p value is the probability that the difference you have observed in your study samples could be due to chance.  Smaller p value = lowered probability that results are due to chance  Statistical Significance

 The size of the p value depends on the size of the sample, so be aware of possible mistakes that can occur in interpreting these values.  Statistical significance does not mean clinical significance.

 Keep it simple  Consistent units  Decimal Points  Include raw numbers and percentages  Always include n  Identify missing data  Group data appropriately

 Keep it simple  Avoid complexity  Clear headings  Scale Carefully  Raw numbers and percentages  Always include n  Group data appropriately

 Basic population characteristics: Pie chart; bar graph  Measures of magnitude including comparisons: Bar chart or box plot

 Frequency: Pie chart; bar chart  Trends over time: Line graph  Distribution of Data: Histogram; Scatter plot  Relationship between two things: scatter diagram

 Whether you are collecting your own data or relying on external sources, there is a difference between compiling/reporting data and analyzing data ◦ Data : petabytes ◦ Reports : terabytes ◦ Excel : gigabytes ◦ PowerPoint : megabytes ◦ Insights : bytes ◦ One business decision based on actual data: Priceless 1

 What is the problem?  What can you improve?  How can you improve?  Have you achieved improvement?  Have we sustained improvement?

 State and national datasets provide important information about key health indicators and can serve as basis for comparison.  However, we must be careful in interpreting and analyzing this data. ◦ Understand limitations  Understand how data is presented ◦ Mean, Median, Mode ◦ Raw sums or percentages

 Level of variables ◦ Individual ◦ Community ◦ State  State level data can help guide decisions, but you must examine individual data in your community to determine if the problem exists at a local level.

 What does data drive? ◦ Assessment ◦ Priority setting ◦ Allocation of resources ◦ Directives to staff and community ◦ Evaluation of clinical outcomes ◦ Basis of QI for providers ◦ Feedback ◦ Sets the groundwork for comprehensive planning

 Assess performance and identify gaps  Understand the needs and opinion of stakeholders  Prioritize problems and improvement projects  Establish overall aims and targets for improvement

 Establish a clear case for the need for improvement.  Data assists in sustained improvement: feedback to reinforce change and demonstrate benefits.