1. Performance Improvement Projects (PIPs) Technical Assistance for Florida Medicaid PMHPs August 21 st, 2007 Christy Hormann, MSW PIP Review Team Project Leader

2. Purpose of the Webinar Provide Technical Assistance for Activities either Partially Met or Not Met overall by PMHPs for the 2006-2007 validation cycle. How to submit PIPs. Provide resources. Address PIP questions and issues.

3. Presentation Outline Review of PIP Activities II, III,VIII, and IX and Examples PIP submissions for the 2007-2008 validation cycle Questions and Answers

4. Activity II: The Study Question HSAG Evaluation Criteria States the problem to be studied in simple terms. Is answerable. In general, the question should illustrate the point of: Does doing X result in Y?

5. Study Question Example Will the implemented interventions, to improve the percent of new children with serious emotional disturbance starting any needed ongoing service within 14 days of a non-emergent assessment with a professional, result in a higher percent of new children with serious emotional disturbance starting any needed ongoing service within 14 days of a non-emergent assessment with a professional?

6. Study Question Example Does providing members with needed follow-up care, and increasing the percentage of discharges with follow-up appointments, improve the rate of members receiving follow-up care within 7 days after discharge from an inpatient stay for a mental illness?

7. Activity III: Selected Study Indicators HSAG Evaluation Criteria The study indicator(s): Is well defined, objective, and measurable. Is based on practice guidelines, with sources identified.

8. Activity III: Selected Study Indicators HSAG Evaluation Criteria (cont.) The study indicator(s): Allows for the study question to be answered. Measures changes (outcomes) in health or functional status, member satisfaction, or valid process alternatives.

9. Activity III: Selected Study Indicators HSAG Evaluation Criteria (cont.) The study indicator(s): Has available data that can be collected on each indicator. Is a nationally recognized measure such as HEDIS ®, when appropriate. Includes the basis on which each indicator was adopted, if internally developed. HEDIS ® is a registered trademark of the National Committee for Quality Assurance (NCQA).

10. Study Indicator Example Study IndicatorDescribe the rationale for the selection of the study indicator: Based on the expertise of the State and the Quality Improvement Committee, the indicator was chosen due to a high number of PIHPs having difficulty meeting the departments established standard. NumeratorThe number of new children with serious emotional disturbance starting any needed ongoing service within 14 days of a non- emergent assessment with a professional. DenominatorAll new children with serious emotional disturbance starting any needed ongoing service after a non-emergent assessment with a professional. First Measurement Period Dates January 1, 2006 – December 31, 2006 Benchmark95% within 14 days based on the state average (91.1%) Source of Benchmark State based standard Baseline Goal95% within 14 days

11. Study Indicator Example Study IndicatorDescribe the rationale for the selection of the study indicator: Because of the high risk to both members and the health plan, XYZ has attempted to improve the rate of follow-up visits post-hospitalization and thereby prevent acute exacerbations from recurring by improving timely and appropriate coordination between systems of care, in the transition from inpatient to outpatient levels of treatment. XYZ calculates NCQAs HEDIS measure of Follow-Up After Hospitalization for Mental Illness at 7 days to monitor progress in this area annually. Numerator Number of ambulatory or intermediate encounters with a mental health practitioner up to 7 days after a psychiatric hospital discharge of an eligible member. Denominator Number of hospital discharges on or before December 1 st of the calendar year for all eligible members 6 years of age or older as of the date of discharge. First Measurement Period Dates Discharge from January 1 through December 1 with a date of service through December 31 of the measurement year. Benchmark 50 th percentile: 38.4% Source of Benchmark National Medicaid HEDIS 2005 Baseline Goal HEDIS 50 th percentile for the measurement calendar year

12. HSAG Evaluation Criteria The data analysis: Is conducted according to the data analysis plan in the study design. Allows for generalization of the results to the study population if a sample was selected. Identifies factors that threaten internal or external validity of findings. Includes an interpretation of findings. Activity VIII: Data Analysis and Interpretation of Study Results

13. HSAG Evaluation Criteria (cont.) The data analysis: Is presented in a way that provides accurate, clear, and easily understood information. Identifies initial measurement and remeasurement of study indicators. Identifies statistical differences between initial measurement and remeasurement. Identifies factors that affect the ability to compare initial measurement with remeasurement. Includes the extent to which the study was successful. Activity VIII: Data Analysis and Interpretation of Study Results

14. Data Analysis Plan Example Data analysis process: The data will be collected and reported on an annual basis. Statistical analysis will be employed to determine whether statistically significant progress is occurring. The resulting baseline and remeasurement proportions will be compared using a chi-square to determine if the difference was statistically significant, with p<.05 as the generally accepted standard for significance.

15. Activity VIII: Interpretation of Study Results Example Baseline Interpretation The baseline rate was 14.1% of members having had follow-up within 7 days after a discharge from an inpatient stay for mental illness during the time period 1/1/03-12/31/03.

16. Activity VIII: Interpretation of Study Results Example Remeasurement 1 Interpretation The baseline rate of 14.1% of members who had follow-up within 7 days after a discharge from an inpatient stay for mental illness increased to 21.7% for the first remeasurement (1/1/04-12/31/04). This represents a statistically significant increase (p=.005).

17. Activity VIII: Interpretation of Study Results Example Remeasurement 2 Interpretation The second remeasurement rate increased to 27.6% from 21.7%. The increase was not statistically significant (p=0.125). The rate of members who had follow-up within 7 days of a discharge has increased each year, with no decline in performance. The PIP has been successful in increasing follow-up after discharge and will be continued. Third remeasurement data will be collected to assess for sustained improvement.

18. Activity VIII: Interpretation of Study Results Example Factors that threatened the validity of the study included acquiring ABCs health plan members and a change in staff members during the data collection phase of the study. Acquiring ABCs health plan members also affected the ability to compare initial measurement with remeasurement. * If there were no factors that threatened the validity of the study or affected the ability to compare measurements, include that in the documentation.

19. Activity XI: Real Improvement Achieved HSAG Evaluation Criteria The remeasurement methodology is the same as the baseline methodology. There is documented improvement in processes or outcomes of care. The improvement appears to be the result of intervention(s). There is statistical evidence that observed improvement is true improvement.

20. Activity IX: Study Summary Results and Improvement Example

21. New PIP Submissions New PIPs were not submitted to HSAG for the 2006-2007 validation cycle. For new PIP submissions, it is important to contact HSAG to obtain the most current PIP Summary Form.

22. How to Submit Continuing PIPs On-going PIPs are those that were submitted to HSAG for the 2006-2007 validation cycle. Highlight, bold, or add text in a different color, and date any new information that is added to the existing PIP Summary Form. Strikethrough and date any information that no longer applies to the PIP study submission. Ensure all Partially Met and Not Met evaluation elements from the previous validation cycle have been addressed in the documentation.

23. PIP Tips 1.Complete the demographic page before submission. 2.Notify HSAG when the PIP documents are uploaded to the secure FTP site and state the number of documents uploaded. 3. Label ALL attachments and reference them in the body of the PIP study. 4. HSAG does not require personal health information to be submitted. Submit only aggregate results. 5. Document, document, and document!! 6.Look for a frequently asked questions on myfloridaeqro.com. 7.If you have additional questions, contact HSAG.

24. Deliverables September 7th:PMHPs notified electronically of submission date with instructions October 5th:Submit PIP studies to HSAG HSAG will be validating two PIPs per PMHP; one clinical and one nonclinical. If the collaborative PIP is clinical, the other PIP chosen for validation will be nonclinical.

25. Resources Frequently asked Questions and PIP information myfloridaeqro.com NCQA Quality Profiles http://www.qualityprofiles.org/index.asp Institute for Healthcare Improvement – www.ihi.orgwww.ihi.org Center for Healthcare Strategies – www.chcs.orgwww.chcs.org Health Care Quality Improvement Studies in Managed Care Settings – A Guide for State Medicaid Agencies www.ncqa.org/publicationswww.ncqa.org/publications National Guidance Clearinghouse www.guidelines.gov www.guidelines.gov Agency for Healthcare Research and Quality http://www.ahrq.gov/

26. HSAG Contacts For questions contact: Cheryl Neel –cneel@hsag.comcneel@hsag.com –602.745.6201 Denise Driscoll –ddriscoll@hsag.comddriscoll@hsag.com –602.745.6260

27. Questions and Answers