1. Introduction to Encounter Data Validation Presenter: Thomas Miller, MA Executive Director, Research and Analysis Team 1

2. Welcome  About me  Rules for engagement  Presentation overview The importance of encounter data Trends in Federal policy CMS protocols Florida EDV study 2

3. Objectives 1. Learn why Encounter Data Validation studies are important. 2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data. 3. Understand the proposed scope of work for Florida Medicaid’s SFY 2013-2014 encounter data validation study. 3

4. 4

5. Importance of Encounter Data  Accurate and complete data are critical to success of managed care programs  Essential for overall management and oversight of Florida’s Medicaid program –Ability to monitor and improve quality of care –Establish performance measures –Generate accurate and reliable reports –Obtain utilization and cost information 5

6. Importance of Encounter Data 6

7.  Used by MCOs and the State for many purposes –Performance measure development and calculation –Performance improvement measurement –Focused studies/quality activities –Rate-setting –Compliance monitoring –Provider practice patterns 7

8. Key Trends  Importance of Federal and State monitoring –Development of core measurement sets Medicare versus Medicaid Health care reform Holding health care accountable  Data, not anecdotes 8

9. Key Trends in the News 9

10. Key Trends  Findings from a recent article in Medicare and Medicaid Research Review, Assessing the Usability of MAX 2008 Encounter Data for Comprehensive Managed Care –Objective: Assess availability, completeness, quality, and usability of encounter data –Results: High rates for reporting by key encounter data types –Conclusions: Completeness and quality of encounter data were high 10

11. Objectives 1. Learn why Encounter Data Validation studies are important. 2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data. 3. Understand the proposed scope of work for Florida Medicaid’s SFY 2013-2014 encounter data validation study.11 11

12. 12

13. Objectives 1. Learn why Encounter Data Validation studies are important. 2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data. 3. Understand the proposed scope of work for Florida Medicaid’s SFY 2013-2014 encounter data validation study. 13

14. EQR Protocol  Developed and refined with the maturation of the External Quality Review program 14

15. EQR Protocol  Specific guideline for External Quality Review Organizations (EQRO) to use when assessing completeness and accuracy of encounter data.  Data submitted by Managed Care Organizations (MCO) to the State 15

16. EQR Protocol  State establishes standards for encounter data  State must establish the following standards: –Definition of “encounter” –Types of encounters –Data accuracy and completeness –Objective standards for data comparison 16

17. EQR Protocol  Five core activities 1.Review state requirements 2.Review MCO’s capability 3.Analyze electronic encounter data 4.Review of medical records 5.Submission of findings and recommendations 17

18. EQR Protocol  Attachment A: Encounter Data Tables Table 2: Data Element Validity Requirements 18

19. EQR Protocol  Five core activities 1.Review state requirements Develop understanding of State-specific policies and procedures for collecting and submitting encounter data Identify data exchange protocols and layouts Evaluate encounter data system interchange flows, including system edits and submission timelines Review existing encounter data quality activities, requirements, and performance standards 19

20. EQR Protocol  Five key activities, continued 2.Review MCO’s capability Develop, conduct, and review MCO’s Information System Capabilities Assessment –Identification of IS vulnerabilities –Key findings address: »Data processing and procedures »Claims/encounter processing and system demonstration »Enrollment Key informant interviews 20

21. EQR Protocol  Five key activities, continued 3.Analyze electronic encounter data STEP 1 - Develop data quality test plan to determine: –Magnitude and type of missing encounter data –Overall data quality issues –MCO data submission issues 21

22. EQR Protocol  Five key activities, continued 3.Analyze electronic encounter data STEP 2 - Verify integrity of encounter data –Macro-level analysis –Encounter file completeness and reasonableness »Volume and utilization by encounter type and service setting »Internal field consistency »General field completeness and validity 22

23. EQR Protocol  Five key activities, continued 3.Analyze electronic encounter data STEP 3 – Generate and Review Analytic Reports –Micro-level analysis –Encounter record completeness and reasonableness »Follows similar analysis as outlined in Step 2 »Analyzing volume/consistency by time, provider, service type 23

24. EQR Protocol  Five key activities, continued 3.Analyze electronic encounter data STEP 4 – Compare findings to state-identified standards –Identification of appropriate benchmark population 24

25. EQR Protocol  Five key activities, continued 4.Review of medical records Verification of the accuracy of coding Protocol assumptions STEP 1 – Determine sampling for medical record review –Identify valid sample size –Encounter- vs. recipient-based samples 25

26. EQR Protocol  Five key activities, continued 4.Review of medical records STEP 2 – Obtain and review medical records and document findings –Procurement efficiencies –Abstraction staff and training –Categorization of errors by level, type, and source –Procurement tracking and abstraction tools 26

27. EQR Protocol  Five key activities, continued 5.Submission of findings Narrative report summarizing findings from Activities 1-4 Actionable recommendations for overall encounter data quality improvement 27

28. Proto what? Questions? 28

29. Objectives 1. Learn why Encounter Data Validation studies are important. 2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data. 3. Understand the proposed scope of work for Florida Medicaid’s SFY 2013-2014 encounter data validation study. 29

30. 30

31. Objectives 1. Learn why Encounter Data Validation studies are important. 2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data. 3. Understand the proposed scope of work for Florida Medicaid’s SFY 2013-2014 encounter data validation study. 31

32. SFY 2013-2014 Encounter Data Validation (EDV) Study Agency for Health Care Administration 32 V ALIDATION OF E NCOUNTER D ATA

33. SFY 2013-2014 Encounter Data Validation (EDV) Study  Review proposed encounter data validation process –Submitted as part of EQR RFP response –Will be conducted in alignment with CMS’ EQR Protocol 4 –Evaluates the accuracy and completeness of encounter data submitted to AHCA by capitated health plans 33

34. SFY 2013-2014 Encounter Data Validation (EDV) Study  Background –Experience –Core competency evaluating data Information system reviews Comparative analyses of MCO and State Medicaid data Medical/clinical record review –Methodology is constructed to provide an effective validation of the quality of data maintained by State agencies within resource requirements 34

35. SFY 2013-2014 Encounter Data Validation (EDV) Study 35

36. SFY 2013-2014 Encounter Data Validation (EDV) Study  Four key steps for conducting successful evaluations –Project implementation –Study design –Data collection & analysis –Reporting & recommendations 36

37. SFY 2013-2014 Encounter Data Validation (EDV) Study  Project Implementation –Kick-off meeting with AHCA Initiated during contract implementation period Review and define overall scope of project Discuss anticipated timelines Define evaluation parameters –Number of MCOs included –Data requirements and limitations –Implementation procedures to validate AHCA’s encounter data –Kick-off meeting with participating MCOs Description of project and finalized study methodology Expectations for MCO involvement 37

38. SFY 2013-2014 Encounter Data Validation (EDV) Study  Study design –Prepare draft methodology including: Study objectives and research questions Data source and collection procedures Measurement methodology Analytic methods Timeline –Review and approval of methodology by AHCA –Develop of detailed analysis plan or technical companion document methodology 38

39. SFY 2013-2014 Encounter Data Validation (EDV) Study  Data collection and analysis – Information systems review Scope to be defined in collaboration with AHCA Identify key encounter data policies and procedures –Selection of key evaluation fields, service groups, and encounter types –Identification of existing/proposed standards –Review of processes affecting data quality Expected to be limited in scope –Focused on building contextual knowledge of systems to facilitate development of effective and actionable recommendations 39

40. SFY 2013-2014 Encounter Data Validation (EDV) Study  Data collection and analysis – Information systems review, continued Request for supplemental documents –Encounter data submission process –Previous studies conducted by AHCA Documentation will be used to assess encounter data quality Used of NCQA ® Roadmap where appropriate 40

41. SFY 2013-2014 Encounter Data Validation (EDV) Study  Data collection and analysis, continued – Encounter data source files Review of State encounter data file layouts Prepare data requirements documents Receive, process, and load encounter data –Final status encounters from the Florida Medicaid Management Information System and Decision Support System (FMMIS/DSS) –Final status claims/encounters from MCO adjudication systems –Includes all claim/service types—i.e., inpatient/outpatient, physician visits, dental, and pharmaceutical 41

42. SFY 2013-2014 Encounter Data Validation (EDV) Study  Data collection and analysis – Comparative data analysis of State and MCO encounter data Evaluates the extent to which encounters submitted by MCOs to AHCA are accurate, complete, and reasonable Preliminary file review –Ensures files are sufficient for processing –Involves the basic checks »Percentage present »Percentage valid »Percentage valid values 42

43. SFY 2013-2014 Encounter Data Validation (EDV) Study  Data collection and analysis, continued – Comparison: State data to MCO data Indicators to measure degree of completeness and accuracy for each encounter type –Overall record matching—percentage of state encounters present in MCO files –Field-level matching—percentage of state encounters with exact value match in MCO file for each select data element »Standard fields include: date of service, recipient ID, provider ID, primary diagnosis, procedure code(s), and payment fields 43

44. 44 SFY 2013-2014 Encounter Data Validation (EDV) Study Table X—Diagnosis Code Matching Rates for Institutional Claims Plan Total Number of Matched Claims Encounter-Level MatchingField-Level Matching: % Correctly Matched % With All Diagnoses Correctly Matched in Both Files In First Diagnosis Field In Second Diagnosis Field In Third Diagnosis Field In Fourth Diagnosis Field In Fifth Diagnosis Field Statewide4,655,81792.1%99.1%82.1%88.2%93.0%94.9% Plan A144,09096.3%97.8%99.0%99.6%99.8%99.9% Plan B500,98099.5%>99.9%99.9%99.8% Plan C2,429,62489.1%100.0%75.4%85.0%91.5%94.9% Plan D737,58792.3%>99.9%68.2%75.3%84.0%89.8% Plan E224,193>99.9% Plan F367,80089.8% >99.9% 89.9% Plan G251,543>99.9% 44

45. 45 SFY 2013-2014 Encounter Data Validation (EDV) Study Table Y—Second Diagnosis Field Code Matching Rates for Institutional Claims MCP Total Number of Matched Claims % Correctly Matched in Both Files % Mismatch Due to: Diagnosis Omitted in State File Diagnosis Omitted in Plan File True Diagnosis Mismatch Statewide4,655,81782.1%0.9%12.1%4.9% Plan A144,09099.0%<0.1%0.0%1.0% Plan B500,98099.9%0.1%0.0%0.1% Plan C2,429,62475.4%0.0%23.2%1.4% Plan D737,58768.2%5.7%<0.1%26.1% Plan E224,193>99.9%<0.10%<0.1% Plan F367,800>99.9%0.0%<0.1%0.0% Plan G251,543>99.9%0.0%<0.1% 45

46. Phew… Questions? 46

47. SFY 2013-2014 Encounter Data Validation (EDV) Study  Data collection and analysis, continued – Medical record review Represents the “gold standard” Evaluation of service level accuracy and completeness Proposed methodology –Only include MCOs operational as of January 2013 –EQRO Contract Years 1, 2, and 3 (7/1/2013-6/30/2016): review one- third of selected plans each year –EQRO Contract Years 4 and 5 (7/1/2016-6/30/2018): review one-half of selected plans each year –Procure and abstraction 25 percent of all sampled records each quarter –Minimum 50 cases reviewed per plan –Target professional, dental, and pharmacy encounters 47

48. SFY 2013-2014 Encounter Data Validation (EDV) Study  Data collection and analysis, continued – Medical record review Sample selection methodology 1.To generate list of randomly selected encounters for medical review, HSAG proposes using data files from comparative analyses 2.Two-stage stratified sampling design used to ensure: »Member’s record is selected only once »Number of encounters included in final sample covers all encounter types and proportional to total distribution of encounters 48

49. SFY 2013-2014 Encounter Data Validation (EDV) Study  Data collection and analysis, continued – Medical record review Sample selection methodology –Identify all users by encounter type per MCO –Determine required sample size of each encounter type based on total distribution of users –Randomly select users form each encounter type based on required sample size –Identify all encounters associated with applicable encounter types for the selected users –Final sample will consist of 50 cases randomly selected from applicable encounter types per MCO per year, OR 1,200 cases for 1/3 of all MCOs being reviewed per year –For each encounter type, HSAG will define specific data elements for validation 49

50. SFY 2013-2014 Encounter Data Validation (EDV) Study  Data collection and analysis, continued – Medical record review Procurement of selected sample records –General Process »Once sample is selected, each MCO to receive list of its study cases »HSAG will match selected date of service for each sampled member with rendering provider »MCOs will procure and submit identified medical records to HSAG for review 50

51. SFY 2013-2014 Encounter Data Validation (EDV) Study  Data collection and analysis, continued – Medical record review Procurement of selected sample records –Two-hour technical assistance call with all participating MCOs –HSAG to review project and procurement protocols –Able to accommodate a variety of procurement methods: »Faxing »Hardcopy submissions »Electronic submission via secure file transfer protocol –Note: HSAG applies strict protocols to ensure security and confidentiality of members’ medical records 51

52. SFY 2013-2014 Encounter Data Validation (EDV) Study  Data collection and analysis, continued – Medical record review HSAG procurement and abstraction tool –Data collection, management, and reporting system HSAG reviewers are experienced: –Clinical nurses –Nurse coders Procurement and abstraction process –Based on established policies and procedures –Continually monitored to ensure validity and accuracy »Inter-rater reliability testing & Rater-to-standard testing »All reviewers must achieve 95% accuracy rate »Variety of reports will be generated, i.e., medical record compliance rates 52

53. SFY 2013-2014 Encounter Data Validation (EDV) Study  Data collection and analysis, continued – Medical record review – analysis of cases Compare electronic encounter data to medical record data Analyze record completeness and the accuracy of coding Four primary indicators for data completeness and accuracy 1.Medical Record Agreement 2.Medical Record Omission (surplus) 3.Encounter Record Omission (missing) 4.Erroneous 53

54. SFY 2013-2014 Encounter Data Validation (EDV) Study  Reporting and recommendations –Prepare draft report of findings including: Indicator results Sub-analysis findings Preparation of supplemental findings for future evaluation by MCOs –Presented for statewide and MCO-specific results –Actionable recommendations for improvement 54

55. Objectives 1. Learn why Encounter Data Validation studies are important. 2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data. 3. Understand the proposed scope of work for Florida Medicaid’s SFY 2013-2014 encounter data validation study. 55

56. Questions