Mental Health Data Alliance, LLC (MHData) June 7th , 2018 DHCS CSI and DCR Data Quality Improvement Project Pre-Submission Validation Mental Health Data Alliance, LLC (MHData) June 7th , 2018

Project Goals Support the ability to submit good data to the current state CSI and DCR mental health data systems Close the feedback loop for counties to validate that they have good data in the CSI and DCR state mental health data systems Improve the value of state CSI and DCR mental health data systems for counties, the state and stakeholders

Approach Over 1.5 years: Provided counties with 2 reports which provide an overview of current data and errors for CSI and DCR Met with counties to review reports and identify potential causes of data patterns and inconsistencies Work with counties and DHCS to improve processes to submit data to DCR and CSI Provide counties with 2 reports which provide an overview of improved data for CSI and DCR

Purpose of Pre-Submission Validation Pre-submission validation allows you to correct errors before submission to CSI to avoid CSI record errors which need to be corrected after submission Some EHRs have pre-submission validation built in Some EHRs suppress the submission of data records which do not pass pre-submission validation Please take our survey about CSI pre-submission validation: https://www.surveymonkey.com/r/CSI-VALIDATION

County comments Sonoma: Pre-submission validation that data will be error free is difficult. DHCS should implement similar pre-submission error checking in their BHIS system. Tehama: DHCS should create a process for pre-submission validation and soft submissions before committing data.

Most Common Fatal Service Record Errors

Most Common Fatal Client Record Errors

Pre-submission Validation of Provider File – Orange County The State of California publishes a list of Mental Health Providers who are authorized to provide services within a specific range of Services (e.g. a range of modes and service function codes). This list is often called “The Provider File”. Each California County has its own unique Provider File. The Provider File is used during the CSI editing process to validate the CSI data submitted by each County. A fatal CSI error will take place if a County submits a CSI service and the provider has not been authorized to perform this service. The Provider File identifies the mode and range of service function codes that may be offered by a registered provider

Pre-submission Validation of Provider File – Orange County Currently the Provider File is published on the ITWS system under the Provider/Legal Entity section. The Provider File is technically a “flat, position sensitive, text file”, meaning that there are no indexes or database structure to the file. Looking up information about a specific provider is difficult.

Pre-submission Validation of Provider File – Orange County A tool was developed by the County of Orange Health Care Agency (HCA) to turn the data in the ITWS Provider File into useful information. Essentially, the Provider Import Tool, reads the ITWS Provider File and creates a Microsoft Access Database that includes a Table containing the provider file information.

Pre-submission Validation of Provider File – Orange County Orange County imports the Provider File in order to edit all CSI services before submitting them to the State. Orange County has developed a CSI editing system that has over 100 CSI edits that identify both fatal and non-fatal errors. The Provider File edit is just one among many other edits that are performed. Orange County edits all CSI data prior to submission, including C, S and P record data. Errors that are found are included in a report back to the data entry source for correction.  Another use of the Provider File Tool would be to make the Provider information more user friendly. Once the data from the Provider File is captured in an Access Database, it could be exported to Excel or other human readable formats.

MHDATA’s CSI Submission File Analysis Tool (FREELY AVAILABLE FROM WWW MHDATA’s CSI Submission File Analysis Tool (FREELY AVAILABLE FROM WWW.MHDATA.ORG WEBSITE) In an effort to strengthen the data collected and reported via the Client and Service Information (CSI) system, the Mental Health Services Oversight and Accountability Commission (MHSOAC) sponsored an effort to provide a tool to assist Counties and Providers with assessing CSI Data Submission Files and CSI Error Files to aide in the identification of data errors and inconsistencies. Note: The tool is now out of data as it uses the DSM IV Diagnoses Axis Codes Download from: https://www.mhdata.org/resources/ https://mhdatapublic.blob.core.windows.net/csi/CSI%20Submission%20File%20Analysis%20Tool%20v2.0.zip

MHDATA’S CSI Submission File Analysis Tool The CSI Submission File Analysis Tool was designed to assist Counties and Providers with assessing CSI Data Submission Files (before submission) and/or CSI Error Files (after submission when errors present) to identify data errors and inconsistencies. The tool allowed users to import CSI record data in either CSI Submission File or CSI Error File formats using the “Import Data” utility. It is provided empty of data and is designed to work with CSI Submission Files and CSI Error Files as defined in the CSI Data Dictionary. The purpose was to organize and display CSI data extracted from CSI Submission Files and CSI Error Files such that users can review records, identify validation errors, and generate validation error reports to aide in the improvement of CSI data quality. The templates and reports were designed for read only viewing of CSI data. Any changes to CSI data had to be entered separately using the County’s existing electronic medical record system or the CSI online system.

Uses of MHDATA’S CSI Submission File Analysis Tool Browsing all Client, Service and Periodic records contained within a CSI Submission File or CSI Error File to review, cross-check and spot-check data values Browsing all Client, Service and Periodic records with identified validation errors contained within a CSI Submission File or CSI Error File to quickly investigate and identify errant data elements Investigating and troubleshooting data submission issues which arise from an incorrectly formatted CSI Submission File Generating an “Error Summary Report” which identifies the total number of errors identified for each data element to identify trends affecting data quality Generating “Detailed Error Reports” which identify records with identified validation errors Generating “Service Record Errors by Provider Reports” to identify records with validation errors by Provider to enhance the ability of Counties to provide specific feedback regarding validation errors to Providers and aide in the identification of barriers to data quality Creating additional queries and reports based on CSI Submission Files or CSI Error Files utilizing the framework and data codes provided within the CSI Submission File Analysis Too

Importing Data

CSI Data Import

Browsing Data

Navigating and Filtering Record Navigation Record Filtering Filter by Error Field

Browse All Client Records

Client Record CSI Error File Errors Section

Browse all Client Records with Validation Errors

Browse All Service Records Template

Service Record CSI Error File Errors Section

Browse All Periodic Records

Periodic Record CSI Error File Errors Section

Generating CSI Submission File Error Reports

CSI Submission File Error Summary Report

CSI Submission File Service Record Errors by Provider Report

Generating CSI Error File Reports

CSI Error File Error Summary Report

CSI Error File Detailed Report

CSI Error File Service Record Errors by Provider Report

CSI/FSP Reports Tab

Generate Service Function Reports Menu

Saved Report Dropdown

Optional Filters

Clearing Data

NEXT STEPS Please take our survey about CSI pre-submission validation: https://www.surveymonkey.com/r/CSI-VALIDATION MHDATA will be providing counties with reports of top errors with additional information for preventing these errors MHDATA is available for technical assistance to help counties implement a pre-submission validation process – contact MHDATA at support@mhdata.org for more information DHCS support: mhsdata@dhcs.ca.gov

DHCS CSI and DCR Data Quality Improvement Project Mental Health Data Alliance, LLC (MHData)