Mental Health Data Alliance, LLC (MHData) January 4th, 2018 DHCS CSI and DCR Data Quality Improvement Project DCR Webinar #6 Issues with XML Submission Mental Health Data Alliance, LLC (MHData) January 4th, 2018

Project Goals Support the ability to submit good data to the current state CSI and DCR mental health data systems Close the feedback loop for counties to validate that they have good data in the CSI and DCR state mental health data systems Improve the value of state CSI and DCR mental health data systems for counties, the state and stakeholders

Approach Over 1.5 years: Provided counties with 2 reports which provide an overview of current data and errors for CSI and DCR Met with counties to review reports and identify potential causes of data patterns and inconsistencies Work with counties and DHCS to improve processes to submit data to DCR and CSI Provide counties with 2 reports which provide an overview of improved data for CSI and DCR

XML Certified Counties & EHRs Alameda Clinician’s Gateway + Insist Los Angeles Internal Marin Clinician’s Gateway Mendocino NetSmart Avatar Monterey NetSmart Avatar Orange Caminar Placer NetSmart Avatar Riverside Internal San Bernardino Internal Santa Barbara Clinician’s Gateway Sonoma Caminar Ventura NetSmart Avatar

XML Issues Experienced by All Submission of assessments out of order are not accepted The online DCR system allows assessments to be completed out of chronological order Does XML support submission of assessments out of chronological order? Is this an XML DCR bug? Submission of further assessments are not allowed once one is missed The online DCR system allows assessments to be submitted after one is missed Does XML support submission allow assessments to be submitted once one is missed? Due to the nature of partnerships, assessments will be missed; is it intended that no further data should be submitted after a missed assessment? Is this an XML DCR bug? Some local systems collect 3Ms outside of the 45 day window but are unable to submit 3M’s to the DCR because they are generally after the closing due date

XML Issues Experienced by All Partners who are reopened after 365 days are not being accepted in the DCR system Change to the DCR at some point that began creating reestablishment KETs for partners who had been gone for a year or more which may have broken the process to submit reestablishment PAFs; is this a bug? Partners who are reopened before 365 days have missing 3Ms, but the DCR will not accept backfilled 3Ms or KETs; the DCR gives an error that partners are not active Does the online system allow backfill of key events and 3Ms? Should the XML system behave similarly or the same; is this a bug? The DCR would not take the flow of multiple discontinuation and reestablishments for a partner as it would only take the most recent discontinuation and reestablishment records while ignoring the others

XML Issues Experienced by All The DCR will not allow overwrite of some assessments; have to delete many partners and start over with submitting all that partners' data XML submitting entities require more documentation, training and guidance on batch submission to correct errors If assessments are deleted in local systems, how do they get deleted in the state’s DCR? If an assessment is “blanked out” in a local system to remove bad data, some local EHRs will not send up a blank KET; would the DCR accept a blank KET record to overwrite and existing KET? Previously, data created from XML batches could only be viewed online, but not corrected; the new BHIS DCR system allows online corrections for XML Counties; more documentation is needed

XML Issues Experienced by All There is not a lot of information provided with the DCR error In order for proper testing of submission of real data, entities require that DHCS support the submission of test files with real data (not deidentified data)

Alameda (Clinician’s Gateway + Insist) Alameda is interested in using the DCR data as a performance quality improvement tool for providers. There are over 300 stagnant partners. Providers often forget to close out partners. If they do remember, they must close out the partners in the two systems in the right order. If they close out the partner in the InSIST system before Clinician's Gateway

Los Angeles (Internal EHR) It would be helpful to have more information and user documentation on XML submission In order for proper testing of submission of real data, it requires that DHCS support the submission of test files with real data (not deidentified data) XML will not allow assessments to be sent out of order There are many 3Ms missing in the state system which exist in the Los Angeles system Assessments deleted in the Los Angeles system may not be removed from the state system Registering a client sometimes fails because of error that previous GUID was used Partners who are reopened after 365 days are not being accepted in the DCR system DCR had stagnant partners for clients who were disenrolled in Los Angeles internal system but not in state system; are disenrollment KETs are being processed and sent to state accurately? There is no process at the county to track or correct DCR rejection errors

Marin (Clinician’s Gateway) Marin is having issues with reporting data correctly from the FSP module in their EHR; for example, child education data is not mapping correctly to XML submission file Some logic is missing in EHR PAF TAY assessment form; the problem is that it does not limit partners to either be legally required to be in school or not, and so it requires both sections to be completed There are “not applicable” options in the EHR which do not exist or map to any state options Some of the Older Adult PAFs are not fully making it into DCR (they show as “pending” in DCR) indicating that an answer under residential information for “tonight” and “yesterday” is needed; however, all variables are correctly entered into the PAF form Second PAFs for clients who were previously in FSP (and more than 12 months have elapsed since discontinuation) are not transferring to DCR and which in turn also prohibits upload of subsequent KETs completed after second PAF

Mendocino (Netsmart Avatar) Working on an older version of NetSmart for the DCR module When the county transferred the clients to an ASO in 2013, the NetSmart Avatar system did not allow a 3M to be submitted once one is missed; there are many 3Ms which are on paper but cannot be entered because a 3M before each was missed

Monterey (Netsmart Avatar) There were a lot of issues at first with NetSmart Avatar because there were a lot of errors related to the PAF

Orange (Caminar) The County feels that the DCR data is very low quality due to a number of issues with the EHR and other limitations The EHR did not allow corrections, but now the corrections are allowed locally, but they have still been unable to correct years of errors related to partners transferring between programs There is a training issue with regard to transferring clients between programs rather than discontinuing the partners The county would like the DCR to allow deletion of KETs County is unable to submit 3M's after due date County is unable to submit 3M's for transfers or those with incorrect PAF dates Partners who are reopened before 365 days have missing 3Ms, but the DCR will not accept backfilled 3Ms or KETs; the DCR gives an error that partners are not active Partners who are reopened after 365 days are not being accepted in the DCR system There are many issues with the transfer of 3Ms and KETS from Caminar to the state Orange county is not able to use Virtual PSCs in the same way they used to use this function There are some providers who host more than one program, which is causing an issue

Placer (Netsmart Avatar) NetSmart Avatar is not able to submit to the DCR data corrections for assessments successfully The DCR will not allow overwrite of some assessments; the county has had to delete many partners and start over with submitting all that partners' data KETs with wrong data cannot be deleted; a workaround is to blank out the KET; however, Avatar will not send up a blank record to overwrite the bad or duplicate record, which prevent Placer from using this method for correcting KETs

Riverside (Internal EHR) The county uploaded large batches of data, but the DCR would not take the flow of discontinuation and reestablishments for a partner as it would only take the most recent discontinuation and reestablishment records The state changed the program names, and now there are replicate programs which are the same County needs to hand clean data to meet the 3M window business rules of the DCR

San Bernardino (Internal EHR) The county uses an internal system which has experienced a number of business rule inconsistencies with the state DCR system, which has resulted in many record errors over the years.

Santa Barbara (Clinician’s Gateway) One provider is not in the DCR

Sonoma (Caminar) There is not a lot of information provided with the DCR error County would be interested in a pre-submission validation checking tool There was a change to the DCR at some point that began creating reestablishment KETs for partners who had been gone for a year or more which may have broken the process for Caminar to submit reestablishment PAFs for partners who had been gone for a year or more There is an error in which the DCR states that the record cannot be changed because the record was created online, but that is a bug because no records were created online for this county

Ventura (Netsmart Avatar) NetSmart Avatar had an issue in that it could not report 3Ms for any partners who existed in the system prior to the switch to using Avatar for submitting DCR data; this issue was resolved and the backlog of 3Ms were submitted

Switching to XML or from XML to Online We will have a webinar on this topic later on February 1, 2018: Flyer Register

Discussion Other issues? Comments? Next Steps? mhsadcrsupport@dhcs.ca.gov

DHCS CSI and DCR Data Quality Improvement Project Mental Health Data Alliance, LLC (MHData) January 4th, 2018