Treatment Perceptions Survey (TPS) DMC-ODS County administrators, please feel free to tailor/edit these slides specifically for your provider network.

Why are Counties/Providers Administering the TPS? Address the data collection needs for the Centers for Medicare and Medicaid Services (CMS) required evaluation of the DMC-ODS demonstration Fulfill the county’s External Quality Review Organization (EQRO) requirement related to conducting a client satisfaction survey using a validated tool Support DMC-ODS Quality Improvement efforts

How was the TPS Developed? Adult version based on San Francisco County’s Treatment Satisfaction Survey Youth version based on Los Angeles County’s Treatment Perception Survey for Youth Input from stakeholders (e.g., county BH administrators, DMC-ODS evaluation Advisory Group, EQRO, DHCS, Youth System of Care Evaluation Team at Azusa Pacific University ) Some survey items and procedures are similar to the Consumer Perceptions Survey (MHSIP) administered on the Mental Health side.

When Do Counties Administer the Surveys and Submit Data to UCLA? Waiver Year Survey Period Dates Deadline Dates for Sending Paper Forms to UCLA Deadline Dates for Submitting Data Files to UCLA 2018 October 1-5, 2018 October 22, 2018 November 19, 2018 2019 October 7-11, 2019 October 28, 2019 November 25, 2019 2020 October 5-9, 2020 October 26, 2020 November 23, 2020 Counties that have executed DMC-ODS contracts (e.g., have gone live) as of August 24, 2018 are required to administer the TPS during the October 1-5, 2018 survey period. Counties that are going live after August 24, 2018 (e.g., October 1, 2018) are invited and encouraged (but not required) to participate in the October 2018 TPS.

Who Should be Surveyed? Youth (12 -17 years old) and adult (18 years old and older) clients who present in-person and receive face-to-face services at providers within the county during the survey period Includes clients receiving face-to-face services outside the office (e.g., field-based settings) during the 5-day survey period Treatment settings: OP/IOP, Residential, OTP/NTP, Detox/WM (standalone), partial hospitalization The procedures for administering the TPS to adults and youth are the same.

Who Should Not be Surveyed with the TPS? Clients who do not receive face-to-face services during the survey period Clients in immediate crisis (e.g., emergency situation)

TPS Forms 14 questions plus demographic items on the TPS Adult form 18 questions plus demographic items on the TPS Youth form 13 languages - English, Chinese, Spanish, Tagalog, Vietnamese, Russian, Arabic, Korean, Armenian (Eastern and Western), Cambodian, Farsi, and Hmong 1-page and 2-page (large font) versions

[REPLACE WITH MOST CURRENT VERSION] Please be sure to use the latest version of the forms, which can be found on the TPS website.

Survey Form Preparation Accurate information is required for UCLA to prepare program-level summary reports for your county. Fill in the information by using: the online fillable feature before printing the Adult and Youth TPS forms (PDFs); or a black or blue pen This section should be pre-filled by the county or providers before offering the form to clients. Please be sure that providers know their CalOMS Provider ID and fill in the appropriate treatment setting. This information is required for purposes of the ODS evaluation and for preparing county- and program-level reports.

Survey Form Preparation, cont’d Print survey forms from PDF files that can be found on the TPS website. Please do not photocopy the survey forms. Print in black/white on plain white paper. Use both sides of the page (double-sided) when printing the 2-page forms. Do not staple or use paper clips to attach the pages.

How Should the Survey be Administered? Offer survey forms to all youth (12-17 years old) and adult clients receiving face-to-face services during the 5-day survey period. Clients need complete only one form during survey Whether the client completes the survey form will in no way adversely affect the services s/he receives. Inform clients that the survey is anonymous. They should not write their names on the forms. Be sure clients use a black or dark blue ball point pen. If the client makes a mistake on the form, s/he should draw an “x” over the incorrect entry. NTPs/OTPs: Because patients typically dose daily, they need complete only one survey form when they present for services during the survey period. Programs do not need to keep track of or monitor who has or has not completed a survey form. Staff can ask the client if s/he has already been offered a survey form during the survey period. If the person indicates that s/he has not been offered a survey, then the staff member can hand her/him a survey form to complete.

What About Client Confidentiality? Direct service staff must not be present while the client completes the survey. A non-clinical staff person, consumer advocate, or volunteer can help the client complete the survey form upon request by the client. Staff are not to influence how a client responds or deny a client the opportunity to complete the survey. Clients are to place completed survey forms directly into a ballot-type survey form collection box or large envelope.

Client Confidentiality, continued Clients receiving services outside the office during the survey period should fill out a survey form and seal it in an envelope (provided by staff). Staff should deposit the envelope into the survey collection box or large envelope with the other completed forms upon returning to the office. Agency staff should package the completed forms for delivery/shipping to the designated county administrator who is coordinating the survey.

Special Note: Client Comments It is the responsibility of the program/county Clinical Director/ Supervisor to review the client comments (prior to sending the forms to UCLA) for anything that may need immediate attention. If sending electronic data to UCLA, please do not include the client comments. UCLA does not have the resources to review client comments, but will make the comments that are written within the comments box on the TPS form available to counties via the county’s folder in Box.

How Should We Submit Paper TPS Forms Data to UCLA for Scanning? Compile completed survey forms from provider network. Complete and email the Shipping Form to UCLA. (UCLA will prepare UPS shipping labels and order supplies, if needed.) Pack the forms in a box/envelope and complete the Cover Sheet. Use new boxes and envelopes. Be sure to seal/tape boxes well to prevent damage during shipping. Send the box(es)/envelope(s) to UCLA. Counties should also review the forms to ensure that the CalOMS Provider ID and Treatment Settings are filled in prior to submitting the forms (or data file) to UCLA.

Shipment Form

Cover Sheet Please be sure to complete and include this form with your shipment.

How Should We Prepare and Submit Electronic TPS Data Files to UCLA? Compile completed survey forms from provider network. Inform UCLA that your county will be submitting data files rather than paper forms for scanning. Follow the instructions posted on the TPS website for preparing and submitting electronic data Codebook Uploading files to your county’s folder in UCLA Health Sciences Box

Data Scanning, Analysis, and Reporting UCLA will scan and analyze the data, and prepare county- and program-level summary reports. Reports prepared within 3 months after receipt of completed survey forms. County access to images of client comments. UCLA will provide counties with access to their raw data files. Images of text written within the Comments box on the form will be provided to the counties. Any comments/writing outside of the Comments box will not be captured in the image. Raw survey data, including demographics, will be provided to the county. UCLA’s Health Sciences Box – a secure HIPAA compliant file sharing platform – will be used to share files between UCLA and each county. Programs that fill in the Program Reporting Unit field will receive Reporting Unit-level reports. Otherwise by default, reports will be prepared according to the CalOMS ID and treatment setting.

County-Level Summary Report: Sample Tables The county and individual program summary reports will be provided to county administrators for their use. They are not intended to be for public release. If counties choose to release these reports to the public, it is recommended they follow the Data De-Identification Guidelines (DDG) prepared by the California Department of Health Care Services. TPS data for youth will also be included in reports provided to the county.

County-Level Summary Report: Sample Tables

County-Level Summary Report: Sample Table

County-Level Summary Report: Sample Table

County-Level Summary Report: Sample Figure

County-Level Summary Report: Sample Figure

Program-Level Summary Report: Sample Comparison to County Figure The program-level summary reports include information similar to the county-level reports, with the addition of a figure comparing the program and county overall average scores.

What is the UCLA Health Sciences Box? It is a secure, HIPAA-compliant file-sharing platform. Each county will have a separate folder. UCLA will send an invitation to individuals identified by the County Administrator to set up a free Box account. County folders will contain county- and program-level summary reports, raw data files (and images of client comments). Counties that scan/enter the data locally will be able to upload their data files into their folders in Box to UCLA. Box accounts expire 6 months after they have been created, so counties may want to download the files.

FAQs Survey Administration Survey Forms Survey Data Submission/Analysis/Access, and Reporting Link: http://uclaisap.org/ca- policy/assets/documents/TPS/FAQs.pdf

What Information is Posted on the TPS Website? MHSUDS Information Notice No: 17-026 and TPS Instructions (Adult), and MHSUDS Information Notice No: 18-032 and TPS Instructions (Youth) Survey forms PDF files Teleform Definition Files FAQs Instructions for shipping forms to UCLA (e.g., UPS shipping labels, supplies, cover sheet) Instructions for submitting electronic data files (e.g., codebook) Instructions for using UCLA Health Sciences Box (HIPAA compliant data sharing platform) Sample County- and Program-level summary reports (TPS – adults) TPS website link: http://www.uclaisap.org/ca-policy/html/client-treatment- perceptions-survey.html

Be sure to check the TPS web site periodically for updates: http://uclaisap.org/ca-policy/html/client- treatment-perceptions-survey.html Contact Cheryl Teruya (cteruya@mednet.ucla.edu) if you have questions or feedback.