For Quality Assurance Committee Delegates SITE VISITOR TRAINING For Quality Assurance Committee Delegates Created by: Megan O’Banion, Psy.D. CAPIC Chair 2015-2016 Last updated by Rene Puliatti, June 2019.

AGENDA Introduction Purpose Expectations Preparation Procedures Documentation Navigating issues Questions?

INTRODUCTION

WHAT IS CAPIC? The California Psychology Internship Council (CAPIC) is an association of clinical psychology doctoral programs and doctoral internship training sites throughout the state of California. Created in 1991, CAPIC’s founders recognized that by working together they could increase the quality of professional psychology training throughout the entire  education trajectory.

FUNCTIONS OF CAPIC? Among its many functions, CAPIC: Facilitates doctoral internship and postdoctoral program quality assurance. Provides standardized application, evaluation, and contract forms used by doctoral internship programs. Implements policies, procedures, and timetables that improve the application / selection process for students, doctoral internship programs, and member schools promotes culturally relevant training. Advocates for increased public awareness of the benefits of student services in the field of mental health.

THE CAPIC BOARD The Board of Directors sets CAPIC policy and oversees  its implementation. They are actively involved in leading CAPIC and representing its members.   The current CAPIC Board of Directors are publicly posted on our website: https://capic.net/board-of-directors/#Board%20of%20Directors CAPIC Board Officers constitute the Executive Committee (EC) along with the Executive Director. The Board Officers include the Chair, Vice-Chair, Secretary and Treasurer. They are elected annually by the Board of Directors themselves and provide leadership for the CAPIC Board.   The current CAPIC Board Officers are publicly posted on our website: https://capic.net/board-of-directors/#Officers

CAPIC OFFICE The CAPIC office is the administrative arm of CAPIC.   It is responsible for running daily CAPIC operations, as well as implementing the policies and directives of the CAPIC Board. Executive Director: René Puliatti (execdirector@capic.net) Assistant Director: Jessica Brown (capicadmin@capic.net)

PURPOSE Membership Criteria & Quality Assurance

MEMBERSHIP CRITERIA CAPIC has adapted its Internship and Postdoctoral membership criteria and clarifications from those of the Association of Psychology Postdoctoral and Internship Centers (APPIC).   Primary differences in CAPIC membership criteria: Encourages but does not require stipends for interns at its doctoral internship programs/agencies.  Explicitly requires comprehensive cultural competency requirements. Ongoing Quality Assurance Review (that includes site visits). https://capic.net/wp-content/uploads/2016/06/DIP-member-criteria-pc-approved.pdf

EXPECTATIONS Site visitor conduct & objectives

SETTING THE STAGE… Site Visitors: Must attend a site visit training session prior to being assigned to a site visit; Represent the QA Committee and are ambassadors of CAPIC; Serve as the reporting and evaluating arm CAPIC. They are the on-site eyes and ears who are charged to evaluate a program’s compliance with CAPIC membership criteria relative to its own mission, goals and objectives; Often develop strong relationships with program representatives over the course of a Site Visit; and Report to the CAPIC office, as well as the QA Committee Chair.

CONFIDENTIALITY QA Reviewers are bound by the signed confidentiality agreement, contract, and the policies, procedures of CAPIC, as well as the ethical and legal obligations of their profession (California Code of Regulations relating to the profession regulated by the California BOP, the Board’s Disciplinary Guidelines and the American Psychological Association’s Ethics Guidelines) The entire Site Visit Process, including information shared by interviewees, is confidential. Site Visitors may discuss the Site Visit with the CAPIC Board and staff only.

ESSENTIAL RESPONSIBILITIES Communication: Timely, professional and accurate Material Review: Online Profile which includes all the uploaded material Onsite tours (of the facilities) Interviews: Program Administrator, Faculty/Supervisors, and Interns/Postdocs. Documentation: Reports Professional Conduct: Site Visitors are invited guests of Internship/Postdoctoral Training Program and serve as representatives of CAPIC. Unprofessional conduct before, during, or after the visit will result in removal of your QA Committee Membership.

PREPARATION I took the job… where do I start?

GETTING ORIENTED CAPIC Office Contact CAPIC Website Access Directory Profile (Therasoft site)

STEP ONE: CAPIC Office Contact The CAPIC Office (Jessica Brown) will provide you the user login information for you to access the aplication/profiles Communicate with the CAPIC Office about your assigned sites, your contact progress and completion status. Please let the CAPIC office know if you are unable to manage the review or are experiencing delays in doing so.

STEP TWO: Accessing CAPIC Online Go to programs.capic.net (our public website/directory) Login at the top of the directory page. Use the login credentials to gain access to the programs directory (the CAPIC office will provide information) in a separate email. Please review all pages and required uploaded documents 2 4 3

PROCEDURES Steps of a site visit

GETTING ORIENTED Materials review Site contact Scheduling the visit

STEP THREE: Documentation Materials … From the QA Reviewer webpage Download a copy of your QA forms: Rating Sheet Site Visit forms Instructions for Board Members These documents are posted on the Quality Assurance page of the CAPIC website

Presenter: Open up and review the RATING SHEET Open up and review the SITE VISIT FORM These sheets can be found on the CAPIC website and/or provided by the CAPIC office

STEP FOUR: Site Materials Review Start with a careful review of the online materials the program provides to CAPIC: Review all application pages (13 pgs for internships and 14 pgs for postdocs) Supporting uploaded documents (located on the 14th page for internships and the 15th page for postdocs) Make notes on your Rating Sheet and Site Visit Form as you review the online materials You will be comparing this online info with the actual situation found at the program during your site visit.

STEP FIVE: Site Materials Review Cont… Supporting Documents: Check page 14 for Internships and page 15 for postdocs and verify the following documents are successfully uploaded. CVs of all supervisors (primary and delegated) Training Manual Didactic Schedule Program’s grievance procedures Program/Agency Brochure Internship certificate of completion with the word “Psychology” Internship-Specific Description/Brochure H2-Yr Narrative (if applicable)

STEP SIX: Contacting the Site Contact the program’s Training Director or designated Contact Person. Document all contacts (email is ideal for this). Introduce yourself (CAPIC representative assigned to complete membership Site Visit). Describe the process. Discuss your review of their uploaded materials. Use this time to ask any questions and resolve any ambiguities you noticed in the materials (anything missing that they need to upload, update, correct, add, remove etc.?) This may take some back and forth (document) Once any clarifications/concerns are resolved, schedule a site visit.

STEP SEVEN: The Site Visit The purpose of the site visit is to ensure the paperwork matches the reality on site, and to address any issues that can only be known from a site visit (e.g. actual presence of supervisors and other relevant staff, training practices, concerns from interns, etc.). Site visits should be scheduled for when a largest majority of all training personnel can attend (including interns/postdocs). Bring questions about their application/profile and your notes on your Rating Sheet and Site Visit Forms. Visits typically take 1.5 - 2 hours onsite and includes: Facility tour Interview with the Training Director Group interviews with supervisors Group interviews with interns/postdocs (as appropriate)

CONDUCTING A QUALITY VISIT Rich Interviews: Introduce the purpose Allow room for questions and concerns Generally, have each stakeholder describe their role in the program, program strengths and pain-points, areas of change or growth, their experience in the program, and recommendations Training director should also be able to address each element on your Rating Sheet as questions Thorough tour: Safety Health records and test security Staff and supervisors adequately present Space allocation for trainees Training and treatment environment Interpersonal dynamics Does it all match how the program is described? Do THEY have questions about CAPIC? (you can provide them with contact information for the Executive Director and/or the CAPIC office)

DOCUMENTATION

IF IT WASN’T WRITTEN DOWN, IT DIDN’T HAPPEN… COMPLETE the Rating Sheet and Site Visit form. If there are shortcomings or issues with an application, follow-up with the program directly for remediation and arrange another review (and site visit if necessary). The CAPIC office can assist with some of this follow-up, but the best person for explaining and resolving any shortcomings is the SITE VISITOR. Send the completed forms, along with you recommendations and any concerns, directly to the CAPIC Office at capicadmin@capic.net or 415-744-1202 (fax). NOTE: After an administrative review, send the documents to the CAPIC Office. The office uploads the completed Site Visit Form and Rating Sheet to the shared drive. This is for internal use only and is accessible to the CAPIC office and Board members, but is not accessible to the public or to CAPIC program members.

NAVIGATING ISSUES

DIPLOMACY When you encounter inaccuracies, errors, mistakes, missing materials, or apparent violation of the membership criteria, assume it is due to a lack of knowledge …. Take some time to kindly educate and help the program make the corrections needed. Feel free to consult with the CAPIC Office. We want to help our members have the best training program they can.

LARGER CONCERNS Not returning your calls/emails Fudging numbers: Overlapping supervision hours, supervisors holding multiple roles in the program, “in-between” hours (e.g. 30 hours per week). False advertising What they say really doesn’t match what you see HIPPA concerns (records) Poor training conditions (abusive) Ethical/Legal violations witnessed Express your concerns: indicate it is necessary for you to consult with the CAPIC office and you will circle-back with them.

RESOLVING CONCERNS Can the shortcomings be adequately addressed via phone or email  (e.g. without another site visit)? Yes if it is documentation/materials related. You would simply need the site to send you their updated and corrected documents for review. Once they pass muster, direct the site to upload them to their CAPIC profile (like they did for their membership application) then confirm they did so. No if the items are related to not meeting with key members of the training program (interns/postdocs), observed safety issues, space allocation and/or records security, etc. then an additional site visit would need to occur. You can always consult with the CAPIC Office directly before taking any steps to resolved issues. Keep us informed… when resolving any issues, copy the CAPIC Office (capicadmin@capic.net). Will the QA Reviewer be further compensated for their time in conducting this follow up? If addressing the shortcomings MUST require another site visit, this must be approved by the CAPIC Office and then the QA Reviewer be compensated for her/his travel. If two visits are needed, will documentation need to be submitted for each visit? Yes. We want the QA Reviewers to do quality work, but we don’t want to promote unnecessary extra visits or work for you…. Diplomacy is key here.

RESOLVING CONCERNS cont. If the concerns remain after your attempts to help the site make the needed corrections, the Chair of the QA committee will raise the issue to the committee then to the Board (as necessary). Decision about removing the site from the match or revoking membership status are determined by the QA Committee and/or Board Vote (as needed).

QUESTIONS?